Data Engineer (Microsoft Fabric | Azure | Databricks)

About This Role

Role Summary **** We are seeking a Data Engineer with hands-on experience in Azure-based data platforms and Microsoft Fabric to build scalable data pipelines and analytics solutions.

The ideal candidate will have strong skills in PySpark, SQL, and modern data engineering frameworks , with exposure to lakehouse architecture, real-time data processing, and cloud-native tools .

[center][/center]

Key Responsibilities 1. Data Pipeline Development

• Develop and maintain ETL/ELT pipelines using Azure-native tools and frameworks.
• Process structured and semi-structured data (JSON, XML, CSV, APIs).
• Implement incremental data loading and transformation logic.
• Ensure data quality, validation, and consistency across pipelines.

[center][/center]

2. Microsoft Fabric & Lakehouse Implementation

• Work with Microsoft Fabric components including:[ul data=1]
• Eventstream
• Eventhouse
• Fabric Pipelines

Implement Lakehouse architecture using Medallion model (Bronze, Silver, Gold).Utilize Delta Lake for efficient storage and processing.[center][/center]

3. Azure Data Platform Development

• Build data solutions using:[ul data=1]
• Azure Data Factory (ADF)
• Azure Databricks
• ADLS Gen2

Ingest data from multiple sources:[ul data=1]ERP / SAP systemsAPIs and external data sourcesSupport cloud-based data integration and transformation.[center][/center]

4. Big Data Processing

• Develop data transformation workflows using:[ul data=1]
• PySpark / Apache Spark

Optimize pipelines using:[ul data=1]Partitioning, caching, and efficient joins

Work on both batch and streaming data processing .[center][/center]

5. Data Modeling & Analytics

• Build analytics-ready datasets for reporting and BI.
• Apply dimensional modeling concepts .
• Deliver curated datasets for downstream consumption.

[center][/center]

6. Real-Time Data Processing (Preferred)

• Implement real-time data pipelines using:[ul data=1]
• Microsoft Fabric Eventstream
• IoT / streaming data sources

Enable real-time dashboards and monitoring use cases.[center][/center]

7. Analytics & Visualization Support

• Integrate data pipelines with Power BI dashboards .
• Support business teams with clean and reliable datasets.

[center][/center]

8. Modern Data Tools & Frameworks

• Work with:[ul data=1]
• dbt for transformation and modeling
• Snowflake for analytics workloads

Develop modular, reusable, and scalable data workflows.[center][/center]

9. DevOps & Best Practices

• Use GitHub / Git for version control.
• Follow CI/CD and deployment best practices.
• Document data pipelines and processes.

[center][/center]

Required Skills & Technologies Core Skills

• Python / PySpark
• SQL (Strong proficiency)
• ETL/ELT pipeline development

[center][/center]

Cloud & Tools

• Microsoft Azure:[ul data=1]
• Azure Data Factory (ADF)
• Azure Databricks
• ADLS Gen2

Microsoft Fabric (preferred)[center][/center]

Data Engineering Concepts

• Lakehouse architecture
• Delta Lake
• Data modeling (basic dimensional modeling)
• Incremental data processing

[center][/center]

Additional Tools

• dbt (preferred)
• Snowflake (good to have)
• Power BI

[center][/center]

Experience Requirements

• 3–5 years of experience in Data Engineering
• Hands-on experience in:[ul data=1]
• Azure data ecosystem
• Databricks / PySpark
• Building data pipelines and transformations

[center][/center]

Educational Qualifications

• Bachelor’s degree in Computer Science / Engineering / IT
• Relevant certifications are a plus:[ul data=1]
• Microsoft Fabric (DP-600 / DP-700)
• Azure Data Engineer (DP-203)
• Databricks certifications

[center][/center]

Soft Skills

• Strong analytical and problem-solving skills
• Ability to work in a collaborative Agile environment
• Good communication skills
• Attention to detail and data accuracy

[center][/center]

Nice to Have

• Exposure to real-time/streaming data pipelines
• Experience with API integrations
• Basic understanding of machine learning workflows
• Experience working with ERP/SAP data