Data Engineer
Role Overview
Build and operate reliable data pipelines that move source data into a lakehouse for analytics and AI consumers.
Deliver automated, performant, secure, governed production pipelines on Azure or Microsoft Fabric, capturing metadata and lineage in Microsoft Purview.
Work with data architects, data quality engineers, business analysts, and AI engineers.
Key Skills for This Role
Full Job Posting
Location and travel
- The role is UAE-based and requires travel.
Assignment duration
- The assignment duration is 6 months.
Role overview
Build and operate reliable data pipelines that move source data into a lakehouse for analytics and AI consumers.
Deliver automated, performant, secure, governed production pipelines on Azure or Microsoft Fabric, capturing metadata and lineage in Microsoft Purview.
Work with data architects, data quality engineers, business analysts, and AI engineers.
Responsibilities
- Build batch, incremental, CDC, and streaming Bronze-layer ingestion pipelines using Azure Data Factory and Fabric pipelines.
- Develop Silver- and Gold-layer transformations using SQL, PySpark, and Spark notebooks, including cleansing, conformance, enrichment, and business rules.
- Implement dimensional models, SCDs, surrogate keys, and data marts to architect designs.
- Integrate data quality checks, validation, quarantine, and reconciliation; publish pipeline metadata and lineage to Microsoft Purview.
- Implement scheduling, dependencies, parameterization, error handling, and alerting.
- Optimize pipeline performance and cost through partitioning, file formats, and compute usage.
- Apply access controls, masking, encryption, and secure credential management to meet UAE data regulations.
- Maintain source-controlled code and deploy through CI/CD with automated tests and environment promotion.
- Support production operations, resolve incidents, maintain runbooks, and coach counterparts for handover.
Must-have qualifications
- 4+ years of data engineering experience; UAE government or regulated-sector experience is preferred.
- Advanced SQL, Python, and PySpark/Spark skills.
- Hands-on experience with Azure Data Factory and Microsoft Fabric, Databricks, or Synapse.
- Strong knowledge of Lakehouse/Medallion architecture, Delta/Parquet, and dimensional modeling, including SCDs and surrogate keys.
- Experience with CI/CD (Azure DevOps or GitHub), source control, and DataOps.
- Familiarity with Microsoft Purview metadata and lineage and data quality tooling.
- Awareness of UAE PDPL, the Dubai Data Law, and DESC ISR.
- Excellent English communication and documentation skills.
Preferred qualification
- Arabic proficiency is advantageous.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Datamatics Global Services Ltd
AI Engineer - Senior
Riyadh, KSA
The employer is seeking a Senior AI Engineer to lead the architecture, development, and production deployment of secure, governed AI and GenAI solutions. Candidates need substantial production AI experience, expert Pytho
AI Engineer
Al Jubayl, KSA
AI Engineer =============== Work Mode : remote (Qatar Consignment ) Language Preference: English Duration: 12-month assignment Role Summary and Overview The AI Engineer is responsible for designing, developing, and deplo
AI Engineer
Riyadh, KSA
The employer is seeking an experienced AI Engineer to build and operate production-grade enterprise AI and GenAI applications. Candidates need 5+ years of software engineering experience, including 2+ years delivering AI
Data Engineer
, UAE
The employer is seeking a Data Engineer to build and operate secure, governed production data pipelines on Azure and Microsoft Fabric. Candidates need 4+ years in data engineering, strong SQL, Python, and PySpark/Spark s
Data Engineer
, UAE
The employer is seeking a data engineer to build and operate governed, production-grade pipelines for analytics and AI consumers. Applicants need 4+ years of data engineering experience, advanced SQL, Python and Spark sk
Data Quality Engineer
, UAE
The employer is seeking a senior data quality specialist to establish an enterprise framework and embed automated controls across lakehouse and analytics layers. Candidates need 4+ years in a relevant data discipline, ad
Data Quality Engineer
, UAE
The employer is seeking a senior data quality specialist to establish an enterprise framework and embed automated controls across lakehouse and analytics layers. Candidates need 4+ years in a relevant data discipline, ad
AI Engineer - Senior
Riyadh, KSA
AI Engineer
Al Jubayl, KSA
AI Engineer
Riyadh, KSA
Data Engineer
, UAE
Data Engineer
, UAE
Data Quality Engineer
, UAE
Data Quality Engineer
, UAE