Sr Data Platform Engineer

Role overview

Work arrangement

• Hybrid arrangement: four days in the office and one day working from home.
• The description identifies the role as located in Abu Dhabi, UAE.

Platform, pipelines, and operations

• Own Azure Databricks workspaces, environments, catalogs, schemas, storage, compute, workload separation, platform standards, and governance.
• Build production pipelines using Databricks, Spark, Python/PySpark, SQL, and Delta Lake; integrate ERP, databases, SaaS, APIs, and files.
• Define ingestion, incremental load, schema evolution, error handling, logging, reconciliation, reprocessing, and recovery patterns.
• Operate production workloads; implement monitoring, alerting, runbooks, incident response, recovery procedures, and operational dashboards.
• Improve reliability, observability, security, performance, scalability, supportability, and cost efficiency.

Quality, leadership, and enablement

• Implement data quality controls and ensure datasets have ownership, classification, lineage, refresh schedules, expectations, and documentation.
• Lead, mentor, and support two Data Engineers; review designs and code and develop team engineering capabilities.
• Partner across business, ERP, product, analytics, security, infrastructure, and DevOps teams to deliver reusable, governed data products.
• Enable trusted data for reporting, automation, AI/ML, and digital applications; advise leadership on platform strategy and technical decisions.

Required experience

• 7+ years of relevant experience in data engineering, data platforms, cloud data architecture, or enterprise data integration.
• Production cloud pipeline experience and strong production experience with Azure Databricks, Delta Lake, and Lakehouse architecture.
• Experience leading or mentoring Data Engineers and providing technical direction across initiatives.
• Experience integrating enterprise sources and reviewing vendor or contractor deliverables.

Required technical skills

• Strong expertise in Azure Databricks, Apache Spark, Python/PySpark, advanced SQL, Delta Lake, and Lakehouse design patterns.
• Knowledge of Unity Catalog, role-based access control, lineage, classification, metadata, and access governance.
• Experience with Azure Data Lake Storage, GitHub or Azure DevOps, CI/CD, secrets management, and cloud integration patterns.
• Strong understanding of data modeling, medallion architecture, data quality, reconciliation, schema evolution, monitoring, and performance optimization.

Preferred qualifications

• Experience with Databricks Workflows, Lakeflow Declarative Pipelines, SQL Warehouses, Serverless compute, MLflow, Databricks Asset Bundles, or infrastructure-as-code.
• Experience with reusable data products, semantic models, governed self-service datasets, Power BI, Tableau, reporting migration, or analytics delivery.
• Exposure to ML/AI pipelines, feature engineering, GenAI, Vector Search, retrieval-augmented generation, automation, or AI-ready data products.
• Industry experience in manufacturing, oil and gas, field services, industrial operations, or ERP-intensive environments is preferred.
• Relevant Databricks, Microsoft Azure Data Engineer, Azure Solutions Architect, or cloud/data certifications are preferred but not required.