AI Data Ops Engineer

Overview

AI Data Ops Engineer

• for a leading Abu Dhabi-based holding group investing heavily in its data and AI capability.
• You'll engineer the pipelines that feed everyday AI assistants and enterprise AI products — and you'll be the technical authority reviewing and signing off vendor-delivered data architectures before they reach production.
• Reports to the AI Solutions Manager.
• What you'll own:
• Engineer batch and stream pipelines using Fabric Data Pipelines / Azure Data Factory, Synapse, or Databricks.
• Implement data quality rules, schema validation, de-duplication, SCD, and reconciliation checks.
• Operationalize lineage, cataloging, and classifications with Microsoft Purview; enforce RBAC and access patterns.
• Automate CI/CD via Azure DevOps or GitHub with environment promotion, infrastructure-as-code (Bicep/Terraform), and secrets management via Key Vault.
• Build feature stores and model-serving data contracts in partnership with MLOps and AI engineering teams.
• Own reliability: alerts, runbooks, on-call rotation, and cost and performance optimization.
• Review and finalize vendor-delivered data pipelines and data architecture for AI projects; ensure compliance with client standards for security, performance, and reliability; approve production readiness.
• Collaborate with delivery partner squads on interface specifications, test data, and delivery checkpoints; support SIT/UAT and production cutover.
• Define and enforce Data Contracts and SLAs per priority dataset (schema, refresh frequency, quality thresholds, reconciliation checks, and consumer expectations).
• Own data incident management: classification, RCA, corrective actions, and prevention of recurring nonconformities.
• Formalize the handshake with AI/ML/DevOps engineers on feature and embedding pipelines, monitoring hooks, and release gates for data-dependent AI deployments.
• What you bring:
• 6–8 years in data engineering with strong SQL and PySpark and cloud-native data services.
• Hands-on experience across the Azure data stack: Fabric/Synapse, Data Factory, ADLS Gen2, Delta Lake/Parquet.
• Bachelor's in Computer Science, Engineering, or equivalent.
• Core skills and tools required:
• Python and PySpark, SQL, Lakehouse patterns, medallion architecture.
• Microsoft Purview: catalog, lineage, classifications; data privacy controls and masking.
• CI/CD with Azure DevOps or GitHub; IaC with Bicep or Terraform; Docker basics.
• Observability: Kusto/KQL, Azure Monitor, Log Analytics; performance tuning and FinOps.
• Production incident response and RCA discipline.
• Required certifications:
• Microsoft Certified: Azure Data Engineer Associate (DP-203)
• Preferred certifications:

• Microsoft Certified: Azure Fundamentals (AZ-900)

• Databricks Data Engineer Associate or Professional

Experience

Salary Range