Data Platform Engineer

Hiring

Location – Abu Dhabi

What This Role Owns

• **Data warehouse build**
• : Azure Data Lake Services + Synapse/Fabric — dimensional models, ingestion patterns, governed access views
• **Data quality monitoring**
• : automated checks on completeness, freshness, row counts, and classification coverage — quality failures pause pipelines, not skip records
• **Additional source ingestion**
• : pipelines for data sources beyond the initial three (research center data, PMO data, procurement systems)
• **Data mesh enablement**
• : governed data products that entities own and the platform makes accessible
• **Agentic platform data readiness**
• : clean, structured data outputs formatted to the contracts defined by the Principal AI Engineer
• **Schema deployment**
• : migration scripts, versioning, blue-green deployment support
• **Pipeline maintenance**
• : monitoring, troubleshooting, and optimizing existing pipelines

Key Decisions

• Data quality check implementation — what to validate, what thresholds trigger alerts
• Warehouse query optimization and partitioning strategy
• Pipeline scheduling and dependency ordering for new sources
• Incident response for pipeline failures (autonomous resolution; escalate to Lead for extended outages)

Required Skills & Technologies

• Python —
• async data pipelines, background jobs, scripting
• PostgreSQL —
• schema design, migrations (Alembic), query optimization
• Azure Data Lake Storage + Synapse Analytics
• dbt —
• transformation, testing, documentation
• Apache Airflow or Azure Data Factory
• Data quality frameworks (Great Expectations, dbt tests, or custom)
• Observability —
• structured logging, alerting, Azure Monitor or Prometheus/Grafana
• Microsoft Graph API —
• SharePoint, M365 data extraction
• Redis —
• queue management, caching
• Docker —
• containerized pipeline jobs
• SQL —
• advanced analytical queries, window functions, performance tuning

Ideal Candidate

• Has built data pipelines that ran in production and survived malformed upstream data without silent failures.
• Defensive by default — assumes every external source will break and codes accordingly. Writes observable systems: if something fails, the failure is visible immediately.
• Works independently once given direction. Communicates blockers early.
• Has warehouse experience — dimensional models, governed views, and the query patterns that serve analytics and reporting.
• What this role does Not Do
• Define platform architecture or engineering standards (Lead Data Platform Engineer)
• Define data models or classification schema (Principal Data Architect)
• Define governance policies (Governance Lead)
• Build API endpoints (Backend Engineer)
• Write AI prompts (AI Engineer)
• Build frontend components (Full Stack Engineer)