Data Engineer – Data Quality & Governance

Role Overview

Key Responsibilities

• Design, build, and maintain **Python-based ETL pipelines** across Bronze, Silver, and Gold layers.
• Develop and operationalise **data quality rules in Python** (validity, completeness, consistency, uniqueness, accuracy, timeliness).
• Contribute to **Informatica IDQ 10.5** activities (mapplets, profiling, scorecards) alongside platform specialists.
• Support **data cataloguing and lineage** activities using Informatica EDC.
• Assist governance workflows in **Informatica Axon**, including business glossary and DQ score visibility.
• Build cross-system DQ checks (referential integrity, reconciliation, deduplication).
• Integrate DQ outputs into **Power BI dashboards**.
• Maintain a **modular, testable Python DQ framework** with unit test coverage.
• Support **Talend-based ingestion** at the Bronze layer.
• Participate in **root-cause analysis** of data quality issues.

Required Skills & Experience

• Strong **Python development** skills (pandas, PySpark).
• Experience with **production-grade ETL pipelines**, logging, and error handling.
• Understanding of **Medallion architecture** and data lake design.
• Working knowledge of **SQL** and relational databases.
• Familiarity with **CI/CD practices** and **Git**.
• Conceptual understanding of:
• Informatica IDQ (Developer Tool, profiles, scorecards)
• Informatica EDC (catalog, lineage, classification)
• Informatica Axon (governance, glossary, stewardship workflows)

Desirable Skills

• Experience with **Talend** ingestion workflows
• Exposure to **AI/ML for Data Quality** (anomaly detection, entity resolution)
• Power BI dashboard development
• Informatica IDQ–Axon integration awareness
• Knowledge of data governance frameworks and regulatory reporting

Additional Information

• Candidates must be comfortable working with **sensitive data** and governance controls.
• Experience in **Agile delivery teams** is preferred.
• Strong documentation and stakeholder communication skills are essential.