Data Engineer

Role overview

Responsibilities

• Build AWS Glue jobs for Bronze-layer ingestion using defined standards and templates.
• Use suitable CDC, full, delta, or snapshot loading methods for source requirements.
• Design historical loads to bring legacy data into the Lakehouse.
• Optimize Glue job performance through DPU allocation, parallelization, and partitioning.
• Coordinate tooling and optimization with platform teams.
• Migrate source tables to Bronze using standards and templates, later leveraging AI-enabled acceleration.
• Version-control jobs and automate production deployment with Git and Terraform.
• Set up source-system connectivity into CDP with source owners; ensure data contracts and monitoring.
• Document work and hand it over to operational support.
• Coordinate patterns and standards with the Data Architect and quality checks with the Data Assurance Lead.
• Partner with platform engineers on tooling and optimization.

Required skills and experience

• Strong experience with AWS Glue, PySpark, and ETL pipeline development.
• Knowledge of Lakehouse architecture and Medallion design principles.
• Familiarity with CDC, delta loads, and historical data ingestion strategies.
• AWS services include Glue, S3, Athena, and Lambda.
• Git and Terraform experience for CI/CD automation.
• Ability to identify automation opportunities and work at pace toward migration targets.
• Strong collaboration and communication skills; experience with Jira and agile ways of working.

Preferred qualifications

• Five or more years in data engineering roles, including hands-on AWS Glue experience, is preferred.
• Financial services or banking domain experience is an advantage.
• Understanding of data quality frameworks such as Soda Core is a plus.