Intermediate Data Engineer

Role overview

Responsibilities

• Build, test, monitor and maintain data pipelines and platform infrastructure using CI/CD, IaC and DevOps practices.
• Apply security controls, encryption, monitoring and secure engineering practices across data and cloud operations.
• Use AWS and PostgreSQL/AWS RDS to deliver and support scalable, reliable data solutions.
• Adopt AI-assisted tools responsibly to improve delivery, quality, documentation, analysis and operational efficiency.
• Automate deployment, testing and operational workflows with GitLab pipelines, version control, Terraform/OpenTofu and scripting.
• Contribute to data quality, lineage, governance, observability and platform reliability.
• Investigate data and operational issues, identify preventative improvements and automate repeatable work.
• Gather business and technical requirements and design data models for reporting, analytics and operational use.
• Contribute to upgrades, migrations, schema updates and platform modernisation.
• Document products and processes, share knowledge, mentor less experienced engineers and contribute to feature deliveries.
• Own work through development, testing, production support, incident resolution and continuous improvement.

Requirements

• Hold or be willing to obtain a relevant AWS certification registerable against the CDL APN.
• Proven experience with AWS services for data engineering and operational support, including services such as RDS, S3, Lambda, ECS, EC2, VPC or CloudWatch.
• Strong SQL skills, including PostgreSQL experience and query optimisation awareness.
• Experience with MySQL and Oracle databases is advantageous.
• Use AI-assisted engineering tools responsibly and be willing to learn and adopt emerging tools.

Tools and technologies

• The description names AWS services including RDS, S3, Lambda, ECS, EC2, VPC and CloudWatch.
• Deployment and infrastructure tools include GitLab pipelines, Terraform/OpenTofu and scripting.
• AI tools mentioned as examples include Kiro and GitHub Copilot, or equivalent technologies.

About CDL