Data Engineer (G7)

Role overview

Responsibilities

• Lead scalable ETL/ELT pipeline design and development using PySpark and Python; optimise, validate and standardise workflows and batch processes.
• Gather complex stakeholder requirements and shape datasets for analysis and insight dashboards.
• Build and manage data supply to systems and applications, including database management and data extraction scripts.
• Lead data and platform transformation or migration, develop team capability, set direction and identify issues.
• Develop cloud data pipelines, products, automation, logging, monitoring and performance assurance.
• Oversee tools for tracking and cataloguing data and metadata, ensuring compliance with business standards.
• Use Agile practices, version control, automation, monitoring, testing and deployment to streamline pipelines and track changes.

Requirements and clearance

• Expert-level Azure data engineering experience, including ETL/ELT pipeline design and technical team leadership.
• Strong knowledge of PySpark and Python.
• Security Check clearance is required.
• Continuous UK residence for at least 3 of the previous 5 years, including the 2 immediately preceding years, is required.

Locations

• The role may be based in Birmingham, Blackpool, Coventry, Darlington, Leeds, London, Manchester, Newcastle upon Tyne or Sheffield.
• London Caxton House is available only to existing DWP employees already based at Caxton House or Sanctuary Buildings.
• Newcastle and Sheffield roles have stated future office moves by the end of 2027 and early 2028, respectively.

Salary and grade

• Salary range is £57,946–£75,664; the advert directs candidates to the benefits section for location-specific breakdowns.
• The role is Grade 7 and includes a Civil Service Pension with a stated 28.97% employer contribution.

Working pattern

• Working patterns listed are flexible working, full-time, job share and part-time.