{bc}
Apply
indeed

Data Engineer (G7)

Department for Work and Pensions (DWP)
London, GBR
Parttime, Fulltime
Onsite
Discovered 4 days ago
Azure data engineeringETL/ELT pipeline designPySparkPythonData engineeringCloud data engineering

Full Job Posting

Role overview

Senior data engineering role in DWP's Data Engineering community.

Lead design and delivery of secure, scalable data solutions that support services used across the UK.

Support reliable, governed and continually improving data, enabling analysts, data scientists and operational teams to access trusted data at scale.

Responsibilities

  • Lead design, development and maintenance of scalable ETL/ELT pipelines using PySpark and Python; optimize, validate and standardize workflows and batch processes.
  • Lead stakeholder data-requirement gathering and shape datasets for analysts' dashboards and insight work.
  • Build and manage data supply across systems and applications, including databases and data-extraction scripts for cross-business use.
  • Lead team delivery of large-scale data and platform transformations or migrations, set direction and identify issues.
  • Develop complex cloud pipelines, products and automation, with logging, monitoring and performance assurance.
  • Oversee data and metadata tracking and cataloguing tools, ensuring compliance with business standards.
  • Lead Agile practices and use version control, automation and monitoring to test, deploy and track pipeline changes.

Requirements and clearance

  • The job summary requests expert-level Azure data engineering experience and ETL/ELT pipeline design experience.
  • The job summary requests technical team leadership experience and strong knowledge of PySpark and Python.
  • Candidates must pass Security Check clearance.
  • The posting states a continuous UK residency condition: at least three of the last five years, including the two immediately preceding years.

Data cataloguing tools

  • The description names Google Cloud Data Catalog, AWS Glue Data Catalog, Azure Data Catalog, Informatica Enterprise Data Catalog, Alation and Apache Atlas as examples of tracking and cataloguing tools.

Possible locations

  • The role may be based in Birmingham, Blackpool, Coventry, Darlington, Leeds, London, Manchester, Newcastle upon Tyne or Sheffield.
  • London Caxton House is available only to existing DWP employees already based at Caxton House or Sanctuary Buildings; other listed locations are available to all candidates.
  • Newcastle and Sheffield roles are initially based at specified existing sites and are expected to move to city-centre locations by 2027 or 2028.

Working pattern

The listed working patterns include flexible working, full-time, job share and part-time.

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Department for Work and Pensions (DWP)

Data Engineer (G7)

London, GBR

Discovered 2 days agoFull-time

Data Engineer (G7)

London, GBR

Discovered 3 days agoFull-time

Data Engineer (G7)

London, GBR

Discovered 3 days agoFull-time

Data Engineer (G7)

London, GBR

Discovered 3 days agoFull-time

Data Engineer (G7)

London, GBR

Discovered 4 days agoFull-time

Data Engineer (G7)

London, GBR

Discovered 4 days agoFull-time

Data Engineer (G7)

London, GBR

Discovered 4 days agoFull-time

Data Engineer (G7)

London, GBR

Discovered 4 days agoFull-time