Data Engineer

Role overview

Responsibilities

• Build repeatable processes to receive, unpack, validate, and ingest data from multiple sources.
• Transform source extracts into consistent structures for migration to one platform.
• Develop maintainable ingestion and transformation code using SQL, Python, PySpark, and dbt.
• Implement automated completeness and accuracy checks throughout the migration.
• Monitor, diagnose, and recover failed or incomplete processing.
• Maintain auditable records of sources, mappings, transformations, and outcomes; apply agreed secure data-handling controls.

Qualifications

• Hands-on experience building and operating data pipelines and delivering reliable processing in a structured migration environment.
• Experience with cloud data pipelines on Azure, AWS, or Google Cloud.
• Experience with Spark, PySpark, Databricks, or comparable distributed processing technologies.
• Experience with modular, tested, documented dbt transformations or an equivalent approach.
• Strong SQL and Python skills for transformations, troubleshooting, validation, and reconciliation.
• Experience with pipeline scheduling, monitoring, and recovery using cloud-native or equivalent orchestration tools.
• Experience with Delta Lake and repeatable batch loading, and with Terraform or equivalent infrastructure as code.

Workplace

• The role offers a hybrid working model with some flexible working.

Benefits

• Benefits include 25 days' holiday, private medical healthcare through Vitality, pension contribution, and life assurance.
• Professional certifications are supported as part of learning and development; retail discounts and Employee Resource Groups are also available.

About Netcompany

Accessibility

• Netcompany offers a guaranteed interview scheme for applicants with a disability who meet the role's minimum criteria; applicants may opt in.
• Reasonable adjustments and additional recruitment support are available on request.