Lead Data Engineer
Role Overview
Howden Group Services is expanding its internal data engineering capability and seeks an experienced engineer to lead complex, high-impact initiatives.
The role focuses on Databricks, Azure data services, metadata-driven solutions, and leadership of technical design and engineering practices.
Key Skills for This Role
Full Job Posting
Role overview
Howden Group Services is expanding its internal data engineering capability and seeks an experienced engineer to lead complex, high-impact initiatives.
The role focuses on Databricks, Azure data services, metadata-driven solutions, and leadership of technical design and engineering practices.
Responsibilities
- Lead metadata ingestion frameworks, orchestration processes, and design and maintenance of ETL processes.
- Build scalable, reusable metadata-driven services and architect solutions using Databricks and Azure services.
- Establish data engineering, CI/CD, and DevOps best practices; optimize pipelines for performance, reliability, and scale.
- Translate stakeholder and analyst needs into data solutions and promote governance and security practices.
- Mentor junior engineers and support continuous improvement.
Required experience
- At least 8 years of data engineering experience, focused on Databricks, SQL, and PySpark.
- Extensive Azure data services experience, including Azure Data Factory, Azure Data Lake Storage, and Azure Functions.
- Experience with metadata-driven pipelines, workflow orchestration, CI/CD, engineering standards, and technical architecture.
- Strong stakeholder management, with the ability to translate business requirements into data solutions.
- Knowledge of data governance, security, and compliance best practices.
Preferred experience
- Infrastructure as code experience with Terraform, Bicep, or ARM templates is preferred.
- Insurance data experience is preferred but not essential.
- Airflow and dbt are given as examples of data orchestration and workflow automation tools.
Workplace information
- The posting mentions that hybrid working may be accommodated, subject to role suitability; it does not establish a definite workplace arrangement.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Howden Group
Senior HRIS Analyst
London, GBR
Howden is seeking a Senior HRIS Analyst to administer, configure and maintain Workday and related HR technology, while supporting global users and system changes. The role requires strong Workday HCM knowledge, experienc
Data Analyst - Specialty
London, GBR
Howden is seeking an early-career Data Analyst to support its Aviation team with data analysis, reporting and analytics. The role calls for strong data foundations, curiosity and a motivation to learn; technical training
Data Analyst Apprenticeship Programme 2027 - Cheltenham
Cheltenham, GBR
Howden is seeking an apprentice interested in data, technology, IT, or coding to learn analytics and data science through an 18-month programme. You’ll support live projects, analyse and check datasets, and create report
Howden UK Broking Head of Data Protection
London, GBR
Howden seeks a Head of Data Protection to lead the function for its UK Broking and UK Reinsurance businesses. The role owns a pragmatic, risk-based framework aligned with applicable data protection legislation and group
Howden Private Capital Due Diligence Graduate Programme 2027
London, GBR
Howden is seeking a recent or final-year graduate for its Private Capital Due Diligence programme, supporting insurance due diligence and live investment transactions. Candidates should be analytical, interested in finan
Howden Private Capital Portfolio Solutions Graduate Programme 2027
London, GBR
Howden is seeking a graduate analyst for its Private Capital Portfolio Solutions team. You’ll research investors and portfolio companies, analyse business information, support client solutions and business development, a
Risk Business Partner
London, GBR
Howden is seeking a Risk Business Partner to help embed its Enterprise Risk Management Framework across UK&I Retail and central functions. The role advises and challenges stakeholders, supports assessments and testing, a
Health & Safety Advisor
London, GBR
Howden is seeking a Health & Safety Advisor to support safety compliance and improve safety culture across its UK and Ireland business. The role covers audits, risk assessments, incident investigations, training and safe
Senior HRIS Analyst
London, GBR
Data Analyst - Specialty
London, GBR
Data Analyst Apprenticeship Programme 2027 - Cheltenham
Cheltenham, GBR
Howden UK Broking Head of Data Protection
London, GBR
Howden Private Capital Due Diligence Graduate Programme 2027
London, GBR
Howden Private Capital Portfolio Solutions Graduate Programme 2027
London, GBR
Risk Business Partner
London, GBR
Health & Safety Advisor
London, GBR