company_site
Technical Delivery Lead
Bits In Glass
London, GBR
Lead · 7+ years experience
Onsite
Discovered 1 weeks ago
airflowbigquerydatadogdbtgcppython
Key Skills for This Role
airflowbigquerydatadogdbtgcppython
Full Job Posting
Roles and Responsibilities
- Lead the engineering and technical strategy for a scalable multi-tenant data integration platform leveraging Apache Airflow and GCP technologies.
- Build and maintain production-grade ETL/ELT pipelines orchestrated with Apache Airflow.
- Utilize container platforms such as GKE and Cloud Run.
- Apply strong software engineering skills in Python, APIs, configuration management, testing, and performance tuning.
- Implement CI/CD pipelines, release controls, automated quality gates, and Infrastructure-as-Code (Terraform).
- Design for reliability including retries, idempotency, dead-letter patterns, and operational resilience.
- Translate complex technical concepts to diverse audiences and align cross-team efforts.
- Work with GCP BigQuery for large-scale data warehousing and analytics.
- Architect cloud solutions using GCP Cloud Architecture best practices. (Desirable: Experience with data lineage/metadata management tools, multi-tenant architectures, data governance, and vendor system integrations).
Required Skills
- 7+ years of experience in data engineering or data platform engineering, with at least 3 years in a technical lead or delivery lead capacity.
- Proven experience owning a technical roadmap and aligning engineering delivery with business priorities.
- Strong hands-on experience designing and building production-grade ETL/ELT pipelines orchestrated with Apache Airflow (or Cloud Composer).
- Strong experience with GCP services including BigQuery, GKE, and Cloud Run, applying GCP cloud architecture best practices.
- Proficiency in Python, with strong software engineering skills across APIs, configuration management, testing, and performance tuning.
- Working experience with Terraform and CI/CD pipelines, including release controls and automated quality gates.
- Experience designing resilient systems using patterns such as retries, idempotency, and dead-letter handling.
- Experience implementing data quality controls, including validation, reconciliation, and exception handling.
- Experience establishing engineering standards for coding, testing, documentation, observability, and security.
- Excellent written and verbal communication skills, with the ability to explain complex technical concepts to diverse audiences and align cross-team efforts.
- Strong analytical, debugging, and problem-solving skills.
Nice to Have
- GCP Professional Data Engineer or Professional Cloud Architect certification.
- Experience designing or enhancing multi-tenant data platforms, including tenant isolation and metadata-driven pipelines.
- Experience with data lineage and metadata management tools such as Dataplex, Data Catalog, or OpenLineage.
- Experience with data governance and data quality frameworks such as DBT or Great Expectations.
- Experience integrating and onboarding vendor or third-party source systems.
- Experience with event-driven or streaming services such as Pub/Sub or Dataflow.
- Experience with observability tooling such as Cloud Monitoring, Cloud Logging, or Datadog.
- Experience in banking, insurance, or other regulated financial services environments.
- Bits In Glass (BIG) operates as a rapidly expanding AI and automation consultancy, maintaining presence throughout Canada, the United States, India, and Great Britain. The firm's technical scope encompasses over ten primary platforms across artificial intelligence, cloud infrastructure, data management, automation solutions, and enterprise software. Our staff consists of skilled engineering professionals dedicated to tackling difficult technical problems, driving joint success, and guiding enterprise organisations through digital transformation with certainty.
Qualifications
- 7+ years of experience in data engineering or data platform engineering, with at least 3 years in a technical lead or delivery lead capacity.
- Proven experience owning a technical roadmap and aligning engineering delivery with business priorities.
- Strong hands-on experience designing and building production-grade ETL/ELT pipelines orchestrated with Apache Airflow (or Cloud Composer).
- Strong experience with GCP services including BigQuery, GKE, and Cloud Run, applying GCP cloud architecture best practices.
- Proficiency in Python, with strong software engineering skills across APIs, configuration management, testing, and performance tuning.
- Working experience with Terraform and CI/CD pipelines, including release controls and automated quality gates.
- Experience designing resilient systems using patterns such as retries, idempotency, and dead-letter handling.
- Experience implementing data quality controls, including validation, reconciliation, and exception handling.
- Experience establishing engineering standards for coding, testing, documentation, observability, and security.
- Excellent written and verbal communication skills, with the ability to explain complex technical concepts to diverse audiences and align cross-team efforts.
- Strong analytical, debugging, and problem-solving skills.
About Bits In Glass
IT Services389 employeesFounded 2002
Designs and implements automated software systems for large enterprises.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Bits In Glass
Solution Architect (Senior AI / Technical Lead)
Calgary, CAN
LeadFull-time
Discovered 1 weeks agoView →
Solution Architect (Senior AI / Technical Lead)
Toronto, CAN
LeadFull-time
Discovered 1 weeks agoView →
Solution Architect (Senior AI / Technical Lead)
Edmonton, CAN
LeadFull-time
Discovered 1 weeks agoView →
Business Analyst Creatio
Calgary, CAN
MidFull-time
Bits In Glass is seeking a Business Analyst to help clients optimize workflows and deliver Creatio CRM solutions. The role combines requirements analysis, process automation, solution configuration, and client enablement
Discovered 1 months agoView →
Business Analyst Creatio
Calgary, CAN
MidFull-time
Bits In Glass is seeking a Business Analyst to gather requirements, optimize customer workflows, and support Creatio CRM solution delivery. The role requires 3–5 years of business analysis experience, CRM and customer su
Discovered 1 months agoView →
Business Analyst Creatio
Toronto, CAN
MidFull-time
Bits In Glass is seeking a Business Analyst for its Creatio team to translate client needs into CRM workflows, automation, and solution improvements. The role requires 3–5 years of business analysis experience, CRM and c
Discovered 1 months agoView →
Senior Data Platform Engineer
London, GBR
Discovered 1 weeks ago
Portfolio Delivery Manager
Calgary, CAN
Discovered 1 weeks agoFull-time
Solution Architect (Senior AI / Technical Lead)
Calgary, CAN
Discovered 1 weeks agoFull-time
Solution Architect (Senior AI / Technical Lead)
Toronto, CAN
Discovered 1 weeks agoFull-time
Solution Architect (Senior AI / Technical Lead)
Edmonton, CAN
Discovered 1 weeks agoFull-time
Business Analyst Creatio
Calgary, CAN
Discovered 1 months agoFull-time
Business Analyst Creatio
Calgary, CAN
Discovered 1 months agoFull-time
Business Analyst Creatio
Toronto, CAN
Discovered 1 months agoFull-time