company_site
Senior Data Platform Engineer
Bits In Glass
London, GBR
Senior · 7+ years experience
Onsite
Discovered 1 weeks ago
airflowbigquerydatabricksdbtdockergcp
Key Skills for This Role
airflowbigquerydatabricksdbtdockergcp
Full Job Posting
Roles and Responsibilities
- Design, build, and maintain scalable data platforms on Google Cloud Platform for financial sector clients.
- Develop and optimize batch and streaming data pipelines using services such as Dataflow, Dataproc, Pub/Sub, and Cloud Composer.
- Architect data lake, data warehouse, and lakehouse solutions, including data modeling and performance tuning in BigQuery.
- Provision and manage data infrastructure using Terraform and CI/CD pipelines.
- Implement data governance, lineage, quality, and cataloging frameworks to meet regulatory and client requirements.
- Apply security best practices across data environments, including IAM, encryption, KMS, data masking, and access controls.
- Monitor, troubleshoot, and resolve platform performance, reliability, and data quality issues.
- Collaborate with client teams, architects, and internal stakeholders to translate business needs into technical solutions.
- Mentor engineers, contribute to technical standards, and promote DataOps and engineering best practices across the team.
- Communicate architecture decisions, progress, and analysis clearly to technical and non-technical audiences.
Required Skills
- 7+ years of experience in data engineering or data platform engineering, with at least 3 years in a senior or lead capacity.
- Strong hands-on experience building and operating data platforms on GCP.
- Working experience with GCP data services such as BigQuery, Dataflow, Dataproc, Cloud Composer (Airflow), Pub/Sub, Cloud Storage, Bigtable, and Cloud SQL.
- Proficiency in Python and SQL, with experience in Spark or Apache Beam.
- Experience designing data warehouse and data lake architectures, including data modeling.
- Working experience with Terraform and CI/CD tooling such as Jenkins, GitHub, or Cloud Build.
- Strong understanding of data security in regulated environments.
- Strong knowledge of DevOps/DataOps and Agile delivery practices.
- Excellent written and verbal communication skills.
- Strong analytical, debugging, and problem-solving skills.
Nice to Have
- GCP Professional Data Engineer or Professional Cloud Architect certification.
- Experience in banking, insurance, or other regulated financial services environments.
- Experience with data governance and quality tools such as Dataplex, Data Catalog, dbt, or Great Expectations.
- Experience with containerization and orchestration (Docker, Kubernetes/GKE).
- Exposure to other data platforms such as Snowflake, Databricks, or Kafka.
- Experience supporting AI/ML workloads or Vertex AI pipelines.
- Bits In Glass (BIG) operates as a rapidly expanding AI and automation consultancy, maintaining presence throughout Canada, the United States, India, and Great Britain. The firm's technical scope encompasses over ten primary platforms across artificial intelligence, cloud infrastructure, data management, automation solutions, and enterprise software. Our staff consists of skilled engineering professionals dedicated to tackling difficult technical problems, driving joint success, and guiding enterprise organizations through digital transformation with certainty.
Qualifications
- 7+ years of experience in data engineering or data platform engineering, with at least 3 years in a senior or lead capacity.
- Strong hands-on experience building and operating data platforms on GCP.
- Working experience with GCP data services such as BigQuery, Dataflow, Dataproc, Cloud Composer (Airflow), Pub/Sub, Cloud Storage, Bigtable, and Cloud SQL.
- Proficiency in Python and SQL, with experience in Spark or Apache Beam.
- Experience designing data warehouse and data lake architectures, including data modeling.
- Working experience with Terraform and CI/CD tooling such as Jenkins, GitHub, or Cloud Build.
- Strong understanding of data security in regulated environments.
- Strong knowledge of DevOps/DataOps and Agile delivery practices.
- Excellent written and verbal communication skills.
- Strong analytical, debugging, and problem-solving skills.
About Bits In Glass
IT Services389 employeesFounded 2002
Designs and implements automated software systems for large enterprises.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Bits In Glass
Solution Architect (Senior AI / Technical Lead)
Calgary, CAN
LeadFull-time
Discovered 1 weeks agoView →
Solution Architect (Senior AI / Technical Lead)
Toronto, CAN
LeadFull-time
Discovered 1 weeks agoView →
Solution Architect (Senior AI / Technical Lead)
Edmonton, CAN
LeadFull-time
Discovered 1 weeks agoView →
Business Analyst Creatio
Calgary, CAN
MidFull-time
Bits In Glass is seeking a Business Analyst to help clients optimize workflows and deliver Creatio CRM solutions. The role combines requirements analysis, process automation, solution configuration, and client enablement
Discovered 1 months agoView →
Business Analyst Creatio
Calgary, CAN
MidFull-time
Bits In Glass is seeking a Business Analyst to gather requirements, optimize customer workflows, and support Creatio CRM solution delivery. The role requires 3–5 years of business analysis experience, CRM and customer su
Discovered 1 months agoView →
Business Analyst Creatio
Toronto, CAN
MidFull-time
Bits In Glass is seeking a Business Analyst for its Creatio team to translate client needs into CRM workflows, automation, and solution improvements. The role requires 3–5 years of business analysis experience, CRM and c
Discovered 1 months agoView →
Technical Delivery Lead
London, GBR
Discovered 1 weeks ago
Portfolio Delivery Manager
Calgary, CAN
Discovered 1 weeks agoFull-time
Solution Architect (Senior AI / Technical Lead)
Calgary, CAN
Discovered 1 weeks agoFull-time
Solution Architect (Senior AI / Technical Lead)
Toronto, CAN
Discovered 1 weeks agoFull-time
Solution Architect (Senior AI / Technical Lead)
Edmonton, CAN
Discovered 1 weeks agoFull-time
Business Analyst Creatio
Calgary, CAN
Discovered 1 months agoFull-time
Business Analyst Creatio
Calgary, CAN
Discovered 1 months agoFull-time
Business Analyst Creatio
Toronto, CAN
Discovered 1 months agoFull-time