Group Technical Architect (US)

Job Summary

Job Title

Data Lead – Azure Databricks / GCP

Job Summary

We are seeking an experienced Data Lead with 14+ years of experience in designing, building, and leading large-scale enterprise data platforms on Azure Databricks and Google Cloud Platform (GCP). The ideal candidate will be responsible for driving data transformation initiatives, defining data architecture, leading engineering teams, and delivering scalable, secure, and high-performance data solutions. The role requires strong leadership capabilities along with deep expertise in modern data engineering, cloud technologies, and analytics platforms.

Key Responsibilities

Data Strategy & Leadership

• Lead enterprise-wide data engineering and modernization initiatives.
• Define and implement cloud data strategies aligned with business objectives.
• Mentor and manage data engineers, solution architects, and technical leads.
• Collaborate with business stakeholders, product teams, and leadership to establish data roadmaps.
• Drive adoption of best practices in data governance, security, and architecture.

Data Engineering & Architecture

• Design and develop scalable data platforms using Azure Databricks and GCP services.
• Architect batch, streaming, and real-time data processing solutions.
• Build and optimize data pipelines using modern ETL/ELT frameworks.
• Define data models, data quality frameworks, and metadata management strategies.
• Ensure high availability, reliability, and scalability of data ecosystems.

Cloud & Platform Management

• Implement end-to-end solutions on Azure and GCP cloud environments.
• Lead cloud migration and data modernization programs.
• Optimize cloud infrastructure utilization and cost management.
• Establish CI/CD, DevOps, and DataOps practices for data delivery.

Analytics & Governance

• Collaborate with analytics and BI teams to enable self-service reporting and advanced analytics.
• Implement data governance, lineage, cataloging, and compliance frameworks.
• Establish monitoring, observability, and operational support processes.

Skill Requirements

Technical Skills

Azure

• Azure Databricks
• Azure Data Factory (ADF)
• Azure Synapse Analytics
• Azure Data Lake Storage Gen2 (ADLS)
• Azure Event Hub
• Azure Functions
• Azure DevOps

GCP

• BigQuery
• Cloud Storage
• Dataflow
• Dataproc
• Pub/Sub
• Composer (Airflow)
• Cloud Functions
• IAM & Security Services

Data Engineering

• Apache Spark (PySpark/Scala)
• Delta Lake
• Lakehouse Architecture
• ETL/ELT Frameworks
• Real-time Streaming
• Kafka
• Data Warehousing Concepts

Programming

• Python
• SQL
• Scala (Preferred)
• Shell Scripting

Data Management

• Data Modeling
• Data Governance
• Data Quality Frameworks
• Master Data Management (MDM)
• Metadata Management

DevOps & Automation

• CI/CD Pipelines
• Terraform
• Infrastructure as Code (IaC)
• Git
• Docker
• Kubernetes (Preferred)
• Experience leading distributed teams of Data Engineers and Technical Leads.
• Strong stakeholder management and executive communication skills.
• Proven ability to drive large-scale data transformation programs.
• Experience managing project delivery, risk mitigation, and resource planning.
• Strong mentoring and team development capabilities.

Other Requirements

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related discipline.
• Azure and/or GCP Cloud certifications preferred.
• Experience working in Agile/Scrum environments.
• Strong analytical and problem-solving skills.
• Excellent client-facing, presentation, and communication skills.
• Experience in enterprise-scale data modernization, cloud migration, and analytics transformation programs.
• Experience with retail, banking, telecom, healthcare, or other large enterprise domains is preferred.