Lead Data Engineer
Role Overview
We are seeking an experienced Senior Data Engineer to drive the performance, governance, and AI-native maturity of our enterprise Data Platform in Databricks.
This is a Databricks-focused Data Engineering role with a working understanding of DevOps practices — designing scalable pipelines, tuning workloads for performance and cost, and operationalizing modern data and AI capabilities on Lakehouse.
The ideal candidate has deep, hands-on Databricks expertise, a strong performance-engineering instinct, and a builder's mindset for AI-assisted operations.
You'll own the Databricks performance and governance standards for the platform, mentor engineers, and shape the direction for AI-native operations.
Key Skills for This Role
Full Job Posting
Position Summary:
We are seeking an experienced Senior Data Engineer to drive the performance, governance, and AI-native maturity of our enterprise Data Platform in Databricks. This is a Databricks-focused Data Engineering role with a working understanding of DevOps practices — designing scalable pipelines, tuning workloads for performance and cost, and operationalizing modern data and AI capabilities on Lakehouse.
The ideal candidate has deep, hands-on Databricks expertise, a strong performance-engineering instinct, and a builder's mindset for AI-assisted operations. You'll own the Databricks performance and governance standards for the platform, mentor engineers, and shape the direction for AI-native operations.
Key Responsibilities:
- Design and develop scalable data pipelines and Lakehouse solutions on Databricks.
- Tune Databricks workloads for performance and cost, including cluster sizing, query optimization, and Delta Lake table design.
- Establish and enforce best practices for partitioning, clustering, and workload isolation.
- Track performance trends, identify high-cost queries, and partner with source teams and end users to resolve long-running loads.
- Design and operationalize Unity Catalog for data governance — access control, lineage, and security.
- Build monitoring and self-healing automation using Databricks-native AI and agentic capabilities.
- Drive CI/CD workflows for Databricks assets, setting DevOps best practices for deployment and release management.
- Lead design reviews and mentor Data Engineers on Databricks best practices and AI-native features.
- Own Databricks vendor coordination — case management, escalations, and release adoption strategy.
What Success Looks Like (First 6–12 Months)
- Within 6–12 months, you'll define the platform's tuning and governance standards, lead design reviews, mentor junior engineers, and shape the AI-native operations roadmap.
Required Qualifications:
- Bachelor's or Master's degree in Computer Science, Information Technology, or equivalent relevant experience.
- 6+ years of data engineering experience with 2+ years hands-on Databricks in enterprise settings.
- Deep understanding of Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and Workflow orchestration.
- Proven ability to tune Spark workloads for cost and performance at production scale.
- Advanced Python (PySpark) and SQL skills.
- Working knowledge of CI/CD practices and DevOps principles applied to data workloads.
- Experience with observability tooling for Databricks.
Preferred Qualifications:
- Experience with Databricks-native AI capabilities and agentic frameworks.
- Familiarity with Databricks Serverless Compute and DBSQL performance tuning.
- A Databricks Certified Professional.
- Exposure to Infrastructure-as-Code is a plus.
Competencies:
- Performance-engineering mindset — measures, tunes, and re-measures.
- Curiosity for AI-native operations and continuous automation.
- Strong sense of platform ownership — quality, cost, and reliability.
- Effective communication with engineering peers, vendors, and business stakeholders.
- Influence outcomes across source teams, vendors, and business stakeholders without direct authority.
More information about NXP in India...
#LI-7013
About NXP Semiconductors
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at NXP Semiconductors
Graduate AI and Software Engineer - Automotive MPUs
Glasgow, GBR
NXP Semiconductors is seeking a graduate engineer to develop and demonstrate AI applications for automotive processors. The role involves deploying machine-learning models to hardware, building software and customer demo
Graduate AI and Software Engineer - Automotive MPUs
Glasgow, GBR
Software Architect
Bengaluru, IND
Director, Frontend Design Enablement
Noida, IND
Americas Mass Market US East Region Leader
, USA
Distribution Business Manager
, USA
Distribution Business Manager
, USA
Digital Validation Lead Engineer
Bengaluru, IND
Safety product development F/M
Noida, IND
