Sr. Databricks Engineer
About the Role
We are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives.
Key Skills for This Role
Full Job Posting
Experience
- 9-15 Years of IT Experience
- Location
- Hybrid
- Job Summary
- We are seeking a skilled Databricks Engineer with minimum of 5+ years of hands-on in Databricks who can design, develop, and optimize scalable data platforms and analytics solutions using the Databricks Lakehouse Platform. The ideal candidate will have expertise in data engineering, ETL/ELT development, cloud technologies preferably Azure, and big data processing to support enterprise analytics and AI initiatives.
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or related field.
- 9-15 years of experience in Data Engineering and Data Warehousing.
- Minimum 5+ years of hands-on experience with Azure Data Engineering technologies.
- Minimum 4+ years of hands-on experience with Azure Databricks and Spark ecosystem.
- Strong understanding of data lake, lakehouse, and cloud-native architecture patterns.
- Experience in handling large-scale structured and unstructured datasets.
- Strong analytical, problem-solving, and troubleshooting skills.
- Key Responsibilities
- Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services.
- Build and optimize ETL/ELT workflows for large-scale structured and unstructured data.
- Develop data models and implement data quality, validation, and governance frameworks.
- Integrate data from multiple sources into a unified Lakehouse architecture.
- Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.
- Implement security controls, access management, and data governance using Unity Catalog.
- Collaborate with business, analytics, and AI/ML teams to deliver trusted data products.
- Monitor, troubleshoot, and resolve data pipeline issues.
- Support CI/CD, DevOps, and infrastructure automation practices.
- Maintain technical documentation and best practices.
- Required Technical Skills Core Technologies
- Databricks Lakehouse Platform
- Apache Spark / PySpark
- Delta Lake
- SQL
- Python
- Data Engineering
- ETL / ELT Development
- Data Modeling
- Data Warehousing
- Data Quality & Validation
- Streaming & Real-Time Processing
- Governance & Security
- Unity Catalog
- Data Lineage
- Row-Level Security
- Access Control & Compliance
- Data Governance Frameworks
- Cloud & DevOps
- Azure / AWS / GCP
- Terraform
- GitHub Actions / Azure DevOps
- CI/CD Pipelines
- Analytics & AI
- Semantic Layers
- Data Products
- BI Platforms
- Machine Learning Support
- Generative AI & RAG Architectures
Key Responsibilities
- Design, develop, and maintain data pipelines using Databricks, Apache Spark, and cloud-native services.
- Build and optimize ETL/ELT workflows for large-scale structured and unstructured data.
- Develop data models and implement data quality, validation, and governance frameworks.
- Integrate data from multiple sources into a unified Lakehouse architecture.
- Optimize Spark jobs and Databricks workloads for performance, scalability, and cost efficiency.
- Implement security controls, access management, and data governance using Unity Catalog.
- Collaborate with business, analytics, and AI/ML teams to deliver trusted data products.
- Monitor, troubleshoot, and resolve data pipeline issues.
- Support CI/CD, DevOps, and infrastructure automation practices.
- Maintain technical documentation and best practices.
- Required Technical Skills Core Technologies
- Databricks Lakehouse Platform
- Apache Spark / PySpark
- Delta Lake
- SQL
- Python
- Data Engineering
- ETL / ELT Development
- Data Modeling
- Data Warehousing
- Data Quality & Validation
- Streaming & Real-Time Processing
- Governance & Security
- Unity Catalog
- Data Lineage
- Row-Level Security
- Access Control & Compliance
- Data Governance Frameworks
- Cloud & DevOps
- Azure / AWS / GCP
- Terraform
- GitHub Actions / Azure DevOps
- CI/CD Pipelines
- Analytics & AI
- Semantic Layers
- Data Products
- BI Platforms
- Machine Learning Support
- Generative AI & RAG Architectures
Preferred Qualifications
- Microsoft Certified: Azure Data Engineer Associate (DP-203).
- Databricks Certified Data Engineer Associate/Professional.
- Experience with Snowflake, Power BI, or Microsoft Fabric.
- Experience in real-time streaming solutions using Kafka/Event Hubs.
- Exposure to Data Governance and Master Data Management initiatives.
- Soft Skills
- Strong stakeholder management and communication skills.
- Ability to lead technical initiatives and drive architecture discussions.
- Experience working in Agile/Scrum environments.
- Excellent documentation and presentation skills.
- Strong mentoring and team leadership abilities.
- Microsoft Fabric
- Power BI
- Azure Event Hubs
- Kafka
- Machine Learning data pipelines
- Data Governance tools such as Purview
About EXL
EXL is a global data and AI company that provides analytics, digital operations, and industry-specific services to enterprises in insurance, healthcare, banking, retail, media, and energy.
Visit company websiteApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at EXL
Manager
Gurugram, IND
Manage marketing analytics projects and ensure alignment with business goals. Ensure adherence to marketing analytics policies and procedures. Provide strategic guidance on marketing analytics practices. Lead a team of m
Assistant Manager
Noida, IND
About the Role We are seeking experienced QNXT professionals to join our healthcare technology team. The ideal candidate will have hands-on experience working with the QNXT platform in either development or testing capac
Lead Assistant Manager
Noida, IND
Lead Assistant Manager – F&A Operations (Insurance P&C) at exl Service.com, handling month‐end closing, journal entries, accruals and balance‐sheet reconciliation.
Senior Executive
Noida, IND
To work on bank reconciliation, Daily cash settlement, maintain cash tracker and do posting on system.
Data Scientist- GenAI Engineer
Noida, IND
We are seeking an experienced Data Scientist with deep expertise in Generative AI and Python to design, develop, and deploy advanced AI models that drive innovation and deliver impactful solutions. This role involves wor
Senior Executive - Claims Audit
Chennai, IND
Senior Executive- Claims Analyst is responsible for reviewing and analyzing medical records and claims to ensure accuracy, completeness, and compliance with regulatory standards. This role involves summarizing medical do
Associate - Business Analyst
Noida, IND
Identify F&A automation opportunities in client environment and able to perform end to end automation operations Core Alteryx + Power BI and Tableau developer with at least 4 years of experience in creating dynamic wor
Executive
Pune, IND
This role is primarily responsible for dealing with customer calls as per the SLAs outlined by the client
Manager
Gurugram, IND
Assistant Manager
Noida, IND
Lead Assistant Manager
Noida, IND
Senior Executive
Noida, IND
Data Scientist- GenAI Engineer
Noida, IND
Senior Executive - Claims Audit
Chennai, IND
Associate - Business Analyst
Noida, IND
Executive
Pune, IND