Big Data Engineer
About the Role
We are looking for an experienced Lead Data Engineer with 7–10 years of hands-on experience to design, build, and maintain scalable data pipelines and processing systems in an on-premises Big Data environment. This is a technical leadership role: beyond strong individual contribution, the ideal candidate will own architecture and design decisions, set technical direction, and mentor and support other developers on the team. The role works closely with cross-functional teams to deliver reliable, high-quality data solutions that support business and analytics needs.
Key Skills for This Role
Full Job Posting
Technical Skills
- Must Have
- 7–10 years of overall experience in data engineering, with a proven track record in technical leadership (design ownership, mentoring, guiding development teams).
- Python – strong hands-on development experience building production-grade data solutions.
- Big Data / Hadoop ecosystem (Hadoop, Hive, Impala, HDFS) – deep, hands-on experience in on-premises environments.
- Apache Spark – solid experience developing and tuning large-scale distributed data processing jobs.
- Job scheduling / orchestration – hands-on experience with CA7 or Control-M (or comparable enterprise schedulers).
- Strong understanding of data structures, ETL processes, and SQL.
- Extensive experience with large-scale data processing and distributed systems.
- Demonstrated ability to make sound architecture/design decisions and to mentor and support other developers.
- Exposure to AI/ML concepts or tools, with a strong willingness to learn and grow in this space.
Qualifications
- Must Have
- 7–10 years of overall experience in data engineering, with a proven track record in technical leadership (design ownership, mentoring, guiding development teams).
- Python – strong hands-on development experience building production-grade data solutions.
- Big Data / Hadoop ecosystem (Hadoop, Hive, Impala, HDFS) – deep, hands-on experience in on-premises environments.
- Apache Spark – solid experience developing and tuning large-scale distributed data processing jobs.
- Job scheduling / orchestration – hands-on experience with CA7 or Control-M (or comparable enterprise schedulers).
- Strong understanding of data structures, ETL processes, and SQL.
- Extensive experience with large-scale data processing and distributed systems.
- Demonstrated ability to make sound architecture/design decisions and to mentor and support other developers.
- Exposure to AI/ML concepts or tools, with a strong willingness to learn and grow in this space.
About EXL
EXL is a global data and AI company that provides analytics, digital operations, and industry-specific services to enterprises in insurance, healthcare, banking, retail, media, and energy.
Visit company websiteApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at EXL
Assistant Manager - Coding Auditor
Chennai, IND
Coding Assistant Manager is responsible for APC & Professional Surgery Audit review. This includes leveraging advanced medical coding analysis, enforcing policy adherence, and driving overpayment recovery programs. Creat
Manager
, IND
Professionals in this group are expected to create new algorithms from scratch. They work with large data sets and relational databases and have excellent problem-solving skills and the ability to analyze issues, identif
Manager - Quality Excellence
Kochi, IND
Manager – Quality Excellence (Kochi) to lead quality governance and process improvement for a US Healthcare Client. Lean Six Sigma GB/BB preferred.
Senior Executive
Noida, IND
The Claims assistant will be responsible for managing a high volume of claims, ensuring timely resolution, accurate documentation, and strong customer/carrier satisfaction. This role involves reviewing claim submissions,
Phone Auditor - Remote
, USA
Phone Auditor – Remote. Responsible for completing audits for Property & Casualty policies (Worker’s Compensation and General Liability) by phone with policyholders, ensuring accurate classifications and exposure amounts
Insurance Client Engagement Lead
Jersey City, USA
P&C Insurance Client Engagement Lead Pay Range: $150K -$190K US Remote
Assistant Manager - Coding Auditor
Chennai, IND
Senior Executive
Bengaluru, IND
Manager
, IND
Business Analyst
Gurugram, IND
Manager - Quality Excellence
Kochi, IND
Senior Executive
Noida, IND
Phone Auditor - Remote
, USA
Insurance Client Engagement Lead
Jersey City, USA