{bc}
Apply
successfactors

Lead Senior Associate | Data Engineering | Hyderabad | Engineering as a Service/ Operate

Deloitte US | Together Makes Progress
Hyderabad, IND
Lead · 4+ years experience
Onsite
Discovered 5 days ago
airflowawscloudwatchdatabricksdatadogeks

Full Job Posting

Responsibilities

  • Pipeline Development & Operations:
  • Design, maintain, and optimize Spark and Flink pipelines running in Databricks and Airflow.
  • Ensure reliability and availability of data workflows in production.
  • Debug and resolve issues in notebooks, jobs, and orchestration workflows. Performance Optimization:
  • Tune Spark jobs, clusters, and data pipelines for performance and cost efficiency.
  • Analyze logs, queries, and workloads to identify bottlenecks and implement improvements. Monitoring & Observability:
  • Build and maintain dashboards, alerts, and metrics with Datadog, CloudWatch, or Prometheus.
  • Proactively monitor pipeline health and escalate issues as needed. Incident Response & RCA:
  • Participate in incident response for pipeline issues, helping diagnose and resolve outages.
  • Contribute to RCA and preventative actions to improve long-term stability. Collaboration & Knowledge Sharing:
  • Partner with data engineering, BI, and platform teams to support delivery and operations.
  • Document best practices, contribute to runbooks, and mentor junior engineers.

Basic Qualifications

  • 4+ years of experience in data engineering or big data platform operations.
  • Hands-on experience with Databricks, Spark, Flink, or Airflow (MWAA) in production.
  • Strong SQL skills and experience with ETL/ELT pipeline development.
  • Familiarity with AWS services (S3, Glue, Athena, EMR, EKS) for pipeline management.
  • Working knowledge of observability tools (Datadog, CloudWatch, Prometheus).
  • Proficiency in Python, Scala, or Java for pipeline development and automation.
  • Familiarity with infrastructure-as-code (Terraform or similar).
  • Strong collaboration and problem-solving skills, with experience in distributed teams.

Preferred Qualifications

  • Experience with incident response and RCA in production data environments.
  • Familiarity with CI/CD pipelines (GitLab, Jenkins, or GitHub Actions).
  • Exposure to data governance, access control, and cost optimization practices.
  • Certifications such as Databricks Data Engineer Associate or AWS Solutions Architect – Associate. Required Education: ● Bachelor’s degree in Computer Science, Engineering, or related technical field (or equivalent experience).

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Deloitte US | Together Makes Progress

Assistant Manager | Financial Due Diligence | Bengaluru | M&A-Transaction Diligence, Execution & Rep

Bengaluru, IND

Discovered Today

Senior Consultant | SAP SD | Ahmedabad | SAP | DSA LS&HC

Ahmedabad, IND

Discovered Today

Team Lead | GCP SRE| Bengaluru | Engineering | Hybrid Cloud Engineering

Bengaluru, IND

Discovered Today

T&T - OIDS - Database / ETL Expert - Senior Consultant - Bhubaneswar

Bhubaneswar, IND

Discovered Today

Team Lead | GCP DevOps | Bengaluru | Engineering | Hybrid Cloud Engineering

Bengaluru, IND

Discovered Today

Assistant Manager | Internal Audit | Bengaluru | Internal audit

Bengaluru, IND

Discovered Today

Senior Consultant | SAP CPI | Bengaluru | SAP | DSA Consumer & TMT

Bengaluru, IND

Discovered Today

Director | Financial Due Diligence | Mumbai | M&A-Transaction Diligence, Execution & Reporting

Mumbai, IND

Discovered Today