{bc}
Apply
successfactors

Manager | Databricks | Mumbai | Engineering | Data Modernization & Migration

Deloitte US | Together Makes Progress
Mumbai, IND
Full-time
Manager · 9–12 years experience
Onsite
Discovered 1 weeks ago
airflowawsazureazure-data-factorydatabricksdbt

Full Job Posting

Key Responsibilities

  • Design, develop, and maintain scalable data engineering solutions using Databricks.
  • Build and optimize batch and real-time data processing pipelines.
  • Develop ETL/ELT frameworks using PySpark and Spark SQL.
  • Implement Delta Lake architecture and data governance standards.
  • Process large-scale structured, semi-structured, and streaming datasets.
  • Optimize Spark jobs and cluster performance for maximum efficiency.
  • Develop reusable frameworks, notebooks, workflows, and data products.
  • Collaborate with business teams, data scientists, analysts, and architects to deliver enterprise-grade solutions.
  • Implement monitoring, alerting, security, and compliance requirements.
  • Participate in code reviews, deployment activities, and production support.
  • Develop technical documentation and mentor junior engineers. Required Technical Skills Databricks & Big Data
  • Azure Databricks
  • Databricks Lakehouse Platform
  • Delta Lake
  • Spark SQL
  • PySpark
  • Databricks Workflows
  • Unity Catalog
  • Delta Live Tables (DLT)
  • Structured Streaming Data Engineering
  • Data Lake Architecture
  • Lakehouse Architecture
  • ETL/ELT Development
  • Data Modeling
  • Data Integration
  • Performance Tuning
  • Data Quality Frameworks Programming Languages
  • Python
  • SQL
  • Scala
  • Java
  • Shell Scripting Cloud Platforms
  • Microsoft Azure
  • Azure Data Factory (ADF)
  • Azure Data Lake Storage (ADLS)
  • AWS
  • Amazon S3
  • AWS EMR
  • Google Cloud Platform (GCP) Databases
  • SQL Server
  • Oracle
  • PostgreSQL
  • MySQL
  • MongoDB
  • Cassandra Streaming & Messaging
  • Kafka
  • Event Hubs
  • Spark Streaming
  • Azure Stream Analytics DevOps & Tools
  • Git
  • Jenkins
  • Azure DevOps
  • CI/CD Pipelines
  • Terraform
  • Unix/Linux Required Qualifications 1. Bachelor's or Master's degree in Computer Science, Engineering, Information Technology, or related field. 2. 9-12 years of IT experience with at least 4+ years of hands-on Databricks and Spark development experience. 3. Strong expertise in distributed data processing and Lakehouse architecture. 4. Experience building enterprise-scale data engineering solutions. 5. Strong knowledge of PySpark, Spark SQL, and performance optimization. 6. Experience with Agile development methodologies. 7. Excellent analytical, troubleshooting, and communication skills.

Preferred Qualifications

  • Experience with cloud-native analytics platforms on Azure, AWS, or GCP. 2. Knowledge of Machine Learning workflows within Databricks. 3. Experience implementing Delta Lake and Unity Catalog. 4. Exposure to Data Governance and Data Security frameworks. 5. Relevant Databricks, Azure, or AWS certifications. Key Competencies
  • Data Engineering Excellence
  • Performance Optimization
  • Solution Design
  • Stakeholder Management
  • Collaboration and Leadership
  • Ownership and Accountability Nice to Have
  • Snowflake
  • Azure Data Factory (ADF)
  • Delta Live Tables (DLT)
  • Unity Catalog
  • MLflow
  • Airflow
  • Kafka
  • Data Mesh Architecture
  • Generative AI / AI Engineering exposure
  • dbt

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Deloitte US | Together Makes Progress

Assistant Manager | Financial Due Diligence | Bengaluru | M&A-Transaction Diligence, Execution & Rep

Bengaluru, IND

Discovered Today

Senior Consultant | SAP SD | Ahmedabad | SAP | DSA LS&HC

Ahmedabad, IND

Discovered Today

Team Lead | GCP SRE| Bengaluru | Engineering | Hybrid Cloud Engineering

Bengaluru, IND

Discovered Today

T&T - OIDS - Database / ETL Expert - Senior Consultant - Bhubaneswar

Bhubaneswar, IND

Discovered Today

Team Lead | GCP DevOps | Bengaluru | Engineering | Hybrid Cloud Engineering

Bengaluru, IND

Discovered Today

Assistant Manager | Internal Audit | Bengaluru | Internal audit

Bengaluru, IND

Discovered Today

Senior Consultant | SAP CPI | Bengaluru | SAP | DSA Consumer & TMT

Bengaluru, IND

Discovered Today

Director | Financial Due Diligence | Mumbai | M&A-Transaction Diligence, Execution & Reporting

Mumbai, IND

Discovered Today