Data Engineer- Azure Databricks- Senior Associate

Industry/Sector

Not Applicable

Specialism

Product Innovation

Management Level

Senior Associate

Job Description & Summary

The Opportunity

Join our Acceleration Center India and help shape the future of business for our diverse client portfolio across geographies and jurisdictions. You’ll work at the heart of global teams across Advisory, Assurance, Tax and Business Services—solving real client challenges through connected collaboration. We’ll help you grow your skills so you can go further. With hands-on learning, cutting-edge tools and an inclusive culture, this is your opportunity to do inspiring work that makes a difference—every day.

As a Data Engineer- Azure Databricks- Senior Associate you will support data modernization efforts by designing, building, and maintaining cloud data pipelines and platform components using Azure Databricks within our Advisory practice. As a Senior Associate you will analyze complex problems, manage assigned workstreams, guide junior team members, and use critical thinking to develop deliverables that align with client needs and the broader business context. In this role at PwC, you will contribute to data integration, ETL, cloud migration, and analytics solutions across Azure environments, while building meaningful client connections and growing your technical skills within Cloud Engineering, Data & Analytics and Product Innovation.

Responsibilities

Designing and building Azure Databricks data pipelines to ingest, transform, and deliver trusted data for analytics and reporting
Developing cloud-based data engineering solutions across Azure data services, data lakes, and modern warehouse platforms
Optimizing pipeline performance, query execution, and data storage patterns to improve scalability and processing speed
Implementing data quality checks, validation logic, and monitoring routines across batch and streaming workflows
Translating business needs into technical data models, integration patterns, and reusable engineering components
Supporting modernization of legacy data environments through cloud migration, consolidation, and platform refactoring
Applying Apache Airflow, Azure Data Factory, and related orchestration tools to schedule and manage data workflows
Reviewing code, troubleshooting pipeline issues, and mentoring junior team members on engineering methods and delivery practices

What You Must Have

At least a Bachelor's degree
At least 4 years of experience
Oral and written proficiency in English required

Must Have:

4+ years of experience working in Azure ADLS, Data Bricks, Data Flows, HDInsight, Azure Analysis services

Should be aware of Data Management concepts and Data Modelling

Good knowledge of Big Data frameworks and related technologies

Experience in Hadoop and Spark is mandatory

Should have expertise in extracting data from different source systems like flat files, XML sources, JSON, Parquet files, Big data appliances, RDBMS, etc.

Good knowledge of Big Data frameworks and related technologies

Experience in Hadoop and Spark is mandatory

Should have clear understanding of DW lifecycle and contributed in preparing

Detail Level Design document

Unit Test plans

Code review reports

Must have strong sql, understand joins, groups, functions & Stored procedures

Must have experience in Unit and Integration Test creation and verification

Proficient in creating documentation related to Project Architecture, Project end User Manual and - Operations Hand off guide

Should have good understanding of Data Migration processes, methods and project lifecycle

Good in Unix Shell Scripting skills

Strong analytical skills and troubleshooting skills

Should be well versed with quality processes and implementation

Excellent communication skills

Strong understanding of Cloud data migration processes, methods and project lifecycle

Good knowledge of Application DevOps tools (Git, CI/CD Frameworks)

Experience in Jenkins or Gitlab with rich experience in source code management like Code - Pipeline, Code Build and Code Commit

Desired Knowledge / Skills:

Certifications in Azure / Databricks

Experience in building stream-processing systems, using solutions such as Storm or Spark

Streaming

Experience in Big Data ML toolkits, such as Mahout, SparkML, or H2O

Experience in Azure functions

Knowledge in Python

What Sets You Apart

Preference for at least one of the following fields of study: Management Information Systems, Computer and Information Science, Systems Engineering, Electrical Engineering, Chemical Engineering, Industrial Engineering, Mathematics, Statistics, Mathematical Statistics
Demonstrating Azure Databricks data engineering experience across cloud platforms
Applying data pipeline, data modeling, and data integration skills in client work
Leveraging Apache Airflow, Azure Data Factory, or Snowflake knowledge
Interpreting complex data challenges and developing practical solutions
Mentoring junior team members while managing assigned workstreams

Travel Requirements

Not Specified

Job Posting End Date