Senior Data Engineer (Databricks Focused)

Opportunity

Data engineering and platform development

• Design, build, and optimize data pipelines using Databricks, Delta Lake, and Apache Spark.
• Develop scalable ETL for structured and unstructured data, ensuring quality and reliability.
• Integrate cloud and on-premises sources using Databricks and Azure Data Lake; automate ingestion, transformation, and validation.
• Monitor, troubleshoot, and improve pipeline performance and reliability; ensure compliance, security, and operational excellence.

Client and data application responsibilities

• Lead client engagements focused on data engineering and analytics solution delivery; translate business requirements into architectures and operational plans.
• Collaborate across teams to integrate solutions into production, and mentor junior team members in data engineering and cloud analytics practices.
• Convert business requirements and analytics prototypes into scalable data solutions.
• Use data modeling and transformation to support analytics and machine learning; document pipelines and contribute to knowledge sharing.

Required qualifications

• Bachelor’s or master’s degree in Computer Science, Data Engineering, Information Systems, or a closely related discipline; strong foundation in data structures, algorithms, and distributed systems.
• At least five years of data engineering experience, including at least three years with Databricks, Apache Spark, and cloud data platforms.
• Production experience deploying and maintaining large-scale data solutions.
• In-depth Microsoft Azure experience, including Azure Data Lake, Azure Databricks, Azure Synapse, and Azure DevOps.
• Strong Databricks experience with Delta Lake, Apache Spark, pipeline orchestration, and production performance and reliability optimization.
• Advanced Python and/or Scala programming and practical use of data engineering libraries and frameworks.
• CI/CD pipeline design and maintenance using tools such as GitHub Actions, Azure DevOps, or Jenkins; familiarity with Terraform or ARM templates for infrastructure automation.
• Strong written and verbal communication, stakeholder engagement, technical explanation, and decision-influencing skills.

Preferred qualifications

• Databricks Certified Data Engineer or Microsoft certification, such as Azure Data Engineer Associate.
• Experience with data governance, data security, and compliance frameworks.
• Consulting experience and client management skills.

Why BDO

• BDO describes a people-first culture with professional development, learning opportunities, and a collaborative environment.
• Benefits include competitive total cash compensation, benefits from day one, flexible personal time off, and wellness reimbursement.
• BDO supports community involvement and charitable initiatives.

Flexibility

• Personnel are expected to spend some time working in the office, at client sites, and virtually, subject to accommodations or alternative arrangements.
• The blended work model is intended to meet role, firm, client, and employee needs.