Sr. Data Engineer

Role overview

Responsibilities

• Design and implement data lakes, data warehouses, analytics platforms, and end-to-end data architectures.
• Define scalable, secure, and performant data integration and transformation strategies.
• Translate business requirements into solutions for analytics, reporting, and AI initiatives.
• Develop data models, ETL/ELT pipelines, and frameworks for structured and unstructured data.
• Lead and mentor data engineers and developers while promoting data management and governance practices.
• Ensure data governance, security, and privacy standards are followed across platforms.
• Optimize existing architectures and processes for performance and reliability.
• Stay current with cloud data services and technologies including Databricks, Snowflake, Azure Synapse, and AWS Redshift.
• Advise clients on architecture decisions and data modernization best practices.

Required skills and experience

• 5+ years of experience in data architecture, data engineering, or analytics solution design.
• Hands-on experience with data lake and warehouse technologies such as Databricks, Snowflake, Redshift, or Synapse.
• Deep understanding of data modeling, data integration, and ETL/ELT design.
• Proficiency in SQL and PySpark/Python or Scala for complex Spark transformations and optimization.
• Understanding of data governance, security, and privacy best practices.
• Experience with Databricks Autoloader, Delta Live Tables, streaming ingestion, and Medallion architecture.
• Experience implementing CI/CD pipelines for Databricks workflows, notebooks, and cluster configurations.
• Experience migrating traditional data warehouses to the Databricks Lakehouse.
• Working knowledge of AWS or Azure data storage, networking, and security concepts.
• Ability to engage clients, present technical solutions, and communicate complex ideas clearly.
• Bachelor’s or master’s degree in computer science, information systems, engineering, or a related field.
• Strong problem-solving, communication, and collaboration skills.

Preferred qualifications

• Experience integrating AI and machine learning with data science workflows.
• Knowledge of data cataloging and metadata management tools.
• Prior consulting or client-facing experience in a technology services firm.

About Bits In Glass