Senior Staff Engineer - Data Platform

Key Responsibilities

• Data Platform Architecture: Own the end-to-end architecture and technical roadmap of the data platform, ensuring scalability, reliability, performance, and long-term maintainability.
• Databricks Engineering: Design, develop, and optimize Databricks-based data solutions using Apache Spark, Delta Lake, and Lakehouse architecture principles.
• Data Pipeline Development: Build and maintain robust batch and streaming data pipelines to support analytics, reporting, machine learning, and operational use cases.
• Distributed Systems: Architect high-throughput, fault-tolerant data processing systems capable of handling large volumes of structured and unstructured data.
• Transactional Systems Integration: Design efficient data integration patterns between transactional databases, analytical platforms, and downstream data consumers.
• Performance Optimization: Identify and resolve bottlenecks in data processing, storage, query execution, and resource utilization to improve system efficiency and control infrastructure costs.
• Data Governance and Quality: Establish standards for data quality, lineage, security, access control, metadata management, and regulatory compliance.
• Technical Leadership: Lead complex engineering initiatives, influence architectural decisions across teams, and establish best practices for data engineering and platform development.
• Cross-Functional Collaboration: Partner with product managers, analytics teams, data scientists, and engineering leaders to translate business requirements into scalable technical solutions.
• Mentorship and Innovation: Mentor senior engineers, conduct architecture reviews, promote engineering excellence, and evaluate emerging technologies to continuously improve platform capabilities.
• Extensive experience in Data Engineering, data platform architecture, and large-scale data processing.
• Strong hands-on expertise in Databricks, Apache Spark, Delta Lake, and Lakehouse architecture.
• Proven experience designing and operating scalable ETL/ELT pipelines and distributed data systems.
• Strong understanding of data modeling, data warehousing, batch processing, and real-time data streaming.
• Experience with cloud-based data platforms, preferably AWS, Microsoft Azure, or Google Cloud Platform.
• Proficiency in SQL and at least one programming language, such as Python or Scala.
• Strong knowledge of data security, governance, reliability, observability, and performance optimization.
• Demonstrated ability to lead architecture initiatives and influence technical strategy across multiple engineering teams.
• 12–17 years of overall experience in software engineering, Data Engineering, or data platform development.
• Significant experience building and scaling enterprise-grade data platforms in complex environments.
• Proven track record of technical leadership, architectural ownership, and successful delivery of mission-critical data systems.
• Excellent problem-solving, communication, stakeholder management, and mentoring skills.

Must-Have Skills

Experience and Qualifications

Qualifications

• Key Responsibilities
• Data Platform Architecture: Own the end-to-end architecture and technical roadmap of the data platform, ensuring scalability, reliability, performance, and long-term maintainability.
• Databricks Engineering: Design, develop, and optimize Databricks-based data solutions using Apache Spark, Delta Lake, and Lakehouse architecture principles.
• Data Pipeline Development: Build and maintain robust batch and streaming data pipelines to support analytics, reporting, machine learning, and operational use cases.
• Distributed Systems: Architect high-throughput, fault-tolerant data processing systems capable of handling large volumes of structured and unstructured data.
• Transactional Systems Integration: Design efficient data integration patterns between transactional databases, analytical platforms, and downstream data consumers.
• Performance Optimization: Identify and resolve bottlenecks in data processing, storage, query execution, and resource utilization to improve system efficiency and control infrastructure costs.
• Data Governance and Quality: Establish standards for data quality, lineage, security, access control, metadata management, and regulatory compliance.
• Technical Leadership: Lead complex engineering initiatives, influence architectural decisions across teams, and establish best practices for data engineering and platform development.
• Cross-Functional Collaboration: Partner with product managers, analytics teams, data scientists, and engineering leaders to translate business requirements into scalable technical solutions.
• Mentorship and Innovation: Mentor senior engineers, conduct architecture reviews, promote engineering excellence, and evaluate emerging technologies to continuously improve platform capabilities.
• Must-Have Skills
• Extensive experience in Data Engineering, data platform architecture, and large-scale data processing.
• Strong hands-on expertise in Databricks, Apache Spark, Delta Lake, and Lakehouse architecture.
• Proven experience designing and operating scalable ETL/ELT pipelines and distributed data systems.
• Strong understanding of data modeling, data warehousing, batch processing, and real-time data streaming.
• Experience with cloud-based data platforms, preferably AWS, Microsoft Azure, or Google Cloud Platform.
• Proficiency in SQL and at least one programming language, such as Python or Scala.
• Strong knowledge of data security, governance, reliability, observability, and performance optimization.
• Demonstrated ability to lead architecture initiatives and influence technical strategy across multiple engineering teams.
• Experience and Qualifications
• 12–17 years of overall experience in software engineering, Data Engineering, or data platform development.
• Significant experience building and scaling enterprise-grade data platforms in complex environments.
• Proven track record of technical leadership, architectural ownership, and successful delivery of mission-critical data systems.
• Excellent problem-solving, communication, stakeholder management, and mentoring skills.