Databricks SME (R-00117)

Performance & Optimization: Optimize data pipelines, configure clusters, and implement best practices for cost and performance.

Governance & Security: Implement security protocols, such as Unity Catalog, IAM, and data access controls.

Technical Leadership: Act as a trusted advisor to engineering teams, mentor staff, and define technical roadmaps.

Collaboration: Work with data engineers and stakeholders to align technical solutions with business requirements.

Technical Expertise: Strong proficiency in Databricks, Apache Spark, Python/PySpark, and SQL.

Cloud Proficiency: Experience with cloud platforms, specifically AWS (S3, Glue) or Azure.

Data Modeling: Expertise in data modeling, data profiling, and ETL/ELT pipeline design

Architecture & Development: Design and implement scalable data pipelines, data models, and Lakehouse architectures.

Implement CI/CD pipelines, use tools like Azure DevOps or Git for version control, and automate routine tasks.

Integrate data from various cloud sources and services (AWS, Azure, GCP) such as Blob Storage, Data Lake Storage, and other databases.

Optimize Spark jobs & monitor system metrics to quickly troubleshoot issues related to the Databricks environment.

Requires 3-5 years of experience in data engineering, data warehousing, or data architecture roles.

US citizenship and ability to obtain and maintain a federal security clearance may be required

Bachelors degree is highly preferred