Data Platform Engineer - Databricks

Position overview

Responsibilities

• Administer workspaces, clusters, jobs, notebooks, pools, SQL warehouses, identities, permissions, and platform security.
• Monitor health, performance, usage, and costs; troubleshoot jobs, integrations, connectivity, and access issues.
• Optimize compute configurations and support integrations with storage, databases, BI platforms, APIs, and enterprise applications.
• Manage configurations across environments and support CI/CD, infrastructure automation, upgrades, testing, and releases.
• Create monitoring dashboards, alerts, audit reports, documentation, and administrative standards.
• Partner on security, governance, compliance, and data protection; advise users and recommend improvements.

Required qualifications

• At least six years of related experience in cloud infrastructure, data platform or database administration, DevOps, or data engineering.
• Hands-on enterprise Databricks administration experience and experience with Azure Databricks, AWS Databricks, or Databricks on Google Cloud.
• Knowledge of Databricks components, Unity Catalog, access controls, identity management, secrets, and data governance.
• Experience monitoring and troubleshooting Databricks and working with cloud storage, networking, security, and identity services.
• Working knowledge of Apache Spark, distributed processing, SQL, and at least one of Python, PowerShell, Bash, or Scala.
• Experience with Git, CI/CD, automated deployment, and Infrastructure as Code tools.
• Bachelor’s degree in Computer Science, IT, Engineering, Data Science, or a related field.

Preferred qualifications

• Relevant Databricks or cloud certification.
• Experience with Delta Lake, Delta Live Tables, MLflow, Lakehouse architecture, or Databricks SQL.
• Experience with Azure DevOps, GitHub Actions, Jenkins, Power BI, Tableau, Snowflake, or SQL Server.
• Experience in regulated environments; familiarity with data lineage, audit logging, disaster recovery, or business continuity.

Key skills

• Databricks administration, Unity Catalog, Apache Spark, cloud infrastructure, cluster management, performance and cost optimization.
• Platform security, identity and access management, CI/CD, Infrastructure as Code, Python, SQL, Terraform, monitoring, troubleshooting, data governance, and technical documentation.

Workplace

• The position is remote.