Senior Software Engineer II

Qualifications

• Lakehouse Platform Ownership: Own the end-to-end architecture of the Databricks Lakehouse, including workspace topology, cluster policies, compute strategy (job clusters, SQL warehouses, serverless), and multi-cloud deployment patterns (AWS/Azure/GCP).
• Unity Catalog & Governance: Design and enforce the global data governance model — catalogs, schemas, access controls, lineage, and audit logging — via Unity Catalog, ensuring consistent metadata management and fine-grained access across business units.
• Data Engineering Standards: Define and enforce best practices for Delta Lake table design, Delta Live Tables (DLT) pipelines, medallion architecture (bronze/silver/gold), and performance optimization (Z-ordering, liquid clustering, partitioning, file compaction).
• Platform Reliability & Automation: Partner with CloudOps and DevOps to industrialize the platform through Infrastructure-as-Code (Terraform/Databricks Asset Bundles), CI/CD pipelines, and monitoring, targeting 99.99% platform availability.
• FinOps & Cost Governance: Own cost transparency and optimization across the Databricks estate — cluster right-sizing, workload isolation, chargeback/showback models, and proactive budget alerting — bringing "Strategic Foresight" to cloud spend.
• Cross-Functional Enablement: Work directly with Data Engineering, Analytics, Data Science, and business stakeholders (Finance, HR, Product) to translate requirements into platform capabilities and self-service patterns.
• Governed AI Enablement: Own the governed rollout of AI use cases and native AI features (e.g., Databricks Genie, AI/BI Dashboards, Mosaic AI) on the platform. You hold the keys to unlocking GenAI and natural-language analytics for the business, safely, within Unity Catalog's governance and access-control boundaries.
• Community & Mentorship: Build and lead a community of practice for Databricks developers across the organization — publishing standards, running enablement sessions, and unblocking delivery friction for engineering teams