{bc}
Apply
workday

Databricks Platform Engineer (AWS)

Vanguard
Hyderabad, IND
Full-time
Mid · 5–8 years experience
Onsite
Discovered 1 weeks ago
awscloudformationcloudwatchdatabricksdelta-lakegit

Full Job Posting

Responsibilities

  • Provision and configure Databricks Accounts, Workspaces, Unity Catalog, and supporting AWS infrastructure.
  • Design, deploy, and manage multi-environment platforms (Dev/Test/Prod) and automate platform provisioning and configuration using Terraform and CloudFormation.
  • Configure and manage AWS services including VPC, Subnets, Security Groups, IAM, KMS, S3, Lambda, EC2, ECS, RDS, CloudWatch, resolving WIZ violations and related cloud services.
  • Implement and govern Unity Catalog, including metastores, catalogs, schemas, external locations, storage credentials, dashboards, Databricks Apps and fine-grained access controls to enable secure data sharing and enterprise data governance.
  • Implement observability and operational monitoring using Databricks System Tables, audit logs, cluster logs, usage metrics, and cloud monitoring solutions to improve platform reliability and user experience.
  • Analyze platform usage patterns, cluster utilization, DBU consumption, and infrastructure costs to identify optimization opportunities and implement cost governance controls.
  • Support migration, onboarding, and enablement of enterprise applications, data engineering workloads, AI/ML platforms, and GenAI use cases onto Databricks while adhering to architecture and security standards.
  • Leverage Databricks Asset Bundles (DAB), Databricks CLI, REST APIs, Git integration, and CI/CD pipelines to automate deployment and lifecycle management of notebooks, jobs, workflows, and platform configurations.
  • Enable AI/ML and GenAI capabilities, including MLflow, Model Serving, Feature Store, Vector Search, and Mosaic AI, while supporting governance, monitoring, and operationalization of machine learning workloads.
  • Implement and govern Unity Catalog, including metastores, catalogs, schemas, external locations, storage credentials, and fine-grained access controls.
  • Enable audit logging, lineage tracking, data access governance, and platform monitoring to meet enterprise standards and analyze DBU consumption, cluster usage patterns, and AWS infrastructure spend to identify optimization opportunities and implement cost governance controls.
  • Provide hands-on troubleshooting and operational support for platform, integration challenges and troubleshoot infrastructure, networking, security, authentication, and connectivity issues.
  • Partner directly with application teams, data engineers, data scientists, architects, and business stakeholders to understand business requirements and recommend platform solutions.
  • Act as a trusted advisor and primary point of contact for Databricks platform capabilities and adoption.
  • Qualifications and Skills
  • Minimum 5 years of experience in cloud engineering, infrastructure management, or cloud operations.
  • Bachelor’s degree (B.E./B.Tech in Computer Science/IT) or equivalent experience.
  • Strong experience with Databricks administration and platform engineering.
  • Deep understanding of Databricks platform components including Unity Catalog, Workflows, SQL Warehouses, Cluster Policies, Delta Lake, Auto Loader, Structured Streaming, and Lakehouse architectures.
  • Hands-on experience with Databricks Asset Bundles (DAB), Databricks CLI, REST APIs, Git integration, and platform automation.
  • Expertise in AWS services including VPC, IAM, EC2, S3, KMS, Lambda, RDS, CloudWatch, and cloud networking.
  • Hands-on experience with Terraform, CloudFormation, Infrastructure as Code (IaC), and automation frameworks.
  • Strong understanding of RBAC, governance, security controls, and enterprise compliance requirements.
  • Experience implementing and supporting CI/CD pipelines using GitHub Actions, Jenkins, or equivalent tools.
  • Proficiency in Python, SQL, Type Script, REST APIs, and scripting for platform automation.
  • Experience supporting enterprise-scale production platforms and mission-critical workloads.
  • Hands-on expertise with GitHub, Bitbucket/Bamboo, AWS Services, Cloud based databases, CI/CD, Pipeline Development, Cloud Platform Development, Cloud Based Application Development
  • Experience Senior Engineer, being able to work independently, passionate about solving points of pain for developer community, scrum/agile methodologies
  • Strong troubleshooting skills in cloud infrastructure, networking, security best practices, and performance optimization
  • Preferred Qualifications /Certifications
  • Databricks Certified Platform Administrator, Data Engineer Associate, or Data Engineer Professional.
  • AWS Solutions Architect Associate/Professional.
  • Experience supporting AI/ML and GenAI platforms including ML flow, Model Serving, Feature Store, Vector Search, and Mosaic AI.
  • Experience working in regulated or highly governed enterprise environments.

Location

  • This role is based in Hyderabad, Telangana at Vanguard India. Only qualified external applicants will be considered.
  • Our mission
  • Vanguard adheres to a simple purpose: To take a stand for all investors, to treat them fairly, and to give them the best chance for investment success.
  • Our commitment to you
  • Vanguard takes the same long-term view of your success—at work and in life—with Benefits and Rewards packages that reflect what you care about, throughout all the phases and stages of your life. Our Total Rewards programs provide you and your loved ones with wellness support for key areas in your life:
  • Financial wellness We're committed to enabling your financial success and provide competitive offers and programs.
  • Physical wellness We're committed to providing benefits that support your physical health and wellness.
  • Personal wellness We're committed to providing resources that help support the full scope of your life.
  • How we work
  • Vanguard has implemented a hybrid working model for most of our employees (crew members), designed to capture the benefits of enhanced flexibility while enabling in-person learning, collaboration, and connection. We believe our mission-driven and highly collaborative culture is a critical enabler to support long-term client outcomes and enrich the employee experience.

Qualifications

  • Senior Engineer, being able to work independently, passionate about solving points of pain for developer community, scrum/agile methodologies
  • Strong troubleshooting skills in cloud infrastructure, networking, security best practices, and performance optimization
  • Databricks Certified Platform Administrator, Data Engineer Associate, or Data Engineer Professional.
  • AWS Solutions Architect Associate/Professional.
  • Experience supporting AI/ML and GenAI platforms including ML flow, Model Serving, Feature Store, Vector Search, and Mosaic AI.
  • Experience working in regulated or highly governed enterprise environments.
  • Location
  • This role is based in Hyderabad, Telangana at Vanguard India. Only qualified external applicants will be considered.
  • Our mission
  • Vanguard adheres to a simple purpose: To take a stand for all investors, to treat them fairly, and to give them the best chance for investment success.
  • Our commitment to you
  • Financial wellness
  • We're committed to enabling your financial success and provide competitive offers and programs.
  • Physical wellness
  • We're committed to providing benefits that support your physical health and wellness.
  • Personal wellness
  • We're committed to providing resources that help support the full scope of your life.
  • How we work
  • Vanguard has implemented a hybrid working model for most of our employees (crew members), designed to capture the benefits of enhanced flexibility while enabling in-person learning, collaboration, and connection. We believe our mission-driven and highly collaborative culture is a critical enabler to support long-term client outcomes and enrich the employee experience.
  • Minimum 5 years of experience in cloud engineering, infrastructure management, or cloud operations.
  • Bachelor’s degree (B.E./B.Tech in Computer Science/IT) or equivalent experience.
  • Strong experience with Databricks administration and platform engineering.
  • Deep understanding of Databricks platform components including Unity Catalog, Workflows, SQL Warehouses, Cluster Policies, Delta Lake, Auto Loader, Structured Streaming, and Lakehouse architectures.
  • Hands-on experience with Databricks Asset Bundles (DAB), Databricks CLI, REST APIs, Git integration, and platform automation.
  • Expertise in AWS services including VPC, IAM, EC2, S3, KMS, Lambda, RDS, CloudWatch, and cloud networking.
  • Hands-on experience with Terraform, CloudFormation, Infrastructure as Code (IaC), and automation frameworks.
  • Strong understanding of RBAC, governance, security controls, and enterprise compliance requirements.
  • Experience implementing and supporting CI/CD pipelines using GitHub Actions, Jenkins, or equivalent tools.
  • Proficiency in Python, SQL, Type Script, REST APIs, and scripting for platform automation.
  • Experience supporting enterprise-scale production platforms and mission-critical workloads.
  • Hands-on expertise with GitHub, Bitbucket/Bamboo, AWS Services, Cloud based databases, CI/CD, Pipeline Development, Cloud Platform Development, Cloud Based Application Development
  • Experience Senior Engineer, being able to work independently, passionate about solving points of pain for developer community, scrum/agile methodologies

Preferred Qualifications

  • Preferred Qualifications /Certifications

Apply for this job in 1 click

Skip the repetitive application forms

Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.

Sarah M.James T.Maya R.

Trusted by over 500,000 job seekers on Base Career

Start Free Today

More from this employer

More jobs at Vanguard

Application Engineer - III

Melbourne, AUS

Discovered TodayFull-time

Application Engineer - III HRCS-MYCE CS

Hyderabad, IND

Discovered YesterdayFull-time

Application Engineer - III HRCS-MYCE CS

Hyderabad, IND

Discovered YesterdayFull-time

Manager, Data Engineering

Hyderabad, IND

Discovered 2 days agoFull-time

Manager, IT Delivery – IAM/IGA Engineering

Malvern, USA

Discovered 2 days agoFull-time

Learning & Development Analyst, Specialist (WT PMO)

Hyderabad, IND

Discovered 2 days agoFull-time

Data Engineer

Malvern, USA

Discovered 2 days agoFull-time

Senior Manager, Application Engineering (Appian Platform)

Hyderabad, IND

Discovered 2 days agoFull-time