Sr Director of Software Engineering- AI Infrastructure Platforms
Key Skills for This Role
Full Job Posting
Job responsibilities
- Lead multiple technology and platform implementations across departments to deliver firmwide AI infrastructure objectives, with a primary focus on training and experimentation platforms operating at enterprise scale.
- Own the design, delivery, and evolution of a Kubernetes ‐ first AI Infrastructure platform, including Kubernetes‐native support for batch and distributed training & Inference jobs, lifecycle management, retry semantics, and failure recovery patterns.
- Standardize AI developer workflows for experimentation, enabling self‐service job submission, reusable templates and golden paths, reproducibility mechanisms, and consistent runtime behavior across hybrid deployment environments.
- Build and evolve platform APIs and automation, including Kubernetes controllers and operators where appropriate, to ensure the platform is safe, scalable, and easy to adopt across teams.
- Drive measurable improvements in GPU availability and utilization through reliability engineering, fleet readiness patterns, and accelerated capacity onboarding.
- Define and implement governance‐based scheduling and placement strategies, including: Multitenant GPU quotas and guardrails, Priority, admission control, and reservation patterns, Preemption policies, Fragmentation reduction and topology‐aware placement (GPU type, MIG, and topology awareness)
- Embed enterprise‐grade security, risk, and control requirements into platform defaults, including IAM and RBAC controls, secrets management, audit logging, policy enforcement, network segmentation, and controlled change management.
- Drive operational excellence by establishing SLIs and SLOs, managing error budgets, leading incident management practices, forecasting capacity, and delivering end‐to‐end platform observability across job lifecycles and GPU telemetry.
- Act as the primary interface with senior leaders, stakeholders, and executives, driving alignment and consensus across competing priorities and complex initiatives.
- Lead multiple engineering teams and managers, building a high‐performing organization with strong engineering standards, scalable operating models, and a culture of accountability and continuous improvement.
- Champion the firm’s culture of diversity, opportunity, inclusion, and respect.
Required qualifications, capabilities, and skills
- 15+ years of engineering experience, including 8+ years of senior engineering leadership experience with responsibility for managing managers.
- Demonstrated experience delivering platform products (beyond foundational infrastructure) with strong adoption, reliability, and operational maturity.
- Experience developing and leading large, cross‐functional engineering teams within highly matrixed and complex enterprise environments.
- Proven track record of leading complex initiatives supporting distributed system design, testing, and operational stability at scale.
- Deep hands‐on expertise with Kubernetes ‐ based platforms, including: Multi‐tenancy, RBAC, admission control, and network policy, Multi‐cluster operations, upgrades, and cluster lifecycle management, Controllers, operators (CRDs), and platform API design patterns
- Experience supporting AI training and experimentation platforms, including: PyTorch and distributed training concepts such as scaling, orchestration, and failure modes, Ray or similar frameworks for distributed experimentation execution,
- Familiarity with Slurm or equivalent HPC or batch schedulers and core concepts such as queues, fair‐share, reservations, and preemption
- Understanding of modern AI inference stacks (for example, vLLM) and how serving constraints—latency, throughput, batching, KV cache behavior, and GPU memory limits—influence training and experimentation platform design.
- Strong understanding of GPU infrastructure fundamentals, including NVIDIA ecosystem capabilities, health and telemetry signals, and scheduling and placement constraints.
- Extensive practical experience with cloud‐native technologies and hybrid infrastructure environments spanning on‐premises and public cloud.
- Experience hiring, developing, coaching, and retaining high‐performing engineering talent.
Preferred qualifications, capabilities, and skills
- Experience operating large‐scale GPU fleets, including heterogeneous accelerator environments.
- Experience delivering hybrid AI platforms across on‐premises infrastructure, public cloud, and specialized accelerated‐compute vendors.
- Experience working at the code level within large‐scale distributed systems.
- This position is subject to Section 19 of the Federal Deposit Insurance Act. As such, an employment offer for this position is contingent on JPMorganChase’s review of criminal conviction history, including pretrial diversions or program entries.
About JPMorgan Chase
Global financial services and investment banking firm.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at JPMorgan Chase
Product Analyst - Card Transactions
Wilmington, USA
As a Senior Product Associate on the Card Transactions team, you will play a multifaceted role in driving end-to-end execution for the product's vision and prioritized roadmap agenda while architecting business solutions
Product Manager for Card Transactions
Wilmington, USA
Vice President, Product Manager on the Card Authorizations Team As a Vice President and Product Manager on the Card Transactions team, you will play a pivotal role in driving end-to-end execution for the product's vision
GRE F&BM P&A - Vice President
Bengaluru, IND
The Global Real Estate Finance team sits within the Chief Administrative Office Finance team and is comprised of finance professionals across the globe. The team is accountable for managing both the expense and capital c
Analytics Solutions Analyst
Mumbai, IND
Shape scalable, business-driven decision-making by delivering complex analytics and remediation initiatives while growing your skills across products, trade flows, and stakeholders. As a Data Analytics Analyst within Mar
FRRA Reg Reporting International-Associate
Bengaluru, IND
You are a strategic thinker passionate about driving solutions in External reporting. You have found the right team. Job responsibilities Required qualifications, capabilities, and skills Preferred qualifications, capabi
Fund Servicing Analyst
Hyderabad, IND
We welcome you to join one of the world’s top five most admired and prominent companies. We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our
Director of Software Engineering
Hyderabad, IND
If you are a software engineering leader ready to take the reins and drive impact, we’ve got an opportunity just for you. As a Director of Software Engineering at JPMorganChase within the Consumer & Community Banking, y
Market and Product Expansion Manager- Commercial Real Estate, Sales Optimization - Vice President
New York City, USA
Be the catalyst for expansion as you pioneer product growth, shape market strategy, and ensure compliance across the CRE franchise. As a Market and Product Expansion Vice President in Commercial Real Estate (“CRE”), you
Product Analyst - Card Transactions
Wilmington, USA
Product Manager for Card Transactions
Wilmington, USA
GRE F&BM P&A - Vice President
Bengaluru, IND
Analytics Solutions Analyst
Mumbai, IND
FRRA Reg Reporting International-Associate
Bengaluru, IND
Fund Servicing Analyst
Hyderabad, IND
Director of Software Engineering
Hyderabad, IND
Market and Product Expansion Manager- Commercial Real Estate, Sales Optimization - Vice President
New York City, USA