Principal Software Engineering Manager - AI Frameworks
Role Overview
As a Principal Software Engineering Manager – AI Frameworks on the team, you will lead and grow a group of engineers working across the AI software serving stack while remaining technically hands-on.
You will set direction and contribute to critical engineering decisions across developer infrastructure, build systems, runtimes, libraries, release pipelines, and application programming interfaces (APIs).
You will be responsible for improving engineering velocity and build reliability, establishing effective system and release practices, and ensuring the team ships high-quality software to production for large-scale model training and inference.
In this role, you will guide the team’s work on benchmarking OpenAI and other large language models (LLMs) across GPUs and Microsoft hardware, translating performance findings into production-ready builds and releases.
You will drive automation across build, validation, benchmarking, regression detection, and deployment workflows, including advanced AI-assisted automation and closed-loop systems that accelerate diagnosis and remediation.
You will partner closely with researchers, product teams, and platform owners to improve developer velocity, reduce time-to-deployment and hardware footprint, and support Microsoft Azure’s capex efficiency goals.
At Microsoft, our mission to empower every person and every organization on the planet to achieve more guides how we partner with customers to deliver trusted, impactful solutions.
With a growth mindset culture, we innovate responsibly and measure success by shared progress people, teams, and customers.
Join us to do meaningful work that changes the world and helps shape what’s next for everyone.
Key Skills for This Role
Full Job Posting
Minimum/Required Qualifications
- Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience.
Other Requirements
- Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.
Preferred Qualifications
- Master’s Degree in Computer Science or related technical field AND 10+ years of software engineering experience, including 6+ years in engineering management, OR Bachelor’s Degree in Computer Science or related technical field AND 12+ years of software engineering experience, including 6+ years in engineering management, or equivalent experience.
- OR Bachelor’s Degree in Computer Science or related technical field AND 12+ years of software engineering experience, including 6+ years in engineering management,
- or equivalent experience.
- 4+ years people management experience.
- Strong technical foundation in software engineering principles, computer architecture, GPU architecture, and hardware acceleration for neural networks, combined with hands-on experience in developer infrastructure such as build systems, dependency management, Bazel or comparable tooling, CI/CD, release engineering, and developer productivity.
- Experience leading teams responsible for end-to-end performance analysis and optimization of LLMs, AI systems, or HPC workloads, including GPU profiling and performance analysis tools, and shipping validated builds through release pipelines into production environments.
- Demonstrated ability to lead cross-team initiatives, align stakeholders, and translate research or platform capabilities into scalable, production-ready solutions.
- Proven people leadership skills, including hiring, coaching, performance management, and career development, with a track record of building high-performing, inclusive teams.
- Working knowledge of AI and ML systems across training, inference, evaluation, and benchmarking, with experience in at least one modern deep learning framework such as PyTorch, TensorFlow, or ONNX Runtime. The candidate should be able to reason about how models, frameworks, runtimes, hardware, and developer infrastructure interact across the end-to-end lifecycle.
- Familiarity with GPU software stacks and acceleration technologies such as CUDA, ROCm, Triton, or equivalent, sufficient to guide technical direction and evaluate tradeoffs. Experience designing advanced automation for build, test, release, benchmarking, and production validation workflows is preferred, including AI-assisted tooling and closed-loop systems that use observed results to drive subsequent actions.
- #AIInfra
- Software Engineering M5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.
- Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here: https://careers.microsoft.com/us/en/us-corporate-pay
- This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
- Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations.
About Microsoft
Multinational technology corporation providing software, cloud, and AI solutions.
Visit company websiteJobs and hiring trendsApply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Microsoft
Solution Sales Advisory IC4
Bengaluru, IND
Senior Critical Environment Electrical Engineering
Toronto, CAN
Senior Software Engineer
Bengaluru, IND
Business Architect - Agent 365
, USA
Construction Project Engineer
Milwaukee, USA
Global Partner Development Manager
Redmond, USA
Solution Engineering (Early in Profession)-Cloud Endpoint
Sydney, AUS
Senior Silicon Supply Chain Program Manager
Redmond, USA