High Performance Computing Software Engineer - Supercomputing
Role Overview
Build foundational compute infrastructure for AI and computational science.
Design, develop, and operate software systems for large-scale AI workloads, including training across 1,000+ GPUs.
Work at the intersection of high-performance computing and machine learning to make infrastructure robust, performant, and developer-friendly.
Key Skills for This Role
Full Job Posting
About the Institute of Foundation Models
The Institute of Foundation Models is a research lab focused on building, understanding, using, and risk-managing foundation models.
The lab advances research, develops AI talent, and contributes to a knowledge-driven economy.
The role
Build foundational compute infrastructure for AI and computational science.
Design, develop, and operate software systems for large-scale AI workloads, including training across 1,000+ GPUs.
Work at the intersection of high-performance computing and machine learning to make infrastructure robust, performant, and developer-friendly.
Job responsibilities
- Design and implement high-performance distributed software for large-scale AI/ML training.
- Optimize low-level system components, including the Linux kernel, GPU/accelerator kernels, and interconnects.
- Develop and tune communication libraries such as NCCL, MPI, UCX, RCCL, and RDMA-based systems.
- Support PyTorch, MegatronLM, and DeepSpeed in large-scale production environments with ML researchers and engineers.
- Contribute to scheduling, orchestration, and job-management systems, including Slurm and Kubernetes.
- Debug and resolve issues across the stack, from kernel and container layers to models.
- Work with hardware vendors, open-source communities, and internal teams to improve performance and reliability.
Skills and experience
- Proven experience developing and optimizing software for large-scale ML workloads; 1,000+ GPUs is preferred.
- Deep understanding of Linux kernel internals and accelerator (GPU) kernel development.
- Proficiency with distributed communication libraries such as NCCL, RCCL, MPI, UCX, SHARP, or Libfabric.
- Experience with ML frameworks such as PyTorch, TensorFlow, JAX, or MegatronLM.
- Strong knowledge of HPC job scheduling and orchestration tools such as Slurm, Kubernetes, or Pyxis.
- Excellent debugging and systems performance-tuning skills.
- Collaborative mindset focused on shared success and technical excellence.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Institute of Foundation Models
Machine Learning Engineer — Pre-training (LLM)
Sunnyvale, USA
Machine Learning Engineer — Reinforcement Learning
Sunnyvale, USA
Research Scientist - World Modeling
Abu Dhabi, UAE
The Institute of Foundation Models seeks a Research Scientist to lead research for its PAN world model project, spanning video generation, action-conditioned models, robotics, and embodied AI. The role includes developin
Research Engineer - NLP
Abu Dhabi, UAE
The Institute of Foundation Models is seeking an NLP research engineer to develop and improve efficient large language models and related AI systems. The work includes research, algorithm implementation, model integratio
Senior MLOps Engineer
Abu Dhabi, UAE
The Institute of Foundation Models is seeking a senior MLOps engineer to build and operate infrastructure and deployment pipelines for LLM, speech, and multimodal systems. The role requires at least four years of relevan
Data & Eval Operations Program Manager
Sunnyvale, USA
Machine Learning Engineer — GPU Kernel
Sunnyvale, USA
Machine Learning Engineer — Pre-training (LLM)
Sunnyvale, USA
Machine Learning Engineer — Reinforcement Learning
Sunnyvale, USA
HPC Engineer
Sunnyvale, USA
Research Scientist - World Modeling
Abu Dhabi, UAE
Research Engineer - NLP
Abu Dhabi, UAE
Senior MLOps Engineer
Abu Dhabi, UAE