Senior MLOps Engineer
Role Overview
Design, build, and maintain ML infrastructure across training, inference, and deployment pipelines.
Own the model lifecycle from data ingestion to real-time serving, ensuring LLM and speech models are deployed efficiently, securely, and reproducibly in Kubernetes environments.
The role calls for hands-on experience with Kubernetes (EKS), Helm, AWS infrastructure, and modern MLOps toolchains.
Speech and voice AI frameworks such as ElevenLabs, Whisper, and RVC are valuable familiarity areas.
Key Skills for This Role
Full Job Posting
About IFM
The Institute of Foundation Models is a research lab focused on building, understanding, deploying, and risk-managing large-scale AI systems.
The institute supports research, education, and industry adoption through scalable infrastructure and real-world applications.
Role
Design, build, and maintain ML infrastructure across training, inference, and deployment pipelines.
Own the model lifecycle from data ingestion to real-time serving, ensuring LLM and speech models are deployed efficiently, securely, and reproducibly in Kubernetes environments.
The role calls for hands-on experience with Kubernetes (EKS), Helm, AWS infrastructure, and modern MLOps toolchains.
Speech and voice AI frameworks such as ElevenLabs, Whisper, and RVC are valuable familiarity areas.
Key Responsibilities
- Design and manage scalable AWS ML infrastructure using EKS, EC2, RDS, S3, and IAM-based access control.
- Build and maintain Kubernetes deployments for LLM and TTS inference using Helm, ArgoCD, and Prometheus/Grafana monitoring.
- Implement and optimize model-serving pipelines using vLLM, SGLang, TensorRT, or similar frameworks.
- Develop CI/CD and MLOps automation for data versioning, model validation, and deployment using tools such as GitHub Actions, Jenkins, or AWS CodePipeline.
- Integrate OpenWebUI, Gradio, or similar interfaces for model demonstrations and internal evaluation tools.
- Collaborate with researchers to productionize TTS, ASR, and LLM-based chat systems.
- Ensure observability, cost optimization, and cloud-resource reliability across environments.
- Contribute to dataset-curation tools, model monitoring, and retraining pipelines.
- Maintain infrastructure as code using Terraform and Helm charts for reproducibility and governance.
- Support real-time multimodal voice, text, and vision workloads across inference clusters.
Required Qualifications and Experience
- At least 4 years of MLOps, DevOps, or cloud infrastructure engineering experience for ML systems.
- Strong proficiency in Kubernetes, Helm, and container orchestration.
- Experience deploying ML models via vLLM, SGLang, TensorRT, or Ray Serve.
- Proficiency with AWS services including EKS, EC2, S3, RDS, CloudWatch, and IAM.
- Solid experience with Python, Docker, Git, and CI/CD pipelines.
- Strong understanding of model lifecycle management, data pipelines, and observability tools such as Grafana, Prometheus, and Loki.
- Strong collaboration skills with ML researchers and software engineers.
Preferred Experience
- Extensive experience with vLLM, Kubernetes, ElevenLabs, Whisper, Gradio/OpenWebUI, or custom TTS/ASR model hosting is preferred.
- Familiarity with multi-GPU scheduling, NCCL optimization, and HPC cluster integration is preferred.
- Knowledge of security, cost management, and network policy in multi-tenant Kubernetes clusters and Cloudflare systems is preferred.
- Prior work in LLM deployment, fine-tuning pipelines, or foundation model research is preferred.
- Exposure to data governance and responsible AI operations in research or enterprise settings is preferred.
Apply for this job in 1 click
Skip the repetitive application forms
Install the Base Career Chrome Extension and autofill job applications across major job boards with your profile.
Trusted by over 500,000 job seekers on Base Career
More from this employer
More jobs at Institute of Foundation Models
Machine Learning Engineer — Pre-training (LLM)
Sunnyvale, USA
Machine Learning Engineer — Reinforcement Learning
Sunnyvale, USA
High Performance Computing Software Engineer - Supercomputing
Abu Dhabi, UAE
The Institute of Foundation Models is seeking an HPC software engineer to build and operate compute infrastructure for large-scale AI workloads. The role covers distributed software, low-level performance optimization, c
Research Scientist - World Modeling
Abu Dhabi, UAE
The Institute of Foundation Models seeks a Research Scientist to lead research for its PAN world model project, spanning video generation, action-conditioned models, robotics, and embodied AI. The role includes developin
Research Engineer - NLP
Abu Dhabi, UAE
The Institute of Foundation Models is seeking an NLP research engineer to develop and improve efficient large language models and related AI systems. The work includes research, algorithm implementation, model integratio
Data & Eval Operations Program Manager
Sunnyvale, USA
Machine Learning Engineer — GPU Kernel
Sunnyvale, USA
Machine Learning Engineer — Pre-training (LLM)
Sunnyvale, USA
Machine Learning Engineer — Reinforcement Learning
Sunnyvale, USA
HPC Engineer
Sunnyvale, USA
High Performance Computing Software Engineer - Supercomputing
Abu Dhabi, UAE
Research Scientist - World Modeling
Abu Dhabi, UAE
Research Engineer - NLP
Abu Dhabi, UAE