Senior LLMOps / AI Platform Engineer

About the company

About the role

Key responsibilities

• Deploy and operate self-hosted LLMs using vLLM, SGLang, and Ollama; optimize inference latency, throughput, concurrency, GPU memory, KV cache, and cost.
• Manage multi-NVIDIA-GPU workloads and deploy AI services on Kubernetes or AWS EKS using Docker and Helm.
• Implement LLM reliability mechanisms, including health checks, monitoring, automated recovery, and model restart or refresh strategies.
• Implement observability with Langfuse or LangSmith, OpenTelemetry, Prometheus, and Grafana.
• Deploy and optimize RAG systems, embedding models, and vector databases such as Qdrant and Milvus.
• Support AI agents and workflows built with LangChain and LangGraph.
• Build and maintain CI/CD pipelines for AI services and infrastructure.
• Troubleshoot production issues across LLMs, GPUs, Kubernetes, networking, and AI applications.

Required skills

• Strong Python and FastAPI experience.
• Experience with vLLM, Hugging Face, and self-hosted LLM deployment.
• Experience with Kubernetes, Docker, Helm, and AWS.
• Experience with NVIDIA GPU inference and performance optimization.
• Experience with LangChain, LangGraph, and LangSmith.
• Experience with RAG, embeddings, vector databases such as Qdrant, and LLM observability and monitoring.
• Experience with PostgreSQL or Redis, GitHub Actions, and CI/CD.
• Strong production troubleshooting skills.