Staff Software Engineer, Inference Performance Optimization, GenAI, DeepMind

Responsibilities

• Analyze and optimize AI inference workloads across the application, model, and distributed fleet infrastructure layers to methodically increase throughput-per-GPU and reduce latency.
• Design and implement inference optimization techniques.
• Investigate and resolve complex model inference performance bottlenecks across the stack.
• Model the latency-to-cost impacts of system variables (such as batch-sizing and utilization goals) and translate these insights into actionable signals that drive production systems.
• Develop investigative tools and metrics (e.g., compute/FLOPs funnels) that track where compute is spent across the fleet.
• - Analyze and optimize AI inference workloads across the application, model, and distributed fleet infrastructure layers to methodically increase throughput-per-GPU and reduce latency. - Design and implement inference optimization techniques. - Investigate and resolve complex model inference performance bottlenecks across the stack. - Model the latency-to-cost impacts of system variables (such as batch-sizing and utilization goals) and translate these insights into actionable signals that drive production systems. - Develop investigative tools and metrics (e.g., compute/FLOPs funnels) that track where compute is spent across the fleet.

Minimum qualifications:

Preferred qualifications:

Qualifications

• Minimum qualifications: - Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, Applied Mathematics, or a related technical field, or equivalent practical experience. - 8 years of experience in software development. - Experience in Python and C++, including navigating, debugging, and modifying serving codebases. - Experience with AI model execution constraints, throughput-latency tradeoffs, memory bandwidth limitations, and modern serving architectures. Preferred qualifications: - Experience with real world LLM inference serving environments or direct contributions to modern open-source inference frameworks (e.g., vLLM, TensorRT-LLM, SGLang, Dynamo). - Experience profiling workloads using standard ML profilers (e.g., PyTorch profiler) and internal trace analysis tools. - Experience with observability and reliability for large distributed systems. - Familiarity with GPU/TPU/accelerator performance concepts (e.g. memory bandwidth, quantization, collective communication, kernel), and can reason their implications to the overall inference serving performance.