Software Development Engineer, AI/ML, AWS Neuron, Model Inference

Role overview

Responsibilities

• Build distributed inference support for PyTorch in the Neuron SDK and tune models for performance and efficiency on Trainium and Inferentia.
• Design and optimize models, frameworks, kernels, and ML operation features for custom accelerators.
• Contribute to distributed system architecture, implementation, profiling, hardware-specific optimization, testing, and production deployment.
• Build infrastructure to analyze and onboard models with diverse architectures.
• Optimize across Neuron hardware generations and implement fusion, sharding, tiling, and scheduling techniques.
• Test models, support continuous releases, and work with customers to enable and optimize their workloads.
• Collaborate across compiler, runtime, framework, and hardware teams; debug issues and improve memory usage and software systems.

Basic qualifications

• Bachelor's degree in computer science or equivalent; at least 3 years of non-internship professional software development experience.
• At least 3 years designing or architecting systems, including design patterns, reliability, and scaling.
• Knowledge of machine learning and LLM architecture, training and inference lifecycles, and model execution optimization.
• Software development experience in at least one of C++ or Python.
• Strong understanding of system performance, memory management, parallel computing, computer architecture, and operating-systems-level software.
• Proficiency in debugging, profiling, and software engineering practices for large-scale systems.

Preferred qualifications

• Master's degree or Ph.D. in Computer Science or equivalent.
• Familiarity with PyTorch, JIT compilation, AOT tracing, CUDA or equivalent low-level kernels, and Triton-like tile semantics.
• Performant kernel development such as CUTLASS or FlashInfer is relevant.
• Experience with production inference serving platforms such as vLLM, SGLang, TensorRT, or similar.

Compensation

• Annual base salary is USD 165,200–223,600 for the listed Cupertino location; compensation may also include sign-on payments and restricted stock units.