Staff + Sr. Software Engineer, Cloud Inference Launch Engineering

About Anthropic

Role overview

Key responsibilities

• Bring up new model architectures and launch them on cloud platforms in sync with the first-party platform.
• Integrate new inference features, including structured sampling and prompt caching, with cloud platforms.
• Fix cross-platform differences in configuration, observability, deployment patterns, and inference behavior.
• Build and own CI/CD for inference servers and load balancers, including shadow traffic, performance baselines, and correctness checks.
• Speed validation and reduce merge-to-production time without sacrificing reliability.
• Use production observability data to find and remediate bottlenecks, cost anomalies, and regressions.

Minimum qualifications

• Significant software engineering experience and a strong background in high-performance, large-scale distributed systems serving millions of users.
• Track record building automation or test infrastructure that measurably improved release velocity or reliability.
• Experience building or operating services on AWS, GCP, or Azure, with exposure to Kubernetes, Infrastructure as Code, or container orchestration.
• Strong interest in LLM serving; prior inference or machine learning experience is not required.
• Able to collaborate across internal teams and external partners, learn quickly, and own problems autonomously.
• Bachelor’s degree or an equivalent combination of education, training, and experience; relevant study may be demonstrated through coursework, training, or professional experience.

Preferred qualifications

• Experience with LLM inference optimization, batching, and caching strategies.
• Experience with capacity-constrained scheduling or shared-resource test infrastructure.
• Understanding of multi-region deployments, request routing, load balancing, and global traffic management.
• Experience working with cloud provider partners across platforms and navigating differences in networking, security, privacy, and managed services.
• Proficiency in Python or Rust.

Compensation

• Annual salary range is $320,000–$485,000 USD.

Workplace policy

• The location-based hybrid policy expects staff to work from an office at least 25% of the time; some roles may require more office time.

Additional information