Staff + Senior Software Engineer, Inference

Anthropic's mission is to create reliable, interpretable, and steerable AI systems. The Inference team builds and maintains the critical systems that serve Claude to millions of users worldwide, operating the industry's largest compute-agnostic inference deployments. The team has a dual mandate: maximizing compute efficiency to reliably serve customer growth while enabling breakthrough research by giving scientists high-performance inference infrastructure. The team tackles complex distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.

• Design, build, and maintain the distributed systems that serve Claude to millions of users worldwide
• Develop resilient, flexible systems that adapt in real time to real world events
• Develop intelligent request routing, load balancing, and traffic management systems across thousands of accelerators
• Maximize compute efficiency across the fleet by autoscaling and orchestrating production, research, and experimental workloads
• Build and operate production-grade deployment pipelines for releasing new models to users
• Provide high-performance inference infrastructure that enables researchers to develop next-generation models
• Integrate new AI accelerator platforms and support inference for new model architectures

• Significant software engineering experience, particularly with distributed systems
• Results-oriented, with a bias towards flexibility and impact
• Willingness to pick up slack, even if it goes outside your job description
• Desire to learn more about machine learning systems and infrastructure
• Thrive in environments where technical excellence directly drives both business results and research breakthroughs
• Care about the societal impacts of your work

• Experience with high-performance, large-scale distributed systems
• Experience implementing and deploying machine learning systems at scale
• Experience with load balancing, request routing, or traffic management systems
• Familiarity with LLM inference optimization, batching, and caching strategies
• Experience with Kubernetes and cloud infrastructure (AWS, GCP, Azure)
• Proficiency in Python or Rust

Annual salary: $320,000—$485,000 USD. Applications are reviewed on a rolling basis with no deadline.

Minimum education: Bachelor's degree or equivalent combination of education, training, and/or experience in a field relevant to the role. Hybrid policy: all staff are expected to be in one of Anthropic's offices at least 25% of the time. Visa sponsorship is offered.