Research Engineer, Model Evaluations

Anthropic is a public benefit corporation focused on building reliable, interpretable, and steerable AI systems. We are looking for Research Engineers to build evaluations that measure Claude's capabilities.

• Design and run new evaluations of Claude's capabilities, including reasoning, agentic behavior, knowledge, and safety properties.
• Build and harden the distributed eval execution platform.
• Own dashboards for monitoring model health during training.
• Debug anomalous eval results and communicate findings.
• Improve tooling and workflows for researchers.
• Partner with research teams across the lifecycle of new capabilities.
• Run experiments on prompting, sampling, and scaffolding.
• Communicate results to internal and external audiences.

• Strong Python programming skills, including production or research infrastructure.
• Experience building or operating distributed systems, data pipelines, or other infrastructure at scale.
• Clear written and verbal communication.
• Comfort operating in an on-call or production-support capacity.
• Interest in steering powerful AI to be safe and beneficial.

• Hands-on experience with large language models such as Claude.
• Background in data visualization and dashboards.
• Experience developing evaluation metrics for language models.
• Experience with observability, monitoring, or experiment-tracking systems.
• Background in statistics and experimental design.
• Experience with large-scale dataset sourcing and processing.
• Experience running or supporting ML training infrastructure.

Annual salary: $500,000—$850,000 USD.

Hybrid policy: all staff expected in office at least 25% of the time. Visa sponsorship is available.