Research Engineer, Anthropic

• We are looking for engineers who can execute rapidly, maintain high throughput, and bring a strong builder mindset to solving complex problems
• The ideal candidate will be able to quickly prototype and iterate on evaluation infrastructure while maintaining high engineering standards
• Have led and conducted fast, iterative experiments with frontier AI models
• Have designed or implemented evaluations that involve sampling + prompting LLMs
• Write clean, well-structured code that others can build upon
• Care deeply about AI safety and responsible development
• Have strong software engineering skills with extensive Python experience
• Have experience working with distributed systems
• Are comfortable defining technical specifications and executing towards them
• Are a self-starter who thrives in fast-paced, collaborative environments
• Are excited about tackling unprecedented technical challenges
• Can balance the urgency of our mission with careful, methodical implementation

Desirable

• We're building a team to develop and run "gold standard" evaluations for catastrophic risks, to make sure we release models that are safe for the world to use
• This work is at the core of implementing our Responsible Scaling Policy (RSP), which defines the technical and operational measures for safely training and deploying frontier AI models
• As a Research Engineer on the Frontier Red Team, you'll be creating evaluation systems that will help us understand and control some of the most capable AI systems ever built
• You will collaborate with domain experts across multiple workstreams including biosecurity, autonomous replication, cybersecurity, and national security
• You'll build, scale, and run evaluations to measure dangerous capabilities in models and determine if and when they cross ASL thresholds and require heightened security measures
• Your work will directly inform decisions at the highest levels of the company and help establish standards that could influence the entire AI industry
• You'll be building systems to evaluate capabilities that have never existed before, requiring creative solutions and rigorous implementation
• Design and implement robust evaluation infrastructure to measure model capabilities and risks across multiple domains
• Lead technical projects to build and scale evaluation systems that could become industry standards
• Collaborate with domain experts to translate their insights into concrete evaluation frameworks
• Build sandboxed testing environments and automated pipelines for continuous model assessment
• Work closely with researchers to rapidly prototype and iterate on new evaluation approaches
• Partner with cross-functional teams to advance Anthropic's safety mission
• Contribute to Capability Reports that inform critical deployment decisions
• Representative Projects:
• Build infrastructure for running large-scale model evaluations across multiple risk domains
• Create tools for rapid evaluation prototyping and iteration
• Contribute to evaluation frameworks that could become industry standards
• Design and implement custom testing environments for specific capability assessments
• Develop monitoring and analysis systems for evaluation results
• Collaborate with domain experts to translate theoretical risks into practical tests, such as cyber ranges and autonomous replication environments

• Applications will be reviewed on a rolling basis