Research Engineer / Scientist, Anthropic

• You want to build and run elegant and thorough machine learning experiments to help us understand and steer the behavior of powerful AI systems
• You care about making AI helpful, honest, and harmless, and are interested in the ways that this could be challenging in the context of human-level capabilities
• You could describe yourself as both a scientist and an engineer
• Have significant software, ML, or research engineering experience
• Have some experience contributing to empirical AI research projects
• Have some familiarity with technical AI safety research
• Prefer fast-moving collaborative projects to extensive solo efforts
• Pick up slack, even if it goes outside your job description
• Care about the impacts of AI
• Education requirements: We require at least a Bachelor's degree in a related field or equivalent experience

Desirable

• As a Research Engineer on Alignment Science, you'll contribute to exploratory experimental research on AI safety, with a focus on risks from powerful future systems (like those we would designate as ASL-3 or ASL-4 under our Responsible Scaling Policy), often in collaboration with other teams including Interpretability, Fine-Tuning, and the Frontier Red Team
• Our blog provides an overview of topics that the Alignment Science team is either currently exploring or has previously explored. Our current topics of focus include..
• Scalable Oversight: Developing techniques to keep highly capable models helpful and honest, even as they surpass human-level intelligence in various domains
• AI Control: Creating methods to ensure advanced AI systems remain safe and harmless in unfamiliar or adversarial scenarios
• Alignment Stress-testing: Creating model organisms of misalignment to improve our empirical understanding of how alignment failures might arise
• Automated Alignment Research: Building and aligning a system that can speed up & improve alignment research
• Representative projects:
• Testing the robustness of our safety techniques by training language models to subvert our safety techniques, and seeing how effective they are at subverting our interventions
• Run multi-agent reinforcement learning experiments to test out techniques like AI Debate
• Build tooling to efficiently evaluate the effectiveness of novel LLM-generated jailbreaks
• Write scripts and prompts to efficiently produce evaluation questions to test models’ reasoning abilities in safety-relevant contexts
• Contribute ideas, figures, and writing to research papers, blog posts, and talks
• Run experiments that feed into key AI safety efforts at Anthropic, like the design and implementation of our Responsible Scaling Policy