Research Scientist / Engineer, Apollo Research
Evals Team
We can sponsor UK visas.
3-5 days a week in office
AI safety organisation
Job no longer available
AI safety organisation
21-100 employees
Job no longer available
We can sponsor UK visas.
3-5 days a week in office
21-100 employees
Company mission
To secure frontier AI systems across development, deployment, and governance.
Role
Who you are
- We’re looking for Research Scientists and Research Engineers who are excited to work on these and similar projects
- Large Language Model (LLM) steering: The core skill of our evals research scientist role is steering LLMs. This can take many different forms, such as:
- Prompting: eliciting specific behavior through clever word choice
- LM agents & scaffolding: chaining inputs and outputs from various models in a structured way, making them more goal-directed and agentic
- Fluent LLM usage: With increasing capabilities, we can use LLMs to speed up all parts of our pipeline. We welcome candidates who have integrated LLMs into their workflow
- Supervised fine-tuning: creating datasets and then fine-tuning models to improve a specific capability or to study aspects of learning/generalization
- RL(HF/AIF): using other models, programmatic reward functions, or custom reward models as a source of feedback for fine-tuning an existing LLM
- Software engineering: Model evaluators benefit from a solid foundation in software engineering. This can include developing APIs (ideally around LLMs or eval tasks), data science, system design, data engineering, and front-end development
- Generalist: Most evals tasks require a wide range of skills ranging from LLM fine-tuning to developing frontend labeling interfaces. Therefore, we're seeking individuals with diverse skill sets, a readiness to acquire new skills rapidly, and a strong focus on results
- Empirical Research Experience: We’re looking for candidates with prior empirical research experience. This includes the design and execution of experiments as well as writing up and communicating these findings. Optimally, the research included working with LLMs. This experience can come from academia, industry, or independent research
- Scientific mindset: We think it is easy to overinterpret evals results and, thus, think a core skill of a good evals engineer or scientist is to keep track of potential alternative explanations for findings. Ideally, any candidate should be able to propose and test these alternative hypotheses in new experiments
- Values: We’re looking for team members who thrive in a collaborative environment and are results-oriented. You can find out more about our culture here
- Additionally, “nice to have” skills include experience related to AI control and cyber security
- For all skills, we don’t require a formal background or industry experience and welcome self-taught candidates
- Depending on your preferred role, we will weigh these characteristics differently, e.g. software engineers don’t have to have research experience, but must have strong software engineering skills
- We intend to hire people with a broad range of experience and encourage applications even if you don’t yet have experience in any of our current team efforts
- We welcome applicants of all ethnicities, genders, sexes, ages, abilities, religions, sexual orientations, regardless of pregnancy or maternity, marital status, or gender reassignment
What the job involves
- We’re looking for research scientists, research engineers, and software engineers who are excited to work on these and similar projects
- We intend to hire people with a broad range of experience and encourage applications even if you don’t yet have experience in any of our current team efforts
- Conceptual work on safety cases for scheming, for example, our work on evaluation-based safety cases for scheming
- Building evaluations for scheming-related properties (such as deceptive reasoning, sabotage, and deception tendencies)
- Conducting evaluations on frontier models and publishing the results either to the general public or a target audience such as AI developers or governments, for example, our work in OpenAI’s o1-preview system card
- Creating model organisms and demonstrations of behavior related to deceptive alignment, e.g. exploring the influence of goal-directedness on scheming
- Designing and evaluating AI control protocols. We have not started these efforts yet, but intend to work on them starting Q3 2025
- Building a high-quality software stack to support all of these efforts. We have recently switched to Inspect as our primary evals framework
Application process
- Start Date: Target of 2-3 months after the first interview
- About the interview process: Our multi-stage process includes a screening interview, a take-home test (approx. 2 hours), 3 technical interviews, and a final interview with Marius (CEO). The technical interviews will be closely related to tasks the candidate would do on the job. There are no leetcode-style general coding interviews. If you want to prepare for the interviews, we suggest working on hands-on LLM evals projects (e.g. as suggested in our starter guide), such as building LM agent evaluations in Inspect
Salary benchmarks
Our take
Apollo Research is an organisation dedicated to understanding and reducing the risks associated with advanced AI systems, particularly those that exhibit deceptive behaviours. It does so by focusing on evaluating AI models for strategic deception, conducting interpretability research to make AI more understandable, and providing technical guidance to global policymakers on how to handle these emerging technologies.
AI is rapidly advancing so the work Apollo is doing has come at a crucial time. As AI systems grow more powerful, the risks associated with them also become more pressing and Apollo's approach, which combines in-depth model evaluation with cutting-edge interpretability research, helps reduce these risks. Its an exciting initiative that not only ensures AI is safe but could also help shape the future of responsible AI development.
Steph
Company Specialist at Welcome to the Jungle
Company
Company benefits
- Competitive salary and equity package, benchmarked by role and experience
- Pension/401(k) with competitive matching
- Life and income protection plans
- Visa sponsorship or relocation support
- Flexible working hours and schedule
- Unlimited paid time off and sick leave
- Private medical insurance
- 6 months' paid parental leave
- Wellbeing support including therapy and counseling
- Annual professional development budget
- Paid work travel, including company-wide retreats and relevant conferences
- On-site lunch, dinner, and snacks on workdays
Company HQ
City of London, London, UK
Leadership
Marius Hobbhahn
(Co-Founder & CEO)Graduated from the University of Tübingen with an MS in Machine Learning before becoming a Research Fellow at Epoch, where they analysed important trends in AI, build quantitative models, and performed AI forecasting.
Christopher Akin
(COO)Previously a Strategy Consultant at CLA Consulting, a VP of Global Sales at BURN Manufacturing, Founder of The Base Project and Locean Labs, and COO of ADWALLS.
Share this job