Senior Product Engineer, Prolific
Agent Training and Evaluations
High-quality human-powered data at scale
Posted today
High-quality human-powered data at scale
201-500 employees
Posted today
201-500 employees
Company mission
To accelerate human understanding.
Role
Who you are
- Strong experience building scalable infrastructure or distributed systems in the cloud
- Solid understanding of how systems are deployed, scaled and secured
- Hands-on experience with integrating and evaluating AI models or agents
- Interest in how AI agents are trained, evaluated and improved
- Bonus: experience with image, video, or audio processing
- Strong product mindset with a focus on delivering business and customer value
- Comfort with ambiguity, and the ability to move quickly from idea to working system
- Strong communication and collaboration skills
- Python experience is useful, but not essential
What the job involves
- As a Senior Product Engineer, you'll be one of the early engineers in our Agentic Training & Evals team
- You'll build the infrastructure that lets us evaluate AI agents, capturing agent traces at scale alongside real human expert work in safe and secure environments
- This is a genuine 0→1 role. There isn't a well-established playbook or existing architecture for you to simply maintain. You'll have significant influence and ownership over the technical direction of the team and the products we build
- You'll work closely with engineering, product and AI research to turn ambiguous problems into scalable systems, and you'll see the impact of what you build directly
- Build tooling for capturing and processing data from agents and humans performing real-world tasks, at significant scale
- Solve hard problems around compute, orchestration, scaling, security and reliability
- Help develop approaches for training, benchmarking and evaluating AI agents
- Work with emerging agentic AI technologies and rapidly evolving models
- Make pragmatic technical decisions in a highly ambiguous environment
- Partner closely with product engineering and AI research to turn customer problems into working products
- Drive direct impact in a fast-moving, high-ownership, cross-functional team
- Key Technologies
- Cloud Platforms: Google Cloud Platform
- Programming Languages: Python, JavaScript, and TypeScript
- Frameworks: FastAPI, Vue.js Container-based and Serverless architectures
- Databases: Postgres SQL
- DevOps and Monitoring: GitHub Actions, Kubernetes, DataDog
Our take
Founded in 2014, Prolific is a marketplace and data analysis company that provides researchers with a network of test participants. From eye-tracking to human-facing AI applications, Prolific’s network acts as both a recruiter for test participants and as a marketplace for researchers scientists to hire said participants.
Founded in 2014, it’s already used by the likes of Google, Stanford, Oxford and Kings College London Universities to test new AI products, providing “stress tests” for its clients. Prolific’s user base now counts over 2000 potential test subjects, whilst researchers leverage its network to launch surveys, tests and evaluation tasks every 2 minutes.
An indicator of Prolific’s success is maintaining of profitability for years after its initial Y Combinator investment. In 2024, it raised $32 million in Series A to maintain its healthy position. This funding will allow it to expand its operations.
Freddie
Company Specialist at Welcome to the Jungle
Insights
Glassdoor (4.2)
Company
Funding (1 round)
Jul 2023
$33.9m
Series A
Total funding: $33.9m
Company benefits
- Bank holiday swaps
- Enhanced sick pay
- L&D budget – £1000 per year
- 25 days annual leave + bank holidays
- Remote Work ( Within UK or US)
Company values
- We win together
- We care deeply about our customers
- We make every minute count
- We're optimistic and find a way
- We embrace change
- We aim high
Company HQ
Kings Cross, London, UK
Leadership
Phelim Bradley
(Co-Founder & CEO)Currently also Venture Partner at Pioneer Fund. Previously Dphil at Wellcome Trust Centre for Human Genetics, Bioinformatics Intern at Illumina and MPhil of Computer Biology at University of Cambridge.
Diversity, Equity & Inclusion at Prolific
- At Prolific, we want all of our employees, researchers, and participants to be included and so we commit to having ongoing discussions about diversity and inclusion and regularly look at areas where we can improve. This policy aims to: - treat all our employees fairly and respectfully, regardless of their job status, location, or work hours ⏰ - avoid any illegal discrimination based on age, disability, gender, gender identity, marriage or civil partnership, pregnancy and maternity, race or ethnicity, religion or belief, sexual orientation, and traveler status. These are protected by the UK Equality Act (2010), Ireland Employment Equality Act (2004), Americans with Disabilities Act (ADA) at the federal level and the New York State Human Rights Law (NYSHRL) at the state level. ⚖️ - refuse to tolerate any kind of illegal discrimination 🙅🏻♀️. This includes pay and benefits, work conditions, grievance and discipline handling, termination, redundancy, parental leave, flexible working requests, employment selection, promotion, training, and other opportunities for development.
Share this job
View 20 more jobs at Prolific