Machine Learning Lead, NewtonX
AI Data Labeling
$180-260k
OTE
Offers Equity
More information about location
AI-powered B2B research
Be an early applicant
AI-powered B2B research
201-500 employees
Be an early applicant
$180-260k
OTE
Offers Equity
More information about location
201-500 employees
Company mission
To uncover world-class knowledge
Role
Who you are
- Deep applied ML experience centered on post-training human data — you've owned a human-data workstream as an applied scientist or ML engineer
- At least 2 years working hands-on with RL, including how tool-use trajectories are rewarded and evaluated. If you're not fluent in RL, this isn't the role — it's the foundation of the core judgment you'll be making
- Working fluency across modern LLM post-training and evaluation: SFT, RLHF/preference data quality, RLVR, chain-of-thought, eval harness construction, contamination handling, statistical significance, and agentic/tool-use evaluation
- Genuine understanding of how training data becomes model behavior — you can reason about what a model will learn from a given dataset, not just whether the data meets spec
- Strong programming foundation: read and reason about an eval harness, write Python comfortably, work with model APIs, prototype scoring pipelines. Not a production engineer, but not hands-off
- Statistical fluency: you know when an effect is real vs. noise and can defend a sample size or significance threshold
- Client-facing presence: you've defended technical design choices in real time to skeptical audiences and adjusted scope without losing rigor. Range matters — you can talk to a Series B CTO and a Fortune 100 AI lead in the same week
- Strong written communication: methodology sections, technical reports, and specs that hold up to expert review
- If the profile above describes you and your passions, we'd love to hear from you!
What the job involves
- NewtonX is rapidly expanding into the AI data annotation and RLHF (Reinforcement Learning from Human Feedback) space
- We are leveraging our core superpower—recruiting the world’s leading domain experts—to provide high-quality, expert-led data labeling for AI labs and enterprises
- Beyond our unmatched B2B recruiting, we utilize powerful, automated project management processes that allow us to scale projects rapidly, adapt to shifting requirements, and manage our subject matter experts with professional, industry-best practices and fair compensation
- This role is the technical owner of data quality
- You will partner directly with the Program Lead and act as technical lead in communicating with clients, interpreting their core AI model testing goals and assisting the Program Lead in creating concrete technical specs that will accomplish these goals
- The core question you own: will the data we produce do its job once the customer trains on it or evaluates with it? Clean data that passes every operational gate can still fail — a training set that yields a weak or misleading signal, or an eval that technically runs but doesn't surface the weaknesses that matter. You are the person who can look at an approved submission and say "this is technically correct and still won't do the job, and here's why."
- You are hands-on and close to the work. This is a foundational role — what it covers will grow as the business does
- Own the judgment of whether task designs and rubrics produce a useful training signal for the consuming method (SFT, RLHF, RLVR, agentic RL, CoT, evals). Catch mismatches between what the data rewards and what the customer is actually training for
- Design the task, environment, and rubric structure for agentic workflows
- Own the ML-level validation of the dataset before delivery — not just whether individual submissions meet spec
- Partner with the Program Lead to convert customer requirements into concrete technical specs: expert profiles, screener trees, task interfaces, task templates, QC rubrics, statistical thresholds
- Partner with the Program Lead to define and defend quality metrics: inter-annotator agreement targets, gold-standard injection rates, statistical power thresholds — providing the statistical and methodological grounding
- Calibrate the ops team on what "good" looks like per engagement; run alignment sessions when standards shift
- Serve as the technical counterpart to the customer's ML, applied science, and product teams. Hold your ground on the technical questions that decide data quality across training and evaluation — task and reward design, data quality for SFT and preference methods, RL and RLVR, contamination, statistical rigor, and agentic workflows
- Help diagnose data-quality questions when a customer reports the data underperformed — reason through whether the issue is the data, the quantity, or something in their training setup, and make the case defensibly
Our take
In business, every decision needs data, but reaching those who have the expertise a company is looking for can be difficult and time-consuming. NewtonX is democratizing access to data and research, by enabling access to high-quality knowledge for all businesses, beyond those who employ a panel of experts.
The premise of NewtonX is that it matches questions to those who are verified to know the answer. Searches are custom and AI-led, and each one teaches the system so that it constantly improves at finding the most relevant professionals. As such, businesses have access to knowledge from a vast array of sources, without having to spend time sifting through irrelevant hits.
NewtonX works with an array of companies, including financial institutions and Fortune 500 consulting firms, showcasing the confidence that the industry has in its platform. It has raised impressive funding over the years, which will ensure that its proprietary search tech continues to be at the forefront of the space.
Freddie
Company Specialist at Welcome to the Jungle
Insights
31% employee growth in 12 months
Company
Funding (last 2 of 3 rounds)
Dec 2021
$32m
SERIES B
Jun 2019
$12m
SERIES A
Total funding: $46m
Company benefits
- Medical, dental, and vision insurance
- 401K 3% match, immediate vesting
- Paid Vacation and Public Holidays
- Paid Sick Days
- Pre-tax commuter benefits
- Health Savings / Flexible Savings Account
- Paid Parental / Family Leave
- Office snacks and refreshments
- Lunch and learns
- Monthly team outings and bimonthly happy hours
- Annual company retreat
- Volunteering
- Virtual fun, social activities (e.g., happy hours, painting, escape rooms, trivia, cooking classes, meditation, and more!)
Company HQ
Murray Hill, New York, NY
Leadership
Sascha J. E.
(Co-Founder & CEO)Former Consultant with McKinsey & Company and with The Boston Consulting Group. Also previously worked as a Financial Business Analyst.
Share this job
View 4 more jobs at NewtonX