Machine Learning Quality Assurance Analyst, OnCorps

Salary not provided

SQL

Python

Junior and Mid level

Boston

ML company integrating decision and behavioral science

Be an early applicant

ML company integrating decision and behavioral science

21-100 employees

B2BArtificial IntelligenceAnalyticsMachine LearningSaaSData Integration

Be an early applicant

Salary not provided

SQL

Python

Junior and Mid level

Boston

21-100 employees

B2BArtificial IntelligenceAnalyticsMachine LearningSaaSData Integration

Company mission

To elevate work and performance by applying AI to handle predictable and laborious calculations, decisions, and tasks.

Job

Company

Role

Who you are

STEM BA/BS
High attention to detail
Ability to think critically and analytically
Ability to follow and understand a complex ruleset
Familiarity with LLMs/Agents is a plus
Knowledge of SQL is a plus
Knowledge of Python is a plus
Comfortable working in fast-paced environments and meeting tight deadlines
U.S. based with work authorization
Primarily desk-based role during standard hours, utilizing PC, video, and phone
Requires flexibility to work in-office and remotely based on business needs

What the job involves

This role is central to how our AI systems learn, improve, and perform
As an ML QA Analyst, you’ll work under the Data Science umbrella and own key tasks across model evaluation, training, and ongoing maintenance for the pipelines and work products built by our DS team
You’ll validate agent behavior, strengthen prompts, ensure high-quality labeled data, and keep our models running reliably in production
It’s an ideal fit for someone with a STEM background, strong analytical instincts, and a passion for precision who wants hands-on experience in a cutting-edge AI environment
This role also offers a clear growth path—successful team members often progress into Data Science roles or transition into Product as Business/Product Analysts
Labeling: Owning data labeling for PDF / Excel-based ML ingestion engines across the company. This includes both training-based labeling and live-run corrective labeling
Agent Evaluation Review: Review evaluation jobs where agents are underperforming, digging into individual logs and traces and providing documentation of possible prompt weaknesses
Agent Regression Testing: Interacting with customer facing “chat” agents. Validating the integrity of an agent’s response to both “happy-path” and adversarial prompts pre-deployment & when model updates are in testing
Dashboarding: Owning client-specific dashboard deployments. Working with product/CS stakeholders to deploy client-specific visualizations. Testing dashboards for errors / logical gaps
Maintenance-Mode Model Upkeep: Initiating and monitoring routine model retraining for models in a steady-state & fixing job failures in production. Working with Data Science to determine appropriate thresholds for escalation
AI Table Annotation

Our take

OnCorps.ai is a machine learning company that integrates decision and behavioural science into its platform to help businesses optimise their operations and drive better outcomes by providing better data-driven decisions that are grounded in a deep understanding of human behaviour and decision-making.

To date, the company's algorithms have reduced labour costs by over 90% and have identified millions of errors. Due to the success of its services, OnCorps.ai has attracted a number of clients with over $9 trillion in assets under management.

With funding raised from several investment organisations, including Guggenheim Investments, OnCorps.ai is utilising the capital gained to broaden the use of AI algorithms across the financial services sector. It's investing in new AI research to continue reducing costs and minimising financial-related risks, while continuing to support the world's leading asset managers.

Steph

Company Specialist at Welcome to the Jungle

Company

Funding (last 2 of 6 rounds)

Dec 2025

$55m

GROWTH EQUITY VC

Mar 2022

$9.6m

SERIES B

Total funding: $72.1m

Company HQ

Prudential / St. Botolph, Boston, MA

Leadership

Bob Suh

(Founder & CEO)

Former CTO at Accenture. Also worked as the President of Dell and as a Managing Director at DXC Technology.

Salary benchmarks

We don't have enough data yet to provide salary benchmarks for this role.

Submit your salary to help other candidates with crowdsourced salary estimates.

Share this job

View 2 more jobs at OnCorps