Machine Learning Quality Assurance Analyst, OnCorps AI

Salary not provided

SQL

Python

Junior and Mid level

Boston

AI agents for financial operations

Open for applications

AI agents for financial operations

21-100 employees

FintechB2BArtificial IntelligenceAnalyticsMachine LearningSaaSData Integration

Open for applications

Salary not provided

SQL

Python

Junior and Mid level

Boston

21-100 employees

FintechB2BArtificial IntelligenceAnalyticsMachine LearningSaaSData Integration

Company mission

To elevate work and performance by applying AI to handle predictable and laborious calculations, decisions, and tasks.

Job

Company

Role

Who you are

STEM BA/BS
High attention to detail
Ability to think critically and analytically
Ability to follow and understand a complex ruleset
Familiarity with LLMs/Agents is a plus
Knowledge of SQL is a plus
Knowledge of Python is a plus
Comfortable working in fast-paced environments and meeting tight deadlines
U.S. based with work authorization
Primarily desk-based role during standard hours, utilizing PC, video, and phone
Requires flexibility to work in-office and remotely based on business needs

What the job involves

This role is central to how our AI systems learn, improve, and perform
As an ML QA Analyst, you’ll work under the Data Science umbrella and own key tasks across model evaluation, training, and ongoing maintenance for the pipelines and work products built by our DS team
You’ll validate agent behavior, strengthen prompts, ensure high-quality labeled data, and keep our models running reliably in production
It’s an ideal fit for someone with a STEM background, strong analytical instincts, and a passion for precision who wants hands-on experience in a cutting-edge AI environment
This role also offers a clear growth path—successful team members often progress into Data Science roles or transition into Product as Business/Product Analysts
Labeling: Owning data labeling for PDF / Excel-based ML ingestion engines across the company. This includes both training-based labeling and live-run corrective labeling
Agent Evaluation Review: Review evaluation jobs where agents are underperforming, digging into individual logs and traces and providing documentation of possible prompt weaknesses
Agent Regression Testing: Interacting with customer facing “chat” agents. Validating the integrity of an agent’s response to both “happy-path” and adversarial prompts pre-deployment & when model updates are in testing
Dashboarding: Owning client-specific dashboard deployments. Working with product/CS stakeholders to deploy client-specific visualizations. Testing dashboards for errors / logical gaps
Maintenance-Mode Model Upkeep: Initiating and monitoring routine model retraining for models in a steady-state & fixing job failures in production. Working with Data Science to determine appropriate thresholds for escalation
AI Table Annotation

Our take

In financial operations, to get perfect oversight on every transaction is unattainable by people. So Oncorps AI supplies an AI-powered platform with the capability to do just this, while also meeting the regulatory requirements of major financial institutions.

The company's agentic solutions are capable of reducing labour and false positives by an impressive 90%, which has enabled Oncorps AI to attract many high-profile clients. The company continues to invest in AI research that will further help reduce costs and minimise financial-related risks.

Steph

Company Specialist at Welcome to the Jungle

Company

Funding (last 2 of 6 rounds)

Dec 2025

$55m

GROWTH EQUITY VC

Mar 2022

$9.6m

SERIES B

Total funding: $72.1m

Company HQ

Prudential / St. Botolph, Boston, MA

Leadership

Bob Suh

(Founder & Chair)

Former CTO at Accenture. Also worked as the President of Dell and as a Managing Director at DXC Technology.

Salary benchmarks

We don't have enough data yet to provide salary benchmarks for this role.

Submit your salary to help other candidates with crowdsourced salary estimates.

Share this job

View 2 more jobs at OnCorps AI