Machine Learning Quality Assurance Analyst, OnCorps AI
Agentic AI platform for fund operations
Job no longer available
Agentic AI platform for fund operations
21-100 employees
Job no longer available
21-100 employees
Company mission
To elevate work and performance by applying AI to handle predictable and laborious calculations, decisions, and tasks.
Role
Who you are
- STEM BA/BS
- High attention to detail
- Ability to think critically and analytically
- Ability to follow and understand a complex ruleset
- Familiarity with LLMs/Agents is a plus
- Knowledge of SQL is a plus
- Knowledge of Python is a plus
- Comfortable working in fast-paced environments and meeting tight deadlines
- U.S. based with work authorization
- Primarily desk-based role during standard hours, utilizing PC, video, and phone
- Requires flexibility to work in-office and remotely based on business needs
What the job involves
- This role is central to how our AI systems learn, improve, and perform
- As an ML QA Analyst, you’ll work under the Data Science umbrella and own key tasks across model evaluation, training, and ongoing maintenance for the pipelines and work products built by our DS team
- You’ll validate agent behavior, strengthen prompts, ensure high-quality labeled data, and keep our models running reliably in production
- It’s an ideal fit for someone with a STEM background, strong analytical instincts, and a passion for precision who wants hands-on experience in a cutting-edge AI environment
- This role also offers a clear growth path—successful team members often progress into Data Science roles or transition into Product as Business/Product Analysts
- Labeling: Owning data labeling for PDF / Excel-based ML ingestion engines across the company. This includes both training-based labeling and live-run corrective labeling
- Agent Evaluation Review: Review evaluation jobs where agents are underperforming, digging into individual logs and traces and providing documentation of possible prompt weaknesses
- Agent Regression Testing: Interacting with customer facing “chat” agents. Validating the integrity of an agent’s response to both “happy-path” and adversarial prompts pre-deployment & when model updates are in testing
- Dashboarding: Owning client-specific dashboard deployments. Working with product/CS stakeholders to deploy client-specific visualizations. Testing dashboards for errors / logical gaps
- Maintenance-Mode Model Upkeep: Initiating and monitoring routine model retraining for models in a steady-state & fixing job failures in production. Working with Data Science to determine appropriate thresholds for escalation
- AI Table Annotation
Our take
Perfect oversight on every financial transaction is beyond human capability, particularly as regulatory demands and transaction volumes continue to grow. OnCorps AI tackles this with an agentic AI platform that investigates discrepancies, recommends resolutions and learns from every exception, reducing manual labour and false positives by up to 90%.
Those capabilities have helped OnCorps with the trust of leading asset managers and fund administrators. Trained on millions of real-world financial exceptions, its AI delivers the specialist expertise needed to streamline reconciliations, reporting reviews and other operational workflows with greater speed and accuracy.
The company's momentum was also reinforced in late 2025 with a $55M growth investment from Long Ridge Equity Partners. The funding will support product innovation, business expansion and the rollout of its AI platform into adjacent capital markets workflows, putting OnCorps in a very good spot for its next phase of growth.
Steph
Company Specialist at Welcome to the Jungle
Company
Funding (last 2 of 6 rounds)
Dec 2025
$55m
GROWTH EQUITY VC
Mar 2022
$9.6m
SERIES B
Total funding: $72.1m
Company HQ
Prudential / St. Botolph, Boston, MA
Leadership
Bob Suh
(Founder & Chair)Former CTO at Accenture. Also worked as the President of Dell and as a Managing Director at DXC Technology.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 4 more jobs at OnCorps AI