Senior Manager of Machine Learning, Coalition
Data Operations
$134.4-200k
Cyber insurance and security for businesses
Open for applications
Cyber insurance and security for businesses
501-1000 employees
Open for applications
$134.4-200k
501-1000 employees
Company mission
To protect the unprotected from technology failures and cyber-attacks, helping manage risk online so they can focus on growth.
Role
Who you are
- 5+ years in ML data operations, data labeling, or a related field (ML engineering, data science, or data engineering with heavy labeling exposure)
- Deep understanding of annotation quality frameworks: IAA, consensus labeling, gold standard evaluation, error taxonomy, and calibration workflows
- Direct experience managing labeling platforms (Label Studio strongly preferred; Scale AI, Labelbox, Prodigy, or similar acceptable)
- Track record managing outsourced labeling vendors or BPOs for ML data production
- Familiarity with common ML labeling tasks: text classification, NER, document extraction, intent detection
- Comfortable working in Python and SQL; bonus if you've built tooling around labeling workflows or quality measurement
- Strong opinions on what makes labeled data good or bad, and the willingness to push back when it's bad
- Experience in insurance, cybersecurity, or fintech is a plus but not required
What the job involves
- Coalition's machine learning models are only as good as the data they're trained on
- This role exists to make sure that data is right
- You'll own labeling quality and methodology across Coalition — designing annotation tasks, defining quality frameworks, and ensuring every labeled dataset meets the standard required to ship production ML models
- You'll manage our labeling platform (Label Studio), work directly with ML and product teams to structure labeling programs, and oversee outsourced labeling vendors to hit quality and throughput targets
- This role reports to the Chief Product Officer and sits at the intersection of product, ML, and operations
- You won't manage internal labelers — all annotation work is outsourced — but you will be the single point of accountability for whether Coalition's labeled data is accurate, consistent, and fit for purpose
- Labeling quality & methodology: Define annotation guidelines, taxonomies, and edge-case protocols for each labeling program. Establish gold standard datasets, inter-annotator agreement (IAA) targets, and audit sampling processes. Identify and remediate mislabeled data in existing datasets
- Platform & tooling: Serve as the primary user and requirements driver for Label Studio — defining project configuration needs, workflow designs, pre-labeling pipeline requirements, and integration points with ML infrastructure. Partner with the data engineering team that builds and maintains the platform
- Cross-functional partnership: Work with ML engineers, data scientists, and product managers to translate model requirements into well-structured labeling tasks. Challenge teams on task design when labeling instructions are ambiguous or likely to produce unreliable labels
- Vendor management: Source, onboard, and manage external labeling vendors and BPOs in coordination with Coalition's operations team. Set quality SLAs, run calibration sessions, and manage feedback loops to labelers. Hold vendors accountable to accuracy, not just throughput
- Measurement & improvement: Define and track operational metrics — label accuracy, IAA scores, cost per label, turnaround time — and use them to drive continuous improvement. Identify opportunities for active learning, model-assisted labeling, and pre-annotation to reduce cost without sacrificing quality
Our take
The cybersecurity industry has been going through a record-breaking period of investment. This trend has emerged as more companies move to cloud-based networks following the increase in out-of-office work, thus increasing the risk of cyber attacks. However, many cutting-edge cyber defences are too expensive for many companies.
Coalition’s joint offering of cybersecurity tools, a risk management platform, and cyber insurance allows companies to manage the ever-present risk of cyber threats.
The company has made a strong start since its 2017 launch, raising more than $800 million in funding. There are several other fast-growth startups in the cybersecurity space, including Wiz and Orca Security, but Coalition’s shrewd insurance-based model is what sets it apart from the competition. The company has set out on a path toward global expansion, branching out from the US and Canadian markets to bring its solutions to the UK, whilst also acquiring smaller competitors like JUMBO.
Steph
Company Specialist at Welcome to the Jungle
Insights
100% employee growth in 12 months
Company
Funding (last 2 of 6 rounds)
Jul 2022
$250m
SERIES F
Sep 2021
$205m
SERIES E
Total funding: $770m
Company benefits
- Paid parental leave
- 401k plan
- Stock option plan
- Open vacation days
- Flexible working hours
- Work from home opportunities
- Health insurance
Company HQ
Financial District, San Francisco, CA
Leadership
Joshua Motta
(Co-Founder & CEO)After studying at the University of Chicago, Joshua worked at Goldman Sachs for a year before moving to Private Equity at Francisco Partners, and working as an IGNIA Partners Investor. He left his role at Cloudflare to co-found Coalition in March 2017.
Share this job
View 18 more jobs at Coalition