AI Data Operations Manager, HackerRank

• Understand the importance of operational excellence, strong organizational skills, and the ability to drive multiple stakeholders who are all going to be busy and distracted
• Can handle ambiguity and drive clarity
• Can strike the right balance between customer requests and internal requests
• Are able to quickly identify and troubleshoot bottlenecks in complex and human-driven processes at scale
• 4+ years in data operations, program management, or content operations for ML, NLP, and AI projects
• Hands-on experience shipping labeled datasets at scale and running quality-control processes, determining the best balance between building and buying tools to scale
• Working knowledge of ML/LLM data formats and annotation platforms (e.g., Labelbox, Scale, custom tools)
• Strong project-management skills and proficiency with SQL or Python/pandas for metric tracking
• Excellent written and verbal communication; able to work with engineers, researchers, and customers

Desirable

• HackerRank’s Machine Learning team is working on the cutting edge of AI
• We’re actively researching and building solutions for a number of exciting initiatives, including plagiarism detection within our Integrity Workstream, how LLMs perform across the SDLC via our ASTRA evaluation harness and benchmark, the types of datasets that can further improve LLMs on software engineering tasks, and finally how far we can take software development agents with our rich datasets
• The AI Data Operations Manager owns the end-to-end pipeline that turns raw content into clean, well-labeled datasets for training and evaluating machine-learning models and LLMs
• You’ll gather data requirements from ML researchers and product teams, work with our Content team to coordinate a global network of subject-matter-expert (SME) labelers, and deliver production-ready datasets to internal stakeholders and customers
• Requirement gathering – Translate model or product needs into clear data specs, timelines, and success metrics
• Labeler & vendor management – Source, onboard, and coach SME labelers or third-party vendors; track throughput and cost
• Process & quality control – Design scalable workflows, run sampling audits, and drive continual quality improvements (e.g., inter-annotator agreement targets)
• Data delivery & documentation – Package datasets with schemas, metadata, and usage guidelines; ensure security and licensing compliance