Senior Data Scientist, Roblox
Generative AI
$217-258.5k
+ Equity
More information about location
3+ days a week in office (San Mateo, CA)
Online gaming and entertainment platform
Posted today
Online gaming and entertainment platform
1001+ employees
Posted today
$217-258.5k
+ Equity
More information about location
3+ days a week in office (San Mateo, CA)
1001+ employees
Company mission
To connect a billion people with optimism and civility.
Role
Who you are
- An advanced Degree and/or PhD in Statistics, Economics, Operations Research, Computer Science, Applied Math, Physics, Engineering, or another quantitative field
- 5+ years of experience in data science or a related field
- Familiarity with GenAI models and GenAI evaluation methods
- Passion for the GenAI field and enthusiasm for continuously improving methods and practices to drive product quality and business impact
- Ability to effectively use AI tools to enhance productivity in research, ideation, coding, and documentation
- Strong learning agility, experience conducting applied research or writing technical papers is a plus
- Proficiency in SQL, Hive, or Spark for transforming and manipulating large datasets
- Experience with scripting languages such as Python or R
- A demonstrated track record of solving open-ended data science and modeling problems that drive business impact and improve user experience
What the job involves
- As a Data Scientist on the team, you will design, build, and operationalize evaluation for GenAI systems, and work with cross-functional teams to improve model performance and the AI data generation flow
- Since AI evaluation is core to GenAI safety, quality, and iteration speed, we are building rigorous and scalable human and model-based evaluation systems that guide product decisions and model improvement
- You’ll combine annotation analysis, design of experiments, causal inference, product analytics, and model-based evaluation methods (such as LLM-as-a-judge / VLM-as-a-judge) to measure quality, safety, and user satisfaction—and translate these findings into model and product improvements
- You’ll also help develop groundbreaking methodologies and tools that advance AI evaluation at Roblox and set industry standards
- Beyond AI evaluation, we proactively explore opportunities and solutions to improve the AI model and data generation flow
- Additionally, we will build agentic workflows and AI agents for data solutions that enable teams to effectively access data, extract data insights, follow best practices, and make data-informed decisions
- If you are a self-starter who is curious, rigorous, and passionate about building innovative solutions that deliver real business value—and thrive in a dynamic, collaborative environment—this role is for you
- Develop and improve evaluation frameworks for GenAI features (text, image, 3D, 4D, agentic workflow), including eval experiment design, eval dataset design, label reliability analysis, results analysis, and online evaluation based on user behavior and feedback
- Establish best practices and guidelines for GenAI evaluation
- Conduct product analytics, online experiments (A/B tests) and causal analyses to quantify GenAI feature impact and identify opportunities
- Build automated evaluation systems, such as research and implement LLM-as-judge and VLM-as-judge methods
- Research and apply state-of-the-art methodologies in GenAI evaluation
- Advance reproducible evaluation tooling to lift evaluation rigor and efficiency at the company
- Proactively explore and develop solutions to improve the AI model and data generation flow, ensuring high-quality input for training and deployment
- Design and implement agentic workflows and AI agents to enable teams to effectively access data, extract data insights, and follow best data practices
- Partner closely with cross-functional teams to align goals, plans, and execution
Our take
Online gaming may be a well-established space, but Roblox has distinguished itself as a leader by empowering its community to build, share, and profit from their own creations, turning game development into an accessible, rewarding ecosystem for millions of users.
In addition to standard games, the platform offers virtual worlds and other experiences to explore and interact with. Its platform provides an online ecosystem for millions to create, socialize, chat, and explore immersive experiences with friends. In fact, in the US its used by two-thirds of children aged 9 to 12.
While the platform does face scrutiny over users safety (particularly given its young audience) the company's efforts to strengthen parental controls and open dialogue through initiatives like its new Parents Council signal a growing commitment to addressing these challenges. At the same time, its financial performance and creator-focused investments underscore a business in very strong health. With record payouts to developers, an expanding older player base, and a platform that increasingly rewards creative talent, Roblox's trajectory remains one of innovation and impressive growth.
Freddie
Company Specialist at Welcome to the Jungle
Insights
25% employee growth in 12 months
Company
Funding (last 2 of 9 rounds)
Jan 2021
$520m
SERIES H
Feb 2020
$150m
SERIES G
Total funding: $922.8m
Company HQ
South San Mateo, San Mateo, CA
Leadership
David Baszucki
(Founder & CEO)Studied Electrical Engineering at Stanford before founding Knowledge Revolution in 1990, where they spent 9 years as CEO. They worked as VP of MSC.Software for 2 years, and started Roblox in 2005.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 127 more jobs at Roblox