Data Scientist, Character.ai

• Our ideal candidate combines deep technical expertise in search and recommendations with practical experience in LLM systems, bringing a data-driven approach to improving user experiences through better content discovery and model performance
• Have 5+ years of experience at chat, social media, or consumer product companies, with substantial focus on ML frameworks
• Have deep expertise in testing, optimizing and measuring Large Language Models, search ranking algorithms, recommendation systems
• Able to design, drive and execute leaning into high velocity experimentation across consumer facing features
• Have a strong intuition for choosing the right questions to ask and quickly driving to key insights, particularly in evaluation of LLM, search and recommendation models and user engagement metrics
• Have extensive history of taking ownership and driving impact through data
• Advanced degree in Computer Science, Statistics, Mathematics, Engineering or related field
• Strong programming skills in SQL, Python and experience with ML frameworks
• Strong background in retrieval, ranking algorithms, and relevance metrics
• Experience with modern search technologies (e.g., Elasticsearch, vector search) and recommendation frameworks
• Expertise in A/B testing and causal inference
• Excellent communication skills and ability to collaborate cross-functionally

Desirable

• We are seeking an experienced Staff Data Scientist with deep expertise in experimentation and ML frameworks (specifically search systems and recommendation engines) to join our dynamic team
• Domain knowledge with large language models (LLMs) and post-training optimization is a key priority as we expand our AI capabilities
• Design and implement search retrieval and ranking algorithms that surface the most relevant content to users
• Develop and optimize recommendation systems for personalized content discovery
• Support post-training optimization of LLMs through techniques like RLHF and DPO
• Create comprehensive offline evaluation frameworks to measure LLM performance, search quality, and recommendation relevance
• Design and execute online A/B testing initiatives to validate improvements across search, recommendations and LLM systems
• Drive continuous improvement in model behavior through quantitative analysis and user feedback
• Collaborate with engineering teams to implement and productionize algorithms