Senior Machine Learning Engineer, Workday

• We’re looking for a seasoned ML Engineer with a strong passion for LLMs and a proven track record of designing and executing complex experiments
• You thrive in a fast-paced environment and are eager to contribute to the advancement of LLM technology
• Master's or Ph.D. in Computer Science, Machine Learning, or equivalent experience
• 5+ years of experience in machine learning engineering, with a focus on natural language processing (NLP)
• 1+ year experience with LLM architectures, training methodologies, and evaluation metrics
• 3+ years’ experience in Python and experience with relevant machine learning libraries (e.g., TensorFlow, PyTorch, scikit-learn)
• Experience with cloud computing platforms (e.g., Google Cloud Platform, AWS, etc)
• 2+ years’ experience with version control systems such as git

Desirable

• You'll be joining the Machine Learning Serving Infrastructure team, the team responsible for building and maintaining Workday's LLM Gateway
• This critical infrastructure powers the seamless integration of large language models across Workday's products and services
• Our team is at the forefront of LLM deployment, ensuring scalability, reliability, and performance for our cutting-edge AI capabilities
• We work with the latest advancements in LLM technology, and this role will be instrumental in evaluating and optimizing the models that drive our platform
• You'll be part of a collaborative and innovative environment, working alongside experienced engineers who are passionate about pushing the boundaries of what's possible with LLMs in enterprise applications
• Your contributions will directly impact the efficiency and effectiveness of our LLM Gateway, ensuring Workday remains a leader in AI-powered solutions
• We're seeking a highly skilled and motivated Senior Machine Learning Engineer to join a team and drive ground breaking experimentation with Large Language Models (LLMs)
• In this role, you'll be the primary architect and executor of experiments designed to rigorously compare and evaluate the performance of various LLMs, particularly focusing on the impact of model updates on our feature set
• You'll be responsible for crafting robust experimental frameworks, implementing automated evaluation pipelines, analyzing sophisticated datasets, and communicating your findings to both technical and non-technical collaborators
• Design and implement comprehensive experimental frameworks for comparing LLM performance across defined features
• Develop automated pipelines for data collection, model inference, and performance evaluation
• Conduct thorough statistical analysis of experimental results to identify significant performance differences
• Build detailed reports and presentations summarizing experimental findings and providing actionable insights
• Collaborate with other ML engineers and product teams to integrate experimental results into product development
• Stay up-to-date with the latest advancements in LLM research and evaluation methodologies
• Optimize and maintain existing evaluation tools and infrastructure
• Define and track key performance indicators (KPIs) for LLM performance
• Document all experiments, results, and methodologies for reproducibility

• End Date: May 2, 2025