Senior Machine Learning Operations Engineer, Fetch

• 5+ years of experience building and operating production ML systems beyond initial deployment, including monitoring, retraining, incident response, and continuous improvement
• Strong production software development skills in Python and proficiency with SQL
• Hands-on experience with Spark/PySpark for large-scale distributed data processing, including feature engineering, historical aggregations, incremental processing, and training dataset construction
• Experience with modern data lake or lakehouse architectures, including S3, Apache Iceberg, or equivalent technologies
• Experience with a workflow orchestration engine for scheduling, dependency management, retries, backfills, and recovery
• Hands-on experience building infrastructure with Terraform
• Experience deploying and operating containerized production workloads on Kubernetes
• Experience building reusable training, evaluation, or deployment infrastructure used by multiple models, scientists, or teams
• Experience implementing model and dataset versioning, experiment tracking, lineage, and reproducible releases
• Ability to distinguish service or infrastructure failures from changes in model effectiveness
• Experience building CI/CD pipelines, monitoring, alerting, rollback mechanisms, and recovery procedures for production ML systems
• Demonstrated ability to balance model quality, delivery speed, reliability, and infrastructure cost
• Strong cross-functional communication and experience partnering with data scientists to preserve scientific validity as models move into production
• Ability to explain a production incident or model degradation issue, how it was detected, and what improvements followed

Desirable

• As a Senior MLOps Engineer, you’ll own the operational foundation that brings ranking and recommendation models from experimentation into reliable production systems
• You’ll partner with Data Science, Product, Platform, and Engineering across the full model lifecycle, including reproducible training, validation, deployment, versioning, monitoring, retraining, and retirement
• This is a hands-on technical leadership role operating at the intersection of ML infrastructure, personalization, and large-scale distributed systems
• You’ll build reusable pipelines, strengthen observability and real-time learning workflows, establish architectural and reliability standards, and make model health and progress accessible through a shared model hub
• Your work will enable Fetch’s Ads Platform to become smarter, faster, and more responsive to changing user behavior over time
• Own the complete model lifecycle from experimentation through validation, deployment, versioning, retraining, rollback, and retirement
• Establish clear model promotion criteria, release processes, and rollback paths that enable teams to ship improvements confidently
• Design and build scalable ML data and feature engineering pipelines, including feature generation, historical aggregations, and training dataset construction, using Spark/PySpark and modern lakehouse technologies such as S3 and Apache Iceberg
• Develop reliable, incremental data processing pipelines with support for data quality validation, lineage, point-in-time correctness, and training-serving feature consistency
• Build monitoring for data quality, drift, prediction quality, service reliability, latency, and infrastructure cost
• Partner with data scientists to define meaningful offline and production evaluation signals, including for use cases with delayed outcome labels
• Use production evidence to improve model performance, platform reliability, deployment speed, and infrastructure efficiency
• Build reusable training, evaluation, and deployment pipelines that support multiple models and modeling approaches
• Develop dependable orchestration workflows covering dependencies, scheduling, retries, backfills, and failure recovery
• Translate scientific requirements into maintainable production systems with clear interfaces, ownership, and operational expectations
• Build and maintain an accessible model hub connecting model documentation, evaluation results, release history, versioning, and monitoring
• Create tested CI/CD workflows, reproducible environments, actionable alerts, incident-response procedures, and operational runbooks
• Define MLOps priorities in partnership with Data Science and Engineering, identifying risks before they become production issues
• Establish reusable engineering standards, mentor teammates, and reduce the operational burden placed on data scientists