Software Engineer, Mirage

• You’re passionate about building foundational systems that enable machine learning at scale — and love solving complex distributed data problems

Desirable

• We’re looking for a Software Engineer to help build and scale the data systems that power our machine learning products
• This role sits at the intersection of data engineering and ML infrastructure: you’ll design large-scale streaming pipelines, build tools that abstract infrastructure complexity for feature developers, and ensure that our feature data is reliable, discoverable, and performant across online and offline environments
• Design and scale feature pipelines: You'll build distributed data processing systems for feature extraction, orchestration, and serving — including real-time streaming, batch ingestion, and CDC workflows
• Feature Extraction: Design and implement reliable, reusable feature pipelines for ML models, ensuring features are accurate, scalable, and production-ready through well-designed SDKs and orchestration tools
• Build and evolve storage infrastructure: Manage multi-tier data systems (e.g. Bigtable for online features/state, BigQuery for analytics and offline training), including schema evolution, versioning, and compatibility
• Own orchestration and reliability: Lead workflow orchestration design (e.g. Pub/Sub, Airflow, Temporal), monitoring, and alerting to ensure reliability at 100M+ video scale
• Collaborate with ML teams: Partner with ML engineers on feature availability, dataset curation, and streaming pipelines for training and inference
• Optimize for performance and cost: Tune GPU utilization, resource allocation, and data processing efficiency to maximize system throughput and minimize cost
• Enable analytics and insights: Support downstream analytics and data science workflows by ensuring data accessibility, discoverability, and performance at scale