Machine Learning Engineer, Lilt

• BS or MS in Computer Science or a related field, or equivalent practical experience
• 3+ years building production backend or ML serving systems in Python, including strong async programming (asyncio) skills
• Hands-on experience with real-time streaming transport: WebSocket or gRPC bidirectional streaming, session state, backpressure, and connection lifecycle handling
• Experience serving ML models in production on GPUs (Ray Serve, Triton, vLLM, or similar), with Docker and Kubernetes
• Experience integrating speech or NLP models into production systems, ideally streaming ASR (partial hypotheses, endpointing, VAD)
• A latency-engineering mindset: you have profiled, instrumented, and optimized a real-time or low-latency system and can reason in per-stage budgets
• Effective use of AI coding agents (Claude Code, Codex, or similar) on top of fundamentals learned the hard way: you let agents do the typing, but you can debug, review, and reason about every line without them, and you know when not to trust them
• US citizenship and residence in the United States (contract requirement)

Desirable

• We are building a new live translation product
• We are looking for an ML Engineer to build the real-time speech translation backend that powers it
• You will own the real-time speech translation backend end-to-end, from live audio input to translated output
• You will build on LILT's production model serving platform (Ray Serve on GPU Kubernetes clusters) and our in-house adaptive machine translation models, working closely with the senior architects of that platform and with our language processing researchers
• The ASR and MT models exist
• Your job is to make them work together as a low-latency streaming system that holds up in production
• This is a hands-on backend engineering role for those looking to own a real-time ML system from end to end, supported by expert guidance and a clear product vision
• Our team adopts an AI-first approach, leveraging agentic coding and AI-driven PR reviews to accelerate development
• We combine this with deep technical expertise, requiring not only expert Python proficiency but also a comprehensive understanding of the entire ML stack, from optimizing neural network architectures to managing production infrastructure on Kubernetes, and making informed, cost-aware decisions on hardware selection
• Real-time pipeline architecture: Build and manage services for high-throughput, real-time audio and text streaming. Handle signal processing, session lifecycles, and concurrency management to ensure robust operation under load
• ML model integration: Integrate and serve streaming speech recognition and machine translation models, collaborating with research teams to ensure models operate within required latency budgets
• Quality and confidence workflows: Develop logic for model-based confidence scoring, routing segments for human intervention as needed, and broadcasting real-time updates and corrections to end-users
• Infrastructure and scale: Architect and scale production ML infrastructure on GPU-accelerated Kubernetes clusters. Implement batching, load balancing, and autoscaling strategies to maintain performance and cost-efficiency
• Latency engineering: Establish comprehensive instrumentation for real-time performance. Identify bottlenecks, optimize system throughput, and drive down end-to-end latency metrics to meet production standards
• Interface and API definition: Define technical contracts and interfaces for audio ingestion and downstream service integrations. Partner with frontend and platform engineering teams to maintain clean, robust integration points
• Collaboration and technical leadership: Drive cross-team alignment by defining clear API interfaces and technical contracts, facilitating effective communication between engineering and product teams to ensure seamless system integration