Senior Machine Learning Infrastructure Engineer, Hippocratic AI

• 5+ years of experience in infrastructure, MLOps, or systems engineering, ideally with time spent in architect or staff-level roles
• Proven experience managing large-scale Kubernetes or Slurm clusters for training or serving ML workloads
• Strong proficiency in Python; familiarity with Go or Rust is a plus
• Hands-on experience with Ray, vLLM, Hugging Face Transformers, and/or custom LLM training stacks
• Deep understanding of GPU scheduling, container orchestration, and workload optimization across heterogeneous hardware
• Experience with inference workloads, benchmarking, latency optimization, and cost-performance tradeoffs
• Familiarity with Reinforcement Learning, particularly RLHF frameworks, is a strong plus
• Contributions to internal platforms that enabled others to train or fine-tune LLMs efficiently

Desirable

• We are seeking a Machine Learning Infrastructure Engineer to design, build, and manage the next-generation training and inference platform for LLMs
• You will be at the heart of building scalable, efficient infrastructure that supports our researchers and engineers in training, serving, and experimenting with large models at scale
• Your work will directly impact our ability to innovate with new architectures and training techniques in production environments
• LLM Training Infrastructure: You'll design and operate large-scale training clusters using Kubernetes and/or Slurm for LLM experimentation, fine-tuning, and RLHF workflows
• Cluster & GPU Management: Own scheduling, autoscaling, resource allocation, and monitoring across high-performance GPU clusters (NVIDIA, AMD)
• Distributed Systems: Build and optimize distributed data pipelines using frameworks like Ray, enabling parallel training and inference jobs
• Inference Optimization: Benchmark and optimize model serving performance with technologies like vLLM, and support autoscaling of inference workloads in production environments
• Platform Reliability: Collaborate with infra and platform engineers to ensure system robustness, observability, and maintainability of ML workloads
• Research Enablement: Partner closely with ML researchers to enable rapid experimentation through flexible and efficient infrastructure tooling