Software Engineer, Modular

• 5+ years of experience working in backend engineering
• Experience working on high scale ML inference infrastructure (traditional AI or genAI)
• Experience with kubernetes and operating your own services
• Ability to create durable, reusable software tools and libraries that are leveraged across teams and functions
• Creativity and curiosity for solving complex problems, a team-oriented attitude that enables you to work well with others, and alignment with our culture
• Strongly identifies with our core company cultural values
• For candidates who fall outside of the listed requirements, we nevertheless encourage you to apply as we may have openings that are lower/higher level than the ones advertised

Desirable

• Our goal is to make inference both the fastest and most scalable and making those systems repeatable to new model architectures
• We're seeking engineers who are passionate about pushing the boundaries of distributed inference systems and enjoy working at the intersection of large-scale systems and machine learning
• We are looking for candidates based on their breadth and depth of experience in backend engineering, AI inference, and distributed systems development
• If this sounds exciting, we invite you to join our world-leading AI infrastructure team and help drive our industry forward!
• Build & ship Modular’s LLM focused inference services using best-in-class inference techniques (eg disaggregated inference, multi-node deployment of large models, high performance networking, high throughput batch processing, etc)
• Build the distributed systems needed to support high performance inference (eg distributed kv-cache, expert parallel request routing & rebalancing, etc)
• Push the envelope for operational excellence with request-to-kernel observability, multi-cloud deployments, clever autoscaling, cold-start optimizations, and more
• Collaborate with our kernels and genAI teams to achieve SOTA application performance by integrating SOTA kernel & serving optimizations with SOTA cluster optimizations
• Build helm charts, kubernetes operators, and more to make a create simple, effective, maintainable deployments