Software Engineer, Modular

• If you’re passionate about shaping the future of AI and creating tools that make a real difference in people’s lives, we want you on our team
• We are looking for candidates based on their breadth and depth of experience in backend engineering, AI inference, and distributed systems development
• If this sounds exciting, we invite you to join our world-leading AI infrastructure team and help drive our industry forward!
• 5+ years of experience working in backend engineering
• Experience working on high scale ML inference infrastructure (traditional AI or genAI)
• Experience with kubernetes
• Familiarity with HuggingFace API and workflows for using community models
• Ability to create durable, reusable software tools and libraries that are leveraged across teams and functions
• Experience in machine learning technologies and use cases
• Creativity and curiosity for solving complex problems, a team-oriented attitude that enables you to work well with others, and alignment with our culture
• Strongly identifies with our core company cultural values

Desirable

• In the Cloud Inference team, we are focused on building end to end distributed LLM inference deployments that are fully vertically integrated with the MAX stack
• Work with Product and partner engineering teams to design and ship new inference server features
• Collaborate with our kernels and genAI teams to achieve SOTA performance at the serving layer
• Help design and develop helm charts and cloud services for scaled LLM inference (intelligent routing, distributed kvcache management, disaggregated inference, etc.)