Senior AI Runtime Engineer , Modular

• 5+ years of experience working on high-performance computing systems
• Experience in C++ programming and complex software systems
• Experience with CPU or GPU runtime optimizations and performance analysis on CPUs, GPUs, or AI accelerators
• Proficiency with one or more profiling tools (CPU or GPU)
• Creativity and curiosity for solving complex problems, a team-oriented attitude that enables you to work well with others, and alignment with our culture

Desirable

• If you’re passionate about shaping the future of AI and creating tools that make a real difference in people’s lives, we want you on our team
• ML developers today face significant friction when deploying trained models
• They work in a fragmented space with incomplete, patchwork solutions that require extensive performance tuning and model-specific optimizations
• At Modular, we are building the next-generation AI platform that will radically improve how developers build and deploy AI models
• A core part of this offering is a platform that enables customers to achieve state-of-the-art performance across model families and frameworks
• As an AI Runtime Engineer, you will own a runtime that operates on various CPU and GPU hardware platforms, optimizing performance for diverse customer AI models
• Design and develop runtime and cross-stack optimizations to improve CPU and GPU efficiency, addressing issues such as CPU overhead, caching, and data locality across multiple devices
• Port the Modular runtime stack to new hardware platforms and develop an API to streamline this process
• Collaborate with the compiler, kernels, serving, and models teams to design core technologies that achieve state-of-the-art end-to-end performance on various CPU and GPU hardware
• Collaborate with the customer success team and engage with customers to understand their performance requirements and use cases
• Collaborate with tooling and infrastructure teams to design systems for automated performance analysis and benchmarking