AI Runtime Engineer, Modular

• 2+ years of experience working on high-performance computing systems
• Experience in C++ programming and complex software systems
• Experience with CPU or GPU runtime optimizations and performance analysis on CPUs, GPUs, or AI accelerators
• Proficiency with one or more profiling tools (CPU or GPU)
• Creativity and curiosity for solving complex problems, a team-oriented attitude that enables you to work well with others, and alignment with our culture

Desirable

• ML developers today face significant friction when deploying trained models
• They work in a fragmented space with incomplete, patchwork solutions that require extensive performance tuning and model-specific optimizations
• At Modular, we are building the next-generation AI platform that will radically improve how developers build and deploy AI models
• A core part of this offering is a platform that enables customers to achieve state-of-the-art performance across model families and frameworks
• As an AI Runtime Engineer, you will own a runtime that operates on various CPU, GPU, and accelerator hardware platforms, optimizing performance for diverse customer AI models
• Design and develop runtime and cross-stack optimizations to improve CPU, GPU, and accelerator efficiency, addressing issues such as CPU overhead, caching, and data locality across multiple devices
• Work with vendor-specific networking libraries to unlock high performance data transfer for multiple topologies
• Collaborate with the compiler, kernels, serving, and models teams to design core technologies that achieve state-of-the-art end-to-end performance on various CPU and GPU hardware
• Collaborate with the customer success team and engage with customers to understand their performance requirements and use cases
• Collaborate with tooling and infrastructure teams to design systems for automated performance analysis and benchmarking