Kernel Engineer, Modular
$135-204k
+ Stock options
2+ days a week in office (Los Altos, CA)
Fast, scalable Gen AI inference platform
Open for applications
Fast, scalable Gen AI inference platform
201-500 employees
Open for applications
$135-204k
+ Stock options
2+ days a week in office (Los Altos, CA)
201-500 employees
Company mission
To have real, positive impact in the world by reinventing the way AI technology is developed and deployed into production with a next-generation developer platform.
Role
Who you are
- Experience in GPU programming and parallel computing
- Solid understanding of CPU/GPU memory hierarchy and performance modeling
- Proficiency in Python or C++
- Familiarity with assembly language
- Hands-on experience with GPU profiling and debugging tools
- Strong foundation in algorithms and data structures
- Deep interest in machine learning technologies and use cases
- Creativity and curiosity for solving complex problems, a team-oriented attitude that enables you to work well with others, and alignment with our culture
Desirable
- Experience with ML framework kernel development (Triton, PyTorch, TensorFlow XLA) and libraries (cuBLAS, CUTE etc.)
- Knowledge of GPU ISA-level optimization (PTX, SASS, GCN)
- Experience with multi-GPU communication frameworks (NCCL, RCCL, MPI)
- Contributions to open-source GPU or HPC communities
- Knowledge of AI algorithms and state-of-the-art LLMs
What the job involves
- ML developers today face significant friction in taking trained models into deployment
- They work in a highly fragmented space, with incomplete and patchwork solutions that require significant performance tuning and non-generalizable/ model-specific enhancements
- At Modular, we are building the next generation AI platform that will radically improve the way developers build and deploy AI models
- As an AI Kernel Engineer, you'll be instrumental in building our core libraries that enable developers to leverage deployment-specific optimizations across diverse model families and frameworks
- You will be crafting high-performance kernels for GPUs and emerging hardware architectures
- You'll drive critical performance improvements through kernel optimizations that:
- Accelerate model latency and maximize throughput
- Minimize communication overhead and optimize resource utilization
- Streamline memory usage through reduced activation volumes
- Enhance data pre- and post-processing pipelines
- Deliver measurable end-to-end performance gains across our model ecosystem
- This role puts you at the cutting edge of AI infrastructure, where your work directly impacts the speed and efficiency of AI applications at scale
- Design and optimize high-performance ML numeric and data manipulation kernels
- Benchmark kernel performance and conduct performance analysis
- Diagnose performance bottlenecks and implement kernel improvements
- Collaborate with framework and compiler teams to deliver end-to-end AI performance that fully utilizes new hardware or GPU capabilities
- Ensure kernel correctness, maintainability, and test coverage
Our take
AI development is booming, but the tools behind it? Not so much. Developers are often stuck dealing with messy, disconnected systems that slow everything down and bump up costs.
To address this, Modular has developed a unified AI platform that streamlines the development process. It's AI Engine enhances the performance of models on CPUs and GPUs, supporting popular frameworks like TensorFlow and PyTorch. And on top of that, it's built Mojo, a programming language that's as easy as Python but much faster (35,000 times faster).
In 2023, the company raised an impressive $100M to power its mission. It's been using this funding to grow its team, support more hardware, and push Mojo even further. With the hopes of making AI development faster, smoother, and a whole lot more fun for everyone.
Kirsty
Company Specialist at Welcome to the Jungle
Company
Funding (last 2 of 3 rounds)
Sep 2025
$250m
LATE VC
Aug 2023
$100m
LATE VC
Total funding: $380m
Company benefits
- A variety of fantastic health benefits (health, dental, vision insurance; life insurance etc) are available
- A 401k plan with up to 5% match
- Free tax advice on Carta
- Generous work-from-home stipend of $1500 to help you improve your home office
- Unlimited paid time off and flexible work hours
Leadership
Chris Lattner
(Co-Founder & CEO)Founder, Architect, Engineer and BDFL for the LLVM Organization. Former President of Engineering and Product at SiFive, as well as working in various roles at Google, Tesla, and Apple.
Tim Davis
(Co-Founder & President)Worked at Google for 5 years, ultimately as Group Product Lead for Google ML. Previously founded Fluc Inc, where they developed the entire ML powered delivery system with their co-founder, and mapping software system CrowdSend.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 14 more jobs at Modular