Machine Learning Intern, d-Matrix
Dynamic KV-Cache Modeling for Efficient LLM Inference
$30 – $59 per hour
Office located in Santa Clara, CA
AI computing and efficiency software
Be an early applicant
AI computing and efficiency software
201-500 employees
Be an early applicant
$30 – $59 per hour
Office located in Santa Clara, CA
201-500 employees
Company mission
To unleash the potential of generative AI to power the transformation of technology.
Role
Who you are
- Currently pursuing a degree in Computer Science, Electrical Engineering, Machine Learning, or a related field
- Familiarity with PyTorch and deep learning concepts, particularly regarding model optimization and memory management
- Understanding of CUDA programming and hardware-accelerated computation (experience with CUDA is a plus)
- Strong programming skills in Python, with experience in PyTorch
- Analytical mindset with the ability to approach problems creatively
- This role is ideal for a self-motivated intern interested in applying advanced memory management techniques in the context of large-scale machine learning inference
- If you’re passionate about optimizing machine learning models and are excited to explore cutting-edge solutions in model inference, we encourage you to apply
Desirable
- Experience with deep learning model inference optimization
- Knowledge of data structures used in machine learning for memory and compute efficiency
- Experience with hardware-specific optimization, especially on custom hardware like D-Matrix, is an advantage
What the job involves
- We are seeking a motivated and innovative Machine Learning Intern to join our team
- The intern will work on developing a dynamic Key-Value (KV) cache solution for Large Language Model (LLM) inference, aimed at enhancing memory utilization and execution efficiency on D-Matrix hardware
- This project will involve modeling at the PyTorch graph level to enable efficient, torch-native support for KV-Cache, addressing limitations in current solutions
- Research and analyze existing KV-Cache implementations used in LLM inference, particularly those utilizing lists of past-key-values PyTorch tensors
- Investigate “Paged Attention” mechanisms that leverage dedicated CUDA data structures to optimize memory for variable sequence lengths
- Design and implement a torch-native dynamic KV-Cache model that can be integrated seamlessly within PyTorch
- Model KV-Cache behavior within the PyTorch compute graph to improve compatibility with torch.compile and facilitate the export of the compute graph
- Conduct experiments to evaluate memory utilization and inference efficiency on D-Matrix hardware
- Develop an efficient support system for KV-Cache on D-Matrix hardware
- Create a torch-level modeling framework for dynamic KV-Cache
- Ensure compatibility of the KV-Cache model with torch.compile and other PyTorch features for optimized graph export
Our take
d-Matrix is redesigning how AI gets done, tackling the limit f traditional chips with its Corsair platform. Using digital in-memory computing and a chiplet-based design, it delivers fast, power-efficient AI interface that's built for today's data-heavy models, without the energy overload.
Recently, the company has been making some serious moves. Its secured substantial funding, with backing from Microsoft, and partnering with HTEC to boost its software capabilities. With Corsair already showing serious speed gains over GPUs like Nvidia's H100, d-Matrix is proving it can deliver real-world performance at scale.
But instead of competing head-on with GPU giants, d-Matrix is focusing on where AI meets people. And with AI growing fast, especially for interactive and generative applications, d-Matrix is building the hardware that helps AI keep up with this demand.
Freddie
Company Specialist at Welcome to the Jungle
Company
Funding (last 2 of 3 rounds)
Nov 2025
$275m
SERIES C
Sep 2023
$110m
SERIES B
Total funding: $429m
Company benefits
- Equity
- Health care
- Flexible time-off
- Paid paternity leave
- 401k retirement plan
- Remote & hybrid working model
- Work with world-leading engineers and researchers
- Peer coaching & development
- Team-building activities
- Happy hours
- Free food and snacks
- On-site fitness center
Company HQ
Santa Clara, CA
Leadership
Sid Sheth
(Co-Founder, CEO & President)Previously Director at NetLogic Microsystems, Director of Marketing at Broadcom Inc and SVP/GM at Inphi Corporation.
Sudeep Bhoja
(Co-Founder & CTO)Previously CTO at Inphi Corporation, Technical Director at Broadcom and Chief Architect at Big Bear Networks.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 20 more jobs at d-Matrix