Senior Staff Machine Learning Engineer, d-Matrix

• BS in Computer Science with 7+ years of strong programming skills in Python and experience with ML frameworks like PyTorch, TensorFlow, or JAX
• Hands-on experience with model optimization, quantization, and inference acceleration
• Deep understanding of Transformer architectures, attention mechanisms, and distributed inference (Tensor Parallel, Pipeline Parallel, Sequence Parallel)
• Knowledge of quantization (INT8, BF16, FP16) and memory-efficient inference techniques
• Solid grasp of software engineering best practices, including CI/CD, containerization (Docker, Kubernetes), and MLOps
• Strong problem-solving skills and ability to work in a fast-paced, iterative development environment
• This role is ideal for a self-motivated engineer interested in applying advanced memory management techniques in the context of large-scale machine learning inference
• If you’re passionate about implementing and optimizing machine learning models for custom silicon and are excited to explore cutting-edge solutions in model inference, we encourage you to apply

Desirable

• D-Matrix is a pioneering company specializing in data center AI inferencing solutions
• Utilizing innovative in-memory computing techniques, d-Matrix develops cutting-edge hardware and software platforms designed to enhance the efficiency and scalability of generative AI applications
• The Model Factory team at d-Matrix is at the heart of cutting-edge AI and ML model development and deployment
• We focus on building, optimizing, and deploying large-scale machine learning modelswith a deep emphasis on efficiency, automation, and scalability for the d-Matrix hardware
• If you’re excited about working on state-of-the-art AI architectures, model deployment, and optimization, this is the perfect opportunity for you!
• Design, build, and optimize machine learning deployment pipelines for large-scale models
• Implement and enhance model inference frameworks
• Develop automated workflows for model development, experimentation, and deployment
• Collaborate with research, architecture, and engineering teams to improve model performance and efficiency
• Work with distributed computing frameworks (e.g. , PyTorch/XLA, JAX, TensorFlow, Ray) to optimize model parallelism and deployment
• Implement scalable KV caching and memory-efficient inference techniques for transformer-based models
• Monitor and optimize infrastructure performance across different levels of custom hardware hierarchy - cards, servers, and racks, which are powered by the d-Matrix custom AI chips
• Ensure best practices in ML model versioning, evaluation, and monitoring