Senior Staff LLM Inference Engineer, d-Matrix

• Have deep intuition for modern generative AI architectures and how to squeeze performance out of them at inference time
• Are familiar with the internals of open-source inference frameworks (vLLM, SGLang, TensorRT-LLM, etc.) and can extend or replace them when needed
• Enjoy pathfinding new use cases — exploring heterogeneous deployment topologies and building early-stage POCs that prove out new ideas
• Are results-oriented with a strong bias toward action; you own problems end-to-end from prototype to optimization to handoff
• Are energized by working at the intersection of novel hardware and frontier models, and want your work to directly influence how next-generation AI silicon is used
• Value clear communication and thrive in a small, high-ownership team environment
• Bachelor’s degree in Computer Science, Electrical Engineering, or a related field, and 10+ years of relevant engineering experience; or equivalent demonstrated experience
• Master’s or PhD in Computer Science, Electrical Engineering, or a related field preferred, with 6+ years of relevant industry experience
• Strong proficiency in Python and C/C++
• Hands-on experience optimizing LLM inference — attention kernels, KV cache, batching strategies, quantization (INT8/FP8/INT4)
• Experience with at least one major inference framework (vLLM, SGLang, TensorRT-LLM, ONNX Runtime, or similar) at a contributor level
• Familiarity with GPU kernel programming (CUDA/Triton) and performance profiling tools

Desirable

• We are hiring end-to-end inference engineers who are comfortable going from a novel research idea to a deployed, optimized system
• You will work at every layer of the inference stack — from kernel-level optimization to distributed orchestration to high-level serving APIs
• Identify and prototype emerging LLM inference use cases suited to heterogeneous hardware deployments
• Build compelling proof-of-concept systems that demonstrate D-Matrix capabilities to customers, partners, and internal stakeholders
• Develop and tune custom kernels and operator-level optimizations to maximize throughput and minimize latency
• Drive quantization, sparsity, and batching strategies tailored to D-Matrix computational model
• Build and maintain inference runtimes, serving frameworks, and evaluation tooling
• Contribute to distributed inference systems: tensor/pipeline parallelism, disaggregated prefill/decode, KV-cache management
• Work closely with hardware architects to provide firmware and compiler teams with actionable inference workload insights
• Partner with product and business development to translate POCs into customer-facing demonstrations
• Contribute to technical publications, whitepapers, and open-source projects that advance D-Matrix visibility