Senior Machine Learning Systems Engineer, OctoML
$175-240k
+Annual bonus & Stock Options
Machine learning acceleration platform
Job no longer available
Machine learning acceleration platform
101-200 employees
Job no longer available
$175-240k
+Annual bonus & Stock Options
101-200 employees
Company mission
To empower more creators to harness the transformative power of ML to build intelligent applications.
Role
Who you are
- We are seeking a highly skilled and experienced Senior Machine Learning Systems Engineer to join our dynamic team
- Bachelor’s, Master’s, or PhD degree in Computer Science, Electrical Engineering, or a related field
- Extensive experience in machine learning systems engineering, with a proven track record of leading successful projects
- Strong programming skills in C/C++ and Python with a robust computer science background
- Experience in LLM inference systems is preferred but not required
- Familiarity with innovative open-source projects like MLC-LLM, vLLM, TGI is a plus
- Experience with machine learning compilers or frameworks such as TVM, MLIR, Pytorch, Tensorflow, ONNX Runtime, TensorRT is preferred
- Excellent problem-solving skills and the ability to work effectively in a fast-paced, dynamic environment
- Strong communication skills, with the ability to articulate complex technical concepts to both technical and non-technical stakeholders
What the job involves
- Our ML engineering team is dedicated to developing the most efficient and feature-packed engines for generative model deployment
- This includes feature enablement and optimization for popular media models, such as Mixtral, Llama-3, Stable Diffusion, SDXL, SVD, and SD3, and thus requires a broad understanding of various system layers from serving API to hardware level
- We achieve this by building systems that innovate new techniques as well as leveraging and contributing to open source projects including TVM, MLC-LLM, vLLM, CUTLASS, and more
- In this senior role, this person will be responsible for contributing to the latest techniques and technologies in AI and machine learning. They will lead projects, mentor junior engineers, and play a crucial role in shaping the future of our ML systems
- Lead and improve model ingestion and robustness at the front end of our core pipeline
- Develop cutting-edge generative AI features and drive innovation in our product offerings
- Analyze the performance characteristics of various generative AI traffics and identify system bottlenecks
- Develop impactful optimizations ranging from system-level to hardware-level, ensuring maximum efficiency and performance
- Create and present groundbreaking demos to showcase our capabilities to stakeholders, customers, and the broader community
- Mentor and guide junior engineers, fostering a collaborative and growth-oriented team environment
- Stay up-to-date with the latest advancements in AI and machine learning, integrating new techniques and technologies into our systems
Our take
It’s no secret AI technology has the capability to revolutionise the world. Now used across many sectors, developers are looking into and continuing to research the possibilities of machine learning. OctoML is an acceleration platform hoping to make AI the most accessible and sustainable version of itself that it can be.
The company has developed a library of products that enables users to implement AI technology into customers' already integrated systems, making it less intimidating for the more amateur user.
The company appointed Lip-Bu Tan, the executive chairman of multinational software company Cadence Design, to its board of directors in 2022. This fresh wealth of expertise will be used toward the company's next phase of expansion. After several rounds of funding gave OctoML the ability to deploy new features among the platform, it’ll be interesting to see what the company decides to do next in its scaling.
Freddie
Company Specialist at Welcome to the Jungle
Insights
Company
Funding (last 2 of 4 rounds)
Nov 2021
$85m
SERIES C
Mar 2021
$28m
SERIES B
Total funding: $131.9m
Company benefits
- Great benefits - Whether you’re into kitesurfing, biking, or board games, we’ve got you covered with low-deductible, inclusive health plans with 100% paid premiums. Plus dental, vision, life, disability, paid parental leave, 401K, Health Savings Accounts, and more
- Time for your life - We strongly encourage use of our generous PTO policy and offer two company-wide “quiet periods” per year to rest and recharge. Enjoy a remote-first culture, family-friendly policies, flexible work schedules, and a generous stipend to outfit your home office.
- Career growth - We understand that career growth isn’t always linear. Octonauts who wish to pursue broader responsibilities at the company are equipped with resources and guidance to develop as leaders.
Company HQ
Pike Pine Retail Core, Seattle, WA
Leadership
Luis Ceze
(CEO)Co-founded Corensic, was a Venture Partner at Madrona Venture Group. Is currently also a Professor at the University of Washington.
Jason Knight
(CSO)Previously a Bioinformatics Scientist at Human Longevity and worked through several roles at Intel Corporation, most recently Head of Software Product - AI Products Group.
Jared Roesch
(CTO)Worked as a Research Assistant at Mozilla, Teaching Assistant at the University of Washington and Research Intern at Microsoft.
Thierry Moreau
(VP Technology Partnerships)Extensive background in research and engineering. Most recently worked as a Postdoctoral Researcher at the University of Washington.
Tianqi Chen
(Chief Technologist)Doctor of Philosophy - PhD in Computer Science. Began as CTO at OctoML before becoming Chief Technologist in January 2022. Also is currently a Professor at Carnegie Mellon University.
Share this job