Distributed Machine Learning Systems Engineer, Together AI

Inference

$160-220k

+ Equity

AWS

Kubernetes

GCP

Python

C++

Rust

Azure

PyTorch

Mid and Senior level

San Francisco Bay Area

Cloud platform for building generative AI

Job no longer available

Cloud platform for building generative AI

201-500 employees

B2BArtificial IntelligenceDeep TechSaaSCloud Computing

Job no longer available

$160-220k

+ Equity

AWS

Kubernetes

GCP

Python

C++

Rust

Azure

PyTorch

Mid and Senior level

San Francisco Bay Area

201-500 employees

B2BArtificial IntelligenceDeep TechSaaSCloud Computing

Company mission

To empower developers and researchers at organizations of all sizes to train, fine-tune, and deploy generative AI models.

Job

Company

Role

Who you are

If you are passionate about designing ML systems that operate at scale and eager to create impactful solutions, we want to hear from you
3+ years of experience in building large-scale, fault-tolerant, high-performance distributed systems
Strong programming skills in one or more of Python, Go, Rust, or C/C++
Excellent understanding of low-level operating systems concepts including multi-threading, memory management, networking, and storage, performance, and scale
Experience with cloud computing platforms (AWS, GCP, Azure etc.) and large-scale infrastructure
Strong problem-solving skills and ability to work in a fast-paced environment
Preferred: Experience with Kubernetes
Preferred: Experience with Pytorch

What the job involves

Together AI is seeking a Distributed ML Systems Engineer to design and build scalable machine learning systems that power our accelerated AI initiatives
This role involves developing large-scale, fault-tolerant distributed systems that handle high-load and high-performance requirements
This position offers the chance to work closely with our AI researchers and infrastructure teams to ensure our systems are robust and efficient
Design and build large-scale, distributed machine learning systems that are fault-tolerant and high-performance
Develop and optimize distributed processing frameworks and storage systems
Collaborate with researchers, engineers, and product managers to integrate ML systems into our infrastructure
Conduct architecture and design reviews to ensure best practices in system design
Implement robust monitoring and logging systems to ensure the health and performance of our ML systems

Our take

Founded in 2022, Together AI is a research-driven AI company that operates a cloud platform for building and running generative AI. The company also contributes to open-source research, models and datasets to advance the use of machine learning.

Together AI's platform enables developers, researchers and businesses to train, fine-tune and deploy generative AI for their needs. With Generative AI gaining a lot of attention recently, the company has some high-profile competitors such as ChatGPT, but its focus on research makes it stand out.

The company has had an incredibly strong start, evidenced by its substantial funding and Unicorn status. It is using these funds to expand its AI Acceleration Cloud product and prepare a large deployment of its GPUs. It will do this whilst securing its position as an open-source model-building platform, with an expectation of 50x company growth over the next five years.

Steph

Company Specialist at Welcome to the Jungle

Company

Funding (last 2 of 5 rounds)

Jul 2026

$800m

SERIES C

Feb 2025

$305m

SERIES B

Total funding: $1.3bn

Company benefits

Competitive health insurance plans
Dental and vision insurance
Pre-tax flexible spending accounts
Mental health support and services
Income protection & retirement
401(k) plan
AD&D insurance
Life insurance
STD & LTD insurance
Monthly team lunches
Flexible time off policy
Team-driven celebrations and events
Monthly commuting stipend + pre-tax bene

Company HQ

Design District, San Francisco, CA

Leadership

Ce Zhang

(Co-Founder & CTO)

Previously Professor at ETH Zurich, Postdoc Researcher at Stanford University and Research Assistant at University of Wisconsin-Madison.

Vipul Ved Prakash

(Co-Founder & CEO)

Previously Senior Director at Apple, CEO and CTO at Topsy and Founder at Cloudmark.

Salary benchmarks

We don't have enough data yet to provide salary benchmarks for this role.

Submit your salary to help other candidates with crowdsourced salary estimates.

Share this job

View 39 more jobs at Together AI