AI Infrastructure Systems Engineer, Together AI

• If you enjoy writing software more than clicking dashboards, obsess over eliminating manual work, and want to build infrastructure that manages tens of thousands of GPUs autonomously, we’d love to talk
• 3+ years building distributed systems, infrastructure platforms, or large-scale backend software
• Strong software engineering skills in Python, Go, or Rust
• Experience building platforms, automation systems, or developer infrastructure
• Experience with Linux, Kubernetes, Terraform, Ansible, or similar infrastructure technologies
• Strong systems thinking with the ability to understand problems across hardware and software
• A passion for solving complex infrastructure challenges through software
• An automation-first mindset—if a task is repeated, your instinct is to build a system to eliminate it
• You’ll Thrive Here If You:
• Love building systems that replace repetitive operational work
• Think of infrastructure as a software engineering problem
• Enjoy solving hard problems with no existing playbook
• Care deeply about performance, reliability, and scale
• Want to build technology that powers frontier AI models

Desirable

• At Together AI, you’ll build and operate one of the world’s largest GPU fleets used for frontier model training and inference. This isn’t a traditional infrastructure role—we’re looking for engineers who love building systems, automating everything, and solving problems at massive scale
• Design and build fleet automation systems that provision, validate, deploy, upgrade, repair, and retire GPU clusters with minimal human intervention
• Build AI Infrastructure Agents that automate deployment, root-cause failures, incident triage, and autonomous remediation
• Develop Fleet Intelligence platforms that continuously monitor hardware health, firmware, networking, storage, thermals, and workload performance to predict failures before they impact customers
• Build software that maximizes GPU availability, utilization, performance, and reliability across thousands of accelerators
• Create automated validation systems for GPUs, InfiniBand/RoCE fabrics, NVLink/NVSwitch, storage, and distributed AI workloads
• Build internal platforms and developer tools that allow infrastructure to be managed through software—not manual operations
• Continuously improve deployment velocity, reliability, and operational efficiency through automation
• Partner closely with hardware, networking, platform, and AI teams to push the limits of AI infrastructure