Director of Engineering, Groq
Compute
$300-375k
+ Equity
Office located in Palo Alto, CA
AI semiconductor chip manufacturer
Open for applications
AI semiconductor chip manufacturer
201-500 employees
Open for applications
$300-375k
+ Equity
Office located in Palo Alto, CA
201-500 employees
Company mission
To increase developer velocity in machine learning by leading the way with simple, fast, innovative solutions.
Role
Who you are
- 10+ years in large-scale infrastructure engineering, including 3+ years leading teams that run business-critical, globally distributed fleets
- Proven leadership experience in highly technical engineering environments, with a track record of delivering innovative platform solutions and effectively leading, motivating, and developing high-performing engineering teams
- Demonstrated excellence in communication, planning, negotiation, and interpersonal interactions across executives, cross-functional stakeholders, and team members, with a strong ability to influence and drive organizational change
- Cloud & hybrid experience: History of building, deploying, and operating compute in data centers in addition to augmenting with Cloud-based workloads - ideally GCP
- Hands-on lineage: you once built or operated clusters yourself (writing Golang operators, CRDs, and CLI tools), graduated into org leadership, and still dive deep when needed
- Experience with on-prem storage technologies and their Kubernetes integrations
- CI/CD leadership: design and run pipelines (GitHub Actions, Buildkite, or similar) that build, test, sign, and promote container images at hyperscale velocity
What the job involves
- Lead the Compute team in building and operating the world’s largest hyperscaler specifically designed for AI inference, tailored to Groq’s hardware across our global data center footprint
- The Compute team develops the foundational compute platform and critical services, including Datacenter Infrastructure Management (DCIM), hardware inventory systems, container and inference orchestration frameworks, observability and monitoring tools, and data center operations automation
- Your role is pivotal in defining, implementing, and scaling Groq’s hyperscaler infrastructure
- Architect the development of Groq’s hyperscale compute platform, ensuring scalability, reliability, and security
- Plan: define and execute technical roadmaps that advance Groq’s capability to manage large-scale general and specialized compute infrastructure efficiently
- Lead highly technical engineering teams focused on container orchestration, hardware provisioning, and platform automation
- Build and grow the organization: attract, hire, mentor, and retain top-tier engineers; shape a culture of automation, simplicity, rapid learning and operational excellence
- Operate the fleet: own production Kubernetes clusters and Storage solutions distributed across several geographic regions, driving SLOs, incident response, and continual improvement
- Ship continuously: enforce robust CI/CD—with container image scanning, automated integration tests, and progressive roll-outs—to keep the platform secure and rapidly evolving
- Collaborate globally with data-center, hardware, and hardware teams to ensure seamless capacity expansions, hardware refreshes, and energy-efficiency initiatives
- Advanced low-latency networking: partner closely with our networking to ensure we champion modern data-plane technologies (Cilium/eBPF, BGP-based service routing, advanced load balancing) for low-latency throughput and high security
Our take
Groq is an exciting semiconductor startup focused on creating advanced chips designed specifically for AI applications, particularly for AI inference, where models use trained data to deliver fast responses.
Groq aims to challenge Nvidia's dominance by focusing on speed and efficiency for AI deployment. While Nvidia's GPUs handle both training and running AI models, Groq's LPUs are optimized for inference tasks. With a recent $640M funding round, Groq plans to scale up and deploy over 100,000 LPUs by 2025.
To achieve this, the company has partnered with companies like Meta and Samsung, as well as sovereign entities like Saudi Aramco, showcasing the company's growth and potential. With its exciting technology, Groq is well-positioned to play a significant role in the AI chip market in the coming years.
Steph
Company Specialist at Welcome to the Jungle
Insights
-19% employee growth in 12 months
Company
Funding (last 2 of 5 rounds)
Aug 2024
$640m
SERIES D
Jul 2024
$300m
SERIES D
Total funding: $1.3bn
Company HQ
Old Mountain View, Mountain View, CA
Leadership
Jonathan Ross
(Founder & CEO)Majored in Mathematics and Computer Science at NYU. Joined Google as a Software Engineer in 2011, and was promoted to Hardware Engineer in 2013. Co-founded Google's Tensor Processing Unit. Was also a Rapid Evaluator at Google X, and then founded Groq in 2016, acting as its CTO before becoming the CEO in 2018.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 3 more jobs at Groq