Site Reliability Engineer, Fireworks AI

Software Engineer

Salary not provided

AWS

Kubernetes

GCP

Python

Bash

Azure

Prometheus

CUDA

Grafana

Mid and Senior level

New York

San Francisco Bay Area

Generative AI platform enabling rapid scaling for developers and businesses

Job no longer available

Generative AI platform enabling rapid scaling for developers and businesses

21-100 employees

B2BArtificial IntelligenceSaaS

Job no longer available

Salary not provided

AWS

Kubernetes

GCP

Python

Bash

Azure

Prometheus

CUDA

Grafana

Mid and Senior level

New York

San Francisco Bay Area

21-100 employees

B2BArtificial IntelligenceSaaS

Company mission

To help every organization unleash the full power of artificial intelligence

Job

Company

Role

Who you are

We’re seeking a highly skilled SRE/PE with deep expertise in Kubernetes (k8s), cloud networking, and infrastructure automation
3+ years in SRE/PE/DevOps roles with production-grade Kubernetes experience
Proficiency in cloud networking (AWS/GCP/Azure VPCs, firewalls, DNS) and service monitoring (Prometheus, Alertmanager, Grafana)
Hands-on experience with incident management and improving system reliability/SLOs
Strong scripting/coding skills (Python/Go/Bash) for automation and tooling
Familiarity with object storage (S3, GCS) and data pipeline integration

Desirable

Experience with GPU clusters (NVIDIA GPUs, MIG, CUDA) and AI/ML workloads
Knowledge of auto-scaling technologies (K8s HPA/VPA) and auto-remediation frameworks
Expertise in service meshes (Istio)

What the job involves

This role will focus on reducing incident response time, implementing auto-remediation, optimizing auto-scaling, and improving cluster efficiency and service health
You’ll design systems that balance performance, cost, and reliability while working onsite with our Redwood City team
Drive initiatives to reduce incident response time through improved monitoring, alerting, and automated remediation
Build self-healing systems and playbooks for common failure scenarios
Lead blameless post-mortems and implement preventative measures
Manage and optimize GPU-enabled Kubernetes clusters for AI/ML workloads, focusing on cost-performance efficiency, auto-scaling, and resource utilization
Debug performance bottlenecks in distributed systems (e.g., network, storage, GPU scheduling)
Strengthen service health by refining cloud networking stacks (VPCs, load balancers, service meshes) and ensuring low-latency communication
Design fault-tolerant architectures to minimize downtime
Enhance service monitoring with tools like Prometheus, Grafana, and custom metrics pipelines
Implement predictive analytics to proactively address system health risks
Build automation for cluster provisioning, scaling, and recovery using Terraform, Argo, and CI/CD pipelines
Develop tools to streamline operational workflows (e.g., automated rollbacks, canary deployments)

Our take

Whilst the development of generative AI is thought to be the reserve of large companies like Open AI, smaller startups such as Firework.AI are making this technology easier to access and develop for smaller companies wanting to handle specific business needs using AI. As an open-source model API, it has one of the largest user bases - with over 12,000 users leveraging Fireworks.AI’s software.

Though lacking the brand name recognition of generative AI developers like Google and Meta, Fireworks.AI differentiates itself in a market crowded with large competitors through its unique approach to deploying its software. It creates smaller, production-grade, customisable generative AI models that can be launched privately and securely, rather than the non-private and inflexible models built by larger companies.

After its 2024 Series B round of $52 million, Fireworks.AI has been valued at over $550 million, an impressive sum for a David in a sea of Goliaths. The company plans to use its funding to develop compound AI systems that can orchestrate across multiple single models. It will be interesting to see how the company evolves with its potentially disruptive business approach to generative AI.

Freddie

Company Specialist at Welcome to the Jungle

Company

Funding (2 rounds)

Jul 2024

$52m

SERIES B

Mar 2024

$25m

SERIES A

Total funding: $77m

Company HQ

Redwood City, CA

Leadership

Lin Qiao

(CEO)

Previously Senior Director of Engineering at Meta, Tech Lead at Linkedin, Research Staff Member at IBM Almaden Research Center and an Advisory Software Engineer at IBM Silicon Valley Lab.

Salary benchmarks

We don't have enough data yet to provide salary benchmarks for this role.

Submit your salary to help other candidates with crowdsourced salary estimates.

Share this job

View 19 more jobs at Fireworks AI