Senior Backend Engineer, you.com
Inference Platform
$200-250k
More information about location
AI search infrastructure for enterprises
Open for applications
AI search infrastructure for enterprises
201-500 employees
Open for applications
$200-250k
More information about location
201-500 employees
Company mission
To help people move faster, think deeper, and do more.
Role
Who you are
- Hands-on production experience with model serving on stacks such as vLLM, SGLang (or similar platforms), with a working understanding of continuous batching, KV and prefix caching, and the tradeoff between time to first token and throughput
- You have tuned a serving stack against live traffic and know what moves p99 in practice, not only in theory
- Fluency across provider APIs: tokenization, prompt-caching semantics, and the differences in how usage and cached tokens are reported and priced
- Experience building or operating a proxy, gateway, or data plane on the critical path of production traffic, with sound instincts for failing open
- Comfort with streaming responses and the parts that get awkward: partial state, mid-stream failures, and work that can only happen once the stream closes
- A strong backend generalist (Python, Kubernetes). We expect you to pick up whatever you have not touched yet
- We weigh trajectory and recent growth more heavily than years of experience
- Experience at an inference provider or GPU cloud, or contributions to an open-source serving stack
- Fine-tuning and evaluating models in production
- Usage metering, billing, or cost optimization at scale
What the job involves
- Every model call from our products and customers passes through one layer: our inference gateway. It allows users to select a model and we intelligently route the call to optimize on cost , latency and high availability. We are now building the control plane above it, to manage and govern access, performance and cost
- You will own and scale that gateway. The work sits between model serving and product: routing and fallback across hosted providers and our own models, cache-aware routing decisions, metering accurate enough to bill on, and enforcement on the request path within a single-digit-millisecond budget
- Own the inference gateway that every model call routes through, including provider and model selection, fallback, and caps
- Build intelligence layer to autonomously pick the right model for the right turn in the task
- Make routing decisions that account for prefix-cache behavior, batching, and time to first token, not only price and availability
- Build cost attribution and metering across providers, including cached-token accounting, accurate enough to bill and enforce on
- Build request-path enforcement that degrades gracefully and never takes down a customer's inference call
- Ship inference-time changes in shadow mode first, with the measurement to prove they work
Our take
The pressure for enterprises to adopt AI is mounting, but even with impressive advancements to the technology it can be difficult to implement effectively. Foundational models fail to meet the needs of enterprises, which require something they can more easily customise.
This is what you.com recognises and addresses, supplying solutions to empower its clients to build the artificial intelligence that best serves them. The company's enterprise AI product, combined with search and agent APIs and consulting, enables each business to adapt AI intelligently to its industry and aims.
Its a valuable proposition that has not gone unnoticed by the industry, with you.com raising a substantial amount of funding in 2025. This will be used to "build the infrastructure for the agentic era", as the company aims to support its clients into this next age of AI-empowered operations.
Freddie
Company Specialist at Welcome to the Jungle
Insights
33% employee growth in 12 months
Company
Funding (last 2 of 5 rounds)
Sep 2025
$100m
SERIES C
Jun 2024
$50m
SERIES B
Total funding: $197.9m
Company benefits
- A remote-first work environment with hubs in California, NYC, and Canada that offer monthly in-person gatherings
- Unlimited PTO with 11 U.S. holidays observed and a week shutdown in December to rest and recharge
- Competitive health insurance plan, where 100% of the policyholder is covered and 75% for dependents
- 12 weeks of paid paternity leave in the US; additional time off is also considered
- 401k program, 3% match coming January 2025
- $500 work-from-home stipend to be used up to a year of your start date
- $1,200 / year Health & Wellness Allowance to support your personal goals
- In-person coworking weeks 1 to 2 times a year
- Chance to collaborate with a team at the forefront of AI research
Company HQ
SoMa, San Francisco, CA
Leadership
Richard Socher
(Co-Founder & CEO)Experience as Chief Scientist of EVP for Salesforce from 2016 to 2020. Founder of AIX ventures since 2021.
Bryan McCann
(Co-Founder & CTO)Experience as a Course Assistant at Stanford University from 2014 to 2016. Lead Research Scientist for Salesforce from 2016 to 2020.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 2 more jobs at you.com