Senior Software Engineer, Upstart

• 6+ years of experience in object-oriented software engineering, with a strong systems engineering background
• Proven experience independently leading impactful, cross-functional, multi-quarter projects with mid-large teams
• Proficiency in backend development with Python (leveraging frameworks like FastAPI, Flask, or similar), microservices architecture, and infrastructure tools like Kubernetes, Docker, and Terraform
• Hands-on experience building ML platforms or infrastructure supporting LLMs or inference systems
• Familiarity with observability tools (e.g., Prometheus, Grafana, Datadog) and data processing pipelines
• Strong written and verbal communication skills and a collaborative approach to cross-functional work
• Strong stakeholder management skills

Desirable

• Upstart’s new LLM Platform team is building foundational infrastructure that democratizes access to generative AI for every product and engineering team across the company
• As a Senior Software Engineer on the LLM Platform team, you will help design and build a unified interface for securely interacting with large language models - abstracting away the complexity of model selection, prompt orchestration, evaluation, and scaling
• You'll help build a centralized GenAI layer that enables engineers to use best-in-class LLMs through clean APIs, configurable pipelines, and intuitive tooling
• Reporting to our Senior Director of Engineering, the team operates with broad scope and high visibility, partnering across Machine Learning, Product, and Compliance to ensure every model integration is performant, secure, and compliant
• Whether it's embedding LLMs in user-facing products or optimizing backend workflows, this team makes it seamless for Upstart Engineering to adopt and scale GenAI company-wide
• Build the LLM Platform powering generative AI use cases across all of Upstart Engineering
• Design and implement reliable, observable infrastructure for model inference, prompt orchestration, and data workflows
• Drive cross-functional adoption of AI tooling by delivering reusable components and intuitive interfaces.Increase developer productivity through automation, metrics dashboards, and streamlined GenAI integrations
• Set the technical roadmap for platform capabilities, balancing innovation with reliability and compliance
• Partner with ML researchers, product engineers, and design to bridge experimentation and production
• Ensure the platform adheres to emerging standards of security, fairness, and explainability for LLM systems