Senior Software Engineer, AssemblyAI

• 5+ years of professional software engineering experience
• Strong proficiency in Go with demonstrated ability to write production-quality code
• Hands-on experience with LLM APIs (OpenAI, Anthropic, etc.) and understanding of prompt engineering concepts
• Experience building or working with API gateways, proxies, and rate limiting systems
• Deep experience with prompt engineering, chain-of-thought, few-shot learning, and other LLM optimization techniques
• Experience building applications with llms or ai agents with cascading architectures
• Solid understanding of software engineering fundamentals:
• Data structures and algorithms
• System design and architectural patterns
• Testing strategies (unit, integration, end-to-end)
• Code review practices and technical collaboration
• Experience with:
• RESTful APIs and distributed systems concepts
• Building or integrating with third-party APIs
• SQL and database design
• Version control (Git) and CI/CD pipelines
• Containerization (Docker) and cloud infrastructure
• Strong communication skills and ability to collaborate effectively with cross-functional teams
• Track record of delivering high-quality, reliable software in a fast-paced, team environment
• Ability to thrive in a startup environment with changing priorities and rapid iteration

Desirable

• We're seeking an exceptional Senior Software Engineer to join our LLM team
• This role is focused on building and maintaining our LLM gateway service—a unified API platform that connects customers to multiple LLM providers
• You'll work on high-impact projects that directly solve customer problems, improve their AI and agentic workflows, and ensure reliable access to the best models for their use cases
• This is a deeply customer-focused role
• You'll work closely with our customer success team to understand customer challenges, help optimize their prompt strategies, build features that address their pain points, and ensure our service reliably delivers value
• We're looking for someone who combines software engineering excellence with genuine curiosity about how customers use AI and a drive to make their lives better
• As a Senior Engineer, you'll drive technical execution within the team, taking ownership of significant features and integrations while mentoring more junior engineers
• You should be passionate about writing clean, maintainable code, implementing comprehensive testing strategies, and building highly reliable systems in service of solving real customer problems
• This role requires close collaboration with customer success, product managers, external API providers, and other engineering stakeholders
• You'll need to balance technical excellence with pragmatic delivery in a fast-paced startup environment where uptime, reliability, and customer success are critical
• Partner closely with the customer success team to understand customer use cases, challenges, and integration needs
• Translate customer pain points surfaced by the CS team into technical solutions and product improvements
• Build features and tooling that directly address customer needs and improve their workflows
• Provide technical guidance and expertise to the customer success team to help them support customers effectively
• Own and deliver complete features and integrations within our LLM gateway service
• Build and maintain integrations with multiple LLM providers and AI services (OpenAI, Anthropic, Google Vertex, AWS Bedrock etc.)
• Write clean, maintainable, well-tested code following best practices
• Design and implement scalable, fault-tolerant solutions with appropriate abstractions
• Proactively identify and address technical debt, reliability issues, and code quality concerns
• Participate in on-call rotation to ensure service reliability and rapid incident response
• Conduct thorough code reviews focused on maintainability, testing, reliability, and architectural concerns
• Ensure proper test coverage across unit, integration, and end-to-end testing levels
• Improve code maintainability and extensibility through targeted refactoring
• Contribute to runbooks, incident postmortems, customer-facing documentation, and knowledge sharing
• Identify bottlenecks and propose solutions to improve team velocity and system reliability
• Work closely with product, customer success, and engineering teams to deliver solutions
• Collaborate with external API providers to troubleshoot integration issues and improve reliability
• Translate ambiguous customer requirements into well-scoped technical work
• Proactively communicate status, risks, and blockers
• Mentor junior engineers on coding standards, reliability practices, and customer-centric thinking
• Participate in architectural discussions and contribute to technical decisions