Staff Software Engineer, Grafana Labs

• Substantial experience designing, delivering and operating production backend systems, with the depth and judgment to lead complex work hands-on
• Strong experience building and operating production backend systems in Go, our primary backend language
• Professional experience contributing to frontend applications using TypeScript and React, or comparable technologies, and willingness to work across the stack as priorities change
• Experience reasoning about distributed systems, including partial failures, retries, consistency and interactions between services
• Experience with relational databases, including data modelling, schema migrations, query performance and safe production changes
• Practical experience deploying and troubleshooting containerised services and jobs in Kubernetes
• Experience managing cloud infrastructure through code and working with CI/CD pipelines, environment configuration and controlled rollouts
• Ability to investigate production problems using logs, metrics and traces, and turn findings into lasting improvements
• A record of delivering projects end to end, clarifying requirements, making trade-offs and following through after release
• Experience working remotely, with clear asynchronous communication and dependable collaboration across time zones
• Ability to learn unfamiliar domains and technologies, adjust your approach as new information emerges, and take ownership beyond your main area of expertise
• Experience writing readable, well-tested software that other engineers can understand, maintain and operate
• Willingness to participate in on-call and take responsibility for the production behaviour of the systems you build
• Experience supporting colleagues through code reviews, mentoring and constructive technical discussions
• You enjoy engineering work where correctness, reliability and business impact matter. You find value in improving an established system, removing a recurring operational problem or making a billing workflow easier to understand and support
• You take ownership from understanding the problem through delivery and production support. You can turn an ambiguous request into manageable increments, make sound trade-offs and follow through without needing each step prescribed. You ask useful questions, seek feedback and raise uncertainty or blockers early
• You adapt as priorities change, learn unfamiliar systems and technologies, and contribute where the squad needs you. You bring a strong backend specialty while remaining willing and able to work on frontend features and supporting infrastructure
• You reason about how a service behaves in production: how it is deployed, where its data lives, what happens when dependencies fail, and how the team detects and recovers from problems
• You communicate clearly in writing and in conversation. In a distributed team, you make decisions, progress and risks visible so colleagues can contribute across time zones. You work independently while staying connected to the squad
• As a Staff engineer, you bring technical depth and sound judgment while remaining close to implementation. You collaborate constructively with other experienced engineers, welcome different perspectives and help the squad reach decisions and move forward
• We embrace thoughtful use of AI-assisted development. We value engineers who use these tools to support exploration, implementation, testing and documentation while taking responsibility for the quality and correctness of the result

Desirable

• Application Core Services (AppCore) is a group within Platform, in the Foundations department. We build and operate systems that support Grafana’s business operations and customer experience, working closely with product teams and internal business partners
• The Usage squad works on billing, rates, usage calculations, usage visibility and cloud marketplace integrations. Our software helps customers understand their consumption and costs, and helps internal teams operate accurate, reliable billing processes
• We are looking for a Staff Fullstack Engineer with strong backend experience who enjoys making complex business systems reliable, understandable and easier to operate. You will help deliver new capabilities while improving existing services, reducing repetitive manual work and addressing edge cases that affect customers and internal stakeholders
• This is a hands-on Staff engineering role within the AppCore Usage squad. You will lead complex technical work, contribute directly to implementation and share ownership with the squad’s other Staff and Senior engineers. Your primary focus will be Usage’s systems and outcomes, collaborating with adjacent teams when the work requires it
• Our ownership includes application code, data stores, deployment configuration and the cloud infrastructure supporting our services. You should be comfortable following a problem across those boundaries and making improvements with the appropriate platform teams
• Your initial focus will be approximately 75% backend and 25% frontend. This balance will evolve with the squad’s priorities. You should be comfortable delivering across the stack, from business logic and APIs through to the interfaces customers and internal teams use
• Design, build and operate backend services and APIs supporting billing, rates, usage calculations, usage visibility and marketplace workflows
• Build and maintain backend services primarily in Go, with Node.js also present in our stack, and contribute to TypeScript/React applications to deliver complete features
• Improve the correctness and reliability of billing and usage workflows, including validation, reconciliation, error handling and recovery
• Identify recurring manual tasks and replace them with maintainable automation, clear operational workflows and appropriate controls
• Maintain and evolve the infrastructure supporting our services, including Kubernetes workloads, scheduled jobs, cloud resources and deployment configuration
• Use infrastructure as code and automated delivery pipelines to make changes repeatable, reviewable and safe across environments
• Investigate production issues across application code, databases and infrastructure, including resource exhaustion, failed jobs, slow queries and replication delays
• Plan and execute service deployments, database changes and infrastructure migrations, including validation, monitoring and recovery plans
• Build and maintain integrations with cloud marketplaces and external business systems where required by the squad’s work
• Deliver projects from understanding requirements and proposing an approach through implementation, testing, rollout and production support
• Make changes traceable and testable, supporting audit requirements and clear explanations of system behaviour
• Collaborate with product management, Support, DealOps, GTM, infrastructure teams and adjacent AppCore squads to clarify requirements and resolve dependencies
• Share technical leadership through design discussions, code reviews, mentoring and practical improvements
• Participate in the squad’s on-call rotation after onboarding. Improve dashboards, alert quality, runbooks and recovery procedures, and follow through on incident findings