Senior Software Engineer, Grafana Labs

• You enjoy working with engineers, as well as with the management structures that are there to support you and enable you and your team to do your very best
• You are comfortable working in a remote-first company; communication is key. For us, working together means being collaborative, friendly, kind, and respectful. We operate by consensus, you can contribute to a discussion but then commit to the team decision
• As such, being such a highly distributed company, means we would love someone who is keen on working with distributed systems, too
• You are eager to learn and grow. There is a lot of room for growth and development, and the team has quite a lot of knowledge to share for those who are wanting to learn
• You approach development holistically. The team owns the full life cycle of our code; from writing design docs, to looking at developer feedback, and integration testing. We appreciate engineers who enjoy looking at the big picture, and also notice the details of the brush strokes. The Platform team mainly works with Go, Python, and Shell
• You have experience with operating your code. Since a lot of operators and developers use our software, having some grounding in both of these spaces really helps us with building better platforms for our users

Desirable

• The Internal Engineering Platform (IEP) delivered by the Platform department provides application engineers with the tools, systems and Kubernetes clusters they need to build, deploy and run their workloads
• Platform roles at Grafana Labs have an eye for engineers with a passion for performance and reliability, and who enjoy taking projects from conception to production
• We organize ourselves into squads to allow focus on: cloud infrastructure and capacity management; security; engineering productivity; monitoring and sustainability; and US Federal compliance
• Because we deploy production services, we have on-call rotations to ensure the health of the system
• Everyone at Grafana Labs tries to incorporate and use our product line up into their day-to-day, so being on call is an important way to understand our system and how people use our products
• We are hiring for the Platform InfraCore squad. This squad provides and owns automation of the provisioning of CSP resources, including networking, Kubernetes clusters and specific CSP resources required by our application teams
• We’re part of a Platform Engineering group that manages infrastructure for the teams that are building some of the most cherished tools - Grafana, Mimir, Loki, Tempo, Pyroscope to name a few
• You enjoy working with engineers, as well as with the management structures that are there to support you and enable you and your team to do your very best
• You are comfortable working in a remote-first company; communication is key. For us, working together means being collaborative, friendly, kind, and respectful. We operate by consensus, you can contribute to a discussion but then commit to the team decision
• As such, being such a highly distributed company, means we would love someone who is keen on working with distributed systems, too
• Kubernetes cluster provisioning and lifecycle management
• Management of cluster networking components: load balancing, NAT, DNS, CNIs, network policies, private connectivity for customers, cross-cluster communication
• Management of scheduling and autoscaling
• Maintaining Crossplane compositions and Terraform modules for CSP resources common to our users. As well the management of versioning and compatibility for Crossplane and Terraform core as well as providers
• Work with our users (Grafana Cloud application teams) to help understand their needs and ensure we’re investing in the right capabilities
• Participation in the Platform department Infrastructure wing on-call rotation
• A few of our upcoming projects:
• Improving K8s efficiency. Considerations include processor architecture, CSP capacity, machine selection. Balancing utilization with idle compute to ensure suitable performance and cost
• Improving network policy adoption and interfaces. Make it easier for our users to create the perfect network policies and tests for their workloads
• Refining/expanding SLOs. Our users need to know exactly what to expect from their IEP. We have explicit measures to indicate when we are and are not on target. We’re continually expanding the picture