Senior Data Engineer, Vivacity

• You are a technical expert with a proven ability to creatively solve complex problems across a broad remit, manage production systems reliably, and navigate ambiguity to deliver impactful solutions
• You have a good understanding of distributed systems, data pipelines, and cloud infrastructure, and you thrive in an environment that values collaboration, continuous improvement, and technical excellence
• You proactively and effectively communicate with both technical and non-technical colleagues, earning trust and influence through your knowledge and proactive, considerate approach
• You are passionate about simplifying complex systems and seek opportunities to balance immediate delivery with long-term improvements
• You continue working on problems, even when you are not familiar with the problem space, and you know when and how to ask for help
• Expertise in Apache Kafka (ideally Kafka Strimzi), including broker management, scaling, upgrades, and integration with high-throughput data pipelines
• Experience managing PostgreSQL databases, including load analysis, query optimisation, and role management
• Experience with modern cloud infrastructure as GCP/AWS, Kubernetes, and Docker
• Proficiency in Unix systems, ideally Linux (we use Ubuntu)
• Strong communication skills, with experience mentoring engineers and collaborating with stakeholders
• Proven ability to resolve technical incidents in unfamiliar production systems
• Technical and process documentation champion
• Experience of operationally managing production software components, including observability, logging, metrics, error reporting, debugging, and live incident management

Desirable

• Reporting to: François-Xavier Decroix (Engineering Manager)
• As a Senior Data Engineer, you will take responsibility for continuously improving critical systems which transport hundreds of gigabytes of data per day and trillions of messages per year
• You will work closely with the broader team to bring innovative new datasets and insights to customers in a reliable, scalable, and cost-effective way
• Your work will be crucial in our mission to empower safe and sustainable transport around the world through data insights
• You will be the go-to person for our cloud data pipeline & underlying infrastructure, especially Apache Kafka and our PostgreSQL database
• You will also work with tech like Kubernetes, Vault, go microservices and other cloud services, as well as on-premise infrastructure
• Your technical expertise will be instrumental in developing and operating these systems for our existing and new products and features; you will be crucial in helping us maintain our market-leading position in the UK, and in scaling internationally
• In this role, you will work within the AI and cloud team, establish technical standards, lead architectural decision-making within your area of expertise, and proactively share knowledge with colleagues
• This role has the option to grow towards either a principal engineering role (a technical specialist leader) or an engineering manager (a team leader with line management) position at Vivacity, through structured career progression pathways
• Your time will be spent roughly as follows:
• 60% - Proactive technical work (e.g. migrating DB hosting provider, new message bus system proof of concept, deploying new algorithms to kubernetes)
• 20% - Reactive technical work (e.g. incident response, unblocking engineers, code review)
• 10% - Technical leadership initiatives (e.g. establishing technical standards, architectural principles, & staff training)
• 10% - Planning and administration

• 30 minute screening interview
• 1.5 hour system design interview where you work together with a VivaCity engineer
• 2.5 hours final round interview, split into a 1 hour technical experience interview, 1 hour soft skills interview, and 30 minutes with Shaun, our Head of Engineering