Lead Data Engineer, AND Digital
+ Share scheme
More information about location
Developing & delivering digital products and platforms
Open for applications
Developing & delivering digital products and platforms
1001+ employees
Open for applications
+ Share scheme
More information about location
1001+ employees
Company mission
To close the world's digital skills gap.
Role
Who you are
- Data Engineering - 7 to 10 years experience building and supporting full-stack data pipelines from source to reporting
- AWS Mastery - 5+ years deep experience with AWS, ideally administration and optimization of RDS / Aurora (Postgres) and Redshift
- Python - 5+ years expert-level Python, with specific experience working on ETL pipelines, including libraries like Pandas and orchestration tools like Airflow or Prefect
- PostgreSQL - 5+ years expert-level SQL and database architecture (partitioning, indexing) for both OLTP and OLAP
- DataOps / Tools - 3+ years expertise in Git, with solid understanding of Git branching and workflow, and schema management (e.g., Liquibase, Flyway, or Drizzle)
What the job involves
- We are seeking a Lead Data Engineer to act as the technical practice lead for the Luna data team. This is an important role responsible for leading a high business priority migration from AND’s existing Redshift environment to our new AWS RDS (Postgres) architecture
- Beyond technical architecture, you will be a mentor and skills builder for the team. You will guide the technical growth of our Data Analysts, transitioning them from SQL-centric workflows to production-ready Python and DataOps practices, while collaborating closely with stakeholders to translate business needs into scalable technical roadmaps
- Technical Leadership & Architecture:
- Lead new Data Pipeline Delivery: Implement the new Python-based ETL pipelines for data movement from Luna’s microservice DynamoDB tables to RDS transactional DB (Postgres OLTP) and subsequently to the RDS data warehouse (Postgres OLAP)
- Platform Optimization: Leverage expertise from previous data setups to optimize the new Postgres OLTP for transactional performance and the OLAP environment for analytics effectiveness, with both being conscious of on-going cost management
- Standard Setting: Establish and enforce DataOps standards, including version control (Git), automated CI / CD deployment, and schema migration management using tools like Liquibase or Drizzle ORM
- Team Enablement & Mentorship:
- Hands-on Coaching: Actively mentor team members, elevating their skills in Python, Git, and engineering workflows through code reviews and workshops
- Code Quality: Conduct rigorous code reviews, providing detailed, educational feedback that explains best practices and clearly outlines required changes to elevate team standards
- AI Engineering patterns: Support the definition and implementation of AI tooling and practices to augment the Data engineering team
- Strategic Agile Delivery:
- Initiative Planning: Break down larger reporting initiatives into manageable epics and stories within our Agile framework (a mix of Scrum for larger work items, and Kanban for smaller continuous flow items)
- Stakeholder Management: Network with business domains to capture requirements and provide strategic guidance on the data platform migration plan
- Product Partnership: Support the Data Product Owner by providing the technical context necessary to prioritize the team's backlog effectively
Our take
As the need for digital transformation continues to grow, many organisations find themselves lacking the skills to keep up. AND Digital has stepped in to help bridge this gap, with a clear mission to close the world's digital skills shortage. Rather than simply delivering digital solutions, they focus on helping businesses understand how to work digitally, so they can continue to grow and adapt long after a project is finished.
The company is standing out from the crowd too, by working alongside their clients instead of working for them. Combining their own people with client teams into one collaborative unit, they identify challenges, build digital products, and share skills throughout the process, so clients can take full ownership of their apps, websites, and future digital work.
Unsurprisingly, this approach has had a strong impact and continues to drive AND Digital's growth. With cross-functional teams across the UK, the Netherlands, and the US East Coast, the company scales based on demand. And their city-based 'Club' model blends local, personalised service with the strength of a global organisation, putting AND Digital in a strong position for the future.
Steph
Company Specialist at Welcome to the Jungle
Insights
Company
Funding (2 rounds)
Apr 2021
$10.9m
EARLY VC
Feb 2020
$15m
EARLY VC
Total funding: $25.8m
Company HQ
Maidenhead, UK
Leadership
Paramjit Uppal
(Founder)Former Founder of Qedis (brought out by North Highland UK in 2013) and Director at Deloitte.
Share this job
View 20 more jobs at AND Digital