Senior Data Engineer, Hack The Box
$140-160k
Ethical hacking community and cybersecurity training
Be an early applicant
Ethical hacking community and cybersecurity training
1001+ employees
Be an early applicant
$140-160k
1001+ employees
Company mission
Hack the Box's mission is to create and connect cyber-ready humans and organisations through highly engaging hacking experiences that cultivate out-of-the-box thinking.
Role
Who you are
- Strong data modelling and warehouse architecture skills (dimensional modelling, event-driven, lakehouse patterns)
- Hands-on experience with GCP data services — BigQuery is a must; Pub/Sub, Dataflow, Bigtable, Cloud Composer are strong pluses
- Production experience with streaming pipelines on Dataflow/Beam, Flink, or Spark Structured Streaming, ingesting from Kafka and/or Pub/Sub
- Solid SQL and strong Python — you write production-quality code, not just notebooks
- Experience with ClickHouse or another columnar OLAP engine in production
- Workflow orchestration experience with Airflow (or Prefect/Dagster)
- Comfortable with dbt or equivalent transformation frameworks
- Experience migrating off legacy warehouses (Snowflake, Redshift, Synapse) onto cloud-native stacks is a plus
- Working knowledge of ML in production — feature engineering, feature stores, model deployment, drift monitoring, retraining
- Docker & Kubernetes experience
- CI/CD mindset, infrastructure-as-code sensibility, and a bias for simple, observable systems
- Bonus: CDC tooling (Datastream, Debezium), Vertex AI / Feature Store
What the job involves
- You will own and evolve our data pipelines on GCP — building new ones, hardening existing ones, improving data quality, and making clean, trustworthy data available across the organisation
- You'll work end-to-end on streaming and batch pipelines, from CDC and event ingestion through transformation, serving, and the feature layer that powers our ML and AI products
- Your day-to-day will include designing ELT/ETL processes on BigQuery and ClickHouse, building real-time pipelines on Pub/Sub and Kafka with Dataflow (and where it fits, Flink/Spark), orchestrating workflows with Airflow, and ensuring data is properly cleaned, modelled, and served for analytics, ML training, and online inference
- You'll partner with ML engineers on feature pipelines, monitoring data drift, and keeping models well-fed and retrained as needed. You'll consume and build REST APIs, integrate with third-party SaaS sources, and treat infrastructure as code
- You will be part of the Data, Analytics & AI team, collaborating closely with Infrastructure, Software Engineering, Product, and ML/AI engineers
- We're in the middle of a GCP-native modernisation — migrating away from Snowflake toward BigQuery, Bigtable, Pub/Sub, and Dataflow — so we're looking for someone who's opinionated about clean architecture, allergic to over-engineering, and comfortable owning systems end-to-end. If retiring a legacy warehouse and standing up its replacement sounds like a good time, you'll fit right in
- ⚔️ Technology tools & weapons you’ll be using:
- Cloud & warehouse: GCP, BigQuery, Bigtable, Cloud Storage
- Streaming & messaging: Pub/Sub, Kafka
- Processing: Dataflow (Apache Beam), with Flink/Spark where appropriate
- Orchestration: Airflow (Cloud Composer)
- Analytical store: ClickHouse
- Languages: Python, SQL
- Modelling & quality: dbt, data quality gates
- Containers & CI/CD: Docker, Kubernetes, GitHub Actions / equivalent
- Legacy (being retired): Snowflake
- The adventures that await you after becoming Senior Data Engineer at Hack The Box:
- Design and build batch and streaming pipelines on Dataflow, Pub/Sub, and Kafka feeding BigQuery, Bigtable, and ClickHouse
- Help drive the migration off Snowflake onto our GCP-native stack — and retire shadow pipelines along the way
- Own the orchestration layer in Airflow, including SLAs, retries, and data quality gates
- Model data for analytics and for ML — including feature pipelines that serve both training and low-latency online inference
- Partner with ML engineers on feature stores, drift monitoring, and retraining workflows
- Capture requirements from stakeholders and translate them into pragmatic, well-scoped data products
- Continuously improve data quality, reliability, observability, and cost efficiency
- Identify new data sources worth acquiring and integrate them cleanly
Application process
- Level 1: To complete level one’s objective, submit your application
- Level 2: Meet the Talent Acquisition team. Level’s objective: highlight your past achievements, ambitions, and values
- Level 3: Meet the hiring team. Level’s objective: connect with the hiring team and share with them your achievements
- Level 4: Complete 2 assignments that align with day-to-day job-related tasks and responsibilities. Part of the assignment is discussing it with the hiring team in a debriefing session, in order to walk the team through your thinking process
- Level 5: Congratulations! Not many reach this level 💪. Level’s objective: have a constructive, final conversation with senior leadership to explore the role and your future at HTB
- Level 6: You've officially received an offer from HTB! To complete the last level and the Quest, all you need to do is accept the offer
- Quest complete. Congratulations, you’re officially one of us 🥳🎉🎇Your next quest: complete the onboarding
Our take
Cybersecurity attacks have ballooned in number recently, shining a spotlight on the urgent need for tighter, sharper cybersecurity measures. Hack the Box’s response has been to create what they call a ‘massive cyberhacking playground’. It’s a cybersecurity skill platform where B2C users can explore gameified cyber training and penetration testing exercises, engage with the infosec community, and explore careers vacancies. B2B customers can access the same training, with additional administrative features.
All in all, it’s a comprehensive offering that allows Hack the Box to sit at the intersection of two lucrative and growing markets. On the one hand, an increase in people carrying out career pivots towards the tech sector, and pursuing online training opportunities. On the other, an increased business focus on (and investment in) improving cybersecurity postures.
Consequently, Hack the Box has seen substantial growth over the past few years, on the back of considerable funding up to series B. It’s touted as the world’s largest hacking community, with over 3.3 million platform members and a growing number of products, including new training programs such as penetration testing labs and an academy. With demand for these skills rising and demand for online teaching equally strong, Hack the Box is operating within a strong market with good growth potential.
Kirsty
Company Specialist at Welcome to the Jungle
Insights
Company
Funding (last 2 of 4 rounds)
Jan 2023
$55m
SERIES B
Apr 2021
$10.6m
SERIES A
Total funding: $69.5m
Company benefits
- 25 annual leave days (paid)
- Paid paternity/maternity leave
- Dedicated budget for training and professional development, participation in conferences
- State-of-the-art equipment
- Full access to the Hack The Box lab offerings; so you can learn how to hack 😉
- Flexibility in working hours
- Private insurance, Dental & Vision, 401K
Company HQ
Folkestone, UK
Leadership
James Hooker
(Co-Founder & Chief Architect)Worked as a web developer with MSO.net, before taking up a Senior Software Engineer position at Holiday Extras. Alongside this work, also researches and publishes vulnerabilities on open source projects.
Haris Pylarinos
(CEO)Previously Security Engineer at Hellenic Air Force and Systems & Network Engineer at Ibiscus Internet Services and then Naftomar Shipping And Trading Co.
Aris Zikopoulos
(Co-Founder)Has held senior consultancy positions at Altec and Space Hellas, along the way working as head of IT and security services at Velto and channel sales manager at ITWAY Hellas.
Share this job
View 4 more jobs at Hack The Box