Staff Software Engineer, Adyen
Distributed Data Systems
Our interview process tends to take about 4 weeks to complete
Payments platform built to grow businesses
Open for applications
Payments platform built to grow businesses
1001+ employees
Open for applications
Our interview process tends to take about 4 weeks to complete
1001+ employees
Company mission
Adyen's mission is to make payments choices easy for merchants.
Role
Who you are
- Scaling and tuning large deployments of Spark-on-k8s and Spark-on-Hadoop
- Hadoop and the HDFS protocol
- Designing and tuning shuffle heavy systems, on yarn, or on k8s via remote shuffle services
- One of the lakehouse file formats (Delta, Iceberg, Hudi)
- OLAP technologies covering at least one of Clickhouse, Apache Druid, Apache Pinot, or Apache Doris
- Open-source contributions in one of the must-have technologies, or other common ones (e.g. Kafka, Cassandra, Trino, etc)
- Team player with strong communication skills
- Ability to work closely with diverse stakeholders you enable (analysts, data scientists, data engineers, etc.) and depend upon (infrastructure, security, etc)
- Demonstrated ability to troubleshoot and resolve issues in large-scale, production environments with distributed systems
Desirable
- Next generation and multi-modal data formats (e.g. LanceDB)
- Building self-service stateful platforms
- Open-source S3 alternatives (e.g. ceph, minio, etc)
- Native or accelerated runtimes for Spark (Apache DataFusion Comet, Apache Gluten, Nvidia RAPIDS, etc)
What the job involves
- We’re looking for an expert with deep knowledge in distributed systems, to both improve operations/scalability of existing offerings, as well as introduce and mature new ones
- The initial focus will be scaling and tuning our Hadoop and Spark infrastructure, in addition to iterating on our OLAP platform, where we currently use Apache Druid
- Over time, we expect this expertise to be useful in a wider range of distributed systems internally
- You’ll be asked to both build new as-a-service offerings, and improve the existing ones. This role is perfect for you if you're passionate about one or all of the following:
- Design, build, and optimize data clusters to ensure scalability, fault tolerance, and high availability. Covering both batch and streaming workloads
- Focus on improving the Spark, Hadoop, Kubernetes, Deltalake, and Druid ecosystems internally
- Evolve these open-source projects internally, with the intention of contributing this code upstream
- Educate and grow Adyen’s internal knowledge in these topics. Covering both peer platform engineers, and the platform users
- Adyen hosts a significant footprint in both scale and variety of distributed systems
- This covers distributed compute (Spark, Trino, Flink), distributed databases (Cassandra, Druid), and distributed file/object storage (HDFS, Ceph Object Gateway)
- These technologies are offered as-a-service internally, towards product engineering, to support them in building and scaling world-class products
Our take
Adyen has been one of the most successful businesses in Europe over the last decade. When it went public in 2018 at a valuation of €7bn, it raised over €1bn and its share price popped on the first day, resulting in a valuation of €14bn. Its valuation has continued to climb by the billions.
Adyen competes with large players like PayPal, as well as outsized startups like Stripe, to power payments and provide other infrastructure to conduct digital transactions. Disrupters in the field have been buoyed by a strong rise in eCommerce activity, where some or all of a transaction by a customer is made either online or by mobile.
Adyen has been one of the companies riding the wave by helping to reduce the friction between a company choosing to take payments online, and actually being able to do it. That can take multiple steps and agreements across numerous countries — Adyen’s pitch is that it essentially handles all of it in the backend as a service for its users. Notable partners have included Autocanteen, a touchless self-checkout solution for the hospitality industry, and BILL, a small business-focused fintech platform.
Steph
Company Specialist at Welcome to the Jungle
Insights
36% employee growth in 12 months
Company
Funding (last 2 of 3 rounds)
Sep 2015
$57.4m
GROWTH EQUITY VC
Dec 2014
$250m
SERIES B
Total funding: $325.8m
Company benefits
- Global exchange program
- Weekly happy hour
- Delicious healthy lunches
- Phantom share package
- Yearly trip to Amsterdam
- Paid holidays
- Work from home opportunities
Company HQ
Nieuwmarkt en Lastage, Amsterdam, Netherlands
Leadership
Pieter Van Der Does
(Co-Founder & CEO)Previously Chief Commercial Officer at Bibit (payments company) before co-founding Adyen in 2006.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 122 more jobs at Adyen