Senior Data Engineer, SpotHero

• We care about your abilities, not how you gained them. You might demonstrate the capabilities below through any combination of relevant professional experience, experience in a research setting, formal education, self-guided learning, open source contributions, or public speaking / writing / teaching experience
• You are able to design and implement high-quality software in Python
• Experience using SQL to read and manage data
• Experience with Airflow, Luigi, Prefect, or other ETL scheduling tools
• You have experience provisioning and managing infrastructure with infrastructure-as-code tools (we use Terraform, but experience with similar tools like CloudFormation, Pulumi, or SaltStack is totally fine!)
• Hands-on experience using multiple data platforms and tools (e.g. Airflow, Hive, Kafka, Postgres, Redshift, S3, Spark, Trino, Airbyte), and experience deploying, monitoring, and maintaining at least one of them
• Experience designing and implementing software (pipelines, services and client libraries) that is run in Docker containers, automatically tested on a continuous integration (CI) system, and versioned in git. You have experience writing shell scripts, Makefiles, or other configuration to glue together these components
• Ability to deploy containerized software in Kubernetes, or sufficient experience in similar technologies like Apache Mesos or Amazon ECS
• Demonstrated experience designing and supporting technology intended to be used by other stakeholders
• Strong ability to communicate on both business and technology subjects

Desirable

• SpotHero is seeking a Senior Data Engineer to join the Data Engineering squad
• This squad interacts with data consumers such as Data Science, Marketing, Engineering, and Business Analysts to provide data platform solutions that meet their day-to-day needs and long term vision
• As a Senior Data Engineer, you’ll focus heavily on backend application development with a focus on building reusable infrastructure services for our stakeholders to enable them to model, store, access, process, and analyze SpotHero’s data
• You’ll also design, instantiate, observe and maintain infrastructure services, both AWS-managed and open source solutions
• As a Senior Data Engineer, you’ll influence the technology choices and patterns established for data-heavy workloads at SpotHero
• Work with our Analytics, Data Science, Marketing and other squads to understand their data storage and processing needs
• Be a hands-on contributor to the design and implementation of our data platform solutions from the infrastructure layer up to the API
• Build robust pipelines that make sure data is where it needs to be, when it needs to be there in a manner that scales
• Build frameworks and tools to help others design and build their own data pipelines in a self-service manner
• Performance testing and engineering to ensure that our systems always scale to meet our needs
• Be a key member of the team focused on hands-on contribution to the implementation and operation of our data platform