Data Engineer, Beehiiv

• You will be a great fit if you have a well thought-out technical perspective that you balance with a practical desire to ship to customers
• We’re seeking a self-motivated teammate who communicates early and often, and strives for the whole team to succeed
• 3–5 years of professional experience in data engineering or a related role
• SQL expertise
• Have worked with and managed production OLAP databases to power internal and/or external real time analytics
• Have worked with Kafka before in capacities of both managing a cluster, and in creating real time streaming applications
• Are proficient in either, Golang, Python, Ruby, or a JVM language
• Have experience with production deployments of services using the above languages that support a RESTful API
• Have experience debugging, scaling, and troubleshooting issues
• Are willing to learn Ruby
• Solid understanding of data modeling, relational databases, and NoSQL systems
• Excellent problem-solving skills and the ability to work both independently and as part of a team
• Strong troubleshooting skills

Desirable

• We are looking for someone to help build out our nascent ad network, removing manual processes, improving analytics and performance and generally proving out our startups biggest bet
• Reporting to: Ads Software Engineering lead
• Creating a new service to abstract database and data modeling complexity away from the main beehiiv codebase
• Creating, maintaining, and cleaning schema definitions for Clickhouse
• Building out unit and integration testing suites and frameworks for the above schema
• Creating internal tooling to manage Avro Schema Migrations
• Creating real time streaming services
• Launching a Lake House architecture for both internal and external customer analytics
• Load testing our system
• Building CI/CD
• Creating an overarching data model and dictionary

• Applied
• Hired