Data Engineer, Lighthouse

• You have experience building and maintaining production data pipelines, with a growing interest in shaping the architecture around them
• You take pride in the software you build, from initial design to production monitoring and continuous improvement
• You have professional proficiency in Python for large-scale data processing and pipeline development
• You have strong knowledge of data cloud technologies and hands-on experience with a major cloud platform like GCP, AWS, or Azure
• You are a forward-thinking builder who uses AI-assisted tools to ship high-quality code faster
• You are fluent in English with excellent communication skills to discuss technical details and explain complex data flows to technical stakeholders
• Not ticking every box? No problem! We value diverse backgrounds and unique skill sets, and we encourage individuals from all walks of life to apply. If your experience looks a little different from what we've described, but you're passionate about what we do and are a quick learner, we'd love to hear from you!

Desirable

• As a Data Engineer, you will be a key player in the end-to-end lifecycle of our large-scale ingestion and transformation engine. You will design, develop, scale, and maintain data pipelines that handle massive data streams, on the order of 100 TB per day.
• This role offers a dynamic mix of advanced greenfield data engineering and improving the stability and quality of our existing pipelines
• Design, build, and deploy scalable data pipelines that ingest data from various sources like APIs, Webhooks, and Cloud Storage
• Implement complex domain modeling and transformation logic to unify disparate data sources into high-fidelity data products
• Contribute to architectural improvements by transitioning legacy processes into modern, scalable, and stable data architectures
• Own the support and troubleshooting of live integrations, from diagnosing hotel and internal issues to communicating clearly with stakeholders during incidents
• Drive operational excellence by improving the reliability and quality of our data pipelines through advanced observability, automated testing, and automation
• Collaborate closely with product, engineering, support, and business teams, as well as customers and partners
• Investigate and resolve complex technical issues by performing deep-dive analysis into our code and systems to identify root causes
• Leverage AI and automation to increase engineering velocity, streamline complex debugging, and handle recurring tasks
• You will join one of our Integration Engineering teams as a data engineer. Our team's mission is to reliably transform, integrate, and store the vast data sources that power our BI products.
• We handle a staggering amount of information, processing over 100TB of data daily and managing a dataset with over 3 trillion hotel rates. You'll be part of a highly talented and collaborative group of about 6-8 data engineers, working together to power Lighthouse's core intelligence