Senior Data Engineer, AllTrails

• Minimum of 5 years of experience working in data engineering
• Expertise both in using SQL and Python for data cleansing, transformation, modeling, pipelining, etc
• Proficient in working with other stakeholders and converting requirements into detailed technical specifications; owning and leading projects from inception to completion
• Proficiency in working with high volume datasets in SQL-based warehouses such as BigQuery
• Proficiency with parallelized python-based data processing frameworks such as Google Dataflow (Apache Beam), Apache Spark, etc
• Experience using ELT tools like Dataform or dbt
• Professional experience maintaining data systems in GCP and AWS
• Deep understanding of data modeling, access, storage, caching, replication, and optimization techniques
• Experienced with orchestrating data pipelines and Kubernetes-based jobs with Apache Airflow
• Understanding of the software development lifecycle and CI/CD
• Monitoring and metrics-gathering (e.g. Datadog, NewRelic, Cloudwatch, etc)
• Willingness to participate in a weekly on-call support rotation - currently the rotation is monthly
• Proficiency with git and working collaboratively in a shared codebase
• Excellent documentation skills
• Self motivation and a deep sense of pride in your work
• Passion for the outdoors
• Comfort with ambiguity, and an instinct for moving quickly
• Humility, empathy and open-mindedness - no egos

Desirable

• Work cross-functionally to ensure data scientists have access to clean, reliable, and secure data, the backbone for new algorithmic product features
• Build, deploy, and orchestrate large-scale batch and stream data pipelines to transform and move data to/from our data warehouse and other systems
• Deliver scalable, testable, maintainable, and high-quality code
• Investigate, test-for, monitor, and alert on inconsistencies in our data, data systems, or processing costs
• Create tools to improve data and model discoverability and documentation
• Ensure data collection and storage adheres to GDPR and other privacy and legal compliance requirements
• Uphold best data-quality standards and practices, promoting such knowledge throughout the organization
• Deploy and build systems that enable machine learning and artificial intelligence product solutions
• Mentoring others on best industry practices