Data Engineer, People Data Labs

• We are looking for individuals who can balance extreme ownership with a “one-team, one-dream” mindset
• If you are looking to be part of a team discovering the next frontier of data-as-a-service (DaaS) with a high level of autonomy and opportunity for direct contributions, this might be the role for you
• We like our engineers to be thoughtful, quirky, and willing to fearlessly try new things. Failure is embraced at PDL as long as we continue to learn and grow from it
• 4-6+ years of industry experience with clear examples of strategic technical problem-solving and implementation
• Strong software development fundamentals
• Experience with Python
• Expertise with Apache Spark (Java, Scala, and/or Python-based)
• Experience with SQL
• Experience building scalable data processing systems (e.g., cleaning, transformation) from the ground up
• Experience using developer-oriented data pipeline and workflow orchestration (e.g., Airflow (preferred), dbt, dagster or similar)
• Knowledge of modern data design and storage patterns (e.g., incremental updating, partitioning and segmentation, rebuilds and backfills)
• Experience working in Databricks (including delta live tables, data lakehouse patterns, etc.)
• Experience with cloud computing services (AWS (preferred), GCP, Azure or similar)
• Experience with data warehousing (e.g., Databricks, Snowflake, Redshift, BigQuery, or similar)
• Understanding of modern data storage formats and tools (e.g., parquet, ORC, Avro, Delta Lake)
• Balance high ownership and autonomy with a strong ability to collaborate
• Work effectively remotely (able to be proactive about managing blockers, proactive on reaching out and asking questions, and participating in team activities)
• Demonstrate strong written communication skills on Slack/Chat and in documents
• Exhibt experience in writing data design docs (pipeline design, dataflow, schema design)
• Scope and breakdown projects, communicate and collaborate progress and blockers effectively with your manager, team, and stakeholders

Desirable

• People Data Labs (PDL) is the provider of people and company data
• We do the heavy lifting of data collection and standardization so our customers can focus on building and scaling innovative, compliant data solutions
• Our sole focus is on building the best data available by integrating thousands of compliantly sourced datasets into a single, developer-friendly source of truth
• Leading companies across the world use PDL’s workforce data to enrich recruiting platforms, power AI models, create custom audiences, and more
• Our customers are trying to solve complex problems, and we only help them achieve their goals as a team
• Our Data Engineering Team is the secret sauce behind all that we do and we are looking for the best of the best
• Build infrastructure for ingestion, transformation, and loading an exponentially increasing volume of data from a variety of sources using Spark, SQL, AWS, and Databricks
• Building an organic entity resolution framework capable of correctly merging hundreds of billions of individual entities into a number of clean, consumable datasets
• Developing CI/CD pipelines and anomaly detection systems capable of continuously improving the quality of data we're pushing into production
• Dreaming up solutions to largely undefined data engineering and data science problems