Senior Data Acquisition Engineer, People Data Labs

• 7+ years industry experience with clear examples of strategic technical problem solving and implementation
• Strong software development architecture and fundamentals for backend applications
• Solid understanding of browser rendering pipeline, web application architecture (auth, cookies, http request/response)
• Solid programming experience: strong grasp of object-oriented design and experience building applications using asynchronous programming paradigms (e.g., async/await, event loops, or concurrency libraries)
• Experience building crawlers
• Proficient in Linux / Unix command line utilities, Linux system administration, architecture, and resource management
• Experience evaluating data quality and maintaining consistently high data standards across new feature releases (e.g., consistency, accuracy, validity, completeness)
• Must thrive in a fast paced environment and be able to work independently
• Can work effectively remotely (able to be proactive about managing blockers, proactive on reaching out and asking questions, and participating in team activities)
• Strong written communication skills on Slack/Chat and in documents
• You are experienced in writing data design docs (pipeline design, dataflow, schema design)
• You can scope and breakdown projects, communicate and collaborate progress and blockers effectively with your manager, team, and stakeholders

Desirable

• We are looking for individuals who can balance extreme ownership with a “one-team, one-dream” mindset. Our customers are trying to solve complex problems, and we only help them achieve their goals as a team. Our Data Engineering & Acquisition Team ensures our customers have standardized and high quality data to build upon
• You will be crucial in accelerating our efforts to build standalone data products that enable data teams and independent developers to create innovative solutions at massive scale.
• In this role, you will be working with a team to continuously improve our existing datasets as well as pursuing new ones. If you are looking to be part of a team discovering the next frontier of data-as-a-service (DaaS) with a high level of autonomy and opportunity for direct contributions, this might be the role for you. We like our engineers to be thoughtful, quirky, and willing to fearlessly try new things. Failure is embraced at PDL as long as we continue to learn and grow from it
• Use and develop web crawling technologies to capture and catalog data on the internet
• Support and improve our web crawling infrastructure
• Structure, define, and model captured data, providing semantic data definition and automate data quality monitoring for data that we crawl
• Develop new techniques to increase speed, efficiency, scalability, and reliability of web crawls
• Use big data processing platform to build data pipelines, publish data, and ensure the reliable availability of data that we crawl
• Work with our data product and engineering team to design and implement new data products with captured data, and enhance and improve upon existing products