Staff Data Infrastructure Engineer, Webflow

• 8+ years of experience as a Data Infrastructure Engineer or related roles like Platform Engineer, SRE, DevOps or Backend Engineer
• Deep expertise in provisioning and managing data infrastructure components like Kafka, Spark, and Airflow
• Extensive experience with cloud services and environments (compute, storage, networking, identity management, infrastructure as code, etc.)
• Strong experience with containerization technologies like Docker and Kubernetes
• Advanced knowledge of infrastructure as code tools like Terraform and Pulumi
• Strong understanding of networking concepts and configurations, including VPCs, load balancers, and endpoints
• Extensive experience with monitoring and logging tools
• Strong problem-solving skills and attention to detail
• Excellent leadership, communication, and mentoring skills

Desirable

• In this pivotal role, you'll lead efforts to build and manage robust, secure, and scalable infrastructure that powers our data operations.
• Your responsibilities will include provisioning, deploying, configuring, managing, and scaling key components such as Kafka, Spark, Airflow, and query engines like Athena.
• You'll work extensively with AWS, containerization technologies, and infrastructure as code tools to ensure our systems run smoothly and reliably.
• Your expertise will be crucial in integrating these components, providing a solid foundation for our data-driven products.
• Additionally, you'll have the opportunity to mentor junior engineers and drive best practices across the team.
• Oversee the provisioning and deployment of infrastructure using Pulumi, ensuring seamless deployment of Kafka, Spark, Airflow, Athena, and other critical systems on AWS
• Design and implement strategies for scaling Airflow, Kafka, and Spark clusters to accommodate increasing workloads and user demands
• Lead efforts in optimizing performance, capacity planning, ensuring fault tolerance, and implementing failure recovery strategies across all infrastructure components
• Configure and manage VPCs, load balancers, and VPC endpoints for secure communication between internal and external services
• Manage IAM roles, apply security patches, plan and execute version upgrades, and ensure compliance with regulations such as GDPR
• Architect and implement high-availability solutions across multiple zones and regions, including backups, multi-region replication, and disaster recovery plans
• Oversee S3 data lake management, including file size management, compaction, encryption, and compression to maximize storage efficiency
• Implement caching strategies, indexing, and query optimization to ensure efficient data retrieval and processing
• Implement monitoring and logging using tools like Datadog, CloudWatch and OpenSearch
• Lead efforts to develop services, tools and automation to simplify infrastructure complexity for other engineering teams, enabling them to focus on building great products
• Participate in all engineering activities including incident response, interviewing, designing and reviewing technical specifications, code review, and releasing new functionality
• Mentor, coach, and inspire a team of engineers of various levels