Principal Data Engineer, Twilio

• 8+ years of software engineering experience, with at least 5+ years focused on large-scale data engineering
• Proficiency in Scala or Java, with hands-on experience building systems using Apache Spark or similar distributed compute engines
• Deep understanding of distributed systems, stream processing, and data pipeline design patterns
• Experience processing large volumes of data using cloud-native and open-source big data tools (e.g., Spark, Kafka, Parquet, Delta Lake)
• Track record of leading technical initiatives and collaborating across engineering and product teams
• Strong problem-solving skills and a bias toward ownership and execution
• Experience with AWS and infrastructure-as-code tools
• Excellent written and verbal communication skills

Desirable

• Join the Segment team as Twilio’s next Principal Data Engineer
• As a Principal Data Engineer, you will lead the technical direction of our core data processing infrastructure
• You’ll be responsible for building and scaling large-scale distributed systems that support Segment’s customer data platform (CDP)
• This role demands deep expertise in data engineering, a strong product sense, and the ability to make high-leverage architectural decisions
• Our systems handle hundreds of thousands of events per second across real-time and batch processing environments
• You'll collaborate closely with cross-functional teams to design and implement scalable data pipelines, high-throughput compute engines, and durable storage solutions that power advanced capabilities like audience segmentation, real-time personalization, and predictive insights
• Architect and implement large-scale data processing systems that power core Segment capabilities
• Design and optimize distributed compute frameworks leveraging Apache Spark, Scala, and cloud-native services
• Drive scalability and performance improvements across our data pipelines, enabling real-time and batch data processing
• Partner with product, infrastructure, and platform teams to build reusable systems that accelerate development across Segment
• Mentor and guide a team of data engineers, setting technical standards and fostering a culture of operational excellence
• Lead end-to-end delivery of major platform features and improvements with measurable business impact
• Ensure systems are designed for observability, maintainability, security, and compliance

• Applications for this role are intended to be accepted until June 1st, 2025, but may change based on business needs