Senior Staff Data Engineer, Warner Bros Discovery

• BS/MS in Computer Science, Engineering, or related field (or equivalent industry experience)
• 10+ years building and operating large-scale distributed data systems
• Deep experience with:
• Streaming and batch data architectures
• Experiment telemetry systems
• Data modelling for analytics and decisioning
• Hands-on experience with modern data stack technologies (e.g., Spark, Airflow, Redshift, Databricks, Snowflake, Delta Lake, etc.)
• Strong programming mindset with a focus on building and enforcing standards that ensure code is maintainable, readable, and extensible over time
• Strong communication skills and ability to partner across DS, Engineering, Product, and Analytics

Desirable

• We are seeking an exceptional Senior Staff Data Engineer to lead the design, development, and scaling of the data and platform systems that power our experimentation, adaptive optimisation, and automated decisioning ecosystem
• This is a high-impact, hands-on technical leadership role that will shape how experimentation data is collected, processed, served, and operationalised across millions of users worldwide
• This role will act as a force multiplier for all Labs initiatives and, in particular, will support the development and productionization of the new Canvas Optimisation system
• As a senior technical leader, you will define the architecture and long-term strategy for experimentation data infrastructure, ensure reliable and cost-efficient data pipelines, and partner closely with Data Science, Engineering, Product, and Analytics teams to scale our platform from hundreds to thousands of concurrent experiments and bandits
• This role would be tasked with reducing Labs data processing costs by ~25% over the year through architectural optimisation, storage strategy improvements, compute efficiency, and intelligent data lifecycle management
• Scale Experimentation Data Platforms:
• Architect and lead the design of scalable, reliable data pipelines supporting large-scale A/B testing, multivariate testing, and adaptive experimentation
• Build and maintain systems that support real-time and batch experiment telemetry ingestion, feature logging, exposure tracking, and outcome measurement
• Design data models and storage strategies optimised for:
• Experiment analysis latency
• Cost efficiency
• Long-term reproducibility
• Governance and auditability
• Enable production-grade pipelines for statistical methods such as CUPED, regression adjustment, and other variance-reduction workflows (in partnership with Data Science)
• Enable Adaptive & Bandit Systems at Scale:
• Build data infrastructure that supports multi-armed bandit decisioning systems, including:
• Low-latency reward signal pipelines
• Feature and context streaming
• Policy logging and replay data stores
• Partner with scientists to productionize bandit frameworks (e.g., Thompson Sampling, epsilon-greedy, UCB) via reliable data services and APIs
• Design systems enabling off-policy evaluation (OPE), replay simulation datasets, and long-term policy evaluation storage
• Canvas Optimization & Personalization Infrastructure:
• Lead data system architecture supporting the Canvas Optimisation platform, including:
• Artwork / creative performance telemetry
• Impression → engagement attribution pipelines
• Near real-time reward computation
• Global rollout observability and monitoring
• Ensure high availability, correctness, and explainability of decisioning-support data feeds
• Cost Optimisation & Efficiency Leadership:
• Drive initiatives to reduce overall Labs data processing costs by ~25%, including:
• Query and job optimisation across compute platforms (Databricks, Spark, etc.)
• Storage tiering and retention policy optimisation
• Data compaction, partitioning, and indexing strategies
• Eliminating redundant or low-value pipeline stages
• Establish cost observability dashboards and cost-to-value monitoring frameworks
• Platform, Reliability & Developer Experience:
• Build reusable data services, libraries, and APIs that:
• Simplify experiment onboarding
• Standardize telemetry schemas
• Enable self-service data access
• Define and enforce SLAs for critical experimentation and decisioning datasets
• Lead data quality frameworks including anomaly detection, reconciliation, and automated validation
• Technical Leadership & Mentorship:
• Serve as a technical thought leader across data platform architecture for experimentation and optimization
• Mentor data engineers and platform engineers on distributed systems design, pipeline reliability, and performance optimization
• Influence cross-org roadmaps spanning experimentation science, platform engineering, and product personalization systems