Senior Data Infrastructure Engineer, Dow Jones

• This is a senior individual contributor role for someone who is strongest in data engineering but comfortable operating across AI infrastructure, retrieval systems, cloud architecture, and product delivery
• Strong experience in data engineering or platform engineering in production environments
• Excellent Python skills and solid SQL fundamentals
• Experience building reliable ingestion and transformation pipelines at scale
• Strong understanding of data modeling across structured and unstructured datasets
• Experience with workflow orchestration tools such as Airflow, Dagster, Prefect, Temporal, or equivalent
• Strong cloud engineering experience in AWS, GCP, or Azure, with clear transferability across platforms
• Experience with infrastructure as code and modern deployment practices
• Experience with distributed systems, event-driven patterns, and data-intensive applications
• Familiarity with search, vector, or retrieval systems used in AI-backed products
• Ability to work cross-functionally and act as a technical leader without losing hands-on depth

Desirable

• Storyful is building the next generation of data and AI products on top of complex, high-volume, multi-source content. We are looking for a hands-on Senior Data Platform Engineer to build the technical foundations that make those products scalable, reliable, and ready for production
• You will design and build the ingestion, processing, storage, and serving layers that power future AI and data products across Storyful
• You will work closely with machine learning engineers, software engineers, product managers, and leadership to turn raw structured and unstructured data into trustworthy product capabilities
• Design and build scalable batch and streaming pipelines for structured, semi-structured, and unstructured data
• Own the ingestion and processing architecture for documents, text, metadata, and other content sources
• Build robust data workflows for parsing, chunking, enrichment, indexing, and retrieval
• Create the platform foundations for AI products, including orchestration, data quality, observability, lineage, and cost-aware processing
• Design storage patterns across object stores, relational databases, search/vector systems, and where appropriate graph or knowledge-based systems
• Partner with ML and product engineering to productionise AI features, agentic workflows, and retrieval-backed user experiences
• Define data contracts, schema evolution practices, and quality controls across services and teams
• Improve reliability, freshness, and traceability of pipelines that feed customer-facing products
• Contribute hands-on code while helping set engineering standards and mentoring other engineers
• What good looks like in this role:
• Raw inputs from multiple sources become clean, versioned, monitorable assets that product and ML teams can trust
• New datasets and content types can be onboarded quickly without fragile one-off pipelines
• AI product features are built on observable, debuggable foundations rather than opaque glue code
• Document processing and retrieval quality improve because content is structured well before it reaches the model layer
• The team has clear standards for pipeline reliability, schema management, testing, and deployment