Senior Software Engineer, Instacart

• 5+ years of software engineering focused on data infrastructure or distributed systems at scale, in a high-growth, data-intensive environment
• Experience in modern data lakehouse architectures and open table formats — Apache Iceberg, Delta Lake, Hudi — with strong understanding of compute/storage trade-offs
• Hands-on experience with distributed query and compute systems (Trino, Spark, ClickHouse) including performance tuning and production reliability work
• Proven depth in event-driven infrastructure: Kafka for high-throughput data ingestion and Flink (or equivalent) for stream processing at scale
• Track record owning and executing major platform transitions, including migration design, phased rollout, and risk management under production constraints
• Experience building business cases for infrastructure investments: cost-benefit analysis, TCO modeling, and presenting recommendations to leadership
• Exceptional written technical communication — clear architecture docs, strategy memos, and cross-team proposals that drive decisions and alignment
• Strong ownership and comfort operating in ambiguity; ability to drive large, multi-team initiatives from concept to production with organizational influence

Desirable

• Instacarts Data Governance & Foundations team builds and operates the core systems that power the company's data ecosystem — a modern data lakehouse at scale, spanning ingestion, stream processing, analytical compute, and self-serve tooling
• You'll join a collaborative team of 6–7 engineers responsible for keeping a highly reliable production platform running today while architecting the infrastructure that will serve the business for the next 3–5 years
• This is a high-ownership, high-autonomy role
• Architectural decisions carry both technical and financial weight, and you'll be expected to drive direction, not just execute it
• You'll work closely with engineering leadership and cross-functional partners across Data Science, ML Platform, Ads Infrastructure, Finance Engineering, and senior stakeholders throughout the organization
• Define and drive multi-year architecture roadmaps for large-scale data ingestion and processing infrastructure, setting technical direction that balances reliability, scalability, and cost
• Own end-to-end platform initiatives — from build vs. buy decisions and migration design through production rollout and risk management — across Kafka-based streaming and Postgres-based systems
• Partner with vendors (Snowflake, Databricks, Confluent) on technical integration, contract evaluation, and TCO modeling to inform infrastructure investment decisions
• Collaborate with various teams to embed governance and compliance controls (SOX, CPRA, GDPR) directly into platform architecture and data lifecycle management
• Optimize infrastructure spend at scale: identify cost reduction opportunities across compute, storage, and pipeline efficiency; manage multi-million dollar infrastructure budgets
• Write compelling architecture documents, strategy memos, and proposals that drive alignment with engineering leadership and senior stakeholders across the organization
• Mentor engineers on the team, model strong engineering culture, and help grow a high-performing data infrastructure organization
• Collaborate with Data Science, ML Platform, Ads Infrastructure, Finance Engineering, and Product teams to ensure the platform meets evolving needs