Staff Software Engineer, Instacart

• 5+ years of software engineering experience building and operating data infrastructure or distributed systems at production scale
• Hands-on expertise with modern data lakehouse architectures and open table formats (e.g., Apache Iceberg, Delta Lake, Hudi) and with distributed query/compute engines (e.g., Trino, Spark, ClickHouse), including performance tuning and production reliability
• Experience with event-driven and streaming infrastructure (e.g., Kafka, Flink) for real-time pipelines and serving systems
• Proven ownership of major platform transitions or migrations (build vs. buy, migration design, risk management) delivered to production
• Ability to build cost/benefit and TCO models for infrastructure investments and to drive alignment via clear architecture docs and strategy memos across multiple teams and leadership levels

Desirable

• Instacarts Data Infrastructure organization builds and operates the systems that power our company’s data ecosystem, including a modern open data lakehouse on Apache Iceberg, a multi-engine compute platform for stream and analytical workloads, and self-serve tooling that helps Product, Data Science, ML, Ads, Finance, and engineering teams move fast with data
• We’re looking for a Staff Software Engineer to join our Data Governance and Foundations Team
• In this role, you’ll serve as a senior technical leader owning the architecture and delivery of our open lakehouse foundation, governance and access patterns, and multi-engine compute strategy—balancing today’s reliability with the next three to five years of scale, maturity, and cost efficiency
• You’ll collaborate closely with engineering leadership and stakeholders across Data Science, ML Platform, Ads Infrastructure, Finance Engineering, Product Engineering, and Security
• You’ll operate with a high degree of ownership in a fast-paced environment where architectural decisions have real technical and financial consequences
• If you thrive on complex, high-scale challenges and roll-up-your-sleeves execution, this is a chance to shape the backbone of Instacart’s data platform
• Our stack includes technologies such as Apache Iceberg, Apache Flink, Trino, ClickHouse, Apache Kafka, Apache Spark, Snowflake, Databricks, Confluent, Airflow, dbt, Delta Lake, Scala, Python, Postgres, and AWS
• You’ll join a focused team of 7 engineers that values pragmatism, clarity, and impact
• Translate Instacart’s data strategy (e.g., monetization, federated access, real-time) into an actionable multi-year architecture roadmap; align with leadership while evolving the platform for scale, maturity, and cost efficiency
• Own the open lakehouse foundation: define and deliver unified table formats, storage governance, and a multi-engine compute portfolio (interactive, batch, streaming) that enables portability and prevents lock-in
• Drive real-time and streaming infrastructure for critical use cases (Ads, Fraud, ML): set deployment patterns, SLAs, and operational practices that balance performance, availability, and spend
• Pioneer AI-native data infrastructure engineering by applying LLM/AI tools to the platform lifecycle—accelerating development, automation, observability, and cost optimization—and partnering to embed AI-powered capabilities into the platform
• Elevate engineering excellence: lead architecture reviews, mentor senior/staff engineers, influence hiring, and clearly communicate complex trade-offs to both technical and executive audiences to ensure cross-org alignment