Staff Software Engineer, Temporal Technologies

• 10+ years building production systems, including significant experience with distributed systems and correctness-critical infrastructure
• Strong experience with replication, consistency, fault tolerance, and failure recovery in distributed environments
• Demonstrated ability to design and implement concurrent, correctness-critical systems with clear invariants and safety guarantees
• Proven track record of leading complex technical projects across teams—setting direction, driving execution, and landing changes safely in production
• Hands-on experience debugging complex production issues involving race conditions, data consistency, partial failures, and performance degradation
• Proficiency writing production-quality concurrent code, preferably in Go (Java/C++ or similar systems languages also welcome)
• Solid understanding of distributed systems fundamentals such as replication, sharding/partitioning, backpressure, failure detection, and durability mechanisms
• Ability to operate with high ownership and minimal oversight, balancing deep technical rigor with pragmatic delivery
• Curiosity and rigor in understanding how systems behave under stress, failure, and scale

Desirable

• We’re hiring a Staff Software Engineer to join the Replication Foundations team, part of Temporal’s Cloud Global Services (CGS) organization
• Replication Foundations owns and evolves Temporal’s core replication stack in Temporal OSS—the distributed systems backbone behind key Temporal Cloud capabilities like High Availability namespaces, cross-cluster and cross-region failover, and migration products that let customers move workloads between self-hosted Temporal and Temporal Cloud
• The team also drives fundamental scalability and reliability mechanisms that define how Temporal operates at large scale
• At the Staff level, you’ll operate as a technical leader in Temporal’s distributed core: shaping protocols and correctness guarantees, leading complex efforts end-to-end, raising engineering rigor across replication foundations, and partnering cross-functionally to ensure OSS primitives unlock current and future cloud products
• This role is deeply technical and requires comfort reasoning about consistency, concurrency, failure modes, performance, and operational safety in correctness-critical systems
• Evolve replication protocols and correctness mechanisms that make failures (and cloud outages) a non-event for customers
• Deliver scalability primitives (e.g., multi-cell namespaces) that unlock the next phase of Temporal Cloud growth
• Raise the bar on safety, observability, and operability of Temporal’s replication layer across OSS and cloud
• Lead the design and implementation of core components of Temporal’s OSS replication stack, from initial design through rollout and long-term operational ownership
• Design and evolve replication protocols that power:
• High Availability namespaces
• Cross-cluster and cross-region replication
• Migration between Temporal clusters (cloud ↔ self-hosted, cloud ↔ cloud)
• Build scalability and reliability capabilities such as:
• Multi-cell namespaces
• Protocols enabling a single namespace to span multiple clusters
• Dynamic split/merge strategies based on usage, hot spots, and capacity needs
• Reason deeply about correctness: consistency models, ordering guarantees, idempotency, failure recovery, and safe rollouts of protocol changes
• Drive cross-team alignment with Cloud Enablement and other CGS teams to ensure OSS foundations support current and future cloud products
• Author high-quality design docs that clarify invariants, trade-offs, failure modes, and operational playbooks for complex changes
• Raise engineering standards through reviews, mentorship, and technical leadership—improving correctness testing, fault injection, and incident readiness
• Participate in on-call/incident response related to replication and core system behavior, helping build durable fixes and prevention mechanisms