Senior Software Engineer, Cohere
GPU Infrastructure, High Performance Computing
CA$285-340k
USA - $200,000-$285,000 depending on location + Equity
More information about location
Enterprise-grade AI solutions for real-world business problems
Open for applications
Enterprise-grade AI solutions for real-world business problems
501-1000 employees
Open for applications
CA$285-340k
USA - $200,000-$285,000 depending on location + Equity
More information about location
501-1000 employees
Company mission
To empower businesses to scale innovation, boost productivity, and drive progress that reaches everyone.
Role
Who you are
- Deep expertise in ML/HPC infrastructure: Experience with GPU/TPU clusters, distributed training frameworks (JAX, PyTorch, TensorFlow), and high-performance computing (HPC) environments
- Kubernetes at scale: Proven ability to deploy, manage, and troubleshoot cloud-native Kubernetes clusters for AI workloads
- Strong programming skills: Proficiency in Python (for ML tooling) and Go (for systems engineering), with a preference for open-source contributions over reinventing solutions
- Low-level systems knowledge: Familiarity with Linux internals, RDMA networking, and performance optimization for ML workloads
- Research collaboration experience: A track record of working closely with AI researchers or ML engineers to solve infrastructure challenges
- Self-directed problem-solving: The ability to identify bottlenecks, propose solutions, and drive impact in a fast-paced environment
- If some of the above doesn’t line up perfectly with your experience, we still encourage you to apply!
What the job involves
- The internal infrastructure team is responsible for building world-class infrastructure and tools used to train, evaluate and serve Cohere's foundational models
- By joining our team, you will work in close collaboration with AI researchers to support their AI workload needs on the cutting edge, with a strong focus on stability, scalability, and observability
- You will be responsible for building and operating superclusters across multiple clouds
- Your work will directly accelerate the development of industry-leading AI models that power Cohere's platform North
- Build and scale ML-optimized HPC infrastructure: Deploy and manage Kubernetes-based GPU/TPU superclusters across multiple clouds, ensuring high throughput and low-latency performance for AI workloads
- Optimize for AI/ML training: Collaborate with cloud providers to fine-tune infrastructure for cost efficiency, reliability, and performance, leveraging technologies like RDMA, NCCL, and high-speed interconnects
- Troubleshoot and resolve complex issues: Proactively identify and resolve infrastructure bottlenecks, performance degradation, and system failures to ensure minimal disruption to AI/ML workflows
- Enable researchers with self-service tools: Design intuitive interfaces and workflows that allow researchers to monitor, debug, and optimize their training jobs independently
- Drive innovation in ML infrastructure: Work closely with AI researchers to understand emerging needs (e.g., JAX, PyTorch, distributed training) and translate them into robust, scalable infrastructure solutions
- Champion best practices: Advocate for observability, automation, and infrastructure-as-code (IaC) across the organization, ensuring systems are maintainable and resilient
- Mentorship and collaboration: Share expertise through code reviews, documentation, and cross-team collaboration, fostering a culture of knowledge transfer and engineering excellence
Our take
Many businesses are now faced with the challenge of adopting advanced AI safely and securely. While AI tools like ChatGPT are making headlines, companies (especially those in regulated industries such as banking, telecoms, or government services) worry about privacy, compliance, and control over their sensitive data. Cohere addresses these concerns by offering AI solutions that can be trained on a company's internal data and, importantly, run on their own infrastructure rather than relying on external cloud providers. And in a world where data breaches and regulatory scrutiny are constant threats, this focus on trust and security is key.
In a competitive landscape dominated by giants like OpenAI, Google, and Elon Musk's xAI, Cohere is setting itself apart by targeting businesses exclusively. Its approach has earned it contracts with some big names, like Royal Bank of Canada, Fujitsu, LG, Oracle, and Dell. By focusing on reliability and enterprise adoption rather than viral consumer appeal, Cohere has carved out a strong niche for itself.
This isn't going unnoticed either, as the company recently raised $500M, boosting its valuation to $6.8B, and brought on board top talent from Meta and Uber to strengthen its leadership team. Alongside this, it's doubled its annual recurring revenue to $100M in 2025 and is aiming to hit $200M by year-end. With its North platform for AI agents and a lean cost structure compared with larger rivals, Cohere is very much ready for more growth.
Steph
Company Specialist at Welcome to the Jungle
Insights
192% employee growth in 12 months
Company
Funding (last 2 of 8 rounds)
Apr 2026
$600m
SERIES E
Sep 2025
$100m
LATE VC
Total funding: $2.1bn
Company benefits
- Six weeks’ paid vacation
- Equity / stock options
- RRSP, 401(k), and Pension Scheme contributions
- Coverage for 100% of your insurance premiums across health, dental, vision, and travel
- Additional coverage for accessing mental health providers/services
- Six months of fully paid parental leave, including adoption and surrogacy
- Financial support for egg freezing and IVF in Canada and the UK
- A monthly fitness and wellness allowance
- Globally dispersed company that supports a remote work culture
- A $2,000 annual education benefit for professional development
- A weekly stipend for meals when working remotely and catered lunch when working from one of our global offices
- A monthly arts and culture allowance
- A monthly quality time allowance
Company HQ
Grange Park, Toronto, ON
Leadership
Aidan Gomez
(Co-Founder & CEO)Aidan received a PhD in Computer Science from the University of Oxford in 2024 and has served as Cohere's CEO since co-founding the company in 2019.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 81 more jobs at Cohere