Head of Machine Learning Infrastructure, Hippocratic AI
Office located in Palo Alto, CA
Large Language Model for healthcare
Open for applications
Large Language Model for healthcare
21-100 employees
Open for applications
Office located in Palo Alto, CA
21-100 employees
Company mission
Hippocratic AI’s mission is to develop the first safest focused Large Language Model (LLM) for healthcare.
Role
Who you are
- The ideal candidate will have deep expertise in infrastructure orchestration, multi-cloud environments, and tools such as Kubernetes and Terraform
- Proven experience in building and managing ML infrastructure platforms, particularly for LLMs or other advanced AI systems
- Expertise in Kubernetes, Terraform, and other IAC tools
- Deep understanding of multi-cloud architectures (e.g., AWS, Azure, Google Cloud) and hybrid cloud solutions
- Strong programming skills in Python, Go, or a similar language, with experience in building automation and orchestration tools
- Familiarity with modern ML frameworks and tools (e.g., TensorFlow, PyTorch, Hugging Face)
- Demonstrated success in leading infrastructure teams and managing large-scale projects
- Excellent problem-solving and decision-making skills
- Strong communication skills, with the ability to convey complex technical ideas to non-technical stakeholders
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field (or equivalent work experience)
- 8+ years of experience in infrastructure engineering, with at least 3 years in a leadership
What the job involves
- We are seeking a highly skilled and innovative Head of ML Infrastructure to lead the design, development, and operation of our orchestration platform for a heterogeneous constellation of Large Language Models (LLMs)
- This role is critical to ensuring that our AI systems are scalable, reliable, and seamlessly integrated into our broader technology ecosystem
- Architect and implement an advanced orchestration platform to manage a diverse set of LLMs efficiently
- Design solutions to optimize performance, scalability, and availability across various deployment environments
- Utilize Kubernetes, Terraform, and other Infrastructure as Code (IAC) tools to automate and manage ML infrastructure
- Collaborate with DevOps and cloud engineering teams to ensure seamless integration with CI/CD pipelines
- Establish robust monitoring, logging, and alerting systems for ML infrastructure
- Design and execute strategies to leverage multiple cloud providers for cost optimization, redundancy, and compliance
- Manage cloud-native services to support model deployment and orchestration at scale
- Work closely with ML engineers to fine-tune model deployment strategies, focusing on latency, throughput, and fault tolerance
- Conduct capacity planning and develop tools for model lifecycle management. Leadership & Collaboration:
- Lead a team of infrastructure engineers, fostering a culture of innovation, collaboration, and excellence
- Act as a bridge between ML research, engineering, and operations teams to align infrastructure capabilities with business needs
- Stay abreast of emerging technologies and methodologies in ML infrastructure and orchestration
Our take
The healthcare system is under increasing pressure to improve performance and patient outcomes, while battling inefficient operations and an ongoing staff shortage. Hippocratic AI hopes to help the industry address these issues, by supplying LLM-powered healthcare agents.
Hippocratic's agents can be employed in a number of scenarios, for example follow-ups with discharged patients, health risk assessments, or pre-op chats. Rather than delivering one conversational agent, the company has specified models that deal with different conditions - just one part of its safety-focussed approach.
By taking measures such as getting real nurses to assess its AI, Hippocratic solidifies this commitment to safety. The company has seen significant investment which it will use to fund further safety tests and develop its product moving forwards.
Kirsty
Company Specialist at Welcome to the Jungle
Company
Funding (last 2 of 5 rounds)
Jan 2025
$141m
SERIES B
Sep 2024
$17m
SERIES A
Total funding: $276m
Company HQ
University South, Palo Alto, CA
Leadership
Munjal Shah
(CEO)Formerly CEO and Co-founder of Health IQ. Also co-founded Like.com, which was acquired by Google, and Andale Inc that was acquired by Vendio and then Alibaba.
Vishal Parikh
(CPO)Co-founded Health IQ, where they worked as CTO, CRO then CPO. Founded and was a Software Architect at Motiv Power Systems.
Meenesh Bhimani MD, MHA
(Chief Medical Officer)Former COO of El Camino Health. Has worked in several roles at O'Connor Hospital, most recently Hospital Executive/COO. Earned their MD at The Johns Hopkins University School of Medicine.
Subhabrata Mukherjee
(Chief Scientific Officer)Was Principal Researcher at Microsoft Research (MSR) Redmond for 4 years, and before that a Machine Learning Scientist at Amazon. Earned a PhD from the Max Planck Institute for Informatics.
Alex Miller
(COO)Previously Co-founder and SVP of Engineering and Sales Operations at Health IQ. Also co-founded and was CTO at Jump Gaming.
Saad Godil
(CTO)Previously spent 15 years at NVIDIA, as Director of Verification (GPU) then Director of Applied Deep Learning Research.
Kim Parikh
(SVP - Data & Content)Former VP of Strategy and Operations at Health IQ. Also worked in High Yield and Leveraged Finance at PIMCO, and Derivatives And Leveraged Finance at Western Asset Management.
Debajyoti Datta
(AI Research Scientist)Earned a PhD in Machine Learning from the University of Virginia. Has worked as a Machine Learning Engineer for x.ai, Research Engineer for Crossing Minds and Machine Learning Fellow at Intuition Machines.
Paul Gamble
(Director of Clinical AI)Was a Clinician Scientist at Google for 3.5 years, and prior to this a Data Scientist at In-Q-Tel.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 21 more jobs at Hippocratic AI