Senior AI Infrastructure Engineer, SWORD Health
Europe
€66.5-104.5k
+ Variable + Equity Share
More information about location
Connecting physical therapists with technology
Be an early applicant
Connecting physical therapists with technology
1001+ employees
Be an early applicant
€66.5-104.5k
+ Variable + Equity Share
More information about location
1001+ employees
Company mission
To free 2 billion people from pain.
Role
Who you are
- If you're passionate about pushing the boundaries of AI infrastructure performance and want to do it in a mission-driven environment where your work directly improves people's health outcomes, we'd love to have you on our team
- 5+ years of experience in infrastructure engineering, with at least 2 years focused on AI/ML workloads in production environments
- Strong experience with Kubernetes for orchestrating GPU-accelerated workloads, including scheduling, resource management, and autoscaling for inference services
- Hands-on experience with model serving and inference optimization frameworks for both real-time computer vision and large language model workloads
- Solid understanding of LLM inference optimization techniques, including speculative decoding, batching strategies, quantization, and inference scaling patterns
- Experience provisioning and managing infrastructure for real-time AI systems, including WebRTC clusters and AI agent architectures
- Familiarity with real-time video/computer vision inference pipelines and the infrastructure challenges of processing continuous visual data streams at low latency
- Familiarity with speech-to-text and text-to-speech serving infrastructure and the challenges of running voice AI at low latency
- Experience with Infrastructure as Code (Terraform or similar) and GitOps methodologies for managing complex, GPU-enabled environments
- Working knowledge of GPU infrastructure - NVIDIA CUDA ecosystem, multi-GPU setups, and GPU monitoring/profiling
- Strong Linux systems fundamentals and networking knowledge, particularly for latency-sensitive, real-time workloads
- Fluent in English (written and oral)
- A proactive, ownership-driven mindset - you see a bottleneck in an inference pipeline and you fix it before it becomes a problem
- Experience with LLM serving engines such as vLLM, SGLang, or LLM-D
- Experience with NVIDIA Triton Inference Server and TensorRT for real-time computer vision workloads
- Familiarity with NVIDIA Riva or similar platforms for STT/TTS serving
- Understanding of speculative decoding, continuous batching, quantization, and model parallelism techniques
- Experience with Istio or similar service mesh
- Experience with Kafka for event streaming
- Experience with Prometheus, AlertManager, and Grafana for monitoring and observability
- Experience with Elasticsearch, Logstash, and Kibana (ELK) for log management
- Experience with Vault for secrets management
- Experience with Redis, MySQL, and DNS management
- Experience provisioning infrastructure on AWS, Azure, or GCP
- Good knowledge of cloud networking including VPC management, routing, NAT, and troubleshooting with tools like TCPdump
- Experience with WebRTC infrastructure and real-time media streaming
- Experience with Python, Go, or similar languages commonly used in ML infrastructure tooling
- Familiarity with SCRUM methodology
What the job involves
- As a Senior AI Infrastructure Engineer at Sword Health, you will own the infrastructure that brings our AI models to life in production
- From optimizing LLM inference and deploying real-time voice AI agents to scaling GPU clusters that serve millions of sessions, your work will directly power the AI Care platform that is transforming healthcare worldwide
- You will sit at the intersection of ML and infrastructure - designing systems that power real-time computer vision for movement analysis, serve large language models for conversational AI, and enable low-latency voice interactions for AI agents
- You'll ensure our models run at the speed and scale our members expect
- This is not a traditional DevOps role; you'll be deeply embedded in AI-specific challenges like inference optimization, real-time video processing, model serving at scale, and GPU workload orchestration
- Design, build, and maintain the inference infrastructure that powers Sword Health's AI products, ensuring models are served with high throughput, low latency, and cost efficiency
- Own the end-to-end deployment pipeline for AI models - from real-time computer vision powering movement analysis to large language models driving conversational AI experiences
- Architect and scale Kubernetes clusters for GPU-accelerated workloads, including autoscaling strategies, resource scheduling, and multi-model serving
- Build and operate the infrastructure behind Sword Health's real-time AI agents, including WebRTC cluster provisioning and deploying speech-to-text and text-to-speech capabilities at low latency
- Drive inference scaling strategies - evaluate and implement techniques such as speculative decoding, continuous batching, and model parallelism to meet growing demand without proportionally increasing costs
- Develop and maintain Infrastructure as Code (Terraform) and GitOps workflows tailored to GPU-enabled, AI-specific environments
- Instrument and monitor AI inference systems, building observability around GPU utilization, model latency, throughput, and error rates to ensure reliability and performance
- Collaborate closely with ML Engineers, Data Scientists, and Product teams to translate model requirements into robust, production-ready infrastructure
- Evaluate emerging AI infrastructure tools, frameworks, and hardware to keep Sword Health at the cutting edge of inference performance and efficiency
- Mentor team members on AI infrastructure best practices, fostering knowledge sharing around GPU workloads, model serving patterns, and production ML systems
Our take
For a number of reasons including cost, only about 30% of patients complete their physical therapy regimens. This results in millions of people not receiving the full breadth of care they need to reach full recovery. And physical therapy practices are missing out on a massive amount of revenue potential.
SWORD Health’s clinical-grade virtual therapy platform pairs physical therapists with FDA-listed wearable technology to deliver personalised treatment plans that are more effective, easier and less expensive than traditional physical therapy. This encourages patients to complete their regimes and allows practices to bolster their income.
The company works with international insurers, health systems and employers, with clients from the Fortune 100 and 500 lists. It is the fastest-growing MSK company and a double-Unicorn, now valued at $4 billion. With a rapidly growing US market, it's using impressive funding to continue its foundational AI model development and expand into ever more areas of healthcare.
Steph
Company Specialist at Welcome to the Jungle
Insights
37% employee growth in 12 months
Company
Funding (last 2 of 10 rounds)
Jun 2025
$40m
LATE VC
Jun 2024
$30m
LATE VC
Total funding: $367.6m
Company benefits
- A stimulating, fast-paced environment with lots of room for creativity
- A bright future at a promising high-tech startup company
- Career development and growth
- The opportunity to work with a talented team and to add real value to an innovative solution with the potential to change the future of healthcare
- A stimulating environment with room for creativity
- A flexible environment where you can control your hours (remotely) with unlimited vacation
- Access to our health and well-being program (digital therapist sessions)
- Remote or Hybrid work policy
Company HQ
Chelsea, New York, NY
Leadership
Virgílio Bento
(Co-Founder & CEO)Co-Founder of EX Capital. Received their PhD in Electrical Engineering from the Universidade de Aveiro.
Márcio Colunas
(Co-Founder & CSO)Co-founded EX Capital and Findster Technologies. Previously worked as an Analyst/Software Engineer at Neuroinova, and was a Researcher at the Universidade de Aveiro.
Share this job
View 17 more jobs at SWORD Health