Staff Machine Learning Engineer, Fullscript

• 6+ years of experience building and implementing machine learning applications in production, including meaningful experience with LLM-powered agents, conversational experiences, or agent-based workflows
• A track record of owning complex technical problems end to end and shaping implementation beyond your immediate code contributions
• Experience designing and deploying AI systems that answer open-ended questions, support follow-up interactions, and operate reliably in production
• Strong experience with LLM application frameworks and tooling, such as LangChain, LangGraph, or similar orchestration and RAG frameworks
• Familiarity with evaluation and monitoring frameworks for LLM outputs, conversational quality, and system reliability
• Knowledge of MCP, agent orchestration patterns, or related approaches for building multi-step AI systems
• Strong proficiency in Python and SQL
• Experience making sound technical decisions around quality, safety, maintainability, and scalability in production AI systems
• Strong communication and collaboration skills, with the ability to work effectively across technical and non-technical stakeholders

Desirable

• We’re hiring a Staff Machine Learning Engineer to join our AI team and help shape the next generation of Fullscript’s AI-powered experiences
• You’ll work on building innovative AI capabilities that help clinicians provide better services and help patients improve their health
• This is a senior individual contributor role for someone who can go beyond implementation
• In addition to building high-quality systems, you’ll help define technical direction, guide architecture decisions, and identify where AI can create meaningful value in clinical workflows
• You’ll work with a high degree of autonomy and partner closely with engineering, product, analytics, and medical stakeholders to deliver scalable, reliable, and clinically useful AI experiences
• Lead the design, development, and deployment of production, multi-turn LLM-powered features, including summarization tools and clinician-facing conversational agents that support follow-up questions and reasoning over clinical context
• Own backend services in Python that integrate LLM agents with Fullscript’s platform and support reliable production use
• Help define technical direction for prompting, grounding, safety, and orchestration strategies used across clinical AI workflows
• Establish and improve evaluation approaches for LLM outputs, including accuracy, hallucinations, edge cases, and overall feature quality
• Shape engineering patterns for model-related workflows, including testing, CI/CD, observability, and version control
• Partner with medical, product, and engineering teams to identify high-value opportunities for AI and turn them into practical, scalable product capabilities
• Work cross-functionally with engineering, analytics, and medical SMEs to refine requirements and ensure data and system design support clinical use cases
• Provide technical leadership across projects by creating clarity in ambiguous problem spaces, guiding tradeoff decisions, and raising the quality bar for the team
• Stay current with the latest LLM research and emerging AI technologies, and help assess where they can be applied effectively at Fullscript