Staff AI Platform Software Engineer, Snorkel AI

• Bachelor’s or Master’s degree in Computer Science, Machine Learning, or a related field
• 8+ years of experience with a good portion of them in AI development, including hands-on work with AI in production systems
• Strong expertise in Python and deep learning frameworks such as PyTorch
• Proven experience leading technical projects and mentoring engineers
• Proficiency with CI/CD pipelines for machine learning workflows
• Deep understanding of LLM architectures, fine-tuning, and deployment methodologies
• Strong communication skills, with an emphasis on scalable and reliable system design

Desirable

• As a Staff AI Platform Engineer, you will play a pivotal role in designing and implementing the core AI pipelines that drive SnorkelFlow’s AI capabilities
• You will act as a technical leader, setting architectural direction, mentoring team members, and collaborating across verticals to ensure the AI Platform aligns with product goals and organizational strategy
• This role requires hands-on experience with large language models (LLMs), requiring deep expertise in AI development, especially in designing and deploying systems that leverage LLMs and more traditional NLP solutions for production workflows
• Technical Leadership
• Provide technical direction for the design and development of AI pipelines, ensuring scalability, robustness, and extensibility
• Serve as a mentor and guide for engineers on the AI Platform team, fostering growth and technical excellence
• Identify and drive high-impact projects aligned with business objectives and product needs
• AI Pipeline Development
• Architect, design, and maintain AI pipelines for labeling, embeddings, training, and deploying models into production
• Lead the development and optimization of MLFlow pipelines for deployment
• Build and deploy foundational models that serve as the backbone for SnorkelFlow’s core product capabilities
• Partner with the Compute Platform team to ensure seamless integration with orchestration tools and infrastructure
• LLM Tooling development:
• Develop and deploy LLM-based systems for production workflows, focusing on efficiency, scalability, and reproducibility
• Create AI training framework pipelines that will allow LLM usage in applications, including fine-tuning, pruning, distillation, and foundational model training
• Integrate APIs from providers such as OpenAI, Anthropic, and Gemini into SnorkelFlow’s pipelines
• Backend Development
• Oversee the integration of backend services for managing LLM calls and API interactions
• Collaborate with the Data Platform team to define data requirements and ensure smooth interoperability
• Work with the Application team to design and implement APIs that power application workflows
• Observability and Monitoring
• Establish observability standards for AI pipelines, including tools and dashboards for monitoring model performance and debugging
• Define key metrics for system health and optimization
• Cross-Functional Collaboration
• Act as a thought leader, collaborating with Data Platform, Compute Platform, Application, Product and other internal teams to deliver cohesive, scalable solutions
• Partner with stakeholders to translate product goals into technical roadmaps and