Senior Technical Product Manager, WellSaid Labs
ML Quality
$130-170k
+ Stock Options
Senior and Expert level
Remote from US
AI text-to-speech technology
Job no longer available
AI text-to-speech technology
21-100 employees
B2BArtificial IntelligenceContentDigital MediaSaaSAdvertising
Job no longer available
$130-170k
+ Stock Options
Senior and Expert level
Remote from US
21-100 employees
B2BArtificial IntelligenceContentDigital MediaSaaSAdvertising
Company mission
To empower everyone with AI voice.
Role
Who you are
- An Experienced Technical Product Manager with a Focus on ML & Model Performance
- You are an experienced product manager capable of managing the requirements, timelines, and overall roadmap of technical products in generative AI quality assurance
- You have a strong technical background and a history of collaborating successfully in ML, TTS, or a related field
- More specifically, in your previous role(s) you have been responsible for overseeing the testing and validation of ML models to guarantee their accuracy, reliability, and ethical implementation within a product
- Ideally, you have experience in audio and can manage technical conversations pertaining to audio signal qualities and methods for measuring and identifying them
- You manage product delivery through the R&D flywheel: from research to development, to testing, to delivery, to feedback, and back to research
- You write clear project plans and define requirements in a way that is comprehensive and can be handed off to engineers to design and develop against
- You rely on your research skills to inform your product requirements, test case suites, and testing plans
- You work well with data engineers and analysts to make informed recommendations and evaluation reports
- You are able to draw connections between model limitations and methods for measuring them
- Further, you take the ethics surrounding AI very seriously and work hard to protect all stakeholders in data gathering, research, and exposure
- You are a creative thinker and strong collaborator who enjoys working with engineering teams to build effective solutions with a meaningful impact on our products
- 2+ Machine Learning Product Management experience at a similar-stage company
- 5+ years in ML, TTS, or related fields with strong instincts for identifying market differentiators
- Tenured understanding and experience with ML research, testing, and deployment
- Experience in audio engineering, analysis, or related field
- Strong cross-team collaboration skills, especially collaborating with ML and Data Engineers
- Experience managing multiple projects at once with a track record of predictable, on-time, on-budget delivery
- Clear, direct communication skills (written and verbal)
- Organization skills that demonstrate you are process-oriented
- Strong ability to create order from ambiguity
- Kind, empathetic, and intentional leadership qualities
- Previous ML engineering experience is a plus!
- If you have experience in audio signal processing or linguistic support at the machine learning level, and some of the qualifications above, we strongly encourage you to apply
- Must be a U.S. Citizen or Permanent Resident
- Must pass a pre-employment background check
What the job involves
- As Senior Technical Product Manager, ML Quality, you will manage the development of products and processes that are used to evaluate data, models, and features developed by the ML team
- Your evaluations may result in reports highlighting major model developments to the broader WS community or in guidance for the development team honing various TTS features
- For example, you might ensure the quality and accuracy of new TTS models before they launch to customers through data research, model testing, and regression analysis
- Defining QA strategy for ML models across many languages: Identifying critical metrics and testing methodologies to assess the performance of TTS models throughout the research & development lifecycle. This includes developing such strategies for any new languages we explore in the future
- Evaluating and validating TTS models: Working with the Applied ML team and selecting appropriate evaluation metrics, running rigorous tests across datasets, experiments, and final models, and interpreting results to identify areas for improvement and/or offer recommendations to the Research team
- Managing data quality: Monitoring data quality, identifying potential biases in training data, and ensuring data preprocessing is done correctly to optimize model performance
- Developing QA automation frameworks: Defining automated testing tools and pipelines to streamline the testing and evaluation processes for TTS models, especially for regression testing
- Monitoring model performance in production: Tracking key metrics post-deployment to identify potential degradation in model performance and take corrective action. This may also require defining the monitoring and tooling needed to do so
- Moderating ethical considerations: Assessing potential biases in ML models and training data, ensuring responsible development practices, and addressing concerns regarding fairness, transparency, and Responsible AI
- Researching and partnering with third parties: Defining test cases, scenarios, and evaluation batches with external evaluation services when appropriate. Recommending evaluation services to leadership with a well-defined testing strategy defending your recommendation
- Collaborate closely with data engineers and software engineers in the ML team to understand model capabilities, identify new methods or metrics for gauging model progression, and create new tools to make the testing and evaluation processes fast and easy to implement
- Write design docs with clear requirements and expectations for engineers to execute against
- Research and collate text and audio test cases from real customer data, product feedback, and active experiments
- Synthesize evaluation results into reports for internal and external publication and/or deliver recommendations to the research team informed by rigorous regression analysis
- Partner closely with Lead Product Manager, ML on overall product roadmap, customer impact, and development recommendations
Application process
- An initial “phone” screen with our recruiting team (45 minutes); if there’s a match we’ll invite you to meet further with the team
- An initial meeting with our Lead Product Manager, Machine Learning (45 minutes)
- An Interview loop with 3-4 interviews (1 hour each) with the team members you will be working most closely with
- All interviews will be remote via Google Meets; we are happy to make accommodations you might need to feel set up for success in our process
Our take
WellSaid Labs offers creative teams a real time text-to-speech engine that leverages AI to produce natural-sounding voice overs and clips, all based on the voices of real people.
The most common rebuttal to this kind of technology is how it can be used with malicious intent – deepfakes being the most pertinent concern. This is precisely why WellSaid Labs has no current intention to bring its product to a consumer market – a wise decision. It’s also worth noting that all its voice samples are provided with consent of the original talent.
Provided the pricing point is right, there’s an obvious attraction to being able to quickly create audio content at an enterprise level. And the fact that the startups first major funding round was oversubscribed indicates that investors see this attraction too. It'll be an interesting company to keep an eye on in coming years.
Steph
Company Specialist at Welcome to the Jungle
Insights
Top investors
67% employee growth in 12 months
Company
Funding (1 round)
Jul 2021
$10m
SERIES A
Total funding: $10m
Company HQ
Northlake, Seattle, WA
Leadership
Michael Petrochuk
(Founder & CTO)Previously founded AI fashion advice app Wardrobe.Tips and location based social network Chirpp. Former Developer and Engineer at vCita and Stylize, also interning at Google and Uber.
Brian Cook
(CEO)Founder of Incredible Capital. Previously co-founded and served as the CEO at Hyperfish, Nintex, and OBS.
Share this job