Machine Learning Research Engineer, Twelve Labs
AI-enabled video understanding technology
Be an early applicant
AI-enabled video understanding technology
21-100 employees
Be an early applicant
21-100 employees
Company mission
To build the best video understanding platform driven by leading perceptual-reasoning research
Role
Who you are
- 10+ years of industry experience (or 7+ with a PhD in a related technical domain)
- A PhD, or a Master's degree, in machine learning or a closely related discipline
- Led teams of 5+ engineers as a technical lead, or formally managed engineering teams comprised of both junior and senior engineers
- Published research/engineering work on LLMs, VLMs, video models, or contrastive multimodal models in top-tier AI conferences such as NeurIPS, ICML, ICLR, etc., or have scaled distributed foundation model data acquisition, training, inference, evaluation, etc
- Expertise optimizing model inference with TensorRT, ONNX, Triton Inference Server, or directly related technologies
- Built Kubernetes-based systems for distributed data/ML workflows or worked extensively with HPC tools such as Slurm
- A passion for, and experience in, both ML modeling and ML/AI systems software engineering
- Strong Python expertise and considerable prior work history with at least one statically typed language (we use Golang)
- An applied bent / are not a pure theoretician: we are an applied science and engineering group at an applied science and engineering company!
- Strong communication skills in written and spoken English
What the job involves
- As an ML Research Engineer at Twelve Labs, you will drive our applied efforts in video embedding and retrieval, multimodal language modeling, and intelligent agents
- You will collaborate closely with other engineers and scientists to build the next generation of Twelve Labs models, services, and infra
- Scaling our models, data, and training + inference platform, while improving the reliability of our core systems, is the essence of the role
- This role is a perfect fit for research minded engineers who want to build SOTA video, vision, and video-language modeling systems!
- Deliver top-notch applied research solutions to problems like VLM finetuning, auto-labeling of video-text datasets, and model-based filtering of said datasets to optimize (end-)model performance
- Collaborate with our science org to optimize the (e.g.) training/inference performance of our core model stack
- Define a systematic prompt generation and selection strategy for our flagship VLM
- Mentor junior engineers/researchers, and hold a high bar around code quality / engineering best practices
- Lead by example in interviewing, hiring, and onboarding passionate and empathetic engineers
- Work across teams to understand and manage project priorities and product deliverables, evaluate trade-offs, and drive technical initiatives from ideation to execution to shipment
- Advance our industry-leading enterprise video solutions by incorporating already-great research into fault tolerant, low latency e2e systems
Application process
- 1) Recruiter Phone Screen
- 2) Initial Technical Assessment
- 3) Final round technical assessment & culture interview
- 4) Reference Checks
Our take
Developing an algorithm that can understand text or images is (relatively) straightforward. However, the challenge escalates when it comes to understanding video, where these modes merge with audio, and context becomes much harder to grasp.
Twelve Labs has developed a machine learning solution that tackles this challenge by making the inner content of videos both indexable for developers and highly searchable for users. This technology could prove immensely valuable, with use cases extending far beyond simple searchability for end users. It could be employed for more accurate monitoring of community guidelines on social media, enterprise knowledge searches, and a deeper understanding of the value of video content.
With recent funding under its belt, Twelve Labs is set for rapid growth. The investment will drive R&D, nearly double its workforce, and advance its video understanding technology. As it expands, Twelve Labs is well-positioned to lead the future of multimodal AI and revolutionize how organizations extract values from video content.
Steph
Company Specialist at Welcome to the Jungle
Insights
Few candidates hear
back within 2 weeks
Company
Funding (last 2 of 6 rounds)
Dec 2024
$30m
EARLY VC
Jun 2024
$50m
SERIES A
Total funding: $107.2m
Company benefits
- Full health, dental, and vision benefits
- Extremely flexible PTO and parental leave policy. Office closed the week of Christmas and New Years
- Remote-flexible, offices in San Francisco and Seoul and coworking stipend
- VISA support (such as H1B and OPT transfer for US employees)
Company values
- Curiosity is power: The complexities of understanding videos present incredibly challenging problems. Curiosity fuels us to create technology that is not only adaptable but also exceptionally robust. Our products have unlocked a powerful new way for developers and enterprises to amplify their understanding of petabytes of data, and we are just getting started
- Embrace the underdog: We strive to be the best by championing innovation, resilience, and empathy. Failure does not intimidate us but encourages us to always speak up and try new things. In the face of adversity, we rally together, and it is our collective purpose that propels us forward achieving things as a team that exceed even our own expectations
- Thought leaders: We're not just part of the industry; we shape it. At Twelve Labs, you will immerse yourself in an environment brimming with creativity and forward-thinking. Each day presents an opportunity to gain invaluable insights, challenge conventional wisdom, and pioneer new frontiers as we push the boundaries of AI with our research
Company HQ
SoMa, San Francisco, CA
Articles
Leadership
Jae Lee
(Co-Founder & CEO)After Interning as a Software Engineer at Samsung and Amazon, they joined the Republic of Korea's Cyber Operations Command as a Lead Data Scientist. Co-founded Twelve Labs after this period of military service.
Aiden Lee
(Co-Founder & CTO)Interned at Korea Advanced Institute of Science and Technology (KAIST) as a Deep Learning Researcher. Completed their National Defense duty as an AI & ML Engineer in the Cyber Operations Command.
Sung Jun (SJ) Kim
(Co-Founder)Served for the Ministry of National Defense as a Lead Software Engineer in the Cyber Operations Command. Before this, they were a Cyber Security Research Scientist at Sungkyunkwan University.
Dave Chung
(Co-Founder)Previously a Project Team Lead at the Institute of East and West Studies (IEWS) at Yonsei University, focusing on the planning and development of the Korean Web 3.0 ecosystem (Funded by the Ministry of ICT).
Soyoung Lee
(Co-Founder)Worked in Risk Assurance for PricewaterhouseCoopers in Seoul before co-founding Twelve Labs.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job
View 5 more jobs at Twelve Labs