Data Engineer, Stability AI
Research
Open source generative AI company
Job no longer available
Open source generative AI company
101-200 employees
Job no longer available
101-200 employees
Company mission
To activate humanity's potential through generative AI.
Role
Who you are
- Proven background within large scale distributed workloads
- Experience with large scale data loading for machine learning training runs
- Experience with cloud storage and file systems; AWS (S3) is strongly preferred, but open to other cloud platforms
- Experience with Python + Pytorch
- Experience with multiprocessing and multithreading python workloads
- Excellent communication skills to effectively collaborate with users, solve issues, and provide guidance
- Attention to detail and the ability to document processes and solutions effectively
- Strong interest in Generative AI
- Experience working with Machine Learning projects and ideally some Deep learning / Comp Vision knowledge
- Experience with dataloading stack (webdataset, torchdata, fsspec, AIstore) and parallel dataframe manipulation using Pyspark/Ray is a plus point
What the job involves
- We are looking for a talented Data Engineer with a focus on scaling efficient distributed workloads
- You will work alongside a growing multidisciplinary team of talented research scientists and machine learning engineers to improve and scale the efficiency within our models
- In this role, you will contribute to groundbreaking projects such as training the largest open language models and be responsible for ensuring data is collected, processed and utilized in the right way
- Clean, normalize, and preprocess data in a scalable, parallelizable way to prepare it for ingestion into our machine learning model training pipelines while ensuring of data quality
- Building and maintaining highly scalable distributed workloads
- Build data pipelines to ingest and process data (e.g. images and text) for feeding into ML models
- AWS Resource Management
- Keep up-to-date with methods regarding how to improve data quality and/or curate data for Image, Video, LLMs etc
Salary benchmarks
Our take
Stability AI is an AI-driven visual art company that designs and implements an open AI tool that creates images based on text input. Through collective intelligence and augmented technology, Stability AI is helping research communities develop cutting-edge AI models for image, language, audio, video, 3D content, biotech and other scientific research.
Working in partnership with Electronic Arts (EA), Stability AI is co-developing next-generation generative AI model and workflows designed to empower artists, designers, and developers across interactive entertainment. By embedding its research teams directly alongside EA's creative talent, Stability AI is advancing image-to-3D, text-to-3D, and volumetric media tools that accelerate world-building, rapid prototyping, and visual storytelling. This collaboration highlights Stability AI's growing leadership i multimodal and 3D generative AI, and its mission to put powerful, creator-centric tools in the hands of global creative industries.
The company is also focused on expanding its creative sector offerings, hiring legendary filmmaker James Cameron to bring an artistic perspective to its suite of storytelling tools. This exciting development will further help Stability AI push the boundaries of AI-driven media and help on its mission to transform visual storytelling.
Freddie
Company Specialist at Welcome to the Jungle
Company
Funding (last 2 of 3 rounds)
Jun 2024
$80m
EARLY VC
Oct 2023
$50m
CONVERTIBLE
Total funding: $231m
Company HQ
Notting Hill, London, UK
Share this job
View 7 more jobs at Stability AI