Senior Data Platform Engineer, Stability AI

Salary not provided

AWS

Docker

Kubernetes

Python

Looker

Kafka

Redis

Postgres

Airflow

Terraform

Redshift

Spark

NumPy

Pandas

Flink

BigQuery

Senior and Expert level

Remote from Canada

Open source generative AI company

Open for applications

Open source generative AI company

101-200 employees

B2CB2BArtificial IntelligenceDeep TechMachine LearningAPI

Open for applications

Salary not provided

AWS

Docker

Kubernetes

Python

Looker

Kafka

Redis

Postgres

Airflow

Terraform

Redshift

Spark

NumPy

Pandas

Flink

BigQuery

Senior and Expert level

Remote from Canada

101-200 employees

B2CB2BArtificial IntelligenceDeep TechMachine LearningAPI

Company mission

To activate humanity's potential through generative AI

Job

Company

Role

Who you are

Must have 7+ years of hands-on experience working with data platforms
Experience being involved in the design and implementation of at least one data platform
Experience in streaming and/or batch analytics
Experience with scalable distributed data processing, management, and visualization tools
Experience using business intelligence tools and data frameworks
Strong working knowledge of Python for both provisioning cloud infrastructure and working with data
We’re a remote-first company, and we need people who can do their best work from home, collaborate effectively, work asynchronously, and are proactive about reaching out to peers to get things done and/or get themselves unstuck
Enthusiasm, a “can do” attitude, and optimism when faced with difficult challenges

What the job involves

We are looking for a Data Platform Engineer to enhance and expand Stability’s data platform using AWS and GCP services. This begins with understanding our needs, selecting the appropriate tech stack and tools, gaining buy-in from stakeholders, and choosing the best long-term solution
This role will act as the technical thought leader for our data platform and help hire and mentor other team members
Architect, implement, and monitor data pipelines, as well as conduct knowledge-sharing sessions with peers to ensure effective data use
Work with team members and leaders to further develop our data strategy
Develop and maintain a data catalog so that we have a full understanding of the data and events within our platform
Maintain infrastructure-as-code (AWS CDK) provisioning, spin up new and/or revised data platform resources and environments to enhance data collection and processing from our applications and data sources
Act as the primary technical decision-maker for the data platform and help hire and support analytical data personnel (e.g. Data Scientist, Data Analyst, BI Developer, etc)
Analyze large amounts of data, both structured and unstructured, and create dashboards, queries, and visualizations
Technical experience in the following areas (with examples):
Containerized compute: Kubernetes (EKS, GKS), ECS, Docker
Data ingestion: Airbyte, Mage AI, AWS Glue
Pipeline orchestration: Airflow, Spark
Data manipulation: Pandas, Numpy, Polars,
Data lake: S3, Iceberg, Delta Lake
Data warehouse: BigQuery, Redshift
Database: PostgreSQL
Data visualization: Looker, Superset/Preset
Data quality (optional): Great Expectations, Monte Carlo
Data catalog (optional): Amundsen, AWS Glue Catalog
Event streaming (optional): Kafka, Flink, Redis, Pub/Sub
Infrastructure-as-code: AWS CDK, Terraform
CI/CD: GitHub Actions

Our take

Stability AI is an AI-driven visual art company that designs and implements an open AI tool that creates images based on text input. Through collective intelligence and augmented technology, Stability AI is helping research communities develop cutting-edge AI models for image, language, audio, video, 3D content, biotech and other scientific research.

Working in partnership with Amazon Web Services, Stability AI's latest release, Stable Diffusion is a text-to-image model that utilizes the Ezra-1 UltraCluster (the world's fifth-largest supercomputer). The model quickly proved to be a success, with four of the top ten applications on Apple's App Store being powered by Stable Diffusion after just one month.

The company is also focused on expanding its creative sector offerings, recently hiring legendary filmmaker James Cameron to bring an artistic perspective to its suite of storytelling tools. This exciting development will help Stability AI push the boundaries of AI-driven media and help on its mission to transform visual storytelling.

Freddie

Company Specialist at Welcome to the Jungle

Company

Funding (last 2 of 3 rounds)

Jun 2024

$80m

EARLY VC

Oct 2023

$50m

CONVERTIBLE

Total funding: $231m

Company HQ

London, UK

Leadership

Emad Mostaque

(CEO)

Graduated from Oxford University with a Master's degree in Mathematics and Computer Science. Founder of Symmitree, a startup aimed at reducing the cost of technology.

Share this job

View 5 more jobs at Stability AI