Data Engineer, Qloo
AI-powered recommendation API
Be an early applicant
AI-powered recommendation API
21-100 employees
Be an early applicant
21-100 employees
Company mission
To demystify the intricacies of global tastes and preferences.
Role
Who you are
- You should be comfortable owning end-to-end data flows, from ingestion and transformation to quality checks, monitoring, and performance tuning
- Bachelor’s degree in Computer Science, Software Engineering, or a related field, or equivalent practical experience
- Experience with Python and distributed data processing using Spark (PySpark) on EMR or a similar environment
- Hands-on experience with core AWS data services, ideally including:
- S3 (data lake, partitioning, lifecycle management)
- AWS Glue (jobs, crawlers, catalogs)
- EMR or other managed Spark platforms
- Athena/Hive and SQL for querying large datasets
- Relational databases such as RDS (PostgreSQL/MySQL or similar)
- Experience building and operating workflows in Airflow (MWAA experience is a plus)
- Strong SQL skills and familiarity with data modeling concepts for analytics and APIs
- Solid understanding of data quality practices (testing, validation frameworks, monitoring/observability)
- Comfortable working in a collaborative environment, managing multiple projects, and owning systems end-to-end
What the job involves
- As a Data Engineer at Qloo, you will design, build, and operate the pipelines that move data from external vendors, internal systems, and public sources into our S3-based data lake and downstream services
- You’ll work across AWS Glue, EMR (Spark), Athena/Hive, and Airflow (MWAA) to ensure that our data is accurate, well-modeled, and efficiently accessible for analytics, indexing, and machine-learning workloads
- Design, develop, and maintain batch data pipelines using Python, Spark (EMR), and AWS Glue, loading data from S3, RDS, and external sources into Hive/Athena tables
- Model datasets in our S3/Hive data lake to support analytics (Hex), API use cases, Elasticsearch indexes, and ML models
- Implement and operate workflows in Airflow (MWAA), including dependency management, scheduling, retries, and alerting via Slack
- Build robust data quality and validation checks (schema validation, freshness/volume checks, anomaly detection) and ensure issues are surfaced quickly with monitoring and alerts
- Optimize jobs for cost and performance (partitioning, file formats, join strategies, proper use of EMR/Glue resources)
- Collaborate closely with data scientists, ML engineers, and application engineers to understand data requirements and design schemas and pipelines that serve multiple use cases
- Contribute to internal tooling and shared libraries that make working with our data platform faster, safer, and more consistent
- Document pipelines, datasets, and best practices so the broader team can easily understand and work with our data
Our take
Powered by AI, Qloo is aiming to understand taste and cultural correlations. While giants like Spotify, Expedia and Netflix dominate their sectors, they inadvertently create "data silos", where valuable information remains isolated. A unified system, however, could help understand and predict diverse personal tastes across various domains.
To confront this, Qloo provides companies with AI-generated correlation data spanning numerous cultural and entertainment domains. Leveraging its platform, a user's taste in one category or genre can seamlessly inform recommendations in another, leading to more personalized experiences. For example, Starbucks is using Qloo to create in-store music playlists tailored to specific neighborhoods, enhancing customer engagement.
With an influx in funding over 2023 and 2024, Qloo is now introducing an accessible self-service interface, aiding small and mid-sized enterprises in charting consumer and taste analytics. Additionally, it has introduced the "multi-person recommendation AI", capable of matching profiles based on individual preferences stored in its database, which further solidifies its position as a leader in personalized recommendation solutions.
Steph
Company Specialist at Welcome to the Jungle
Company
Company benefits
- Champion your health: We offer 100% health insurance coverage, with the ability to join group dental and vision for a nominal fee.
- Invest in your future: Qloo’s competitive 4% 401K matching program helps you build a financial foundation for the years ahead.
- Generous PTO: With 20 days of PTO and 10+ annual company holidays, we encourage taking time to rest and investing in your life outside of work.
- Thrive in Soho: Our sun-lit, spacious office in Soho boasts standing desks, team lunches every Thursday, monthly outings, and a tight-knit community of hard workers.
- Parental Leave: Whether you’re welcoming a child through birth or adoption, Qloo supports all new parents with three months of paid parental leave.
- Ownership Mentality: We provide stock options to all positions, ensuring that everyone, regardless of level, has a stake in our success and shares in our achievements.
Company HQ
SoHo, New York, NY
Leadership
Alex Elias
(Founder & CEO)Previously served as a Principal at APE Capital for 4 years before founding Qloo in 2012.
Salary benchmarks
We don't have enough data yet to provide salary benchmarks for this role.
Submit your salary to help other candidates with crowdsourced salary estimates.
Share this job