Senior Data Pipeline Engineer, Clever
$200-240k
+ Bonus
AWS
Python
Postgres
Airflow
Terraform
Unity
RDS
Senior and Expert level
Remote from US
Real estate agent matching service
Open for applications
Real estate agent matching service
101-200 employees
B2CB2BMarketplaceReal Estate
Open for applications
$200-240k
+ Bonus
AWS
Python
Postgres
Airflow
Terraform
Unity
RDS
Senior and Expert level
Remote from US
101-200 employees
B2CB2BMarketplaceReal Estate
Company mission
To make selling or buying a home easier and more affordable fo everyone.
Role
Who you are
- You've spent 8+ years in data engineering or data platform roles, with at least 2-3 years focused on infrastructure ownership (not just writing transformations)
- You're comfortable being the primary infrastructure owner on a small team. You see this as an opportunity to shape strategy, not a burden
- You’ve administered Databricks on AWS and have familiarity with the suite of Databricks tools, including Unity Catalog
- You have deep, hands-on AWS experience — not just using services, but configuring VPCs, managing security groups, setting up peering, and debugging networking issues
- You've managed Terraform in production and understand state management, module structure, and the discipline required to keep IaC reliable
- You've administered Airflow (ideally MWAA) and have opinions about DAG design, monitoring, and failure handling
- You're pragmatic about tooling: you'll maintain what works, improve what's fragile, and propose changes only when the ROI is clear
- You communicate clearly with non-infrastructure stakeholders and can explain tradeoffs without jargon
- Qualifications
- 8+ years of experience in data engineering or data platform roles
- Databricks administration and lakehouse architecture
- Apache Airflow administration and DAG development (MWAA preferred)
- AWS (EC2, RDS, VPC, S3, MWAA): our AWS environment is central to all data operations and requires deep, hands-on expertise
- Terraform (infrastructure as code): managing multiple repos across RDS, CDC, and orchestration infrastructure
- Python (PySpark, general scripting)
- PostgreSQL / RDS management, including replica promotion, security group configuration, VPC peering
- VPN setup and management (Pritunl on EC2)
- SSO configuration
- Excellent communication skills, able to influence across teams and levels
- Experience with SOC-2 compliance infrastructure requirements is a plus
- Real estate, fintech, or marketplace industry experience a plus
What the job involves
- We’re hiring a Senior Data Pipeline Engineer to own and evolve the data infrastructure that powers Clever's core data products, from the pipelines that feed our agent benchmarking tools to the warehouse architecture behind our company's highest-priority initiatives
- You'll be the primary owner of Clever's data platform: AWS infrastructure, orchestration (Airflow/MWAA), Databricks lakehouse, and the ingestion pipelines that bring operational and third-party data into the warehouse. This is a high-autonomy role where you'll shape infrastructure strategy, not just maintain what exists
- This role reports to the Data Engineering Manager and serves as a critical partner across data engineering, product, and operations, helping Clever scale reliable data infrastructure that directly enables revenue-driving products and company priorities
- Own and operate Clever's data platform infrastructure. Manage AWS services (EC2, RDS, VPC, S3, MWAA), Terraform-managed infrastructure-as-code, and Databricks administration. You are the go-to person for keeping these systems running securely and cost-effectively
- Maintain and improve data pipeline reliability. The Airflow/MWAA orchestration layer automates all ETL/ELT jobs feeding Databricks. You'll monitor, triage, and resolve pipeline failures — and proactively improve reliability so failures happen less often
- Build and extend data ingestion pipelines. Design and implement ingestion for new operational data sources (e.g., telephony, CRM, transaction data) that directly support Clever's speed-to-match initiative (P1 company priority)
- Manage database infrastructure. Administer PostgreSQL/RDS instances, including replica promotion, security group configuration, and VPC peering. Ensure databases are performant, secure, and properly networked
- Support security and compliance. Maintain infrastructure aligned with SOC-2 requirements, including VPN management (Pritunl), SSO configuration, and access controls. Respond to audit findings that require infrastructure changes
- Collaborate with the data engineering team. Partner closely with data engineers and data analysts to ensure smooth handoffs between infrastructure and pipeline/transformation work. Provide technical mentorship on infrastructure best practices
- Drive infrastructure strategy. Evaluate opportunities to reduce complexity (e.g., consolidating orchestration, optimizing cloud spend) and propose a forward-looking platform roadmap
- In 3 months…
- You've onboarded to Clever's AWS environment, Terraform repos, Airflow/MWAA setup, and Databricks workspace, and can independently triage and resolve pipeline alerts
- You've established a working relationship with the data engineering team and understand how each data product depends on the infrastructure you now own
- You've completed a documented audit of the current infrastructure state, identifying any immediate risks or technical debt
- In 6 months…
- Pipeline reliability has measurably improved: fewer unresolved alerts, faster mean-time-to-resolution, and no extended outages caused by infrastructure gaps
- You've delivered at least one new data ingestion pipeline for an operational data source supporting speed-to-match and/or algorithmic content
- You've proposed and begun executing on an infrastructure roadmap that addresses cost optimization, security posture, and reduced single-points-of-failure
- In 12 months…
- Clever's data platform is more resilient, better documented, and less dependent on any single person than when you started
- Operational data ingestion is running reliably for multiple new sources, enabling the data team to support Bench Score, Market Score, and speed-to-match with fresh, trustworthy data
- You've made meaningful progress on at least one strategic infrastructure initiative (e.g., orchestration consolidation, Databricks optimization, or compliance automation)
Our take
Real estate commissions have been kept artificially high for decades as a result of uncomfortableness from consumers to negotiate and archaic pricing systems. Clever is changing all that with its discount brokerage service.
Using an algorithm along with “concierge” staffers, Clever matches homeowners with agents who have experience in their price range and neighborhood. As a result, agents get more sales and Clever effectively negotiates on behalf of the consumer.
It competes with other discount brokerages like Redfin, which attracts customers through aggregated realtor-sanctioned listings and provides discounts to home sellers through eventual rebates. However, Clever is earning market share from Redfin through its differentiated strategies: intuitive customer experience and a portfolio of online content brands reaching millions of readers with expert real estate advice and education.
Steph
Company Specialist at Welcome to the Jungle
Insights
-15% employee growth in 12 months
Company
Funding (last 2 of 7 rounds)
May 2023
$2m
LATE VC
Dec 2021
$8m
SERIES B
Total funding: $20m
Company HQ
Central West End, St. Louis, MO
Leadership
Luke Babich
(Co-Founder & CEO)Previously Co-Founder & Principal Investor of Arch Buyers. Also previously the Director of Strategy & Sales at L&T Co.
Share this job
View 1 more job at Clever