Principal Data Platform Engineer, Exactera

• Unity Catalog: Production experience with multi-catalog governance, metastore design, and lineage tracking
• Data Structuring: Experience designing and building unified schemas across multiple disparate product lines
• Delta Lake: Expert-level experience with Z-ordering, compaction, liquid clustering, and performance tuning at multi-TB scale
• Delta Live Tables: Strong hands-on experience building declarative ETL pipelines, including change data capture and expectations/constraints
• Databricks Workflows: Experience with job orchestration, scheduling, and operational monitoring
• Business Intelligence: Experience enabling company-wide analytics and reporting with modern business intelligence tools and maintaining source of truth data and metrics
• PySpark & Databricks SQL: Strong proficiency for code review, performance tuning, and query optimization
• 5-8 years in data engineering or data platform roles, with 3+ years hands-on Databricks experience
• Track record leading at least one significant platform build or migration project
• AWS experience (S3, IAM, VPC) with ability to collaborate on infrastructure decisions
• Infrastructure-as-code experience (Terraform preferred)
• Demonstrated ability architecting data platforms from first principles and defending technical decisions
• Strong written and verbal communication— document architecture decisions and present to both technical and business stakeholders

Desirable

• As Principal Data Platform Engineer, you'll architect and implement our centralized data platform on Databricks
• You'll establish governance patterns using Unity Catalog, optimize for cost and performance at scale, and enable our existing Data Engineers to build confidently on the platform
• This is a data infrastructure role—focused on pipelines, storage, governance, and platform operations
• We operate multiple product lines (Transfer Pricing, R&D Services, RoyaltyStat, Provisioning), each with distinct databases containing enterprise financial data—journal entries, general ledgers, and financial statements
• Our immediate challenge is migrating multi-terabyte datasets from legacy systems to a unified Databricks lakehouse while establishing governance patterns that enable multi-product operations at scale
• Data Structuring: Design data models and implement unified schemas across multiple disparate product lines
• Unity Catalog Architecture: Design and implement multi-catalog governance strategy supporting data isolation, cross-product data sharing, and comprehensive lineage tracking across our product portfolio
• Delta Lake Optimization: Establish patterns for Z-ordering, compaction, and liquid clustering at multi-TB scale. Define table structures, partitioning strategies, and retention policies that balance query performance with storage costs
• ETL Pipeline Framework: Build declarative pipeline patterns using Delta Live Tables. Create orchestration workflows for ingesting data from internal sources such as SQL databases and S3
• Third Party Integrations: Integrate with third party data sources such as ERP systems (Netsuite etc.) and external data providers (S&P etc.) with automated ingest, robust error handling and monitoring
• Platform Operations: Implement cost monitoring and optimization strategies, establish data quality frameworks, create self-service patterns enabling Data Engineers to work independently while maintaining governance standards
• Key Legacy Product Migrations: Lead the architecture for migrating multi-terabyte datasets from legacy systems to Databricks—establishing patterns that will be reused across multiple product lines
• Multi-Product Data Architecture: Design Unity Catalog structures enabling secure data separation between product lines while allowing controlled cross-product analytics where appropriate
• Cost-Efficient Scale: Build infrastructure that scales efficiently—through intelligent caching, query optimization, and compute management strategies that avoid linear cost growth
• Platform Reliability: Establish monitoring, alerting, and data quality validation ensuring the platform operates reliably as foundation for both analytics and AI workloads