Data Engineer, Leadtech

• Bachelor's degree in Computer Science, Software Engineering, or a related field (equivalent experience is valued)
• We are looking for an experienced Data Engineer with 5+ years of professional experience and a solid technology background using Java or Python as a primary language
• 3+ years of programming experience with Java / Python
• Experience in data engineer design and delivery with cloud based data Warehouse technologies, in particular Snowflake, or Redshift,BigQuery
• Development with cloud services, especially Amazon Web Services
• Demonstrable experience in designing and implementing data pipeline architectures based on Kafka in cloud environments, preferably AWS
• Deep understanding of distributed systems and high availability design principles
• Experience in building and optimizing data pipelines using technologies like Apache Kafka, Apache Flink, Apache Spark, etc., including real-time processing frameworks such as Apache Flink or Apache Spark Streaming
• Excellent communication and teamwork skills
• Ability to independently and proactively solve problems

Desirable

• In this role, you will design, build, and maintain scalable, secure, and high-performance cloud-based data pipelines to support real-time and batch analytics within our payments platform. You will work closely with product owners, data scientists, and cross-functional engineering teams to translate business requirements into robust data models and ETL/ELT workflows. Your day-to-day work will include architecting and implementing Kafka-based streaming pipelines, processing event streams and orchestrating data ingestion and transformation jobs on AWS. You will leverage Snowflake as our central data warehouse
• Define and enforce best practices for data ingestion, cataloging, and lineage across our cloud infrastructure (AWS S3, Glue, EMR, Lambda, etc.)
• Develop and maintain real-time processing applications using Kafka (Producers, Consumers, Streams API) or similar technologies to aggregate, filter, and enrich streaming data from multiple sources
• Collaborate with development and analytics teams to understand and fulfill the company's data requirements
• Develop ecient and scalable data pipelines for data transformation and enrichment
• Implement monitoring and alerting mechanisms to ensure the integrity and availability of data streams
• Work with the operations team to optimize the performance and eciency of the data infrastructure
• Automate management and maintenance tasks of the infrastructure using tools such as Terraform, Ansible, etc
• Stay updated on best practices and trends in data architectures, especially in the realm ofreal-time data ingestion and processing
• Monitor and troubleshoot data workflows using tools such as CloudWatch, Prometheus, or Datadog—proactively identifying bottlenecks, ensuring pipeline reliability, and handling incidentresponse when necessary
• Ensure data quality and performance
• Define and test disaster recovery plans (multi-region backups, Kafka replication, Snowflake Time Travel) and collaborate with security/infra teams on encryption, permissions, and compliance