Data Engineer, Leadtech

• We are looking for an experienced Data Engineer with at least 5 years of professional experience and a solid technology background using Java or Python as a primary language
• Bachelor's degree in Computer Science, Software Engineering, or a related field (equivalent experience is valued)
• At least 3 years of programming experience with Java / Python
• Experience in data engineer design and delivery with cloud based data Warehouse technologies, in particular Snowflake, or Redshift, BigQuery
• Experience in a wide range of DB technologies such as, DynamoDB, Postgres,and Mongo
• Development with cloud services, especially Amazon Web Services
• Demonstrable experience in designing and implementing data pipeline architectures based on Kafka in cloud environments, preferably AWS
• Deep understanding of distributed systems and high availability design principles
• Experience in building and optimizing data pipelines using technologies like Apache Kafka, Apache Flink, Apache Spark, etc., including real-time processing frameworks such as Apache Flink or Apache Spark Streaming
• Excellent communication and teamwork skills
• Ability to independently and proactively solve problems

Desirable

• In this role, you will design, build, and maintain scalable, secure, and high-performance cloud-based data pipelines to support real-time and batch analytics within our payments platform
• You will work closely with product owners, and cross-functional engineering teams to translate business requirements into robust data models and ETL/ELT workflows
• Your day-to-day work will include architecting and implementing Kafka-based streaming pipelines, processing event streams and orchestrating data ingestion and transformation jobs on AWS
• You will leverage Snowflake as our central data warehouse
• Design, build, and maintain scalable data pipelines in AWS to support operational and analytical use cases
• Define and enforce best practices for data ingestion, cataloging, and lineage across our cloud infrastructure (AWS S3, Glue, EMR, Lambda, etc.)
• Develop and maintain real-time processing applications using Kafka (Producers, Consumers, Streams API) or similar technologies to aggregate, filter, and enrich streaming data from multiple sources
• Define data schemas, partitioning strategies, and access patterns optimized for performance and cost
• Collaborate with development and analytics teams to understand and fulfill the company's data requirements
• Implement monitoring and alerting mechanisms to ensure the integrity and availability of data streams
• Work with the operations team to optimize the performance and efficiency of the data infrastructure
• Automate management and maintenance tasks of the infrastructure using tools such as Terraform, Ansible, etc
• Stay updated on best practices and trends in data architectures, especially in the realm ofreal-time data ingestion and processing
• Monitor and troubleshoot data workflows using tools such as CloudWatch, Prometheus, or Datadog—proactively identifying bottlenecks, ensuring pipeline reliability, and handling incidentresponse when necessary
• Ensure data quality and performance
• Define and test disaster recovery plans (multi-region backups, Kafka replication,Snowflake Time Travel) and collaborate with security/infra teams on encryption,permissions, and compliance