Data Engineer, LineTen

• Proven experience as a Data Engineer, working on complex data integration, transformation, and storage solutions
• Expertise in Azure data services, including Azure Data Factory, Data Lake, Synapse Analytics, and Azure Functions
• Strong SQL proficiency for data modelling, transformation, and optimisation
• Programming skills in C# for ETL, automation, and data pipeline development
• Experience with self-serve reporting platforms (e.g., Power BI) and backend data model design for analytics
• Deep understanding of data governance principles, including security, privacy, and compliance
• Knowledge of modern data architectures, such as data mesh, data fabric, event-driven systems, and streaming architectures
• Experience supporting AI/ML workflows by preparing, cleaning, and transforming large datasets

Desirable

• We are looking for a highly skilled and innovative Data Engineer to join our team and play a critical role in building and optimizing our data infrastructure
• As a Data Engineer, you will design and implement modern data solutions that enable the organization to leverage data at scale
• Your work will support advanced analytics, reporting, and AI/ML initiatives, driving key business insights and innovation
• This is a hands-on technical role requiring deep expertise in Azure data services, scalable data pipelines, and data governance
• You will work with structured and unstructured data, enabling self-service reporting and ensuring that data is secure, accessible, and optimised for analytics
• The goal of this role is to provide internal and external data services that allow anyone to self-serve
• Design, implement, and manage scalable and reliable data pipelines to integrate and process data from multiple structured and unstructured sources
• Develop and maintain Azure-based data infrastructure, including Azure Data Lake, Synapse Analytics, Data Factory, and Azure Functions
• Optimise data ingestion, transformation, and storage to support real-time and batch processing workloads
• Ensure data quality, consistency, and integrity through robust data governance policies
• Develop and execute a data lake and data warehousing strategy, ensuring efficient storage, accessibility, and management of large datasets
• Enable self-serve analytics by integrating data with visualisation tools such as Power BI, supporting teams in creating insightful data models and reports
• Design and implement efficient backend data models to support enterprise reporting and analytics solutions
• Collaborate with data scientists and analysts to prepare, clean, and transform large datasets for AI/ML model training and predictive analytics
• Work on data preparation workflows that enhance machine learning pipelines and support data-driven innovation
• Optimise data solutions to enable scalable machine learning operations (MLOps) and advanced analytics
• Implement and enforce data governance policies, ensuring compliance with security, privacy, and regulatory frameworks (e.g., GDPR, SOC 2, ISO27001)
• Develop strategies for data cataloging, lineage tracking, and metadata management to improve data discoverability and compliance
• Monitor, troubleshoot, and optimize data pipelines to ensure high performance, scalability, and cost efficiency
• Implement cloud cost optimization strategies for data storage, processing, and analytics workloads
• Contribute to CI/CD pipelines for data workflows, ensuring smooth deployments and operational efficiency
• Automate infrastructure provisioning using Terraform, ARM templates, or Bicep to streamline cloud deployments