Data Engineer

We are seeking an experienced Data Engineer (GCP & Azure) on a contract basis to support the design, development and optimisation of moder cloud-based data platforms and data products. This is a fully remote opportunity, offering the chance to work within a collaborative Agile environment on large-scale data engineering initiatives.

Key Responsibilities

• Design, build, test and maintain scalable data pipelines and data products.
• Develop and optimise ETL/ELT processes for integrating data from multiple sources.
• Design and implement cloud-native data solutions, preferably within Google Cloud Platform (GCP).
• Support migration of data platforms, pipelines and services from on-premises environments to the cloud.
• Develop and maintain data models, data warehouses and data integration solutions.
• Implement CI/CD processes and automated deployment pipelines.
• Use workflow orchestration tools such as Airflow, Prefect, Dagster or Cloud Composer.
• Monitor, troubleshoot and optimise platform and pipeline performance.
• Collaborate with technical and non-technical stakeholders to translate business requirements into robust technical solutions.
• Contribute to code reviews, testing, technical documentation and software engineering best practices.
• Ensure compliance with data governance, security and regulatory requirements, including GDPR.

• Strong programming skills in Python and SQL (R experience desirable).
• Experience with relational databases and data warehousing technologies.
• Proven experience building and supporting production-grade ETL/ELT pipelines.
• Strong understanding of data modelling principles, including conceptual, logical and physical models.
• Experience with Git-based version control systems such as GitHub, GitLab or Azure DevOps.
• Experience working in Agile delivery environments.
• Experience with cloud platforms, preferably Google Cloud Platform (GCP).
• Experience with workflow orchestration tools including Airflow, Prefect, Dagster or Cloud Composer.
• Knowledge of containerisation technologies such as Docker.
• Experience working with large-scale datasets and distributed processing technologies such as Spark, Dataproc and BigQuery.
• Understanding of DevOps and DataOps principles.
• Experience integrating systems through APIs and data services.
• Strong troubleshooting and performance optimisation skills.

• Experience with metadata management, data catalogues and lineage tools.
• Experience implementing data quality, profiling and validation frameworks.
• GCP Data Engineer and/or GCP Architect certification.
• Experience within highly regulated or data-driven environments.

• Degree in Computer Science, Data Engineering, Software Engineering or a related technical discipline, or equivalent commercial experience.

Robert Walters Operations Limited is an employment business and employment agency and welcomes applications from all candidates