Data Engineer (3 Year Contract)
Indexed description
We are looking for a Data Engineer to design, build, and optimise modern cloud-native data platforms that power enterprise-scale analytics and AI initiatives. You will play a key role in developing scalable data architectures, implementing high-performance data pipelines, and enabling data-driven decision-making across the organisation.
Working within a collaborative engineering team, you will contribute to Data Lakehouse architectures, cloud migration initiatives, DataOps practices, and the integration of AI capabilities into enterprise data platforms while mentoring junior engineers and driving engineering best practices.
- Design and implement scalable, secure, and highly available cloud-based data platforms.
- Architect and develop modern Data Lakehouse solutions.
- Work closely with stakeholders to gather business requirements and translate them into scalable technical solutions.
- Produce and maintain architecture documentation and technical standards.
- Develop and maintain robust ETL/ELT pipelines using modern data engineering frameworks.
- Build reliable, production-grade data ingestion and transformation workflows.
- Optimise pipeline performance, scalability, reliability, and cloud cost efficiency.
- Implement automated data quality validation, monitoring, and alerting.
- Design cloud-native Data & AI solutions aligned with business and technical objectives.
- Lead cloud migration initiatives and support complex multi-cloud deployments.
- Integrate AI capabilities into enterprise data platforms.
- Evaluate emerging technologies and recommend modern cloud-native approaches.
- Build and manage cloud infrastructure using native cloud services.
- Implement Infrastructure as Code (IaC) using Terraform.
- Ensure platforms comply with security standards, governance policies, and operational best practices.
- Support platform reliability, monitoring, and continuous improvement.
- Mentor and guide junior data engineers on technical implementation and best practices.
- Participate in architecture reviews and contribute to the evolution of the organisation's data strategy.
- Collaborate with cross-functional engineering, analytics, and business teams.
- Bachelor's degree in Computer Science, Information Technology, Computer Engineering, or a related discipline.
- Minimum 3 years of experience in data engineering, data architecture, or enterprise data platform development.
- Strong hands-on experience building production-scale data pipelines and cloud-native data platforms.
- Strong proficiency in:
- Python
- SQL
- Apache Spark
- Experience with modern data engineering tools such as:
- Apache Kafka
- Apache Airflow (or similar workflow orchestration tools)
- Good understanding of:
- Cloud computing principles
- Infrastructure as Code (Terraform)
- Containerisation
- Microservices architecture
- Distributed systems
- Cloud networking
- Identity & Access Management (IAM)
- Cloud security best practices
- Experience with DataOps, Data Lakehouse, or modern data platform architectures.
- Familiarity with AI/ML platform operations including MLOps or LLMOps.
- Strong analytical, problem-solving, and communication skills.
- Ability to translate business requirements into scalable technical solutions.
- Experience with Amazon Web Services (AWS).
- Experience with AWS services such as:
- S3
- Glue
- Lake Formation
- SageMaker
- Bedrock
- Amazon Quick Suite
- AWS certifications such as:
- AWS Certified Solutions Architect – Professional
- AWS Certified Data Engineer – Associate
- Experience with machine learning deployment pipelines and MLOps.
- Knowledge of metadata management and data governance frameworks.
- Familiarity with serverless computing, edge computing, or IoT architectures.
- Experience with business intelligence or data visualisation platforms.
- Work on enterprise-scale cloud and AI data platforms.
- Design and build modern Data Lakehouse architectures.
- Be part of large-scale cloud transformation initiatives.
- Collaborate with experienced engineers on cutting-edge data and AI technologies.
- Opportunity to mentor engineers while shaping the future of enterprise data platforms.
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search