Senior Data Engineer – Data Analytics Platform 80-100% (f/m/d) - (Contract through our external payroll partner with immediate start for 12 months with possible extension)
Indexed description
In our team, you will be a key contributor to building our Data Platform as a foundation for Data Analytics and AI.
YOUR CHALLENGE
- Design, develop, and maintain data pipelines and backend services for real-time decisioning, reporting, data collecting, and related functions
- Data Requirements and Modeling
- Data Management and Transformation
- Produce high-quality, well-tested, and secure code
- Develop and maintain software designed to improve data governance and security
- Develop processes designed to ensure Data Security and Data Quality
- Bachelor’s or Master’s degree in Computer Science, Data Analytics, Information Systems, or a related technical field; or equivalent training and professional experience
- 5+ years of experience in ETL/ELT, data warehousing, Business Intelligence, and integrating data processing/workflow management tools into pipeline design
- 5+ years building and maintaining end-to-end data systems using Python, Scala, or similar programming languages
- Solid experience in utilising SQL for data analysis, investigating data issues, diagnosing root causes, and designing effective solutions
- Experience in working with cloud-based data technologies, preferably on Microsoft Azure, within continuous integration and delivery (CI/CD) environments
- Demonstrated experience working with large-scale datasets using Databricks, PySpark, Spark Streaming, and Delta Lake
- Hands-on experience with structured, semi-structured, and unstructured data across various storage systems, including relational databases (RDBMS), data warehouses, in-memory caches, and document databases
- Familiarity with cloud storage solutions such as Azure Data Lake and Blob Storage
- Solid understanding of data modelling principles (experience with Data Vault is a plus) and strong skills in system design, implementation, and testing
- Experience with event-driven architectures (e.g., Kafka, Event Hubs, Apache Flink) and containerised microservices platforms (e.g., Kubernetes, Docker, Helm Charts)
- Knowledge and practical use of dbt (data build tool), experience with BI tools
- Excellent communication and collaboration skills, ability to provide technical leadership to other developers
Is this not quite what you are looking for? Set up a job alert by creating a candidate account here.
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search