Software Engineer - Data - Data Mesh & Lakehouse

Software Engineer - Data | Data Mesh & Lakehouse

About the Role

We are looking for a Software Engineer - Data to join a central Data Delivery team building the foundation of a global Data Mesh platform based on a Lakehouse architecture.

You will work on the engineering layer responsible for ingesting, processing and provisioning source-aligned data products into central data catalogs, enabling teams across the organisation to consume trusted data for analytics, reporting, machine learning and other data-driven applications.

A key part of the platform is enabling scalable data consumption through zero-copy data sharing, while maintaining strong governance, security and data quality standards.

What You'll Be Working On

• Design and build scalable batch and streaming data pipelines.
• Develop ingestion solutions for source-aligned data products.
• Work with Apache Spark and Databricks within a modern Lakehouse environment.
• Implement data ingestion strategies including Full Loads, Delta Loads and Change Data Capture (CDC).
• Build and maintain streaming pipelines using technologies such as Kafka, Flink or Confluent.
• Manage datasets stored across Google Cloud Storage and Azure Blob Storage.
• Enable secure zero-copy data sharing through technologies such as Databricks Unity Catalog and BigQuery.
• Implement data governance and access-control models including RBAC and attribute-based access control.
• Design solutions capable of handling complex schema evolution, including backward and forward compatibility.

Tech Environment

Data & Processing: Apache Spark, Databricks, BigQuery

Streaming & Ingestion: Kafka, Flink, Confluent, Airbyte

Cloud & Storage: GCP, Google Cloud Storage, Azure, Azure Blob Storage

Orchestration & Platform: Airflow, Kubernetes

Governance: Databricks Unity Catalog, RBAC, ABAC/CBAC, Data Contracts

CI/CD: GitLab, Azure DevOps, JFrog Artifactory

Quality & Security: SonarQube, Snyk

What We're Looking For

• Strong professional experience in Data Engineering or Software Engineering focused on data platforms.
• Hands-on experience with Databricks and Apache Spark.
• Experience designing distributed data pipelines in cloud environments.
• Strong understanding of Lakehouse architectures.
• Experience or strong knowledge of Data Mesh principles and Data Products.
• Experience with batch and streaming ingestion patterns.
• Knowledge of CDC and incremental data processing strategies.
• Experience dealing with schema evolution in production data pipelines.
• Understanding of modern data governance and access-control models.
• Experience with Airflow, Kubernetes and CI/CD.
• Comfortable working collaboratively through code reviews and technical design discussions.