Data Engineer - Azure Databricks
Indexed description
Senior Data Engineer (Azure Databricks)
Lead design and architecture of large-scale data infrastructure and pipelines (Data Lakehouse).
Transform complex business requirements into automated, secure data solutions (ETL/ELT) primarily in Azure.
Optimize data storage and compute costs across Databricks clusters and Data Lake.
Troubleshoot and resolve production pipeline failures and data quality issues.
Drive best practices in data governance, security, and pipeline reliability.
Mentor and coordinate with global teams (data scientists, analysts, operations).
Required Skills & Experience
5–8 years (often 8+ in practice) in data engineering or platform roles.
Expert hands-on experience with Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
Deep SQL and data modelling skills (OLAP, OLTP, dimensional modelling).
Proficiency in Python/Scala for Spark jobs and data transformation.
Understanding of Azure Data Lake Storage optimizations and Azure resource cost tuning.
Experience implementing CI/CD and IaC for data pipelines.
Knowledge of data security/compliance (RBAC, encryption, GDPR principles).
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search