Azure Databricks Engineer/Lead/ Architect

🚀 Hiring: Azure Databricks Data Engineer| HCLTech 🚀

We are seeking a highly skilled Azure Databricks Data Engineer to build and optimize modern data platforms leveraging the Lakehouse architecture. If you are passionate about designing scalable data solutions, implementing robust ETL/ELT pipelines, and enabling data-driven insights through Databricks, we'd love to hear from you.

📍 Location: Chennai | Bengaluru | Hyderabad| Noida | Pune

💼 Experience: 5 – 15 Years

What We're Looking For

Key Responsibilities

• Design and implement Data Lakehouse architectures using the Medallion Framework (Bronze, Silver, Gold layers).
• Develop high-performance data pipelines using PySpark notebooks and Databricks.
• Build and manage Delta Live Tables (DLT) pipelines for reliable and efficient data processing.
• Implement automated data ingestion using Autoloader for batch and streaming workloads.
• Ingest and process data from diverse sources including files, databases, APIs, and streaming platforms.
• Perform complex ETL/ELT transformations to support analytics and reporting requirements.
• Ensure data governance, security, and access control using Unity Catalog.
• Monitor, optimize, and troubleshoot pipelines for performance, scalability, and cost efficiency.
• Implement robust data quality checks, validations, and monitoring frameworks.
• Develop and optimize advanced SQL queries for large-scale data workloads.

Mandatory Skills

✅ Azure Databricks

✅ Data Lakehouse / Medallion Architecture (Bronze-Silver-Gold)

✅ PySpark Development

✅ Delta Live Tables (DLT)

✅ Databricks Autoloader

✅ Unity Catalog & Data Governance

✅ ETL / ELT Pipeline Development

✅ Advanced SQL

✅ Data Quality & Validation Frameworks

✅ Data Ingestion (Files, Databases, Streaming Sources)

✅ Pipeline Performance Tuning & Optimization

📩 Interested candidates can share their updated resumes or refer suitable professionals from their network.

Please fill out this form

Keywords: Azure Databricks | Data Engineer | PySpark | Delta Live Tables | DLT | Autoloader | Unity Catalog | Data Governance | Data Lakehouse | Medallion Architecture | ETL | ELT | Advanced SQL | Data Quality | Delta Lake