Data Engineer

Job Summary:

We are looking for a skilled Data Engineer with strong experience in Databricks to design, build, and maintain scalable data pipelines and analytics platforms. The ideal candidate will work closely with data scientists, analysts, and business stakeholders to deliver reliable, high-quality data solutions.

Key Responsibilities:

• Design, develop, and optimize ETL/ELT pipelines using Databricks and Apache Spark
• Build and manage Delta Lake architectures (Bronze, Silver, Gold layers)
• Ingest data from multiple sources (APIs, databases, streaming, files)
• Optimize Spark jobs for performance and cost
• Implement data quality, validation, and monitoring frameworks
• Collaborate with analytics and ML teams to support data science workloads
• Ensure data security, governance, and compliance best practices
• Automate workflows using Databricks Jobs and CI/CD pipelines

Required Skills & Qualifications:

• Strong experience with Databricks and Apache Spark (PySpark/Scala)
• Proficiency in Python and SQL
• Experience with Delta Lake and Lakehouse architecture
• Hands-on experience with cloud platforms (Azure / AWS / GCP)
• Knowledge of data modeling, data warehousing, and big data concepts
• Familiarity with orchestration tools (Airflow, ADF, etc.)