Senior Data Engineer

Job Title: Senior Data Engineer / Data Engineering Lead

Experience: 15+ Years

Location: new york

Employment Type: W2

Job Description

We are looking for an experienced Senior Data Engineer / Data Engineering Lead with 15+ years of experience in designing, developing, and implementing scalable data solutions. The ideal candidate will have strong expertise in data engineering, cloud platforms, ETL/ELT, data warehousing, big data technologies, and modern data pipelines.

Key Responsibilities

• Design and develop scalable, high-performance data pipelines and ETL/ELT processes.
• Develop and maintain enterprise-level data warehouses, data lakes, and data platforms.
• Work with business and technical teams to understand data requirements and translate them into technical solutions.
• Develop data integration solutions using tools such as Informatica, Talend, SSIS, Azure Data Factory, or similar technologies.
• Build and optimize data pipelines using Python, SQL, Spark, and PySpark.
• Design data solutions on cloud platforms such as AWS, Azure, or GCP.
• Implement data ingestion, transformation, cleansing, validation, and reconciliation processes.
• Work with large datasets using Apache Spark, Databricks, Kafka, or other Big Data technologies.
• Develop and optimize complex SQL queries, stored procedures, and data models.
• Design dimensional and relational data models for analytical and reporting requirements.
• Implement data quality, governance, security, and performance standards.
• Troubleshoot data pipeline failures and resolve performance and data-quality issues.
• Mentor junior and mid-level data engineers and provide technical leadership.
• Participate in architecture discussions, code reviews, technical documentation, and production support.
• Collaborate with DevOps, Data Science, BI, and application development teams.

Required Skills

• 15+ years of overall IT experience, with strong experience in Data Engineering.
• Strong expertise in SQL and relational databases.
• Hands-on experience with Python / PySpark.
• Strong knowledge of ETL/ELT concepts and data integration.
• Experience with Data Warehousing, Data Lakes, and Data Lakehouse architecture.
• Strong experience with one or more cloud platforms:
• AWS
• Microsoft Azure
• Google Cloud Platform (GCP)
• Experience with Apache Spark / Databricks.
• Strong knowledge of data modeling and database architecture.
• Experience with Kafka or other streaming technologies.
• Experience with CI/CD, Git, and DevOps practices.
• Strong understanding of data security, governance, and quality.
• Excellent communication, analytical, and problem-solving skills.

Preferred Skills

• Snowflake
• Azure Synapse / Microsoft Fabric
• AWS Redshift / Glue / EMR
• Databricks
• Apache Airflow
• Kafka
• Informatica
• Terraform
• Docker / Kubernetes
• Power BI / Tableau
• NoSQL databases such as MongoDB or Cassandra
• Experience with Agile/Scrum methodology

Education

Bachelor’s or Master’s degree in Computer Science, Information Technology, Engineering, or a related field.

Ideal Candidate

The ideal candidate should be a hands-on Data Engineering Lead/Architect capable of owning end-to-end data solutions, working with modern cloud and Big Data technologies, and providing technical leadership across complex enterprise data projects.