Enterprise Data Engineer | Cloud (AWS, Azure) | Big Data (Spark, Hadoop) | ETL & Data Pipelines | SQL & NoSQL
Indexed description
Software Requirements
Required:
- Strong understanding of data management concepts, cloud platforms (preferably AWS or Azure), and scalable architectures.
- Hands-on experience with programming languages such as Python, Java, or Node.js.
- Practical experience with big data tools like Apache Spark, Hadoop, Flink, or similar frameworks.
- Working knowledge of databases such as SQL (MySQL, SQL Server, PostgreSQL) and NoSQL databases (e.g., MongoDB, DynamoDB).
- Experience with data orchestration and pipeline tools such as Apache Airflow, Luigi, or comparable frameworks.
- Familiarity with version control systems such as Git and collaboration tools like JIRA and Confluence.
- Knowledge of containerization (Docker, Kubernetes) and infrastructure as code (Terraform, CloudFormation).
- Experience in deploying and managing data pipelines on cloud platforms like AWS Glue, Azure Data Factory, or GCP Dataflow.
- Familiarity with stream processing tools like Kafka or Kinesis.
- Exposure to data security protocols and compliance standards (GDPR, HIPAA, etc.).
- Design, develop, and maintain large-scale data pipelines, ETL workflows, and data integrations to support analytics, reporting, and operational needs.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver reliable solutions.
- Optimize and monitor data pipelines for performance, scalability, and data quality.
- Implement data governance, validation, and cataloging processes to ensure data integrity and security.
- Automate deployment, testing, and data infrastructure changes using CI/CD practices.
- Participate in architecture discussions, technical reviews, and documentation to support data ecosystem growth.
- Stay informed of emerging data technologies, industry standards, and best practices, and incorporate relevant innovations.
Technical Skills (By Category)
Programming Languages:
- Essential: Python, Java, or Node.js
- Preferred: Spark (PySpark, Spark Scala), SQL for data manipulation
- Essential: SQL database management (MySQL, PostgreSQL, SQL Server)
- Preferred: NoSQL databases (MongoDB, DynamoDB)
- Preferred: AWS (Glue, S3, EMR), Azure Data Factory, GCP Dataflow
- Essential: Apache Spark, Kafka, Hadoop ecosystem components
- Preferred: Dask, Flink
- Essential: Git, Jenkins, CI/CD pipelines, Agile/Scrum practices
- Preferred: Terraform, Docker, Kubernetes, DataOps tools
- Awareness of data encryption, access controls, and compliance frameworks such as GDPR, HIPAA, and data masking best practices.
- Minimum of 5+ years developing and maintaining enterprise data pipelines and big data solutions.
- Proven experience in designing scalable ETL workflows, integrating cloud data services, and optimizing data processes.
- Demonstrable success in deploying data solutions that support reporting, analytics, and machine learning initiatives.
- Industry experience in finance, healthcare, retail, or enterprise sectors highly desirable; relevant open-source or academic projects also acceptable.
- Develop, test, and deploy scalable data pipelines and ETL workflows.
- Collaborate with business and data science teams to gather requirements and deliver data solutions.
- Monitor data pipelines and optimize for performance, reliability, and security.
- Troubleshoot technical issues, perform root cause analysis, and apply fixes.
- Automate deployment and infrastructure provisioning procedures.
- Maintain detailed documentation of data architecture, workflows, and operational guidelines.
- Proactively research emerging data tools and platforms to recommend innovation.
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, or related disciplines.
- 5+ years of experience supporting enterprise data ecosystems, especially on cloud platforms.
- Experience with big data frameworks, cloud data services, and automation tools.
- Certifications in cloud platforms (AWS Data Analytics, Azure Data Engineer, GCP Data Engineer) are advantageous.
- Strong problem-solving, analytical thinking, and communication skills.
- Critical thinking to design innovative and scalable data architectures.
- Leadership skills to mentor junior staff and guide data projects.
- Effective stakeholder management for cross-team collaboration.
- Adaptability to rapidly evolving data technologies and organizational needs.
- Ownership of data quality, security, and compliance standards.
- Time management skills to effectively prioritize tasks and meet project deadlines.
All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.
Candidate Application Notice
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search