CommIT
Himalayas · Posted 11d ago
Senior Big Data Engineer
Continue to application
Add your email once, then Caio opens the original posting.
Indexed description
Description
We are looking for a Big Data / Data Engineer to join our growing team. In this role, you will be responsible for the design and development of our big data processing engine a heavy ETL pipeline handling complex logic including geo-location analysis, running across both cloud and on-prem environments.
Requirements:
- 4+ years of experience in Data Engineering
- 2+ years of hands-on experience with Scala (must) or Java (optional)
- 2+ years of experience with SQL and NoSQL DBs
- Hands-on experience with Spark & PySpark (must)
- Experience with Airflow (must)
- Experience with AWS and/or GCP
- Experience with Kubernetes (k8s) and on-prem deployments
- Solid understanding of distributed systems design, pipeline architecture, and partitioning strategies
- Deep understanding of Agile principles, practices and values
- Solid understanding of design patterns, software development techniques, and clean code practices
Requirements
Advantage:
- Experience with Apache Iceberg, Helm Charts, or data catalog tools
- Experience with Spring Framework
- Experience with microservice architecture
- Experience working with DevOps and CI/CD practices
- Customer-facing experience or comfort working directly with clients
- Bachelor's, BSc or Master's degree in Computer Science or Engineering
Originally posted on Himalayas
Create a free Caio profile to unlock the full index and keep your job-search signal for future recommendations.
Unlock free search