CommIT
Himalayas · Posted 8d ago
Senior Big Data Engineer
Continue to application
Add your email once, then Caio opens the original posting.
Indexed description
Description
We are looking for a Big Data / Data Engineer to join our growing team. In this role, you will be responsible for the design and development of our big data processing engine a heavy ETL pipeline handling complex logic including geo-location analysis, running across both cloud and on-prem environments.
Requirements:
- 4+ years of experience in Data Engineering
- 2+ years of hands-on experience with Scala (must) or Java (optional)
- 2+ years of experience with SQL and NoSQL DBs
- Hands-on experience with Spark & PySpark (must)
- Experience with Airflow (must)
- Experience with AWS and/or GCP
- Experience with Kubernetes (k8s) and on-prem deployments
- Solid understanding of distributed systems design, pipeline architecture, and partitioning strategies
- Deep understanding of Agile principles, practices and values
- Solid understanding of design patterns, software development techniques, and clean code practices
Requirements
Advantage:
- Experience with Apache Iceberg, Helm Charts, or data catalog tools
- Experience with Spring Framework
- Experience with microservice architecture
- Experience working with DevOps and CI/CD practices
- Customer-facing experience or comfort working directly with clients
- Bachelor's, BSc or Master's degree in Computer Science or Engineering
Originally posted on Himalayas
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search
Want help applying to roles like this?
Search Caio for free. If the repetitive CV tweaking gets heavy, Daniel can help set up Caio Agent.
Ask about Agent
