Data Engineer
Indexed description
Responsibilities
- Participate in cross-functional meetings to define project objectives and scope
- Analyze and prioritize business requirements related to data
- Design and develop data extraction, transformation, and loading pipelines (ETL/ELT)
- Develop applications for extracting data from various sources and formats (JSON, XML, PDF, etc.)
- Write clean, efficient, and scalable code, adapting to new technologies as needed
- Implement data quality methodologies throughout ingestion and transformation processes
- Optimize services, queries, and processes to reduce operational costs and improve performance
- Validate the functionality, integrity, and security of data engineering processes
Specialization in Data Engineering (preferred).
5 to 8 years of experience in similar roles.
Hands-on experience with batch and streaming data processing.
Intermediate-Advanced English proficiency.
Technical Skills:
- Programming languages: Java, Scala, Python, and SQL
- Experience with data processing frameworks (Spark, Flink, etc.)
- Knowledge of SQL and NoSQL databases (PostgreSQL, Cassandra, MongoDB, etc.)
- Experience with orchestration and ETL tools (Airflow, Airbyte, Stitch, etc.)
- Experience handling large-scale data transformations across multiple formats
- Design and architecture of Data Lakes, Data Warehouses, and Lakehouse solutions
- Data modeling (star schema, snowflake schema, OBT)
- Development and consumption of REST APIs
- Agile methodologies and version control (Git)
- Experience with web scraping projects
- Knowledge of additional languages such as Go or JavaScript
- Experience with streaming and messaging systems (Kafka, RabbitMQ)
- Background in data science projects
Create a free Caio profile to unlock the full index and keep your job-search signal for future recommendations.
Unlock free search