恩士迅信息科技(上海)有限公司
Linkedin · Posted 10d ago
Data Engineer（Spark,Scala,Pyspark）
Continue to application
Add your email once, then Caio opens the original posting.
Indexed description
该职位来源于猎聘 Responsibilities
- Deliver solutions for online data extraction, data integration and data management.
- Develop and maintain high-performance, scalable utilities to support technology research and data transformation.
- Optimize data processing protocols and systems for better efficiency and maintainability.
- Contribute to the establishment and maintenance of distributed computing platform and big data services
- Writing of documents that clearly explain how algorithms should be implemented, verified and validated. Requirements
- Experience in a field encompassing Distributed computing, Data Analytics, Data Transformation
- Data Engineer with hands-on experience in Hadoop, SQL, Hive/Impala, Spark (Scala/ Python/ Java), Parquet/Avro/HBase and HDFS.
- Analyse and address issues relating to capacity constraints and performance related items on SQL and Spark ETL processing.
- Strong in Linux/Unix Commands and Scripting (Shell Scripting and Python/Scala).
- Experiences in automation of batches with Shell scripts + REST APIs (triggering and result parsing).
- Experience in using AWS products like S3 Storage.
- Strong problem-solving skills with the can-do attitude to be able to come up with solutions for various problems on a wide technical area.
- Knowledge of professional software engineering practices & best practices for the full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search
Want help applying to roles like this?
Search Caio for free. If repetitive applications get heavy, Managed Job Search adds supervised execution for $99/month.
View Managed Job Search
