Data Engineer and Analyst
Indexed description
The Data Engineer and Analyst must have good listening skills to be able to understand business needs and
interpret those needs into an efficient and effective application design.
Responsibilities
- Consults with users, clients and other technology groups on issues and recommends advanced programming solutions.
- Accountable for all phases of development process, analysis, design, construction, testing and implementation.
- Provides user and operational support on applications to business users.
- Solves / works complex problems / projects through in-depth evaluation of business processes, system processes and industry standards; performs root cause analyses.
- Engages in post implementation analysis of business usage to ensure successful system design and functionality.
- Provides sound understanding of Big Data apps development concepts and principles and knowledge of concepts and principles in other technology areas.
- Provides evaluative judgment based on analysis of factual information in complicated and unique situations.
- Directly impacts the business by ensuring the quality of work provided by self and others.
- Impacts own team and closely related work teams. Exhibits sound and comprehensive communication and diplomacy skills to exchange complex information
- Strong leadership, interpersonal, influence, negotiation, and written and verbal communication skills required.
- Utilize ad-hoc techniques to perform on-the-fly analysis of data
- Support the division & department deliverables
- Work closely with Department & Section Head to set department direction and AAM according to division and corporate vision
- Work closely with all levels of staff to achieve corporate goals and objectives
- Writes advanced SQL including some query tuning
- Experience in the identification of data quality
- Experience working with Stored Procedures
- Some database design experience is helpful
- Implementing data Compression techniques for space saving
- Experience designing and building complete ETL processes moving and transforming data for ODS, Staging, and Data Warehousing
- Experience with Informatica ETL tool
- Proficient in how to deploy to and debug a production Data Lake environment
- Experience with real time and batch data ingestion frameworks using (Kafka/Flink/Hudi/Hive/Clickhouse)
- Informatica/Shell Scripting / SSIS Packages/ SQL experience
- Experience with Python, Anaconda and Jupiter Notebook skillset is highly desirable
- Hands on Experience with open-source software platforms Linux
- Agile development processes experiences a plus
- Hands on experience in building SSIS packages
- Minimum of 4-6 years of experience in Data Engineering ETL, SQL and Big data preferably within the Financial Services industry.
- At least 1-2 years in a leadership or supervisory capacity.
- Proven track record of delivering complex, large-scale data projects (e.g., system migrations, regulatory reporting).
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search