Senior Data Engineer
Indexed description
This effort will result in a scalable solution offering four key capabilities that include but are not limited to, a secure single source of truth PI claims data repository, interactive data visualization capabilities (financial, operational, experience), integrated early alert and management remediation controls and an advanced data center that enables the use of advanced analytics and leveraging AI/ML to modernize data.
About The Role
As a Senior Data Engineer, you will be responsible for the development of complex data sources and pipelines into our data platform (i.e. Snowflake) along with other data applications (i.e. Azure, Airflow, etc.) and automation.
The Senior Data Engineer will work closely with the data, Architecture, Business Analyst, Data Stewards to integrate and align requirements, specifications and constraints of each element of the requirement. They will also need to help identify gaps in resources, technology, or capabilities required, and work with the data engineering team to identify and implement solutions where appropriate.
Careers with Optum offer flexible work arrangements and individuals who live and work in the Republic of Ireland will have the opportunity to split their monthly work hours between our Dublin or Letterkenny office and telecommuting from a home-based office in a hybrid work model.
What You Will Do
- Integrate data from multiple on prem and cloud sources and systems. Handle data ingestion, transformation, and consolidation to create a unified and reliable data foundation for analysis and reporting
- Develop data transformation routines to clean, normalize, and aggregate data. Apply data processing techniques to handle complex data structures, handle missing or inconsistent data, and prepare the data for analysis, reporting, or machine learning tasks
- Implement data de-identification/data masking in line with company standards
- Develop monitoring tools to automate error handling mechanisms to ensure data integrity and system reliability
- Utilize data quality tools like Great Expectations or Soda to ensure the accuracy, reliability, and integrity of data throughout its lifecycle
- Create, maintain & monitor data pipelines using Airflow & Snowflake as primary tools
- Understand data requirements and design optimal SQL stored procs to fulfil the use-cases
- Creating logical & physical data models to ensure data integrity is maintained
- CI CD pipeline creation & automation using GIT & GIT Actions
What You Will Bring
- Bachelor's degree or higher in Database Management, Information Technology, Computer Science, or a related field
- Proven hands-on experience as a Data Engineer
- Demonstrated proficiency in SQL, including experience utilizing window functions and advanced SQL capabilities
- Demonstrated knowledge of data modeling techniques, data warehousing principles, data architecture, data masking, and implementation methodologies
- Demonstrated solid proficiency in Python
- Proven hands-on experience developing data pipelines and building ETL, ELT, and data integration solutions using Snowflake and complex SQL queries
- Proven experience utilizing programming languages such as Python, Scala, Java, or similar technologies
- Proven experience developing CI/CD pipelines using Git, GitHub Actions, or similar version control and automation technologies
- Proven experience working with relational and non-relational database technologies
- Demonstrated solid communication skills
- Demonstrated knowledge of the U.S. health insurance industry, particularly in claims adjudication and Payment Integrity
- Proven solid experience in Data Engineering
- Proven experience orchestrating data workflows using Airflow, Docker, and Kubernetes for data ingestion, processing, and transformation
- Demonstrated familiarity with Azure services, including Blob Storage, Functions, Azure Data Factory, Service Principals, Containers, and Key Vault
- Proven experience working within Agile and Scrum delivery environments and collaborating with high-performing teams
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search