Data and AI Architect

JOB DESCRIPTION- Data and AI Architect

Position: Data & AI Architect

Location: Vietnam - working hybrid

Job Type: Full-Time

Work timings: Singapore Time zone

Role Summary:

We are looking for an experienced Data and AI Architect to lead the data platform modernization and AI implementation programs covering discovery, assessment, design and migration strategy for transitioning an On-Premises Data Warehouse platform to a target Modernized On-Premises/Hybrid/Cloud Data Warehouse or Data Lakehouse platform. Additionally, the role will also cover AI use case detailing and implementation using Cloud data and AI platforms. This individual will be responsible for assessing the current environment, proposing a viable migration strategy with high-level architecture blueprints and designing of a target Data lakehouse solution with AI capability overlay. The individual should be capable of evaluating technological options and providing cost-benefit analyses. This individual should also have sufficient knowledge of data ecosystem and AI/ML concepts covering data engineering, data governance, AI model implementation, tune the AI/ML models, training and visualization. The ideal candidate should have over 15+ years of experience in IT, with majority or specialization in data and AI architecture design, solutioning, sizing and migration for on premises and cloud-based solutions.

Key Responsibilities:

• Discovery & Assessment:

o Conduct a thorough evaluation and assessment of our customer’s current data warehouse platform, to identify key pain points, challenges, business use case, and opportunities for improvement.

o Assess SQLs, ETLs, cubes and other entities, integrations, consumers, data models and structures that need migration and transformation to move to new lake house.

o Assess use case requirements, data readiness for AI implementation. Identify if it is Gen AI or AI/ML requirement. Identify integration requirements and feasibility.

o Understand customer requirements to adopt and implement the target state.

o Identify security, governance and non-functional requirements.

o Identify data migration requirements and risks if any.

• Design Architecture

o Design Architecture for Data lakehouse on platform chosen for implementation.

o Design AI blueprint architecture with integrations, security and governance requirements. o Define design decisions, dependencies and policies to be implemented.

o Define environment sizing, integrations and non-functional design decisions.

o Design devops integrated with the overall data platform and AI solution covering tools, configuration, sizing and set up.

o Design code repository and knowledge base structure to be adopted for delivery.

o Lead design walkthrough and seek customer sign off.

o Review the low level design and ensure implementation is aligned to design.

o Present to customer on solution and technical detailing.

• Design Migration Strategy and Solution:

o Design a comprehensive migration strategy that outlines the approach, mapping, data model, cutover strategy, timeline, and potential risks associated with the transition to a target data warehouse or Data Lakes or to new AI model/solution.

• Implementation

o Govern the implementation through configuration, design adoption, testing and integration.

o Develop and Configure the ETLs, transformation logic, integrations

o Unit testing and support UAT

o Cutover strategy implementation and ensuring smooth cutover

o Post production support

• Technology Evaluation:

o Research and evaluate various data and AI platform options, including on-premises, hybrid, and cloud only solutions, to recommend the most suitable technology alignment with customers’ business objectives.

o The recommendation of technology should be based on perform cost-benefit analyses and recommend the most viable technology stack with rationale.

• Data Strategy:

o Develop strategies for data acquisition, integration, transformation, pipeline creation, and data mart creation.

o Develop a data strategy that defines the organization's data assets, data quality standards, and data usage policies.

• Governance & Security:

o Design a robust data governance and security framework to ensure data integrity, confidentiality, and compliance with relevant regulations.

• Stakeholder Communication:

o Collaborate with stakeholders to understand requirements and provide updates.

o Ensure alignment with business objectives and technical needs.

• Project Methodology:

o Should have understanding of project delivery methodologies such as Agile and DevOps to propose Implementation Timeline.

Must Have Skills and Knowledge:

Skill/Knowledge

Description

Experience

10+ years in Data analytics, Data Lake and Data warehouse, Visualization domains and AI/ML majority of which is in designing and implementing production-grade Data engineering, analytics and AI/ML solutions.

Domain Knowledge

Should have knowledge in at least 2 domains expertise from Banking/ Insurance/ Fintech/ Manufacturing/ Logistics/ Telecom/ Media

Architecture Blueprint

As Architect, delivered at least 4 programs leading the design and implementation of data warehouse, data lake solutions and AI/ML solutions, with a deep understanding of data modeling for OLAP application, ETL processes, analytics and AI applications. Ability to design high-level architecture blueprints for data platforms and applied AI.

Migration Expertise

As Architect, led at least 2 programs for Data Warehouse migration or modernization.

End-to-End Data Processing

Good understanding of data domain concepts like data ingestion, data governance, data catalog, data classification, data engineering, data analytics and data visualization

Data Science and Analytics

Good understanding of AI and ML and its integration with data platforms.

Platform Skills

Must have platform design and implementation experience in

1. (Azure Data Fabric, ADF and Power BI)

2. Snowflake

3. AWS Bedrock or Claude or similar AI model implementation

4. Azure AI foundry

Technical Skills

AWS Cloud, Azure Cloud, Fusion, Fivetran, Airbyte, DBT, Airflow, Python, Pyspark, SQL

Governance & Security

Experience in developing and implementing data governance frameworks, including data quality standards, metadata management, and data security measures for Data Warehouse Application.

Problem-Solving

Excellent problem-solving skills to identify and address challenges related to data migration and cloud adoption.

Communication Skills

Strong verbal and written communication skills.

Certifications

Professional Architect certifications from at least one of Leading Cloud Provider (Azure, AWS, Snowflake)

Good to Have Skills and Knowledge:

Skill/Knowledge

Description

DWH Technologies

Familiarity with Programing technologies like Python, Java, etc. Big Data Technologies like Hadoop, Spark, or Kafka. Experience of working with Relational and Non-Relational Database, Columnar Database, NoSQL Database, Graph Database, etc.

On-prem and Cloud Data Platforms

Experience with Databricks or AWS would be a plus as would be good working knowledge of on-prem data platforms