Principal Engineer Data & AI
Indexed description
Key Responsibilities
Architecture & Platform Leadership
- Drives and supports architecture decisions for AI, automation, and data platforms.
- Define and maintain reference architectures, design standards, and reusable frameworks.
- Design automation involving external applications or sites, APIs, and internal applications through scalable microservices.
- Lead implementation of robust lakehouse/warehouse supporting analytics, automation, and AI workloads for agentic AI.
- Establish patterns for batch and streaming pipelines, event‑driven architectures, and scalable data access.
- Design and build robust data pipelines using technologies such as Azure Data Factory, Azure Data Lake, Snowflake, Databricks, SQL, Spark, Python, or other similar technologies.
- Implement strong data quality, lineage, observability, governance, and auditability standards.
- Deliver curated datasets, semantic models, and data products for analytics and downstream systems.
- Lead development of Intelligent Document Processing (IDP), RAG pipelines, GenAI‑driven architectures, and NLP based querying.
- Make the enterprise context and data available easy for business consumption and decisions.
- Develop and identify meaningful insights through “big data”, assists in the creation of required ETL pipelines and data structures for Azure Data Lake, Databricks or snowflake, and Data Factory.
- Design and deploy AI agents, GenAI models, IDP models, and workflow‑driven AI automation.
- Implement and manage MLOps and LLMOps pipelines for training, deployment, monitoring, and governance.
- Integrate AI/ML solutions into systems using APIs, microservices, queues, MCP, and containers.
- Build secure, compliant RAG architectures with vector search and prompt/version management.
- Lead the full lifecycle: discovery, architecture, development, testing, deployment, and support.
- Ensure adherence to enterprise security, DevOps, compliance, and data governance standards.
- Monitor and optimize performance, reliability, and cost of AI/automation platforms.
- Collaborate with the department leader and technical lead to drive technical direction and architecture decisions aligning with standard tech stacks.
- Bachelor’s degree in Computer Science, Data Science, Analytics, or related field (Master’s degree is preferred).
- 7+ years of software, data, AI or platform engineering experience.
- 5+ years building data engineering, AI automation, or cloud‑native solutions.
- Proven experience delivering production AI systems, including ML, MCP servers, GenAI, LLM, IDP, NLP, and RAG‑based architectures.
- Strong hands‑on expertise in Python, SQL, React, C#, Java, and other frameworks.
- Strong experience defining enterprise data structures, data migration across tools, metadata catalogs, and data governance standards, with hands‑on implementation of multi‑tier (raw, curated, consumption) data lake or warehouse architectures.
- Deep experience with most of the Microsoft Azure Services (Data platforms, Docker, Functions, Kubernetes, Azure Containers, Function Apps, ASB, App Services, GitHub, and ML studio).
- Strong stakeholder communication and technical leadership skills.
- Azure Data Factory, ADLS Gen2, Synapse/Fabric, Azure Databricks, Snowflake
- SQL, Python, Java, C#, React, Power Automate, Workato
- Playwright, Azure ML, Azure OpenAI, Document Intelligence
- Docker, Kubernetes, GitHub Actions, CI/CD, semantic models, vector databases
- LangChain, Hugging Face, scikit-learn, TensorFlow, PyTorch, Keras, Hugging Face, OpenCV, NLP, MCP, NLTK, Airflow, Spark, Mistral, ML studio, shell scripting, UAMI, Yaml, and advanced libraries and other open-sources.
- Event‑driven systems (Service Bus, Container Apps, AKS, ACS, KEDA, Event Grid, etc.)
Create a free Caio profile to unlock more results and save your role and location preferences.
Unlock free search