Platform Engineer – CI/CD Gate & Online Evaluation

Platform Engineer – CI/CD Gate & Online Evaluation

Project Overview

Our customer is a multinational corporation with more than a century of history and operations in over 180 countries. As part of its transformation journey, the company is introducing a portfolio of Reduced-Risk Products (RRPs) for more than one billion consumers worldwide.

Intellia supports the customer by engineering a comprehensive software ecosystem powering innovative IoT products, digital experiences, and enterprise platforms. Our teams build core platform components for eCommerce, Digital Marketing, IoT, and Digital Engineering solutions.

As a Platform Engineer – CI/CD Gate & Online Evaluation, you will join the Core Architecture Team and play a key role in enabling reliable, scalable, and governed AI deployments by integrating automated quality evaluation into enterprise CI/CD pipelines and production environments.

The platform consists of shared services and applications that accelerate software delivery by addressing common SDLC challenges while providing engineering teams with standardized technologies, best practices, governance, and compliance capabilities.

About the Role

We are looking for a Platform Engineer – CI/CD Gate & Online Evaluation to help establish automated quality controls for enterprise AI platforms and agent-based systems.

In this role, you will integrate evaluation workflows into deployment pipelines, configure online quality monitoring, and build operational visibility for AI performance, reliability, and compliance. Working closely with Platform Engineers, DevOps Engineers, AI Engineers, and Governance teams, you will ensure AI agents meet quality standards before deployment and continue to perform reliably in production.

Responsibilities

• Design and implement CI/CD quality gates using AWS AgentCore Evaluation to validate AI agents and workflows before deployment.
• Integrate evaluation execution into GitHub Actions and deployment pipelines to automate release decisions.
• Configure and maintain on-demand evaluation workflows for pre-release quality validation and regression detection.
• Implement and manage online evaluation capabilities, including production sampling strategies and evaluation scheduling.
• Build Amazon CloudWatch dashboards to visualize evaluation scores, quality trends, and AI agent performance over time.
• Configure CloudWatch alarms and notifications to detect quality degradation, performance regressions, and operational issues.
• Collaborate with AI Engineering teams to define evaluation thresholds, release criteria, and quality acceptance standards.
• Integrate evaluation results into enterprise monitoring, observability, and operational reporting workflows.
• Analyze evaluation outcomes to identify regressions across AI agent versions, prompts, tools, and workflows.
• Develop automation that enables continuous validation of AI systems throughout the software development lifecycle.
• Partner with Platform Engineering and DevOps teams to improve deployment safety, operational readiness, and governance controls for AI workloads.
• Contribute to enterprise AI quality engineering standards, deployment best practices, and monitoring frameworks.

Requirements

Experience

• Bachelor's degree in Computer Science, Software Engineering, Information Technology, or a related field.
• 4+ years of experience in Platform Engineering, DevOps Engineering, Site Reliability Engineering (SRE), or Cloud Engineering.
• Hands-on experience designing and maintaining CI/CD pipelines.
• Experience implementing deployment quality gates and release automation.
• Experience designing CloudWatch dashboards, alarms, and monitoring solutions.
• Experience working with AWS cloud services in production environments.

Technical Skills

• AWS AgentCore Evaluation API (CreateEvaluation) for on-demand evaluation workflows.
• GitHub Actions integration for automated deployment pipelines.
• CI/CD pipeline automation and release orchestration.
• Online evaluation configuration, including production sampling strategies.
• Amazon CloudWatch dashboards, metrics, alarms, and monitoring.
• Quality gate implementation and automated deployment validation.
• Observability and operational monitoring.
• Regression detection and release validation.
• Infrastructure automation and scripting.
• Git version control and modern DevOps practices.

Nice to Have

• Experience with AWS AgentCore Evaluation online mode and sampling configuration.
• Experience comparing AWS AgentCore Runtime versions using automated evaluation for regression detection.
• Experience with CloudWatch Embedded Metric Format (EMF) for structured evaluation metrics.
• Familiarity with enterprise AI governance and Responsible AI practices.
• Experience supporting AI platforms, LLM applications, or agent-based systems.
• Knowledge of Infrastructure as Code (Terraform, AWS CloudFormation, or AWS CDK).

Why Join Us?

• Help build enterprise-scale AI quality engineering and deployment standards.
• Work with cutting-edge AI platforms, cloud-native technologies, and DevOps automation.
• Design automated quality gates that improve the reliability and safety of AI systems.
• Collaborate with highly skilled Platform, AI, DevOps, and Cloud Engineering teams.
• Make a direct impact on the delivery of enterprise AI solutions used at global scale.