Senior AI Site Reliability Engineer

About Us

At YTL AI Labs, we build sovereign AI models that perform on par with the world’s best- while staying grounded in local needs, values, and context. Our flagship model, ILMU, is designed to be culturally aware, contextually intelligent, and fluent in Bahasa Melayu, delivering cutting-edge solutions that empower Malaysian businesses with intelligence that truly understands the market and the people they serve.As pioneers of sovereign AI, we believe every nation should have the power to shape its own intelligenc - guided by its people, priorities, and principles.

About the Role

As a Senior AI Site Reliability Engineer, you will build, operate, and scale the core infrastructure powering ILMU and the AI runtime layer that drives model serving, inference workloads, retrieval pipelines, and agent execution. You will be hands-on in ensuring the reliability and performance of our infrastructure across the cloud, on-prem GPU clusters, and hybrid deployments, so that our LLM inference, agentic workflows, and platform services run with industry-leading uptime and efficiency.

You will help establish and strengthen SRE practices across the organisation: contributing to standards for availability, SLIs/SLOs, incident response, observability, performance optimisation, and capacity planning. You will work closely with platform engineering, AI research, product, and security teams to drive operational excellence across all stages of the AI lifecycle, from offline training pipelines to real-time inference, and mentor junior engineers along the way.

This role requires strong systems engineering expertise and cloud and container orchestration proficiency to ensure the core infrastructure powering ILMU can scale reliably to millions of users and enterprise partners.

Key Responsibilities:

Reliability & Systems Architecture

• Design and scale highly available infrastructure supporting LLM runtime, model hosting, vector search, and platform microservices
• Build resilient traffic routing, multi-AZ/region failover strategies, and model fallback pathways for inference continuity
• Contribute reliability-focused design decisions across compute, storage, networking, and GPU environments
• Implement and uphold SLOs, SLIs, SLAs, and error budgets across services and model-serving pathways

Infrastructure Operations & Automation

• Operate Kubernetes-based environments across cloud and on-prem GPU clusters, ensuring consistent configuration, security, and operational readiness
• Automate deployment workflows, CI/CD pipelines, and standardised rollout processes that ensure safe, repeatable delivery
• Build and enhance incident response automation, auto-remediation, and operational runbooks
• Work with engineering leads to support deployment safety, change management controls, and production-readiness checklists

Observability, Monitoring & Performance

• Build and maintain end-to-end observability for model servers, vector DBs, agent frameworks, and platform
• APIsImplement logging, tracing, metrics pipelines, and real-time health dashboards across all runtime systems
• Contribute to performance engineering initiatives: benchmarking, load testing, cost telemetry, and model throughput optimisation
• Investigate and resolve systemic bottlenecks across compute, GPU, networking, and storage paths

Security, Compliance & Platform Governance

• Apply infrastructure security best practices across API gateways, IAM, VPCs, and hybrid-cloud environments
• Collaborate with security teams to implement compliance controls (SOC 2, ISO 27001, PDPA)
• Maintain rigorous auditability, access controls, secrets governance, and overall platform hardening

Collaboration & Mentorship

• Mentor junior SREs and infrastructure-focused engineers
• Partner cross-functionally with AI researchers, platform engineering, backend, and product teams to ensure end-to-end reliability
• Participate actively in incident management, postmortems, and continuous improvement programs with a blameless culture
• Champion operational excellence, engineering discipline, and reliability-first thinking

Skills & Qualifications

Must-Have

• 5+ years in SRE, infrastructure engineering, platform engineering, DevOps, or equivalent roles
• Strong expertise in Kubernetes, Terraform, and cloud-native architectures (AWS/A
• zure/GCP)Deep knowledge of observability tools (Prometheus, Grafana, OpenTelemetry, ELK, Datadog, etc.)
• Strong background in CI/CD, GitOps, infrastructure-as-code, and container orch
• estrationProven experience with availability engineering, incident management, and scaling distributed systems
• Solid understanding of networking (VPCs, VPNs, load balancers, service mesh), hybrid connectivity, and routing
• Strong command of Linux systems internals, performance debugging, and distributed systems
• Understands on-prem infrastructure (SAN, Proxmox, firewalls, switches, routers) and/or DevOps tooling (ArgoCD, public cloud components, K8s, LGTM monitoring stacks), and/or model inferencing/benchmark testing experience (bonus)

Bonus

• Experience supporting GPU clusters or LLM inference/model-serving workloads
• Familiarity with vLLM, SGLang, Triton, TGI, or other model-serving platforms
• Experience operating on-prem, high-performance compute clusters
• Knowledge of cost telemetry, infra budgeting, and capacity planning for large-scale systems
• Experience with compliance frameworks (SOC 2, GDPR, HIPAA, PDPA)
• Background in database performance, replication, and distributed storage systems

What Success Looks Like

• ILMU's model-serving and platform infrastructure achieves consistently high availability (99.5%+ or defined SLO targets)
• Deployments are predictable, low-risk, and fully traceable, with automated verification and rollback mechanisms
• Full-stack observability is implemented: metrics, logs, traces, dashboards, model performance telemetry
• Engineering teams detect issues proactively — before customers report them
• Every significant incident results in a clear, blameless postmortem with measurable reliability outcomes
• GPU cluster utilisation, inference performance, load testing, and capacity planning are stable, predictable, and cost-aware
• Infrastructure scales seamlessly as ILMU's traffic grows across enterprise and national deployments
• You operate as a trusted, proactive reliability partner: shaping system design, not just reacting to operational issues