Lead AI Automation Engineer (Next-Gen QA)

CredServ is building the AI Operating Layer for Embedded Receivable Workflows. We are transforming B2B trade by deploying specialized Agentic AI to automate the entire receivables lifecycle—from Partner Onboarding and Credit Underwriting to Voice Collections and Digital Legal recovery.

The Role: Lead AI Quality Engineer

You are not just testing AI; you are using AI to redefine how testing is done.

We are looking for a pioneering Lead QA Engineer who bridges the gap between deterministic software and non-deterministic AI Agents. You will use AI copilots to hyper-accelerate automation, migrate legacy suites to self-healing frameworks, and build the infrastructure that ensures our core services and autonomous agents operate flawlessly, safely, and within strict financial regulations.

What You Will Do (Core Responsibilities)

• Next-Gen Automation & Core QA
• AI-Assisted Scripting & Regression: Utilize AI copilot tools (e.g., Cursor, GitHub Copilot) to rapidly generate, maintain, and scale automated regression, integration, and functional test suites.
• Modern Frameworks: Architect and scale automation using Playwright or Cypress, transitioning away from and migrating legacy Selenium suites to AI-native, self-healing frameworks.
• Visual & API Testing: Implement Visual AI testing (e.g., Applitools) to catch UI anomalies across multiple devices. Own the end-to-end API testing strategy for our deterministic rule-based logic and core banking integrations.
• Root Cause Analysis: Perform deep triage and root cause analysis of pipeline failures and flaky tests using AI-powered log analysis and observability tools.
• Agentic AI Testing Strategy
• Onboarding Agent: Test conversational KYC flows, document extraction (OCR/NLP) accuracy, and self-serve onboarding logic.
• Underwriting Assistant: Validate the AI's ability to draft accurate credit summaries and reason codes for "Human-in-the-Loop" (HITL) limit reviews.
• Voice Collection Bot: Measure voice latency (Time-to-First-Token), intent recognition, and compliance-grade empathy for recovery calls.
• Digital Legal Module: Verify the automated triggering and tracking of real-time legal notices and case updates.
• Non-Deterministic QA & Guardrails
• LLM-as-a-Judge: Implement hallucination metrics, context precision (RAG testing), and semantic similarity checks to ensure AI outputs never breach regulatory guardrails.
• Red Teaming & Security: Write adversarial test cases to evaluate prompt injection vulnerabilities and data leakage prevention.
  

• Experience: 5+ years of proven experience as an SDET or QA Automation Engineer, with at least 1+ years specifically testing LLMs, RAG systems, or deploying AI-assisted QA workflows.
• Automation Mastery: Deep expertise in Playwright or Cypress. Strong understanding of core QA principles, including the Page Object Model (POM), exhaustive regression testing, and robust CI/CD integration (GitHub Actions/GitLab CI).
• Coding Proficiency: Strong programming skills in Python (for the AI evaluation stack) and/or Node.js/TypeScript (for UI/API automation).
• AI QA Tooling: Hands-on experience with AI coding assistants (Cursor), Visual AI (Applitools), and LLM evaluation frameworks (e.g., LangSmith, DeepEval or Ragas).
• Financial Precision: Deep understanding of testing "Human-in-the-Loop" workflows, complex reconciliation logic, and "exception-clearing" AI where accuracy is non-negotiable.
  

You will be defining how global supply chains and enterprises evaluate and trust Artificial Intelligence. If you want to work at the bleeding edge of Next-Gen Automation, Voice AI, and B2B trade, this is your place.

• Competitive Salary & Equity
• Flexible Work Options
• Comprehensive Health & Wellness Benefits