AI Test Analyst

12-Month Contract | Brisbane | Hybrid Working

An established organisation is expanding its technology capability and is seeking a AI Test Analyst to support the delivery of a suite of intelligent software solutions used across multiple business functions.

This is an opportunity to join a collaborative delivery team working on applications that incorporate advanced AI and machine learning capabilities into enterprise platforms. The successful candidate will play a key role in ensuring these solutions perform reliably under a wide range of real-world conditions and meet high standards of quality before release.

Rather than focusing solely on traditional functional testing, you'll be responsible for evaluating how intelligent systems behave, respond to changing inputs and integrate within broader business processes.

Key Responsibilities

• Develop practical test approaches for AI-enabled agentic software solutions across multiple concurrent initiatives.
• Design realistic business scenarios that challenge system behaviour under varying conditions.
• Assess the consistency, accuracy and appropriateness of AI-generated responses.
• Investigate unexpected outcomes and identify behavioural trends that may impact business users.
• Perform exploratory and risk-based testing to uncover defects beyond standard scripted testing.
• Validate intelligent components within integrated enterprise environments.
• Support test planning, requirements clarification and definition of acceptance criteria.
• Prepare meaningful test evidence and communicate findings to technical and non-technical stakeholders.
• Work closely with delivery teams to triage issues and support timely resolution.
• Contribute to the ongoing evolution of testing practices for emerging technologies.

About You

You are an experienced Test Analyst who enjoys working on technically challenging projects and understands that testing AI-enabled solutions requires a different mindset from conventional software testing.

You'll ideally bring:

• Demonstrated experience as a Test Analyst within large or complex technology environments.
• Recent commercial experience testing AI, machine learning or intelligent automation solutions.
• Experience validating software where outputs are probabilistic or context-dependent rather than deterministic.
• Strong skills in scenario-based, exploratory and integration testing.
• Experience working across interconnected enterprise applications and services.
• Excellent analytical and critical thinking skills with an ability to identify subtle behavioural issues.
• Confidence working with delivery teams to refine requirements and define quality outcomes.
• Strong written communication and documentation skills.
• Experience working within Agile delivery environments.

Experience testing systems that leverage AI to support decision-making, automation, recommendations or conversational experiences will be highly regarded.

Applicants should be comfortable evaluating system behaviour where responses can vary based on context or input and have experience applying structured testing techniques to intelligent software rather than solely traditional business applications.

If you've been looking for an opportunity to apply modern testing practices to the next generation of enterprise software, we'd like to hear from you.