Quick start

AI Testing Probe turns Jira stories into reviewed Playwright tests in your own repository. AI drafts every step, a named person approves every step, and the tests that run in CI contain no AI. This page explains the six stages and follows one example story from start to finish.

What AI Testing Probe is

AI Testing Probe is a governed test engineering platform. It takes a requirement, either a Jira story or free text, and moves it through a flow with human gates until it becomes an executable, maintainable Playwright test.

The platform is built on three rules:

• No AI output becomes a test asset without human approval. Findings, test cases, code, repairs and bug reports are all drafts until the right role approves them.
• Every AI call, tool call, approval and rejection is recorded in a hash-chained, append-only audit log.
• No AI runs in CI. The pipeline runs only approved, deterministic test code. There is no model key, agent or model SDK in the CI and test images.

Outputs stay in your tools: test cases go to Jira or Xray, test code goes to your Git repository as pull requests, and results come back as standard JUnit XML and Allure reports. The generated code is plain Playwright with your page objects, so it keeps working without the platform.

The six stages

Each stage has an AI part, a human part and a system part.

Stage	What happens	Who decides
1. Requirement	The story is read from Jira (read-only) or entered as free text.	System
2. Pre-review and consistency check	AI finds ambiguity, missing acceptance criteria and contradictions, and compares the story with the application’s code and data. Every finding carries evidence.	Analyst decides; product owner approves in Jira
3. Test cases	AI drafts cases linked to acceptance criteria.	Analyst reviews and publishes
4. Playwright code	An agent verifies each step in the running app; code is filled into a template and opened as a pull request. Break-to-Verify checks that the test catches a fault.	Engineer reviews and merges
5. CI	Approved tests run on PR, push, deploy and nightly triggers. Results go to the Control Tower.	System
6. Triage	Deterministic rules classify failures first, AI second. Locator breaks get a repair PR; real defects get a bug draft.	Engineer merges repairs; tester files bugs

One story, end to end

This walkthrough uses a fictional example: the Acme Retail web shop and the story STORY-142 Order cancellation. The story says a customer can cancel an order until it ships, gets a refund to the original payment method, and receives a confirmation email.

1. The analyst pulls STORY-142 from Jira. The platform shows the description and the acceptance criteria AC1 to AC4.
2. The analyst starts the pre-review. AI reports that the story allows cancellation for 24 hours, while the code and the settings table allow a shorter window. It also reports that the confirmation email in AC3 is missing from the code. Each finding links to a file and line or to a read-only query result.
3. The analyst marks one finding “fix the document” and another “code is wrong”. Document fixes are written back to Jira; the code defect becomes a bug draft. The product owner answers in Jira and adds the approval label.
4. AI drafts test cases. The analyst approves, edits or rejects each one, then publishes the approved cases to Jira or Xray.
5. The engineer generates Playwright code for the approved cases. Each case becomes one pull request.
6. Break-to-Verify runs each test against the normal app, a deliberately broken app and the normal app again. The engineer merges, and CI takes over.

What you need for a pilot

A pilot usually starts with one web application and a few real stories. You need:

• A Jira Cloud project with stories that have acceptance criteria. Xray Cloud is optional; without it, cases are published as Jira Test issues.
• A GitHub repository for the test code, with GitHub Actions. The platform opens pull requests through a GitHub App and never merges them.
• A Playwright page object library and stable test IDs in the application. If they don’t exist yet, setting them up is the first pilot step.
• A test environment of the application that the platform’s browser agent and test runner can reach.
• A Linux host with Docker for the platform services. Today the platform is installed with Docker Compose on a single server.
• A model provider account. Today that is Anthropic Claude through the model gateway.

Pilots run on synthetic or non-production data. See Integrations setup for accounts and permissions, and the Pilot page for scope.

How to read the maturity labels

Every capability on this site carries one of three labels. The label is written as text next to headings, so it also appears in search results and Ask AI answers.

• Live-verified: evaluated against a live model and run on real systems such as Jira Cloud, Xray, GitHub and GitHub Actions. The evidence and date are listed on the Capabilities page.
• Demonstrated: works end to end and is shown in demos, either on a live platform or with recorded model outputs (REPLAY mode). The live model evaluation is not complete.
• Roadmap: not available yet, or only designed. Roadmap items are described, never documented as how-to steps.

Demos may run in REPLAY mode. In REPLAY, the output of each AI step comes from a curated record whose input has been verified, and the interface marks every such output with a REPLAY badge. The model gateway, logging, schema validation and human gates run through the same code path as in live mode. The Capabilities page is the single, current source for every label.

Where to go next

The documentation is organised in four groups.

• Start: this page and Concepts, which defines Approval Gates, roles, evidence-backed findings and the audit log.
• Flow: Requirement review, Test cases, Code generation and Break-to-Verify, one page per stage with the decisions each role makes.
• Platform: Architecture covers the services, network isolation, the model gateway and what data reaches the model.
• Setup: Integrations setup lists what Jira Cloud, Xray, GitHub and Sauce Labs (Roadmap) need, with least-privilege permissions.

If you are evaluating security, start with the Security page, which summarises data flow, isolation and audit for reviewers. For questions about pricing, duration or commitments, use the contact page; those topics are settled in a pilot conversation.