TR Book a 30-min call

How it works

From Jira story to reviewed Playwright test, one gate at a time

AI Testing Probe takes a requirement through six stages: requirement, pre-review, test cases, code, CI and triage. AI drafts at each stage, a named person decides, and deterministic system steps run, check and record. Outputs stay in your own Jira, Xray and Git repository.

The flow

Six stages, every step accounted for

For each step: what happens, who does it, which tool it stays in and what it produces.

  1. Stage 1

    Requirement

    A Jira story, or a free-text requirement, enters the flow with its acceptance criteria.

    1. Pull the story

      The analyst pulls stories from the configured Jira project. The platform reads Jira with a dedicated read-only account. Requirements that do not live in Jira can be pasted as free text.

      Who
      HumanSystem
      Stays in
      Jira Cloud (read-only)
      Output
      A requirement with its acceptance criteria, linked back to the Jira issue

    Docs: Requirement review

  2. Stage 2

    Pre-review and consistency check

    Before any test is written, AI checks the story and compares it with the application. Every finding carries evidence.

    1. Pre-review

      Demonstrated

      AI reads the story for ambiguity, missing acceptance criteria and contradictions. Story content is treated as untrusted data, and the output must match a JSON schema.

      Who
      AI
      Stays in
      AI Testing Probe
      Output
      Draft findings

    2. Consistency check

      Demonstrated

      AI compares the story with the application code, read with a read-only token, and with data from restricted, masked read-only queries. Each finding cites a file and line or a query result. The platform re-verifies that evidence without a model and drops any finding it cannot confirm.

      Who
      AISystem
      Stays in
      Your repository and database, read-only
      Output
      Evidence-backed findings

    3. Decide and approve

      The analyst marks each finding "fix the document" or "code or data is wrong". Document fixes are written back to Jira by a write account limited to the project; code defects become bug drafts. The product owner approves the corrected story in Jira.

      Who
      HumanSystem
      Stays in
      Review portal and Jira
      Output
      An approved story, plus bug drafts for code defects

    Docs: Requirement review

  3. Stage 3

    Test cases

    AI drafts test cases from the approved story. Only approved, unchanged cases reach your test management tool.

    1. Draft cases

      Demonstrated

      AI drafts positive, negative and boundary cases in a fixed schema. Each case references the acceptance criteria it covers, and the platform lists any criterion that no case covers.

      Who
      AI
      Stays in
      AI Testing Probe
      Output
      Draft test cases tagged AI-generated

    2. Review

      The analyst approves, edits or rejects every case. Edits are kept as a field-level diff, and each approval is bound to the content hash of the case.

      Who
      Human
      Stays in
      Review portal
      Output
      Approved test cases

    3. Publish

      A publish preview lists every action first, without writing. Approved cases are then written to Jira Test issues or to Xray Cloud and linked to the story.

      Who
      HumanSystem
      Stays in
      Jira or Xray Cloud
      Output
      Linked test cases, with the audit chain hash in a story comment

    Docs: Test cases

  4. Stage 4

    Code

    Approved cases become plain Playwright tests in your repository, each one checked against a deliberately broken app.

    1. Verify each step in the app

      Demonstrated

      An agent walks through the steps in the running application with an allowlisted set of browser tools and maps each step to a symbol in your page object library. A locator that matches no known symbol stops the case and sends it to a human.

      Who
      AISystem
      Stays in
      Your test environment
      Output
      A verified step-to-page-object mapping

    2. Fill the template

      The model does not write TypeScript. The spec file is produced from a template, values go in as type-checked literals, and assertion values come from the approved case. Compiler, linter and AST guard checks run before a pull request opens.

      Who
      System
      Stays in
      Toolchain
      Output
      One spec file per case, opened as a pull request

    3. Break-to-Verify

      Demonstrated

      The test runs on the normal app, on the app with a controlled fault switched on, and on the normal app again. A test that stays green while the app is broken can only merge with an explicit, logged acceptance.

      Who
      System
      Stays in
      Isolated test runner
      Output
      A passed, weak or failed result on the pull request

    4. Review and merge

      The engineer reviews the code and the mapping table and merges through your protected main branch. The platform's bot opens pull requests but never merges them.

      Who
      Human
      Stays in
      GitHub
      Output
      A merged, plain Playwright test in your repo

    Docs: Code generation

  5. Stage 5

    CI

    Your pipeline runs approved, deterministic tests. There is no model in it.

    1. Run the approved suite

      The same suite runs on pull request, push, deploy and nightly triggers. CI and test images contain no model key, no agent and no model SDK, and an automated security check confirms it.

      Who
      System
      Stays in
      GitHub Actions (Sauce Labs cloud runs: Roadmap)
      Output
      Standard JUnit XML, Allure and Playwright reports

    2. Collect results

      Demonstrated

      The platform finds each run, downloads its artefacts, writes the result as one comment on the pull request and shows it in the Control Tower.

      Who
      System
      Stays in
      Control Tower
      Output
      Run history across stories, pull requests and runs

    Docs: Architecture

  6. Stage 6

    Triage

    Failures are classified by rules first and AI second, and every fix goes through review.

    1. Classify the failure

      Demonstrated

      Deterministic rules classify the failure first. The model is called with failure details and log excerpts only when no rule matches.

      Who
      SystemAI
      Stays in
      AI Testing Probe
      Output
      A classified failure

    2. Repair PR

      When a test ID changed but the element's role and accessible name did not, the platform proposes a pull request that changes only the test ID. A human reviews and merges it. Nothing heals silently.

      Who
      SystemHuman
      Stays in
      GitHub
      Output
      A reviewed repair PR

    3. Bug draft

      A failure where the expected result differs from the actual one is never repaired. It becomes a bug draft that is first searched against existing issues. The tester files it, links it, edits it or rejects it.

      Who
      AIHuman
      Stays in
      Jira
      Output
      A filed or linked bug

    Docs: Concepts

The core rule

No AI output becomes a test asset without human approval. Every AI call, approval and rejection is recorded in a hash-chained audit log.

Data flow

What goes where, and what reaches the model

The platform runs in your environment. Only one service calls the model, and it sends only what the current step needs.

Data flow between your tools, AI Testing Probe and the model provider Your Jira and Xray, Git repository, application under test and application database sit on the left. AI Testing Probe runs in your environment in the middle, with the core worker and model gateway, the review portal and an isolated test runner. The model provider, Anthropic Claude today, sits on the right and is reached only by the core worker. Numbered arrows match the list below the diagram. Your tools AI Testing Probe · your environment Model provider Jira / Xray stories, cases, bugs Git repository + CI plain Playwright, no AI Application under test test environment Application database read-only role Core worker model gateway · only AI caller Review portal people approve and merge Test runner no internet, no secrets Anthropic Claude swappable model gateway 1 2 3 4 5

The numbered flows

  1. Stories and acceptance criteria are read with a read-only account. Only after a human decision does a project-limited write account send fixes, approved cases and bugs back.
  2. Application code is read with a read-only token. Generated tests and repairs arrive as pull requests from a GitHub App that never merges.
  3. Data checks run as single SELECT queries through a read-only role, on allowed tables and columns, with row and time limits and masked results.
  4. The core worker sends the model only what the current step needs. Drafts must come back in a JSON schema, and nothing becomes a test asset before a person approves it.
  5. Generated tests run against your test environment from a runner with no internet access and no secrets. In CI, the same approved tests run without any model.
What reaches the model at each step
StepSent to the modelNot sent
Requirement reviewStory text, acceptance criteria and the product owner's answersCredentials, database access
Consistency checkRelevant code from your repository and masked results of restricted queriesRaw database contents, write access
Test case generationThe approved story, acceptance criteria, product owner answers and test dataUnapproved story versions
Code generationCase steps and the agent's view of the page through allowlisted browser toolsScreenshots, which are kept as evidence for reviewers
Triage and bug draftsFailure details and CI log excerpts, only when no rule could classify the failureFailures that a rule already classified
CI runsNothing. No model runs in CI.Everything

Read the architecture: services, network isolation and the model gateway

Start with one story.

In a 30-minute call we look at your stack and a few of your stories, and tell you honestly whether a pilot fits.

Book a 30-min call Read the security overview