Integrations setup

AI Testing Probe connects to your existing tools through adapters: Jira Cloud for requirements, Jira or Xray Cloud for test cases, GitHub and GitHub Actions for code and CI, Sauce Labs for cloud browsers (Roadmap), and a model provider through the model gateway. This page lists what each integration needs and the minimum permissions to grant.

Principles for every integration

The same rules apply to every connection, whichever tool it is.

• Separate read and write identities. Reading and writing use different accounts or tokens. The write identity is limited to the project or repository the platform works in.
• Credentials live in one place. Model keys and write credentials are given only to the core worker service. The toolchain, the test runner and CI never receive them, and an automated security check confirms this after every change.
• Secrets stay out of Git. Credentials are kept in the environment file on the platform host, which is never committed, or as CI secrets. Commands that verify a connection never print secret values.
• Same code, simulator or real. Every external system sits behind a port. During evaluation and development, a local simulator can stand in for Jira, Git and CI; the real adapter uses the same client code. Switching is a configuration change.
• Honest labels. AI output that does not come from a live model is always shown as REPLAY.

When a pilot ends, rotate or revoke all tokens and keys that were issued for it.

Jira Cloud

Jira Cloud holds the requirements and receives document fixes, product owner questions and bugs.

Project. Use a project with the Story and Bug issue types. A Test issue type and a Test link type are recommended; without them, the platform falls back to another issue type and to Relates. The workflow needs a status in the “Done” category. Acceptance criteria should be under an “Acceptance Criteria” heading in the description, as numbered lines such as AC1: .... If they live in a custom field, plan a short adaptation.

Accounts.

Account	Permissions
Read	Browse projects, in the target project only. It must not hold add-comment, create or edit permissions in any project.
Write (bot)	In the target project only: browse, create, edit, add comments, link and transition issues
Product owner (a person)	Comment on and label stories

Each account creates its own API token in its Atlassian account settings. The product owner must be a different Atlassian account from the read and write accounts. Approvals are matched by account ID or email, never by display name.

Product owner approval settings

Product owner approval is a human gate given in Jira, so it needs a little configuration.

• Allowed approvers. List the Atlassian account IDs of the people who may approve. If the list is empty, no approval is accepted on real Jira.
• Bot account ID. Register the write account’s ID so that a label it adds is never counted as approval.
• Role mentions (optional). Account IDs for the analyst, product owner, engineer and tester roles let platform comments mention the right people. A mention is a notification only; it grants no rights.

The approval itself is a label on the story. The platform reads the issue’s change history to confirm who added it. It never adds this label itself, and it removes the label if it was added by the bot account.

Every comment the platform writes begins with an AI Testing Probe: marker, so platform comments are easy to tell apart from human comments even when the write account belongs to a named person.

Xray Cloud

Xray Cloud is optional. Without it, approved cases are published as Jira Test issues with steps in the description.

To publish structured manual tests with steps:

1. Make sure the Jira Cloud setup above is complete. Xray links the test to the story through Jira.
2. Install Xray Cloud on the site and enable it for the target project.
3. In Xray’s global settings, create an API key (client ID and secret) bound to the platform’s write account.
4. Add the key to the platform host’s environment file and restart the services.

The publish preview in the portal then shows Xray as the target. After the first publish, each case appears as a manual test with steps, linked to the story.

In a project where Xray is installed, the Test issue type belongs to Xray. Publishing to Jira Test issues still works in such a project, but those tests have no Xray steps. Choose one target per project and keep it.

GitHub and GitHub Actions

GitHub holds the generated tests and runs CI. The platform uses four identities, each with the narrowest scope.

Identity	What it does
GitHub App (bot)	Creates branches and pull requests for generated tests and repairs, comments, closes superseded PRs, updates branches, triggers workflows and reads artefacts. It never merges.
Human merge identity	Merges from the portal on behalf of an engineer. Belongs to a team allowed to bypass the ruleset.
Read-only token	Lets the consistency check read application code.
Read-only mirror token	Lets the toolchain fetch commits for checks and Break-to-Verify.

Install the App only on the test repository, with Contents and Pull requests write, Actions read and write, and Checks read.

Ruleset for main: changes only through pull requests; at least one approval, code owner review and approval of the last push; required checks for tests, the AST guard and static analysis; no force-push or deletion. Only the human team may bypass. Push the initial content before applying the ruleset.

CI triggers and Sauce Labs

The same approved suite runs on four triggers, with no AI in any of them.

Trigger	Source
Pull request	Opening or updating a PR
Push	A merge to main
Deploy	A repository_dispatch event, sent by your deploy pipeline or from the portal
Nightly	A scheduled run, or started manually from the portal

The platform finds each run, downloads its artefacts and writes the PR result as a single comment on the pull request. Results are standard JUnit XML, Allure and Playwright HTML reports.

Sauce Labs (Roadmap) is built to run the same tests on cloud browsers as a separate GitHub Actions job. Store the Sauce username and access key only as GitHub Actions secrets, never on the platform host, and set the data centre region as a repository variable. The Sauce job is non-blocking and runs one session at a time. Sauce Labs can lag behind the newest Playwright release, so its job may pin an older supported Playwright version than the rest of the repository. The Sauce Labs adapter is ready as a GitHub Actions job but has not been shown in a demo yet (Roadmap).

Model provider

The model gateway talks to the model provider. Today the supported provider is Anthropic Claude.

• Create a dedicated workspace and API key for the platform in the provider console. If the key belongs to a workspace, also configure the workspace ID.
• The key is given only to the core worker service. The security check confirms that no other service can see it.
• Choose the model mode: replay uses curated records and ignores any key; live calls the provider and fails clearly if there is no key; auto uses live when a key is present and replay otherwise.
• Before a prompt bundle is used live, its evaluation set must pass against the live model. The evaluation run has a hard budget cap.
• Personal-data masking in the model gateway (national ID, IBAN, phone, e-mail) is available and must be switched on whenever customer data is processed; pilots can run on synthetic or non-production data.

Azure OpenAI and other model providers (Roadmap). The gateway is provider-independent and provider-specific code lives only in adapters. Other providers are not supported today; the date is set by pilot demand.

Not available today

The following integrations are not part of the product today. They are listed so that you can raise them early in a pilot conversation.

• Azure DevOps (Boards, Repos, Pipelines, Test Plans) (Roadmap). The date is set by pilot demand.
• Other test management tools, such as TestRail or Zephyr. Test management is a port, so new adapters follow the same interface, but none exists today.
• Single sign-on (OIDC or SAML) for the portal. Portal users sign in with platform accounts and role-based access control.
• Jira Data Center or Server. The Jira adapter targets the Jira Cloud REST API with API tokens.

If a tool you use is missing, the adapter scope can be discussed for your pilot. The Integrations page shows the current label for every integration.