Code generation

Code generation turns approved test cases into Playwright tests in your repository. An agent verifies each step in the running application and maps it to your page object library, then the code is filled into a template. The model does not write TypeScript, and every test arrives as a pull request that a human reviews and merges.

Prerequisites

Code generation builds on what your test codebase already has. Before the first run, the repository needs:

• A Playwright page object library. Generated tests call page object methods and locators; they do not invent selectors. The platform reads the library into a symbol table and maps every step to an existing symbol.
• Stable test IDs on the elements the tests interact with. Locators based on test IDs, together with the element’s role and accessible name, are what make deterministic repair possible later.
• A reachable test environment of the application for the browser agent and the test runner.
• A protected main branch with required checks, so that merges go through review.

If the page object library or the test IDs are missing, setting them up is the first step of a pilot. Code generation is available for web applications with Playwright today. Desktop and mobile code generation are on the roadmap (see the last section).

Starting a generation run (Demonstrated)

The engineer selects one or more approved cases and starts generation. A case is eligible only when:

• It is approved or published.
• Its content hash still equals the approved hash.
• The story’s product owner approval is still current.

If an open or merged pull request already exists for the same approved content, the case is not generated again. Each new generation creates a new branch and a new pull request from the current main. Any earlier open pull request for the same case is commented on and closed.

For the example story STORY-142 Order cancellation, the engineer might select three approved cases: cancel a paid order, a shipped order shows “Request return”, and cancellation at the end of the window. The live model evaluation for the code agent is pending. The current label is on the Capabilities page.

The verification agent

For each case, an agent walks through the steps in the real application using a browser, through Playwright MCP. The agent’s tools are tightly limited:

• Only an allowlisted set of browser tools is available. Tools that run arbitrary code, evaluate scripts or read files are never given to the model.
• Navigation is restricted to the application’s origin, and the agent’s browser can reach only the application under test.
• Each run has a bounded number of tool calls.
• The agent uses a dedicated test user. Shared state, such as a cart, is reset before each verification, and verifications run one at a time.

Every step is recorded with its tool, arguments (with secret fields masked), a short summary and a screenshot. Screenshots are taken outside the model loop and are kept as evidence for the reviewer; they are not sent to the model.

Each locator the agent produces is cross-checked against the page object symbol table. If the locator does not match a known symbol, the step is marked unverified, no pull request is opened, and the case goes to a human.

Template-bound code

The model’s job ends with a verified mapping from case steps to page object symbols. The TypeScript file is then produced from a template, not written by the model.

• Steps become calls to existing page object methods and locators from your library.
• Values are inserted only as type-checked literals, so text from a case cannot inject code into the template.
• Assertion values come from the approved case text, not from what the agent observed. If the application shows something different, the test fails, which is intended.
• Each case produces one spec file in a generated-tests folder, named after the story and the case.

An illustrative result for STORY-142 looks like this:

test('STORY-142 C2: paid order can be cancelled', async ({ orders }) => {
  await orders.open('A-1001');
  await orders.cancelButton.click();
  await expect(orders.status).toHaveText('Cancelled');
});

The mapping table, which shows each step, its page object symbol and whether its value appears in the approved case, is shown next to the code in the portal and in the pull request description.

Local checks

Before a pull request is opened, the toolchain runs a set of deterministic checks on the generated file:

• tsc in strict mode and eslint.
• The AST guard, which checks the structure of the change against the rules for bot pull requests. The same guard runs again as a required check in CI.
• A Playwright run of the new test, repeated several times against the test environment.

The outcome is classified:

Outcome	Result
All checks pass	A pull request is opened
The same test both passes and fails	Marked flaky; no pull request
The test fails consistently on an assertion	Marked defect candidate; no pull request
A human requested changes or rejected	Recorded as human rejection

The toolchain service that runs these checks never executes code itself. It sends runs to an isolated test runner with no internet access and no secrets.

Defect candidates

A defect candidate is one of the most useful results of code generation. It means the test expects the behaviour that was approved in the story, and today’s application consistently does something else.

In the STORY-142 example, suppose the analyst marked “the confirmation email is missing” as a code defect during the requirement review. The case “Cancelled order sends a confirmation email” is generated against the intended behaviour, and its local run fails in every repeat. The test shows the expected value next to the actual value.

The platform does not try to make the test green. It does not change the assertion, and it does not open a pull request. The candidate goes to a human, and during triage it can be linked to the bug draft that came from the review finding. The finding the analyst made in the document and the failure the test caught end up in the same Jira issue.

Pull requests and merge

Each case that passes local checks gets its own branch and pull request, opened by a bot identity such as a GitHub App. The bot opens pull requests but never merges them.

The pull request description carries traceability:

• The story, the case and its approver.
• The approved content hash and the audit chain head.
• The step mapping and the local check results.
• A note when the AI output came from REPLAY.

The only changed file is the generated test. Merging requires the protected branch’s required checks to be green and a human approval. The portal shows the merge gate and lists anything that blocks it, such as a red check, a branch behind main, or Break-to-Verify still running. After a merge, other open bot branches are updated onto the new main, and a push run validates main with the full suite.

Desktop, mobile and Selenium (Roadmap)

The following are not available in the product today:

• Desktop (FlaUI) and mobile (Appium) code generation (Roadmap). The date is set by pilot demand. Antrek Labs delivers desktop and mobile test automation as a service today.
• Selenium to Playwright migration (Roadmap). The product generates new Playwright tests; it does not convert existing Selenium suites. Migration is available today as an Antrek Labs service.

Requirement review and test case generation do not depend on the application’s technology, but they have not yet been validated on a mobile or desktop pilot. If these capabilities matter to you, raise them in the pilot conversation so they can be set as pilot conditions.