Pairing with AI for Testing

What this checklist covers

Setting Up the Pairing

An AI pair has no memory of your codebase, no access to your Jira, and no idea which of your conventions are load-bearing. Almost every disappointing session traces back to context that was never supplied rather than to the model.

Choosing the Mode · Setup

• The role is decided deliberately - AI as driver writing code you review, or AI as navigator questioning the approach while you write.
• AI drives best on mechanical work: fixture scaffolding, parameterised data, converting a spec table into cases.

Context You Must Supply · Setup

• The actual requirement is provided, not a paraphrase - a summarised acceptance criterion loses precisely the edge conditions worth testing.
• Existing test conventions are shown by example, because a described convention gets approximated and a demonstrated one gets copied.

Boundaries & Data Hygiene · Setup

• What may be pasted into a hosted model is settled in policy before the first session, not judged case by case under deadline.
• Production data, patient data and live credentials are never pasted; a masked or synthetic equivalent is prepared instead.

Prompting for Test Work

Testing prompts differ from general coding prompts in one way: you usually want breadth of cases rather than one good answer, and models default to the happy path unless pushed off it.

Test Design Prompts · Prompting

• The technique is named explicitly - boundary values, equivalence partitions, decision table, state transition - which produces systematic coverage instead of an assorted list.
• Negative, boundary and error cases are asked for separately, since an unqualified request returns mostly happy paths.

Prompting for Automation Code · Prompting

• Framework and version are stated up front, or the output lands on a deprecated API from whichever version dominated the training data.
• An existing test from the suite is supplied as a style exemplar, which fixes structure, naming and helper usage in one move.

Iteration & Refinement · Prompting

• Failures are fed back with the actual error output rather than 'it didn't work', which is the difference between a fix and a guess.
• A badly-started thread is restarted rather than argued with, because early wrong context tends to persist through the whole conversation.

Verifying AI Output

The defining risk of AI-assisted testing: a test that passes, reads well, and verifies nothing. Reviewing generated tests demands more suspicion than reviewing a colleague's, not less, because the surface polish is uniformly high.

Reviewing Generated Tests · Verification

• Every generated test is watched to fail for the right reason before it is trusted - a test that has never failed has never been shown to test anything.
• Assertions are checked to verify behaviour rather than restate the implementation, since a test asserting what the code does is tautological.

Detecting Hallucinated APIs & Selectors · Verification

• Unfamiliar library methods are checked against real documentation, because confidently-named methods that do not exist are a routine output.
• Every generated selector is verified against the live DOM rather than assumed present.

The Oracle Problem · Verification

• Expected results come from the requirement or a human, never from asking the model what the output should be - that closes the loop and verifies nothing.
• AI is not used to generate both the implementation and its test's expected values from the same understanding, since a shared misreading then passes cleanly.

Team Practice & Governance

Making the practice sustainable at team scale: where it genuinely pays, how to keep skills from hollowing out, and the policy questions someone will eventually ask you about.

Where AI Pairing Pays Off · Practice

• Highest return is on volume work - test data generation, parameterised expansion, translating specs into skeleton cases.
• It is strong on unfamiliar syntax and on explaining a failure in someone else's framework, which shortens onboarding materially.

Skill Retention & Review Norms · Practice

• Nobody merges test code they could not explain line by line, which is the practical floor that keeps review meaningful.
• Juniors are paired with humans as well as AI, since a model will supply an answer where a mentor would have supplied the reasoning.

Attribution, Licensing & Policy · Practice

• AI involvement is disclosed where the organisation or client requires it, particularly for regulated deliverables.
• Generated code is reviewed for licence-incompatible reproduction of training material where the output is long or distinctive.

AI Pairing Tooling

The tools available for the practice above. The category splits into general coding assistants that happen to write tests and test-specific platforms that embed AI in the runner itself.

Coding Assistants · Tools

• Claude Code runs in the terminal and IDE with repository-wide context and tool access, which suits multi-file test refactors rather than single-file completion.
• GitHub Copilot is strongest at inline completion inside a file, and correspondingly weaker at reasoning across a whole suite.

Test-Specific AI Platforms · Tools

• Testim and Mabl apply self-healing locators, which reduces maintenance and correspondingly reduces the signal when the UI genuinely changed.
• Applitools performs visual comparison with tolerance for benign rendering differences, catching the layout defects assertion-based tests never see.

Agentic & Browser Tooling · Tools

• Playwright MCP lets an assistant drive a real browser and read the accessibility tree, which removes most invented-selector failures at the source.
• MCP servers for Jira, GitHub and the test management tool let the assistant read the actual ticket rather than a pasted summary.

Model Access · Tools

• Frontier models are used for reasoning-heavy work such as test design and root-cause analysis, where capability gaps are most visible.
• Smaller and faster models are adequate for mechanical transformation, and cost far less per run in bulk.