Review checklist for AI-generated tests

An AI assistant writes a Playwright test in a few seconds. The test compiles, runs, goes green… and sometimes checks very little. This 7-point checklist is for reviewing a generated test the way you would review a colleague's code, before merging it. Free, printable, no sign-up.

Download the PDF (French)

It complements our guide Playwright MCP: testing with AI, which shows how an assistant explores a page and then generates a test. For an overview, see AI in software testing.

The 7 points of the checklist

await page.getByRole('button', { name: 'Place order' }).click();
await expect(page).toHaveURL(/\/order/);

await page.getByRole('button', { name: 'Place order' }).click();
await expect(page.getByRole('heading', { name: 'Order confirmed' })).toBeVisible();
await expect(page.getByTestId('order-total')).toHaveText('49.90 €');

await page.locator('div.form > div:nth-child(2) > input').fill('lea@example.com');
await page.locator('//button[contains(@class,"btn-primary")]').click();

await page.getByLabel('Email address').fill('lea@example.com');
await page.getByRole('button', { name: 'Sign in' }).click();

// account found during exploration, shared by all tests
await page.getByLabel('Email address').fill('john.smith@example.com');

test.beforeEach(async ({ request }) => {
  // account created through the API before each test, specific to this test
  const res = await request.post('/api/test/users', {
    data: { email: `test-${Date.now()}@example.com` },
  });
  expect(res.ok()).toBeTruthy();
});

await page.getByLabel('Password').fill('Password123!');

await page.getByLabel('Password').fill(process.env.TEST_USER_PASSWORD!);

// playwright.config.ts
use: { baseURL: 'https://www.my-site.com' },

// playwright.config.ts
use: { baseURL: process.env.BASE_URL ?? 'http://localhost:3000' },

Test the application and write all the necessary tests.

Open http://localhost:3000/login. Describe the fields and the messages.
Try an incorrect password and report the exact message.
Do not open any other page and do not submit any other form.

// generated test, run once, green: merged as is

// one-off check: we falsify the expected result…
await expect(page.getByTestId('order-total')).toHaveText('0.00 €');
// …the test must fail; then restore the correct value.

The 5 defects to spot first

• Fragile selectors: CSS or XPath based on the page structure instead of getByRole, getByLabel or getByTestId.
• Empty assertions: the test checks that an action took place, not its result.
• Fixed waits: page.waitForTimeout(3000) instead of an assertion that retries, such as await expect(locator).toBeVisible().
• Hard-coded data: accounts, identifiers and amounts copied from the exploration and shared between tests.
• Happy path only: no error case, empty field or boundary value.

How to use this checklist

1. Print the PDF or keep this page open during the pull request review.
2. Go through the 7 points for every generated test, not just the first one.
3. An unticked point blocks the merge: fix the test, or note why the point does not apply.
4. Add the checklist to your definition of "done" for any test written with an assistant.

To design the test cases themselves before automating them, start from a test acceptance book (FR): expected results written before execution, fixed data, clear statuses.

Frequently asked questions