Fixing flaky Playwright tests: causes and patterns

Start at the failed attempt

A flaky test passed in the end, so its last attempt shows nothing wrong. What you need is in the attempt before it: the error, the line it failed on, and what the page looked like.

Record that for failing attempts in playwright.config.ts:

use: {
  screenshot: "only-on-failure",
  trace: "retain-on-failure",
},

Inside defineConfig. trace: "on-first-retry" records less, but belongs to the retry rather than to the failure.

With a trace, the Trace Viewer shows each action with the page before and after it, the network requests and the console.

In Squally each attempt of a test in a run has its own tab - Attempt 1, Retry 1 and so on. An attempt shows its error with Playwright's code frame and, where they were recorded, its steps, screenshot, browser console and test output, and a link that opens its trace in the Trace Viewer.

A run's Errors tab bundles its errors by category - Assertion, Timeout, Element not found, Network, Other - and the Error explorer follows one error across runs and tests.

Fixed waits → web-first assertions

page.waitForTimeout() pauses for a fixed time: long enough on a fast machine, too short on a loaded CI runner. Playwright's API reference calls tests that wait for time inherently flaky (Page).

// Flaky: hopes two seconds are enough
await page.getByRole("button", { name: "Add to cart" }).click();
await page.waitForTimeout(2000);
expect(await page.getByTestId("cart-count").textContent()).toBe("1");

// Waits until the text is there, up to the expect timeout
await page.getByRole("button", { name: "Add to cart" }).click();
await expect(page.getByTestId("cart-count")).toHaveText("1");

Web-first assertions such as toHaveText, toBeVisible and toHaveURL retry until the condition holds or the timeout runs out (Assertions). Actions such as click already wait for the element to be visible, stable and enabled (Auto-waiting).

When the page waits for the server, wait for that response: page.waitForResponse() waits for the request you name, not for a guess at how long it takes.

A missing await

Without await, a Playwright call returns a promise and the test moves on. The assertion may run before the click it depends on, or after the test has ended.

// Flaky: the assertion is not awaited
expect(page.getByText("Saved")).toBeVisible();

// Correct
await expect(page.getByText("Saved")).toBeVisible();

A linter catches this before it flakes: Playwright's best practices recommend the @typescript-eslint/no-floating-promises rule (Best practices).

Fragile selectors

A selector that depends on position, generated class names or markup breaks when the page renders in a different order, or a moment later.

// Fragile: the second button in a div with a generated class
await page.locator("div.css-1x2y3 > button:nth-child(2)").click();

// What the user sees
await page.getByRole("button", { name: "Checkout" }).click();

Role, label and text locators describe the page the way a user does (Locators). Locators are strict: an action on a locator that matches more than one element throws instead of picking one, so an ambiguous selector fails every time rather than now and then.

Shared state

Tests that read data another test wrote, or that expect to run in a given order, pass alone and fail together - or the other way round.

• Give each test its own data: create it in the test or in a fixture, with a unique name.
• Do not depend on the order of tests in a file.
• Each test gets a fresh browser context, so cookies and storage do not carry over between tests. Data on the server does.

A fixture sets the data up for one test and removes it afterwards:

const test = base.extend<{ account: Account }>({
  account: async ({}, use) => {
    const account = await createAccount({ name: `test-${crypto.randomUUID()}` });
    await use(account);
    await deleteAccount(account);
  },
});

createAccount and deleteAccount stand for your own API calls.

Parallel workers colliding

Playwright runs test files in parallel worker processes (Parallelism). Two workers that sign in to the same account, write the same file or change the same record get in each other's way - depending on timing, which is what makes it flaky.

testInfo.parallelIndex numbers the workers from 0; use it to give each worker its own account or data set. Where tests must share something, test.describe.configure({ mode: "serial" }) runs a group in order, one test after the other, at the cost of speed.

Slow third parties → mock

Payment providers, maps and feature-flag services answer slowly at times, or limit how often a CI runner may call them. A test that depends on them flakes with them.

await page.route("**/maps.example.com/**", (route) =>
  route.fulfill({ json: { eta: "2 days" } }),
);

Mock APIs covers page.route and replaying recorded responses from a HAR file. Playwright's best practices advise against testing third-party dependencies you do not control.

Confirm the fix over the next runs

One green run proves little for a test that failed one run in ten. Before merging, repeat it: npx playwright test tests/cart.spec.ts --repeat-each=20 --retries=0. Without retries a failure shows as one; twenty green runs in a row are evidence, not proof.

After merging, watch the next CI runs: the test should stop needing retries.

In Squally, a test's page shows its Flaky rate and Failure rate for the period, its runs per day and every run with its result, filterable by Passed, Flaky and Failed; the Test explorer's Last 10 runs show the latest results at a glance.

A run's Compare tab sorts its tests against another run into groups such as Newly failing, Still failing, Newly flaky, Still flaky and Fixed.

Which test to fix first: What flaky tests cost. How to find them: How to detect flaky tests in Playwright. In the docs: Common causes and Debugging basics.