Know which tests are flaky.
And what they cost you.

Squally watches every CI run of your Playwright suite and shows which tests are least stable and what their retries and failures cost.

Free during the betaHosted in the EU

The real Squally layout, with example data.

One reporter line
squally-reporter in playwright.config.ts. Your CI stays as it is.
Stored in the EU
Runs, results, screenshots and traces in Frankfurt and the EU.
No code access
The GitHub app posts checks and comments. It never reads your code.
Shards as one run
However many shards a CI run has, Squally counts it once.
Three questions after every red build

Numbers instead of guesses.
Your team decides.

Squally counts across your CI runs and shows its work. Whether a test is worth fixing stays your call.

01

Flaky or broken?

Stability, flaky rate and failure rate for every test, across its CI runs.

shows today’s deal
Stability74.2%
Flaky rate19.4%
Failure rate6.4%
02

What does it cost?

The time lost to retries and failures, per test, so the most expensive one comes first.

applies coupon11min
shows today’s deal4min 12s
checkout total1min 31s
03

What changed?

Compare two runs down to one test and one attempt.

#63 compared with #62master
Newly failing2
Newly flaky0
Still failing1
Fixed1
Pull requests

The answer lands
where you review.

A check on every commit and one comment per pull request, updated with each run. Flaky and failing tests are named, with a link to the attempt.

  • Counts that match the run: passed, flaky, failed, skipped
  • Shards as one run, re-runs linked to the run they repeat
  • Checks and pull requests only — it never reads your code

Checks and comments go to GitHub repositories, whatever CI runs the tests — Azure Pipelines too. GitLab merge requests: not yet.

Squally — Run #63 · 66 passed · 1 flaky · 2 failed · 1 skipped
Details
squally-checks bot commented · edited
Squally · Run #63 · 66 passed · 1 flaky · 2 failed · 1 skipped
Failed
three items get the 10% multi-buy discount once (chromium)
three items get the 10% multi-buy discount once (firefox)
Flaky (passed on retry)
shows a delivery estimate (chromium)
View run #63 in Squally
Posted by Squally – updated in place on every run of this pull request.
AI

From red test
to the next step.

Ask your agent about runs, tests and errors. The MCP server answers with the evidence attached.

  • Read-only, with a key you can revoke
  • Runs, tests, errors and every attempt of a failure
  • Runs locally next to your editor
demo-shop — claude — 100×28
Claude Code · connected to squally MCP (7 tools)
❯ Why is the last run on master red?
› Looking up the latest failed run on master.
● squally · squally-find-run (projectId, branch: "master", status: "failed")
└ run #63 · 66 / 70 passed · 2 failed · chromium + firefox
● squally · squally-debug-failure (projectId, runId, testName: "three items get the 10% multi-buy discount once")
└ Error: expect(getByTestId('total')).toHaveText
└ Expected: "$26.97" Received: "$29.97"
● squally · squally-get-test-metrics (projectId, testName: "three items get the 10% multi-buy discount once")
└ failure rate 100% (36/36) · flaky rate 0%
› Not flaky — a real bug. The total is the full price: the 10% multi-buy discount is never applied. Fix the discount rule in cartTotals() in src/store.ts.
❯ ▋
Reports

A Monday mail
your manager reads.

What still fails, what got flaky, what got slower — in one email a week. Or a report of a single run to print or paste into a ticket.

Weekly report · Demo Shop · master
2 tests still fail since Tuesday
Stability
94.9%
Runs
51
Flaky
11
Time lost
11min
Needs attention
three items get the 10% multi-buy discount oncestill failing
shows a delivery estimatepassing again
Setup

Five minutes,
one reporter.

  1. 1
    Install the reporter
    One dev dependency that runs inside Playwright — no agent, no upload step.
  2. 2
    Add it to your config
    Keep your other reporters. Squally runs next to them.
  3. 3
    Add one CI secret
    The ingest key from Squally. No URL to set.
  4. 4
    Push
    The next CI run shows up in Squally while it is still running.
setup
$ npm install -D squally-reporter
added 2 packages in 2s
// playwright.config.ts
import { defineConfig } from "@playwright/test";
export default defineConfig({
reporter: [
["list"],
["squally-reporter"],
],
});
# .github/workflows/e2e.yml
env:
SQUALLY_INGEST_KEY: ${{ secrets.SQUALLY_INGEST_KEY }}
$ npx playwright test
Running 70 tests using 4 workers
70 passed (1.4m)

FAQs

Does Squally run my tests?

No. Your CI runs them as today; the reporter sends the results to Squally after each test.

Do I have to change my CI pipeline?

Barely. You add one reporter line to playwright.config.ts and the key as a CI secret. Where your CI does not hand secrets to every job, as on GitHub Actions or Azure Pipelines, one env line passes it to the tests. No new steps or jobs.

Where is my data stored?

In the EU. Runs and results in Frankfurt, screenshots and traces in an EU bucket.

What leaves my CI?

Test results with names, durations and retries, branch, commit and author name, error messages and stacks with a few lines of the failing test code, console output, and screenshots and traces if you record them. Runs are kept 30 to 90 days by plan; screenshots and traces 7 days, 21 for failed tests.

Only Playwright?

For now, yes. Squally reads Playwright results through its own reporter.

Does it work with sharded runs and retries?

Yes. Shards of one CI run become one run, and a test that passes on a retry counts as flaky, not as passed.

What does it cost?

The beta is free. Prices come with the launch.

Stop rerunning red builds
to find out if they mean it.

Join the waitlist. We let teams in a few at a time and help with the setup.

We use your address only for the waitlist and the invite. Privacy