How a fix gets kept
- Confirm the finding while the bug is still there. Mark it Real problem, under More on the finding.
- The next Regression check run turns it into a kept check. Regression check runs with every press of Check now after your first run, and makes a check from each confirmed finding that has steps a browser can re-run.
- The check fails once. The bug is still there, so the first run fails, and that failure is the proof that the check can see the bug.
- You fix the bug. From then on the check passes.
- If the bug comes back, the check fails again, and the original finding reopens as ship-blocking, with what the browser saw.
If the bug is already fixed, use Keep watching this, under More on a confirmed finding. It re-drives the finding’s steps in a real browser straight away and keeps the check. From then on it runs with every Quick check.
Why a check has to fail first
A check that has only ever passed might be watching the wrong thing. With a typo in what it looks for, it would pass forever by finding nothing. So a kept check does not count as protection until it has been seen to fail against the bug it guards.
The usual reason a check has never failed is that it was kept after the bug was already fixed. The Regression check report under Runs lists those as unproven and does not count them. Prove a check while the bug is still there.
The rule works the other way too. If a finding is still open but its check cannot find the bug, the report flags the disagreement: either the fix landed and nobody closed the finding, or the check is not seeing the bug.
Opinions, such as “the headline is vague”, are never kept this way, because no browser can settle them.
Visual changes
The second half of every Regression check run photographs each page and compares it with the last look you accepted. Pages that match pass. A page that changed waits for your review: open Visual changes from the IT page and accept or reject each one.
- Accept: I meant this makes the new look the one later runs compare against.
- Reject: this is a regression files it as a finding, with the before, the after and the difference.
The first time a page is photographed, that photograph becomes the look to compare against. Your product’s first run skips Regression check, because there is nothing to compare with yet: your next run photographs every page, and the run after that shows what moved.
Where you see them
- The card at the foot of the sidebar says how many kept fixes are holding or came back, and whether anything keeps checking the product on its own. Press it to see them.
- Findings, filtered to Kept, lists every finding you asked nth Labs to keep checking, each marked kept, holding or came back.
- The finding itself carries a Watching this card with the last result and when it ran.
- Runs has each Regression check run’s report: every kept check, whether it has been seen to fail, and every page that moved.
Running them on a schedule
Kept checks run with every Quick check and every press of Check now. To run them automatically, add a schedule in Settings, Runs. Schedules come with a paid plan: weekly on Pro, daily on Team and Agency. Scheduled runs appear in Runs like any other.
Checking one fix right now
Regression check stops fixed bugs coming back. To find out whether one fix has landed, press Fixed? Verify it on the finding. When the finding has steps a browser can re-run, the browser decides, and that costs nothing. When it does not, the AI looks at the page again, which needs a paid plan. The finding closes only on a confident yes. More in Fix prompts and verifying.
Free on every plan
Regression check needs no AI. A kept check either finds the bug or it does not, and gives the same answer every time, so it can run on every press and on a schedule at no cost.
From the command line
If you run nth Labs yourself from the repository:
node cli.cjs guard # make and run every kept check, then compare the pages
node cli.cjs contracts list # the kept checks
node cli.cjs contracts new <findingId> # keep one confirmed finding now
node cli.cjs contracts export # the kept checks as ordinary Playwright tests
node cli.cjs changes accept <id> # a change on screen you meant
node cli.cjs changes reject <id> # a change on screen you did not
node cli.cjs verify <findingId> # is this one fixed, right now?contracts export writes each kept check as a Playwright test you can run without
nth Labs.