What a finding carries
| Part | What it is |
|---|---|
| Title | What went wrong, in plain words, written as its effect on a visitor rather than the rule that caught it. |
| Page | Where it happened, such as pricing or /pricing. |
| What the browser saw | The frame the browser captured, and the steps it took when a browser can repeat them. |
| Severity | Ship-blocking, Worth fixing or Polish. |
| How it is known | Proved in a browser, measured by code, or a model’s judgement. A tester’s reaction is marked opinion. |
| Fix prompt | A prompt for your coding tool that ends in how to check the fix worked. |
| History | How many runs have seen it, and whether it came back after a fix. |
| Id | A short code such as 6359b5, which the command line and your coding tool use to name the finding. |
When the same defect shows up on several pages, the Findings list folds it into one row that says on how many pages. Fix it once and each page closes as it is verified.
How severity is decided
| Severity | What it means |
|---|---|
| Ship-blocking | A first-time visitor could bounce on this. |
| Worth fixing | Friction that costs you some of them. |
| Polish | Real, but nobody leaves over it. |
Severity is a claim about consequence, not about how hard the problem was to find. A department’s To do list calls ship-blocking Serious; the two words mean the same thing.
What makes a finding ship-blocking
Proof is required. A finding can be ship-blocking only when something other than a model established it: a real browser repeated the steps and saw it happen, or code measured it on the page. The top issue on Home says which, as Reproduced in browser or Verified in browser.
- A claim nothing has checked is filed as Worth fixing at most. Every word of it is kept, and it waits under Needs your eye for your answer.
- An opinion, a tester’s reaction or a judgement about how the product reads, is never ship-blocking and can never become a regression check.
- A claim that failed its check before it was filed is held out of the list, and the run’s report names it.
What confidence means
Confidence is a ranking, not the chance that a finding is real. The figure is calculated from what the finding has earned: reproduced in a browser, seen again by later runs, back again after a fix, and your own verdicts. No model’s opinion goes into it, and everything from a first run starts at the same figure.
A percentage nobody has checked reads like a measurement, so the Findings list shows one only after your verdicts have calibrated it. Until then, a finding under Needs your eye says its confidence is not measured yet. In Review, press Enter for the details: the figure and what it rests on.
Running nth Labs yourself, ask for the reasoning behind any finding:
node cli.cjs actions why <id>That prints the confidence, the severity and every line behind the finding’s rank. Each line is something nth Labs observed or you decided.
Measuring accuracy
Measured precision, how often nth Labs has been right on your product, is counted from your verdicts and from nothing else. Until there are enough verdicts to support a rate, you get a dash, or a note of how many more are needed, instead of a number.
node cli.cjs calibrationFrom the command line, the calibration command shows whether the confidence figures match your own verdicts and, from repeated runs, estimates how much of your product no run has met yet. The command uses only what is already on disk, costs nothing, and does not print a rate it lacks the evidence for.