The three you will use most
| Button | You are saying | What happens |
|---|---|---|
| Real problem | It is real, and it needs fixing. | Stays on your list, and counts as nth Labs being right. Once it is fixed, it can be kept under watch. |
| Not a problem | It is not real. | Leaves your list. The only verdict that counts as nth Labs being wrong, and the reason you give is shown to future runs. |
| Won’t fix | It is real, and you are not fixing it. | Leaves your list, and still counts as nth Labs being right. Your note stays on the finding and is never shown to future runs. |
Other verdicts
| Button | You are saying | What happens |
|---|---|---|
| Fair point | Worth knowing, but not a defect. | Leaves your list. Counts as nth Labs being right, and is never used to teach future runs. |
| Duplicate | Real, but already reported. | Leaves your list, and counts as nth Labs being right. |
| Can’t tell | You cannot tell whether it is real. | Stays open, and is left out of the accuracy figures. |
| Skip | Not now. | Moves on to the next finding. Nothing is recorded. |
Snoozing is not a verdict. Later, on an opened finding, puts it away for a week, and H on the list puts a finding away until the next run or a day you pick. Either way it stays open and comes back on its own.
Where you give them
- Review. On Findings, press the Review button (it carries the count, as in Review 12 findings) for one finding at a time, keyboard first: R Real problem, N Not a problem, W Won’t fix, F Fair point, D Duplicate, C Can’t tell, S Skip and U to undo. V verifies the finding in a browser when its steps can be repeated, and Enter shows the evidence.
- The Findings list. A new finding waits for your answer. 1 accepts it as a real problem, 2 marks it a duplicate of a finding you pick, and 3 declines it with a reason: Not a problem, By design, Can’t reproduce or Won’t fix. X picks several rows to answer at once.
- One finding, opened. Not a problem is one of its three buttons. Real problem and Won’t fix are under More.
By design counts like Won’t fix, and Can’t reproduce counts like Not a problem. A verdict given in Review or on the list shows an Undo as it is saved, so you can reverse a mistake with one press.
Not a problem changes future runs
Not a problem is the only verdict that counts as nth Labs being wrong, and the only one that changes what the next run does. When you give it, you are asked why it was wrong. That sentence is shown to future runs on this product, so the same mistake is less likely to be made again.
Use it whenever a finding is wrong. If you only ever mark findings real, future runs learn nothing from your verdicts, and the accuracy figures stay unmeasured however many runs you do.
Won’t fix and Not a problem are different
Won’t fix says the finding is real and you are not fixing it, so nth Labs was right. Not a problem says the finding is wrong. Counting them as one would make the accuracy figure meaningless, so they are separate buttons.
A Won’t fix note is kept on the finding for the next person who reads it, and is deliberately never shown to future runs: the defect is real, and you still want to hear about a new one like it.
What happens next
- A finding you ruled off the list stays off it. When a later run sees it again, its row says so (seen again, kept off your list) instead of filing it as new.
- A fix that comes back reopens. A finding marked fixed that a later run finds again is reopened, and marked came back.
- A real problem can be kept under watch. After you mark one real, Review offers Keep watching when a browser can repeat its steps. Once the fix is in, nth Labs checks it again with every Quick check and tells you if it returns. Fix prompts and verifying has the rest.
From the command line
Running nth Labs yourself, three of the verdicts are commands: confirm is Real problem, wontfix is Won’t fix and reject is Not a problem. The rest are in the console.
node cli.cjs actions list
node cli.cjs actions confirm <id>
node cli.cjs actions wontfix <id> "known, tracked in LIN-402"
node cli.cjs actions reject <id> "the field is hidden on purpose"
node cli.cjs actions why <id>The note on wontfix and reject is optional. It records your reason for the next person who reads the finding.