Security at nth Labs

Three questions about your data

Each account’s data lives in its own directory, so one account cannot read another’s. An access check can be forgotten when a new route is added; a directory boundary cannot.

Two cross-tenant defects here had exactly the shape that choice exists to prevent.

Where can it go?

Only to addresses the guard allows. The browser fetching your URL runs on the machine holding your state and secrets, so each address is judged before the browser moves and after it lands.

lib/urlguard.cjs · assertSafeUrl

Who can read it?

The account that filed it, and nobody else. Scoped by directory rather than by a filter each route has to remember, so there is no query to forget.

lib/tenant.cjs · stateDir

Whose API key is used?

On the hosted console, ours. You bring no key and pick no model. Checks that need AI draw on your own plan, shown in validations, never dollars.

On a console you run yourself, yours: your key, the model you pick and the provider’s bill, with nothing metered by us.

lib/modelroute.cjs · modelFor

Two of the eight checks send your product to a model

The two that read and reason send what a model has to look at: your page text, and the frames the browser captured. The other six send what runs recorded, or nothing.

What is sent on runs that need a model

• Page text from the pages the run opened on the product you pointed it at.
• Screenshots of those pages, signed in if you handed over a test account, which means frames of a product nobody outside your team has seen.
• Fresh frames when you ask it to verify a fix, with the selectors and text the surface is now made of, and only where re-running the recorded recipe in a browser did not settle it first.
• To one third-party AI provider, under that provider’s own terms. On the hosted console it is the one we route the check to (today OpenAI), and which provider is our choice, not a setting on your account. On a console you run yourself, it is the provider of the model you picked, on your key.
• The report’s written summary, when its run holds a model key (ours, on the hosted console). The free Quick check never writes one. That one call sends your findings’ titles and each check’s latest score and headline, never a page and never a frame. The titles were written by a check that had read your pages, so some of that text is carried in them.
• The product reading, when it is out of date. After a run that holds a model key, one call rewrites what nth Labs understands your product to be, using what runs recorded, such as its description, page paths and titles, finding titles and run headlines. Never a frame, and never after the free Quick check.

What is never sent, in any mode

• Page health and Regression check call no model at all. Dead controls, Core Web Vitals, links, console errors, accessibility and the screen-by-screen visual diff are measured in a real browser, without a model.
• Your findings, frames and verdicts stay in your own account directory. They are never sold, and never trained on unless you opt in to research use.
• No telemetry, no crash reporter, no usage ping, no licence check. Nothing here calls us on its own, which means there is no setting to find and turn off.
• Card details. Plans and top-ups are paid for on Stripe’s own checkout page. What this console stores is the ledger entry your payment made; it never sees a card number.

Every address, and what the guard does with it

The guard was asked about each of the 256 IPv4 blocks directly, once for each /16 inside it, and each block is coloured by the answer that came back.

Four gates, checked in order

Pointing a browser inward is how a testing tool becomes a way to read the machine it runs on. The path traced below is a real refusal.

How a hostname is judged

A public hostname can resolve into private space whenever its owner chooses. So nth Labs judges every address the name resolves to, and one private answer refuses the run.

1. The scheme is checked first

   Only http and https reach anything. file:, gopher: and data: are refused outright: they are attempts to read the machine nth Labs runs on, not a product to test.

   https: · SCHEME OK

2. The name is resolved

   The target is the set of addresses behind a hostname, not the name itself. A public name can resolve into private space, so the check never judges the text you typed.

   yourproduct.com · RESOLVED

3. Every resolved address is judged

   Not only the first one. Otherwise a name that returns one public address and one private one would let its owner choose which one the browser reaches.

   2 answers · BOTH PUBLIC

4. The landing address is judged again

   After every navigation the address is judged again, before the capture runs. Staying on the origin is allowed; leaving it for a private address is refused.

   302 → 169.254.169.254 · REFUSED

CAPTURE NEVER RUNS · NOTHING KEPT

Limitations

A DNS answer that flips between the check and the fetch is not caught: the browser driver does not expose the socket. The redirect case is, and test/browser/security.cjs proves it.

The same address, five ways to write it

Five notations of the cloud metadata endpoint, the main target of SSRF. The guard parses addresses into bytes rather than matching text, because a regex can miss a spelling nobody thought of.

nth Labs runs on our hosted console, or on your own machine

The two modes are different threat models, and the guard is stricter in the one where the URL came from a stranger.

On the hosted console

The URL is untrusted input from any account, so private space closes and every address behind the name has to be public. Your findings and your settings sit in your own account directory.

On your own machine, if you run it yourself

The console binds to the loopback interface, and hosting is a mode you turn on rather than a config value you can mistype. Your findings and your settings are files only you can read.

Running it yourself is optional, and needs a copy of the repository, which is private. The Run it yourself guide says how to ask for one.

Limitations

Your product’s sign-in password is encrypted at rest only when the operator sets LABS_SECRET_KEY. Otherwise owner-only file permissions protect it. Give the console a throwaway test account.

Security practices

The sections below describe what the code does, by area. Every claim in them is a line in a file of the nth Labs repository, which is private.

What we store, and for how long →

Product security

A run opens a URL somebody else typed, in a browser, on the machine that holds every other account’s state. That is the threat model this section addresses.

• One of those callers has no account. The free check on the landing page runs from a public endpoint, so a URL can reach a browser here with no session behind it. Nothing the engine can do is narrower: one page, no crawl, deterministic checks only, no model key reachable from it. The fence around it is its own, the address is judged by the same guard below before anything launches, the child process is killed at ninety seconds, two may run at once, twenty-five an hour from one address, and two hundred an hour in all. Its state is a scratch directory that is deleted when it finishes.
• The name is resolved before it is judged. A public hostname may resolve into private space; loopback, link-local, the private ranges and the cloud metadata endpoints are refused, and one private answer refuses the whole run.
• The address landed on is checked again. DNS can change between the check and the fetch, and a redirect can arrive somewhere the first gate never saw. The pre-flight check is the first gate, not the only one.
• Model output is untrusted input. Every model answer is escaped at every render site, capped in length at the store boundary, and never interpolated into generated code.
• Both surfaces send a Content-Security-Policy, the console with a per-response nonce, and it adds HSTS on a secure connection.

Data security

Scoping is a boundary rather than a filter each route has to remember. Two cross-tenant bugs here had exactly that shape, and both were routes that forgot.

• One directory per account, not a row in a shared table. The id in the path comes from the system’s cryptographic random number generator and is validated on the way in, so no id can walk out of the accounts root.
• State files are written owner-only, so nothing else on the box reads them.
• The audit log refuses secrets by field name. A password, hash, token, key, cookie or session handed to it is dropped and the refusal written in its place.
• An archive you move can be redacted, and the redactor names the fields it cleared rather than promising about a file it did not understand.

One limitation: your product’s sign-in password is encrypted at rest only when the operator sets LABS_SECRET_KEY. Otherwise owner-only file permissions protect it. Give the console a throwaway test account.

Data privacy

Five of the eight checks call no model and send nothing themselves. The list below says what the other three send.

• Page health and Regression check run entirely in a browser. They call no model and send nothing, though a run of one on its own can be followed by the product reading above.
• Report is free too, and adds a written summary when its run holds a model key. That one call sends your finding titles and scores, never a page and never a frame. The free Quick check never makes it.
• Site review and User simulation send page text and screenshots to one AI model, and to nobody else. On the hosted console it is ours, on our key, and counts against your plan. On a console you run yourself, the provider is yours, the key is yours, and the bill is yours.
• Findings are files, not rows in our database. On the hosted console they sit in your own account directory; on a console you run yourself, there is no server of ours to send them to.

Access control

• Passwords are salted scrypt hashes. A digest that is cheap to compute is cheap to attack; scrypt is slow and memory-hard on purpose.
• Secrets are compared in constant time. Comparing with === leaks the prefix through how long the comparison took.
• The session cookie is HttpOnly and SameSite=Strict, and carries Secure over HTTPS. The OAuth state cookie is scoped to the auth path alone.
• A read-only viewer cannot write config, spend, or erase. A test enforces this.
• Sign-ins are rate-limited with an escalating lockout, and account creation is limited too. Each new account costs a directory and about a tenth of a second of scrypt, which is a denial-of-service primitive if it is free.
• Signups on a hosted box are closed by default. An invite link is a one-time credential (checked for shape, workspace, expiry and prior use), and a cookie that is merely present opens nothing. The link is read at the form and only spent when an account is seated.
• Two-step sign-in, for password accounts. Turn it on in Settings, Account: signing in then also asks for a six-digit code from an authenticator app, and ten single-use recovery codes cover a lost phone. A code works once. A workspace owner can require it for everyone in the workspace who signs in with a password. Google, GitHub and company single sign-on rely on the second step at the provider.
• Coding-tool tokens end, and can be read-only. You choose how long a token lasts when you make it, up to a year, and a workspace owner can set a shorter limit that also ends older tokens. A read-only token reads findings and reports and cannot start a check or change anything you set. Admins see and can revoke every token in the workspace.
• Company sign-in over SAML 2.0 as well as OpenID Connect. For identity providers that speak only SAML. Assertions must be signed by the certificate the workspace owner set, and each one is accepted once, for this console and for the sign-in it answers.

Application security

The product holds that a claim nobody re-tested is not evidence. The same rule applies to our own code.

• A guard must be seen to fail before it counts. The test is written against the unfixed code and watched going red; a test written after the fix proves the code passes the test, not that the test would have caught the bug.
• A security fix ships with the exploit as its regression test. The test replays the attack that worked and requires it to fail.
• Every push of the product’s code runs a secret scan before anything else in the chain that gates it.
• Roughly 504 test modules (388 unit, 116 browser) across the engine, the console and the CLI, plus a browser suite driving real Chromium.

Infrastructure security

• A copy you run yourself is local by default. Its console binds to the loopback interface, and hosting is a mode you turn on rather than a config value you can mistype.
• The two modes are different threat models. The guard is stricter in the one where the URL arrived from a stranger.
• No build step and no bundler. Plain CommonJS on Node 20+, so the dependency surface an attacker inherits is exactly what the lockfile lists.

The browser it drives

The endpoint that matters here is the browser a run steers, because that is the thing holding your product’s session.

• A signed-in run uses the test account you saved. The browser types its email and password into your sign-in form, and refuses to type the password on any other domain. Run it yourself and you can import a session exported from your own browser instead, so the sign-in happens where you are, through your own SSO or 2FA.
• Every step it took is in the transcript, in the tester’s own words, next to the frame it was looking at when it took it.

Network and egress

• Outbound: your product and the AI provider, plus the services a feature in use needs, such as Stripe for a payment or Slack for an alert. There is no telemetry endpoint, and no analytics beacon on the console.
• Inbound on a console you run yourself: nothing. Loopback only, unless you turn hosting on.
• Inbound on the hosted console: the sign-in routes, Stripe’s webhook, and the free check. Everything that reads or changes an account needs a session, or a token issued for it, such as your coding tools’ token or a share link. The free check is the only one that does work for a caller with no account, which is why it is fenced the way the threat model above describes.
• The site you are reading contacts nobody but us. Static HTML, no tag manager, no session recorder, no pixel, no cookie, and the fonts are served from this domain rather than a font CDN. Beyond this domain, a page may reach only our console, which the pricing page asks for the current plans.

Where it is kept, and for how long

If you run it yourself, data stays on your disk, in whatever country your disk is in. The hosted console is covered below.

• There is no automatic retention window. Until you delete something, it is kept. The one thing that outlives an account is its billing ledger, because that a payment happened has to survive seven years for bookkeeping, though everything it paid for is gone.
• Backups are nightly where the operator has configured them, encrypted before they leave the box, kept for the last thirty nights plus one a month for a year.
• Deletion reaches the backups only on request. For up to a year after you delete something a copy exists inside a backup archive: not readable by the console, not selectively restorable, and touched only if the whole service has to be rebuilt. Ask and it is done by hand.
• The hosted console runs on Railway; payments go to Stripe; these pages come off Vercel; model calls go to whichever provider the run’s model belongs to. nth Labs is operated from Toronto, so data on the hosted console is stored and processed outside Canada, in the United States.

The full table, including the lawful basis for those transfers, is on the privacy page.

If something goes wrong

• We tell you by email. The message goes to the address on your account, with what happened, what of yours was in it, what has been done and what you should do.
• Within seventy-two hours to the relevant authority, where the UK and EU rules set that deadline. We would rather send an incomplete account inside it than a complete one afterwards.
• You can report a vulnerability. If you find a way into somebody else’s data here, write to us.

What is not in place yet

We do not yet have a SOC 2 report, an ISO certificate, an independent penetration test, or a third-party audit of anything above.

Stored secrets are encrypted at rest only when the operator sets LABS_SECRET_KEY. There is no bug bounty.

To report a way into somebody else’s data here, write to hello@nthlabs.dev, the same address the privacy page gives. We will answer, and we will say what we fixed.