Evening session: tests that read every package, so the module I write next month is already checked

The Starting Point

Finished the security review earlier in the day: 45 findings, each one closed with its own regression test.

Which protects exactly the code that exists today. The platform has fifteen packages and will have more. A regression test says "this bug is fixed". It says nothing about the module somebody writes next month.

So: guards.

What Actually Happened

Guards that read the source rather than exercise it

Each one reads every package's routes, controllers, source and views, and fails on a shape rather than on a behaviour:

Routes — the business unit is set before model binding; every route is authenticated, a share link, or on a two-entry public allow-list; unit middleware on every signed-in route; the module-enabled check on every module route; the API group's throttle, active-user and token-owner checks.
Controllers — every routed action makes its own authorisation decision, or is listed with the reason it doesn't need to.
Raw output and URLs — unescaped output and stored URLs only where listed, or passed through the safe-URL helper.
Business-unit scope — scope escapes and raw queries on unit-scoped tables only where listed; commands and jobs that touch unit-scoped models must run inside a unit context.
Plus validation scoping, CSV output and mass assignment.

Two rules make the allow-lists survive contact with reality:

Every entry carries its reason. Not a path on a line — a path and a sentence saying why this one is allowed to break the rule.

Stale entries fail too. An allow-list that still names a file that no longer breaks the rule is an allow-list nobody has read. If the exception isn't needed any more, the guard says so.

And each guard was seen failing against a planted violation before being trusted. That's the fourth or fifth time I've written that sentence in these notes and it's still the only thing that makes a guard worth having.

They run as their own early, labelled stage in the check, so a violation fails in seconds rather than after the full suite.

Four gaps on the first run, and one was a real finding

A token could do what its owner may not.

This is the one worth the whole evening. The API endpoints checked only the token's ability — and anybody signed in may mint any declared ability. So a person without the permission to manage billing could mint a token carrying that ability and create records through the API that they could not create through the interface.

The fix has three parts, and the third is the one that matters:

Every ability names the permission its owner must hold.
The token screen offers and mints only the abilities whose permission you hold.
A middleware re-checks on every request, reading the abilities from the route's own middleware — so a new module is covered without anybody remembering to add it.

Which means taking a permission away disarms the tokens made while it was held. A token is a delegation of what you can do, not a capability of its own.

The other three were the same shapes I'd already fixed, in places round one missed:

Two API endpoints kept working while their module was switched off. The web routes had been fixed; the API halves hadn't.
The report menu still validated with a plain exists: rule, so another unit's page could be filed into a menu.
One document read an account through an unscoped raw query. The other document of the same kind was fixed in round one, and this one was missed.

That last one is the argument for the guards in a single line. I fixed the bill document by hand, in round one, having found it by reading. I did not find the contract document, which is the same bug in the same shape, because reading is not exhaustive and I am not a grep.

What Worked

Writing the rule down next to the guard that enforces it. The external build rules — the document handed to anything built green-field that will later arrive here as a module — now has a section stating each rule and the guard that checks it. So "every routed action authorises" isn't advice, it's a thing that will fail somebody's build.

Three open questions raised rather than fixed. The scope guard found three lookups crossing business units, and all three need a product decision before a fix — whether SLAs and a particular tag are set per unit or per deployment, and whether a uniqueness rule should be per unit.

They went on the tracker as questions. A guard that finds something needing a decision has done its job; guessing at the answer because the guard is red is how you ship the wrong semantics quietly.

What I Learned

A regression test protects the past. A guard that reads source protects the future. Both, not either.
An ability is not a permission. If anybody signed in can mint any ability, your API authorisation is whatever the token says it is. Bind every ability to a permission its owner must hold, and re-check it on the request.
Revoking a permission should disarm tokens minted under it. Otherwise "remove their access" is a half-measure with a long tail.
Allow-list entries need reasons, and stale entries should fail. Otherwise the list grows and nobody ever reads it again.
Fixing a bug by hand fixes one instance of it. I fixed one document and missed its twin. The guard found the twin in seconds.
Run guards early and label them. A structural violation shouldn't wait behind 1,600 tests.
When a guard surfaces a question rather than a bug, record the question. Don't pick an answer to make it green.

Tech Notes

Stack: PHP 8.4, Laravel, PHPUnit
Guards read routes, controllers, package source and views; each has an allow-list where every entry carries a reason, and unused entries fail
bin/check runs them as a separate early stage
API: each ability declares the permission its owner must hold; middleware reads abilities off the route's own middleware so modules are covered by default
Every guard verified by planting a violation and watching it go red