Testing & the Gates

What this covers / who it's for. The map of how this app is verified: the architecture-as-tests strategy, every structural guard and what it enforces, the RLS test lifecycle, how to run the suite (parallel vs serial vs filtered), the four gates every change must pass, and the extra CI-enforced lint gate on the workflows themselves. The how-to detail (writing tests, Pest idioms, factories) lives in the pest-testing skill and the stables-app-development testing reference — this page is the map, not the manual.

The strategy: architecture as tests

The repo's non-negotiables (see How a change ships) are not review conventions — they are executable guards in tests/Architecture/ that fail the suite the moment code violates them. Two design choices make the guards durable:

1. Discovery from disk, not hand-lists. Helpers in tests/Pest.php (module_namespaces(), module_layer_namespaces()) enumerate every module, model, and presentation layer from the filesystem — a new module is guarded automatically, without editing any test. The one deliberate exception is ModuleBoundaryTest, whose dependency rules are hand-maintained: when you add a module, you must slot it into the graph.
2. A meta-guard against vacuous guards. GuardCoverageTest proves the discovery itself works — every module, model, and layer is actually enumerated, and every model file resolves to a loaded class — so a guard can never silently pass by scanning nothing.

Exemptions are always per-class, reviewed allow-lists inside the guard (never a dropped namespace), and the guards keep their own allow-lists honest (stale entries fail).

The guard table

Guard	Location	Enforces
StrictTypesTest	tests/Architecture/	declare(strict_types=1) in every file under app/ and every module's src/
ReturnTypeTest	tests/Architecture/	Native return types on every method, no mixed fallback; FQN exception list only for deliberate generic escape hatches (e.g. the bus's dispatch()/ask())
CqrsTest	tests/Architecture/	Every Command/Query implements its contract, is immutable, and routes to a real handler; never both Idempotent and WithoutTransaction
CqrsBoundaryTest	tests/Architecture/	The read/write boundary (D22/D24): presentation layers (module Http + Livewire) never touch Eloquent, the query builder, or the DB facade — only handlers do; raw DB is confined to the bus + sanctioned internals
ModuleBoundaryTest	tests/Architecture/	The acyclic module dependency graph, pointing down to the Foundation shared kernel — the one hand-maintained guard: new modules must be slotted into its rules
AuthorizationCoverageTest	tests/Architecture/	Every CQRS message is gated by a declared permission (or a conscious allow-list entry); every guarded API route declares ->requires(); the allow-lists themselves stay honest
ModelConventionsTest	tests/Architecture/	Every Eloquent model carries HasPublicUuid + SoftDeletes + Blamable unless class-exempted (D25/D26)
TenantIsolationTest	tests/Architecture/	The conventions that make RLS correct: tenant-scoped models use FillsCurrentTenant, central models (e.g. the backoffice User) never carry the tenant trait
EventConventionsTest	tests/Architecture/	Domain events are final, implement the contract, carry no Eloquent models, have unique wire names; queued jobs/listeners carry only serialization-safe primitives
IntegrationContractsTest	tests/Architecture/	src/Integration/Contracts publishes only interfaces, and no contract method returns an Eloquent model or builder (D52)
RationaleCoverageTest	tests/Architecture/	Compliance-significant commands (freeze/unfreeze/closure, risk-rating change, case disposition) carry a mandatory audited rationale (D46)
GuardCoverageTest	tests/Architecture/	The meta-guard: module/model/layer discovery is non-vacuous, no model file is silently dropped
SchemaConventionTest	tests/Feature/	The realized database schema (not migration source): every table has bigint id + public uuid (uuidv7 default), deleted_at, and the blamable columns — per-convention exemption lists
RlsCoverageTest	app-modules/tenancy/tests/Feature/	The tenants:verify-isolation guard itself: every tenant-scoped table has an enforced RLS policy; a new tenant table without one fails the suite

The structural guards do not replace the per-model runtime isolation test: every new tenant-scoped table also ships a feature test proving create as tenant A, read as tenant B → empty (see Tenancy & RLS).

The RLS test lifecycle

Tenant/RLS feature tests never use RefreshDatabase — RLS DDL (the NOBYPASSRLS role, grants, FORCE-RLS policies) is schema-level and would be torn down by it (D14). Instead, tests/TestCase.php provides:

• $this->refreshTenantDatabase() in setUp() — the schema, role, grants, and policies are provisioned once per process (TenantDatabaseState::provisionOnce()), and each test starts from a fast TRUNCATE ... RESTART IDENTITY CASCADE that leaves schema and policies intact. This replaced the historical per-test migrate:fresh + tenants:rls (~39 min → ~1.6 min suite).
• $this->actingAsClient($user, $tenant) — authenticates on the client guard and presents the tenant's public uuid in the X-Tenant header, resolving the request into that tenant's RLS scope. The standard setup for tenant-API endpoint tests.
• $this->runInTenant($tenant, $callback) — runs a callback inside a tenant's RLS scope (thin wrapper over tenancy()->run()) for arranging/asserting tenant-scoped data.

Experimental "fast reset" toggle (D108, default OFF). The between-test reset truncates ~70 tables every test, most of them already empty. Behind two native env toggles (TEST_FAST_RESET, TEST_FAST_RESET_PARANOID, read via getenv() so docker compose exec -e TEST_FAST_RESET=1 app php artisan test --parallel --compact reaches an already-running container), the reset instead probes which tables actually hold rows in one round-trip and truncates only that dirty subset; TEST_FAST_RESET_PARANOID re-probes all tables afterwards and fails loudly on any residue. Default off means a normal run is byte-for-byte unchanged; the path is instrumented (/tmp/fast-reset-stats.log inside the app container — docker compose exec app cat /tmp/fast-reset-stats.log) so the go/no-go on flipping the default rests on the measured dirty-set distribution. Note: prefixing the make target (TEST_FAST_RESET=1 make test-parallel) is a silent no-op — host env does not cross docker compose exec. See D108 for the two behavioural deltas (untouched-table sequences; the silent-bleed window the paranoid check guards) and the rejected alternatives.

Running the suite

Command	What it is	When
make test-parallel	Full suite in parallel (~2 min); each worker gets a private stables_testing_test_{token} database + its own Redis logical DB index, provisioned by App\Providers\ParallelTestingServiceProvider	The default local verification
make test	Full suite, serial (~12 min)	When parallel contention is suspected
make test-filter FILTER=Name	One test/class, serial, no per-worker overhead	The inner iteration loop
make test-coverage	Suite with Xdebug coverage; gate with ARGS="--min=90"	Before merge; ratchet the floor toward 100

Parallel notes: keep worker count ≤ 16 (Valkey has 16 logical DBs; pass ARGS="--processes=12" on bigger hosts), and remember that parallel sessions/worktrees share the stables_testing database — concurrent full-suite runs contend and produce failures that look like isolation bugs but aren't. Stagger them (see Local development).

Guardrails against a wedged run (D106): make test and make test-parallel both refuse to start on top of an already-running suite and point at make test-kill to clear it; both also clean up in-container on INT/TERM, so a killed non-TTY (agent/CI) run can't leave zombie pest processes behind. Postgres safety valves (idle_in_transaction_session_timeout + lock_timeout, both 120s, applied by App\Providers\ParallelTestingServiceProvider) bound a cross-connection self-deadlock that is invisible to Postgres's own deadlock detector — one worker's pgsql_logs session left idle-in-transaction blocking its sibling pgsql session's between-test TRUNCATE. The dev/CI Postgres also runs with fsync=off/synchronous_commit=off/full_page_writes=off (docker-compose.yml — a durability tradeoff never appropriate outside dev/CI), and the dev image enables OPcache for the CLI SAPI (docker/Dockerfile.dev) so both long-lived paratest workers and freshly spawned artisan/pest processes skip re-parsing the framework.

The four gates

Every change must pass all applicable gates, run inside Docker:

#	Gate	Command	Always?
1	Test suite	make test-parallel	Always
2	Code style (Pint)	make lint	Always (fixes in place locally; CI's static job runs the same check directly — vendor/bin/pint --test, no container)
3	Static analysis (PHPStan/Larastan, level 6)	make stan	Always
4	RLS regeneration + isolation proof	php artisan tenants:rls + php artisan tenants:verify-isolation	After any migration touching a tenant-scoped table, or any change to the RLS machinery (the tenancy module, config/tenancy.php)

CI additionally runs make seed-verify (db:seed --force, twice) between make migrate and the suite, so a seeder change that only breaks on a fresh database is caught there rather than on the preview deploy — it is not part of the routine four, but run it locally whenever you touch a seeder.

CI runs the same gates via the same make targets — split across three jobs in .github/workflows/ci.yml (D99 established the real-compose-stack pipeline; D100 split it across runners for speed and cost):

• tests ("Test suite") — on a Blacksmith ARM runner (blacksmith-8vcpu-ubuntu-2404-arm). Boots the real dev compose stack — the Dockerfile.dev app image baked native arm64 (matching the arm64 production images, tagged stables-app:ci via buildx-bake with a per-arch ci-dev-image-arm64 layer cache this job single-writes), postgres:18-alpine, valkey/valkey:8-alpine, and minio (+ the one-shot minio-init bucket creator, so the real-driver S3 integration tests execute rather than silently skipping), with a docker-compose.ci.yml override that idles the app container. Runs inside that container: a real APP_KEY written into .env before the stack boots (compose snapshots .env via env_file, so the key must exist first) → make composer-install → the Vite manifest built on the runner (npm ci + npm run build, after Composer install because resources/css/app.css @imports vendor/livewire/flux CSS — and empirically required: six operator-facing tests 500 with "Vite manifest not found" without it) → make migrate → make seed-verify → make test-parallel (8 workers on the 8-vCPU box). make seed-verify runs db:seed --force twice against the database make migrate just built: the first pass is the literal reproduction of migrate --seed on a fresh database (previously exercised nowhere but the preview deploy), the second proves the seeder is re-runnable. Any non-zero artisan exit fails the job.
• static ("Static analysis") — on a free GitHub-hosted ubuntu-latest runner via setup-php 8.5, no compose stack. First make lint-workflows (actionlint — see below; a one-shot container, the job's only Docker use, and it needs no PHP so it fails fast), then vendor/bin/pint --test + vendor/bin/phpstan analyse --memory-limit=1G — the same checks with the same result-affecting flags (only output formatting differs) as make lint-check / make stan, so results can't drift from local. A deliberate parity-for-speed tradeoff (these are pure static analysis, no service dependencies); PHP drift still fails fast on the ^8.5 platform constraint at composer install. It generates a real APP_KEY too (Larastan boots the app to resolve bindings).
• test-packages ("SDK packages (in-repo suites)") — on a small Blacksmith ARM runner (blacksmith-2vcpu-ubuntu-2404-arm). make test-packages auto-discovers every package under packages/stables/ carrying a phpunit config and runs its suite in the same app container — today seven SDKs (utila-sdk, conduit-sdk, sumsub-sdk, fingerprint-sdk, seon-sdk, sfox-sdk and sovera-sdk); a new SDK needs no Makefile/workflow edit. The PackageTestCoverageTest architecture guard (run by the main suite) reddens if a package ships without a suite and no documented exception, so the lane can never silently skip one. A read-only consumer of the ci-dev-image-arm64 cache.

A first job, changes ("Detect changed paths") — always-running, free, GitHub-hosted — gates the three build jobs: each declares needs: changes + a fail-open if: (!cancelled() && (needs.changes.result != 'success' || needs.changes.outputs.code == 'true')), so a docs-only change skips all three, while a crashed classifier runs them rather than skipping (a job skipped via if: still reports success, satisfying a future required status check — this is deliberately not paths-ignore, which would never run and leave the check pending forever). The classifier diffs the changed files (with git diff --no-renames, so a rename can't collapse a code path into a docs-looking one) against the branch's last green run — an anchor read via the actions: read REST API, scoped to the changes job only, with the attacker-controlled branch name run through a character allowlist and passed as URL-encoded query fields — and is fail-open: every ambiguous state runs the full pipeline. It acts at push / PR-synchronize granularity — the whole head-state diff is classified at once, so intermediate commits get no separate run and a docs commit pushed together with a code commit runs everything. It also covers direct pushes to main (anchor scoped to branch=main); the Aikido security gate reports independently and is never skipped. This is the change that lets docs-only follow-up commits within a PR stop re-running once the code is green, while never wrongly skipping. The full rationale is D101; the behavior is:

Scenario	Build jobs	Why
Any changed file is code — PHP/JS/config, .github/workflows/**, Makefile, composer.json/lock, the skills-package PHP/composer.json	RUN	Any path off the docs-only list trips code=true; a mixed docs+code diff runs everything
All changed files docs-only + green anchor resolvable + base unmoved	SKIP	Skip route 1 (anchored): all three conditions hold. Jobs report skipped, which counts as success for required checks; no Blacksmith credits spent
Docs-only push after a red run	RUN	The anchor is the older green run, so the still-broken code sits inside anchor..head and trips code=true — a docs push can't turn a red PR green
Docs-only push after a cancelled run	RUN (usually)	A cancelled run is not a success, so it can't anchor; the diff vs the older green (or the fallback) still contains the unproven code
Docs-only, but the base branch advanced since the anchor	RUN	Base-ancestry guard (PR events): the live base tip is no longer an ancestor of the anchor, so the merge is re-tested against the new base
Merge-from-main pushed into a PR	RUN	The base moved, so the base-ancestry guard trips; and if main's delta carried code it is in the anchor..head diff too
Rebase / force-push (anchor sha unresolvable)	RUN / SKIP	Anchor GC'd/unreachable → fall back to the event base and classify the whole-PR diff: its code trips code=true, but skip route 2 (the anchor-less fallback) still SKIPS a genuinely docs-only whole-PR diff; an unresolvable before-sha fails open
Branch creation (before sha all-zeros)	RUN	Fail-open: no meaningful range to diff
No green run yet in the branch's history	RUN / SKIP	No anchor → fall back to the whole-PR (base..head) / push (event.before..head) diff and classify that: a code-bearing PR runs, but skip route 2 (the anchor-less fallback) still SKIPS a genuinely docs-only whole-PR diff
Actions API / base-tip fetch / git diff failure	RUN	Fail-open at every ambiguous step
The gate job itself times out or errors	RUN	The build jobs' condition treats any non-success gate result as fail-open (needs.changes.result != 'success')
Empty diff	RUN	Fail-open: nothing to classify

The docs-only pattern list (maintained in the changes job bash; keep it and D101 in sync with the workflow): wiki/**, docs/**, any **/*.md anywhere (module READMEs, CLAUDE.md, skill markdown, the .claude/.agents/.cursor mirrors), the .claude/.agents/.cursor trees (non-markdown too), .github/ISSUE_TEMPLATE/**, the PR template (.github/PULL_REQUEST_TEMPLATE.md or .github/PULL_REQUEST_TEMPLATE/**), and the licence (LICENSE / LICENSE.*) — the template/licence patterns are anchored, so a lookalike like .github/PULL_REQUEST_TEMPLATEx.php is code. Everything else is code — including .github/workflows/**, the Makefile, docker files, lockfiles, and the non-markdown files under packages/stables/skills. The anchor query is same-repo-and-branch scoped (fork PRs never reach the Blacksmith jobs; a code-bearing fork PR can't false-green — its Blacksmith checks stay pending).

Because make migrate (in the tests job) chains migrate → tenants:rls → tenants:verify-isolation (the last exits non-zero on drift), gate 4 is a hard CI gate in the test pipeline — run against the freshly migrated dev-schema DB (a valid proxy, since the test DBs are migrated from that same schema): a PR that adds a tenant-scoped table without an ENABLED+FORCED policy fails the build. The preview deploy runs the same fail-closed gate too (issue #87); the still-open half — wiring it into the staging/production deploy workflows and the production entrypoint — is tracked in docs/tracking/devops-hardening.md §2. You remain gate 4 locally: run tenants:rls + tenants:verify-isolation after any tenant-scoped schema or RLS-machinery change before you push. (Once branch protection is configured on main, its required checks should be the three job names above — none is configured as of this writing; see PR #73's post-merge checklist. The old "Build & Test" / "utila-sdk package" checks no longer exist either way.)

The extra CI lint gate: workflow linting

One more gate applies only when you touch .github/workflows/**. make lint-workflows lints them with actionlint in a one-shot Docker container — needing no compose stack at all, unlike the gates that run in the app container. It is pinned by image digest (version 1.7.12 kept as a comment beside it), not by tag: third-party code executing in CI follows the same rule as the SHA-pinned actions, and a tag in a personal Docker Hub namespace is repointable by its publisher. The container runs sandboxed — read-only whole-repo mount, --network none, --security-opt no-new-privileges, no secrets — because that mount spans the repo root and locally includes the gitignored .env and full git history. A pinned image rather than a marketplace action keeps the supply-chain surface where D101 left it. CI runs the identical command as the first step of the static job (before setup-php, so a malformed workflow fails in seconds). It is CI-enforced but not one of the four gates above: it validates delivery-pipeline YAML, not application code. Run it locally before pushing any workflow change — which the D101 skip gate classifies as code, so the full pipeline runs anyway.

Be precise about its reach — verified against actionlint 1.7.12:

Caught	Workflow schema errors; invalid event names/activity types; unknown runs-on: labels (the Blacksmith ARM labels are declared in .github/actionlint.yaml — add a row when a new runner size is adopted); bad steps.<id>.outputs.* and other typed-context references; malformed ${{ }} expressions
Not caught	Typos in github.event.* property access (loosely typed — github.event.label.nmae passes); anything inside run: bodies, because the bundled shellcheck pass is off

The shellcheck pass is disabled (-shellcheck=) so the gate lands with a zero-finding baseline instead of an ignore list. With it enabled the workflows report 53 pre-existing findings, all audited as non-behavior-affecting: 46 SC2086 + 3 SC2046 quoting findings, 2 SC2016 (JMESPath literals in AWS --query arguments, where the single quotes are correct), and an SC2221/SC2222 pair for a shadowed case arm in ci.yml's own docs-only classifier. 36 of the 53 are in deploy-preview.yml, the rest spread across sync-secrets.yml, destroy-preview.yml, deploy-production.yml (4 each), deploy-staging.yml, ci.yml (2 each) and build-image.yml (1). Clearing them is not just a re-quoting job — the per-group breakdown and the fix for each are tracked as docs/tracking/devops-hardening.md §8; the rationale for the gate's shape is D103.

Coverage

The goal is 100% coverage of app/ + app-modules/*/src (the <source> set in phpunit.xml): fully cover what you touch, never let coverage regress. Measure with make test-coverage (sets XDEBUG_MODE=coverage); enforce a floor with ARGS="--min=90" and ratchet it up — don't put --min=100 on the default run until coverage actually is 100%. Details and the fast unit-vs-feature split: the testing reference.

---

← Engineering wiki index