Skip to main content

D100 — CI test execution moves to Blacksmith ARM runners; static gates stay GitHub-hosted (RESOLVED)

Architecture decision record. Status, thematic clusters, and how to record a new ADR: the decision log index. Refines D99 (which established the real-compose-stack pipeline) by splitting it across runners; the pipeline as a whole is mapped for readers on Testing & the gates. Runner economics and the still-open deploy-time RLS gate are tracked in docs/tracking/devops-hardening.md §2/§4.

Context​

D99 made CI boot the real dev compose stack and run every gate inside the app container through the make targets — killing toolchain drift, but as a single job on a GitHub-hosted standard 2-core runner. Two costs followed from the 2-core box:

  • The suite is CPU-bound and the runner gave it two cores. make test-parallel defaults to the runner's CPU count, so a 2-core runner runs 2 paratest workers — the full parallel suite took ~18 min, and the whole job (including the cold arm64-less image build) ran ~21.5 min.
  • Scaling up on GitHub is not free. GitHub's included free minutes apply only to standard runners; larger GitHub-hosted runners (4/8-core) bill per-minute from the first minute and never draw on the free-minutes allowance. So "just use a bigger GitHub runner" converts every CI minute into a paid minute.

A pricing comparison surfaced a cheaper axis. Blacksmith (a third-party CI-runner vendor, installed here as a GitHub App) grants a free tier of 3,000 2vCPU-x64-minute credits/month that do apply to larger runner sizes proportionally, and ARM runners are discounted — an ARM 2vCPU-minute costs 0.625 credits vs 1.0 for x64 — on faster cores. That makes a large ARM Blacksmith runner both faster (more workers) and cheaper (discounted, credit-covered) than a large GitHub-hosted runner, while GitHub's own free minutes still cover a lightweight static-analysis job.

Separately, moving to ARM is a parity improvement: build-image.yml already bakes the arm64 production images, but D99's CI built/ran x64 — so CI and prod diverged on architecture.

Decision​

Split the D99 pipeline into three jobs, placing each on the runner that fits its shape — the compose stack + suite on a large Blacksmith ARM runner, the pure static gates on a free GitHub-hosted runner, the SDK package suites on a small Blacksmith ARM runner.

  • tests ("Test suite") → blacksmith-8vcpu-ubuntu-2404-arm. The full compose stack (D99) with a native arm64 image bake — matching the arm64 production images from build-image.yml, so CI now mirrors the production architecture too. Layer caches are per-arch, so the bake uses a dedicated ci-dev-image-arm64 scope; this job is the single writer of that cache. It runs make composer-install → the runner-side Vite build (npm ci + npm run build) → make migrate (the migrate → tenants:rls → tenants:verify-isolation hard gate) → make test-parallel (defaults to 8 workers on the 8-vCPU box, within Valkey's 16-logical-DB limit), plus failure-only stack diagnostics. The npm build stays — empirically required (see Consequences).
  • static ("Static analysis") → GitHub-hosted ubuntu-latest via setup-php 8.5, no Docker. vendor/bin/pint --test + vendor/bin/phpstan analyse --memory-limit=1G — the same checks with the same result-affecting flags (only output formatting differs) as the make lint-check / make stan targets, so results can't drift from local. This is a deliberate parity-for-speed tradeoff at the user's request: pint/phpstan are pure static analysis with no service dependencies, so they don't need the container, and running them on a free GitHub runner keeps them off the Blacksmith credit budget. PHP-version drift still can't slip through — composer install enforces the ^8.5 platform constraint and fails fast on a mismatch. The job needs a real APP_KEY: Larastan boots the app to resolve container bindings, and an empty key makes the encrypter binding unresolvable (it produced false "undefined method encryptString/decryptString" errors on the first live run), so it generates a key exactly like the container jobs.
  • test-packages ("SDK packages (utila, conduit, sumsub)") → blacksmith-2vcpu-ubuntu-2404-arm. make test-packages runs the utila-sdk, conduit-sdk AND sumsub-sdk suites inside the bake-built app container (PHP 8.5 arm64). It is a read-only consumer of the ci-dev-image-arm64 cache (cache-from only, no cache-to — single-writer discipline). This replaces the old runner-hosted utila-sdk job and newly gates conduit + sumsub, which were previously not wired to CI at all. Owning the deviation: the user's original split put this SDK job on GitHub-hosted runners, but the follow-up requirements ("run it in container as well", "use arm images") make that impractical — an arm64 container on GitHub's x64 standard runners means QEMU emulation. The Blacksmith 2vCPU ARM placement was surfaced during the session, and the user's subsequent lint/stan-only correction left the package job in-container; moving it back to GitHub-hosted (x64 image or QEMU) remains an easy one-line change if free-minute economics ever dominate.
  • Third-party runner = an explicit vendor-trust decision, made by the user. Security caveat: the repo source and a read-only GITHUB_TOKEN execute on Blacksmith infrastructure. This is acceptable because CI carries no repo secrets — the pipeline needs none (a freshly generated APP_KEY, the compose stack, and public package installs), and the default token stays contents: read.

Consequences​

  • CI now matches the production architecture (arm64), not just the PHP/Postgres/Valkey majors — closing the last CI↔prod parity gap that D99 left (x64 CI vs arm64 prod).
  • The conduit-sdk and sumsub-sdk suites are gated for the first time. The old workflow only ran utila-sdk; a comment (and the tracking doc) noted the gap. All three now run in-container.
  • Live-run timings (2026-07-12). tests went green in 13m40s including the cold arm64 build (vs ~22 min on the GitHub 2-core runner); test-packages green in 5m07s. Faster and on discounted, credit-covered ARM minutes.
  • Once branch protection is configured on main, its required-check names must match the new jobs. The old checks "Build & Test" and "utila-sdk package" no longer exist; the required checks should be "Test suite", "Static analysis", and "SDK packages (utila, conduit, sumsub)". None is configured as of this writing — see PR #73's post-merge checklist. A stale required-check name silently stops gating.
  • The Vite manifest is empirically required, so the npm build stays. A proposal to drop npm ci/npm run build was refuted: with the manifest and public/hot absent, six operator-facing tests (Fortify auth screens + the backoffice dashboard) return 500 with "Vite manifest not found" because the suite renders @vite Blade views with no Vite::fake. The build runs after make composer-install (the flux CSS @import needs vendor/).
  • No repo secrets execute on third-party infrastructure — the pipeline was already secret-free, which is what makes the vendor-trust decision tolerable.
  • The static job's setup-php extension list is a maintained parity surface. It mirrors Dockerfile.dev's extension set and must be updated together with it — drift fails loud via Larastan boot errors, not silently.
  • Fork PRs will not get Blacksmith runners (a repo-owner-billed app): the tests and test-packages jobs would sit queued until GitHub's stale-job cancellation while only the static job runs. External contributions are expected to be maintainer-rerun; revisit if the repo ever takes fork PRs routinely.
  • Queued-time exposure. timeout-minutes counts only after a runner picks the job up, so a Blacksmith outage leaves the two container jobs pending (static analysis, being GitHub-hosted, still reports); merges block until Blacksmith recovers or the jobs are cancelled.
  • Credit overage. Beyond the 3,000 monthly credits Blacksmith bills pay-as-you-go (Ubuntu ARM $0.0025/min per 2vCPU) — watch usage in the Blacksmith dashboard.
  • Docs-only changes skip these build jobs (spending no Blacksmith credits) via the changes ("Detect changed paths") change-detection gate — its last-green anchor + base-ancestry mechanics and the full decision table are recorded in D101.
  • Architecture note for contributors. amd64 dev machines now build/test x64 locally while CI tests arm64 (matching prod); ordinary failures reproduce fine locally, and only genuinely arch-sensitive bugs could diverge — a rare new failure mode to be aware of.

Alternatives rejected​

  • All-GitHub, just use a larger runner. Larger GitHub-hosted runners bill per-minute and never draw on the free-minutes allowance (those cover standard runners only), so this is the most expensive option — the exact cost the Blacksmith move avoids.
  • GitHub-hosted ARM for the static job. GitHub-hosted ARM runners' availability and free-minute status for private repos is unverified, so the static job stays on the known-free ubuntu-latest x64 (architecture is irrelevant for pure static analysis).
  • Drop the npm build from the suite job. Refuted empirically — six operator-page tests 500 without the manifest (above).
  • x64 Blacksmith runners. An x64 2vCPU-minute costs 1.0 credit vs 0.625 for ARM, so x64 spends more of the free-tier budget and loses the arm64 prod-architecture parity — strictly worse on both axes.

← Decision log index