Building the test-codebase mirror

The other two leaves in this topic are about proving your fork change holds. This one is about the codebases you prove it against: the newer Drupal cores and distributions you install, clone and migrate to exercise BOA end to end. Building that set by hand — track upstream, resolve every version, work around each distro's Composer quirks, package, publish — is exactly the kind of manual procedure BOA wraps in a tool, and staticbuild (aegir/tools/bin/staticbuild) is it.

It is a first-class shipped tool, deployed on every SKYNET box the same way edgetest is: fetched by _fetch_versioned and symlinked to /usr/local/bin/staticbuild (BOA.sh.txt:1466), so on any current box it is just staticbuild on the PATH. It is kept everywhere so the mirror source always carries the current builder, but it only does anything on the mirror-source VM, where the built codebases are published to /var/www/static/{core,distro,dev/{dev,lts,pro}} — the tree the fleet's read mirrors replicate and installs fetch from.

Nothing is hardcoded: each distribution is built from its Composer spec (farmOS from its GitHub release), each vanilla core from its minor's latest patch, the core minors are auto-discovered, and the versions Composer actually resolves are read back out of the build and stamped into the tarball names. The only settings to review are the Composer specs and the core floor/exclude in the configuration block at the top of the script.

Run as root, build as the Octopus user

staticbuild is fail-closed about who it runs as. It refuses to start unless it has both:

• root (staticbuild:205) — packaging, the per-build /usr/bin/php switch, and the writes into /var/www/static/ all need it;
• a real Octopus build user (_require_o8, staticbuild:208-217) — o8 by default, -u USER overrides. It must be an existing system account with a /data/disk/<user> home, and wget/curl must be present.

The builds themselves run as that user under /data/disk/<user>/static/<MM-DD>/; only the packaging and publishing steps use root. Crucially, it never scans /data/disk to guess an account — a bare o* glob there would run builds as an arbitrary customer instance, so a missing or wrong -u is a hard error, not a guess. A single-run lock (/run/staticbuild.lock) guards against re-entrancy, and an EXIT trap restores the original /usr/bin/php symlink and releases the lock even on a failed or interrupted run.

Subcommands

staticbuild check              # report the latest upstream versions a build would pull
staticbuild all                # build every target, package, publish
staticbuild build [name ...]   # build all, or only named targets
staticbuild package            # clean + tar (cores keep core/profiles, distros strip)
staticbuild distribute         # copy tarballs to /var/www/static/{core,distro,dev/{dev,lts,pro}}
staticbuild backdrop           # build + package + publish ONLY the Backdrop family

check is read-only and needs no root — run it each cycle to see what a build would pull before committing to one. Options precede the action: -d MM-DD picks the build-day directory (default today), -u USER the build user, -f forces a rebuild of an already-built platform, and -C skips the opening composer clearcache.

What it builds

Six distributions to distro/, each from its own upstream, named by the authoritative distribution profile version (not the create-project template's, which is often a -dev alias) plus the resolved core version, e.g. localgov-4.0.2-11.4.4:

Raw vanilla cores to core/, one per supported minor at its latest patch, named drupal-<version>. Minors are auto-discovered: every released major.minor at or above the floor (_CORE_FLOOR, 10.2) minus the exclude list (_CORE_EXCLUDE, 11.0), so the newest minor is always built without editing the script — currently 10.2–10.6 and 11.1–11.4. Set an explicit _CORE_MINORS list only to pin a fixed matrix.

The Backdrop family (see below), built alongside everything else by staticbuild all.

Versions, and the older-core fallback

Every build resolves upstream live. staticbuild check reports what each distro and the core matrix would pull — drupal.org's release feed is the authoritative source for most distribution profile numbers, GitHub for farmOS, and Packagist for the core minors (and for commerce_kickstart, which has no drupal.org stable release). The distro version stamped into the tarball name is read from the built composer.lock, and the core version from the built docroot's Drupal.php VERSION constant, so the name always reflects what was really installed.

Because the newest core sometimes breaks a distribution — a distro's bundled core patch goes stale on a just-released core, for instance — a distro that fails to build on the latest core is retried pinned to progressively older core minors, and the newest one that builds is kept (_with_core_fallback, staticbuild:310-331). Each retry starts below the core the failed attempt had resolved, wipes the lock and vendor tree, and pins both drupal/core and drupal/core-recommended (a distro may require drupal/core directly, so pinning only the recommended metapackage would conflict rather than override).

Per-distribution quirks

Each of these is a real Composer-level workaround the manual procedure needed and the tool now applies automatically. Distributions build under php85 and vanilla cores under php84 — staticbuild swaps the /usr/bin/php symlink per build and restores it on exit.

• allow-plugins — every distro gets composer config allow-plugins true (_composer_prepare). The distros pull varying plugin sets (composer/installers, cweagans/composer-patches, installers-extender, …), and current Composer blocks any unlisted plugin, failing the install; enumerating them per-distro is fragile.
• advisory blocking off — policy.advisories.block false is set for the batch because commerce_kickstart enables Composer's security-advisory blocking, which refuses every advisory-affected core or dependency and also defeats the older-core fallback (older cores carry advisories). These are throwaway test platforms, so it is turned off. Commerce Kickstart also builds in two phases — install, add the centarro/certified-projects layer, then re-resolve and install.
• openculturas — its distribution ships no example.sites.php (one is copied in from the platform's own core scaffold, falling back to a sibling build), and its core patch no longer applies on current core while composer-patches 2.x has no per-patch skip, so it is built unpatched via extra.composer-patches.ignore-dependency-patches. The build also flattens the openculturas-distribution wrapper so its inner profile becomes the install profile.
• thunder — the template thunder/thunder-project versions independently from the actual distribution thunder/thunder-distribution (e.g. 5.0.0 vs 8.4.0); the platform is named by the distribution.
• cms — drupal_cms's post-update-cmd cleanup exits non-zero, so its drush require runs with --no-scripts; the whole build runs non-interactively so the "move this requirement?" prompt (drupal/cms dev-requires drush) never hangs it.

The stages it automates

staticbuild all runs three stages in order; build, package and distribute run them individually against a build-day directory.

1. build — composer clearcache, then for each target: composer create-project into a hidden working dir (farmOS is a release tarball instead), allow-plugins and the advisory/patch config, composer update --no-install then composer install --no-dev, with the older-core fallback wrapping the install. Vanilla cores install immediately and add drush; distributions stage without installing until the fix-ups are applied.
2. package — strip the artefacts that must never ship inside a platform tarball (sites/all/drush, stray o_contrib*, PID files, sites.php, …), then gzip the raw cores first, keeping their core/profiles, then strip the stock core/profiles/* from the distributions (they ship their own install profile) and gzip those. Re-running package is idempotent — cores always keep their profiles.
3. distribute — route each tarball by kind: drupal-*.tar.gz and backdrop.tar.gz to core/, bee.tar.gz and backdrop-drush-extension.tar.gz to every per-tree dir — dev/dev/, dev/lts/, dev/pro/ — everything else to distro/.

Once published, register the built platforms in Ægir (paths like MONTH-DAY/drupal-11.4.4) and run site install, clone and migration against them — the three tasks the whole set exists to exercise. How BOA registers a pre-built mirror tarball as a platform is covered on The staged setup.

The Backdrop family

staticbuild is also the Backdrop-family builder (BOA.sh.txt:1462). Backdrop is not Composer-based, so it is built apart from the Drupal distros and cores — no Composer, no /usr/bin/php switch, just git / wget / unzip — and always fetched at its newest upstream, published version-less so BOA never falls behind. staticbuild all builds it alongside everything else; staticbuild backdrop does a lightweight build + package + publish of only this family, scoped so it never touches unrelated build-day content. Three artefacts, always rebuilt at the latest upstream tag (pin any with the matching _*_TAG in the config block):

• backdrop — Backdrop CMS core (backdrop/backdrop), from its latest GitHub release backdrop.zip. Repackaged version-less as backdrop.tar.gz (extracts to backdrop/), classified as a core, with the resolved version written to backdrop.txt so BOA can name the platform while still fetching the version-less tarball. The Backdrop redis contrib module is baked into modules/ for Valkey/Redis object-cache support (its cache implementation is Backdrop-native; the Drupal redis edge is not compatible). Published to core/.
• bee — the native Backdrop CLI (backdrop-contrib/bee), from its latest git tag, packaged version-less as bee.tar.gz (bee.php at the root). Published to dev/{dev,lts,pro}/ — each box fetches from its own tree dir.
• backdrop-drush-extension — the Backdrop Drush extension (backdrop-contrib/backdrop-drush-extension), from its latest git tag, packaged version-less and shipped pristine — BOA applies its own PHP 5.6 de-hint and __DIR__ include fix on deploy. Published to dev/{dev,lts,pro}/.

Downstream, OCTOPUS.sh.txt:160 resolves _BACKDROP_V from that published backdrop.txt stamp, so satellites name Backdrop platforms backdrop-<ver>-{dev,stage,prod} against the current release; the version pinned in OCTOPUS.sh.txt is only a fallback for when the mirror is unreachable at make time.

varbase is disabled

The Varbase builder is present in the script but unwired — removed from the distribution set and from staticbuild check. The upstream Vardot/varbase-project template no longer yields an installable build: a fresh resolve pins a core the Varbase profile rejects, pinning the core the profile wants fatals on a drifted dependency during install, and the older line OOMs then hits a final-class fatal. The last working varbase-10.1.0-11.3.12 build is kept on the mirror and must not be rebuilt until upstream stabilises; the builder is left in place, commented, for an easy re-enable.

Related

• Building & testing BOA changes — the four test gates for a BOA change, including edgetest, staticbuild's documented sibling tool, and the disposable-VM gate where the codebases built here get installed and migrated.
• Verifying the drush extension filter — a worked end-to-end verification of a security-sensitive backend change.
• The staged setup — the consumer side: how a box fetches a pre-built mirror tarball and registers it as a platform.
• The serial & fetch pipeline — how staticbuild itself ships: _fetch_versioned, the fNN serials, and how a bump propagates to the fleet.
• Commands and Variables — the consolidated tool and _VAR tables.