Backup-task failures
Symptom — an Ægir Clone, Migrate, or Backup task aborts with:
Could not back up sites directory for drupal.
This is a fault at the source site, not the target: the same backup step runs first in Clone and Migrate, so a backup failure shows up across all three task types. Six distinct root causes, in rough order of frequency.
Cause 1 — permissions / ownership
The backup process must read every file under sites/<domain>/ and write the
tarball. Ownership drift (Composer wrote files with the wrong owner, a manual
copy left wrong perms) breaks it. Repair with the fix-drupal-* scripts —
equals-form flags only — against the real platform path under
/data/disk/<USER> (the --site-path target must contain settings.php or the
script exits 1):
# As oN.ftp
fix-drupal-site-permissions.sh --site-path=/data/disk/<USER>/static/<platform>/sites/<domain>
fix-drupal-site-ownership.sh --site-path=/data/disk/<USER>/static/<platform>/sites/<domain>
# Or whole-platform if the drift is broad
fix-drupal-platform-permissions.sh --root=/data/disk/<USER>/static/<platform>
fix-drupal-platform-ownership.sh --root=/data/disk/<USER>/static/<platform>
Then re-run the failing task.
Cause 2 — disk full
df -h /data/disk/<USER>
df -h /home/<USER>.ftp
Near 100 %, the backup can't write the tarball. Common space to reclaim:
# Old backup tarballs
ls -lt /data/disk/<USER>/backups/ | head
# Old per-platform site trees
ls -lt /data/disk/<USER>/static/
# Tenant DB dumps under quota
du -sh /data/disk/<USER>/static/files/dbackup/* 2>/dev/null
A tenant can shorten DB-backup retention via
/data/disk/<USER>/static/control/dBackupCycle.info. For headroom rather than
cleanup, resize the data disk.
On a box with an attached data disk, migratefs frees the root partition by
moving data, never destroying it: it relocates each account's
/data/disk/<oN>/static/files store and the shared /data/disk/arch (SQL
dumps, cluster backups) onto the single real mountpoint under /mnt and
symlinks them back — on any error the real directory is left in place.
The
default run is a DRY plan; migratefs --apply performs the relocation (it
pauses the Ægir queue via /run/boa_queue_stop.pid and drains tasks first;
--grace default 15 s).
Operator-only, never automated: it assumes exactly one
attached mountpoint under /mnt and refuses to run when more than one is
present (even with an explicit --target), and the arch relocation defers
while a backup writer
(duplicity, mydumper, sequential_backups, dbackup,
mysql_cluster_backup) is active.
Full runbook: migratefs storage relocation.
Cause 3 — stray sites/<domain>.restore from a prior failure
A previous failed Clone/Migrate may have left sites/<domain>.restore behind,
and the next backup tries to include it and fails:
ls -la /data/disk/<USER>/static/<platform>/sites/ | grep '\.restore'
rm -rf /data/disk/<USER>/static/<platform>/sites/<domain>.restore
Then re-Verify and retry. The shell-free alternative — including the gated
orphan-database clean-up the rm leaves behind — is the platform's
Import leftover → Purge leftover flow described in
Task failures.
Neither route removes the <domain>.restore nginx vhost and Drush alias the
same failed task can leave behind — those are reaped separately, and only when
enabled. See
The leftover vhost and alias go separately.
Cause 4 — symlinks pointing outside the site dir
Some modules create symlinks (Compass-generated CSS, etc.) pointing outside the site directory; the backup tar can't follow them.
# As oN.ftp — inspect, then remove links pointing outside the site tree
find /data/disk/<USER>/static/<platform>/sites/<domain>/files -type l -ls
rm /path/to/offending/symlink
Then retry the task.
Cause 5 — MyQuick-era archives carry no single .sql dump
A backup task produces a self-contained archive with the database inside
whenever the run carries a database-bearing mode, and every task queued through
the Ægir front end does: the panel Backups form opens on Site files with
classic mysqldump DB, and the task runner stamps that same mode on any
backup task that reaches it without one — so API-queued and scheduled backups
inherit it too. The automatic safety copy a Restore takes first is forced to
that mode explicitly.
Three shapes still hold no dump. The deliberate Site files without any DB
mode skips the dump by design, as its name promises. A run that carries no mode
at all on a MyQuick account — the safety copies Clone, Migrate and Delete take,
and a backend drush @site provision-backup invoked outside the task queue —
dumps per-table with mydumper into the shared transient store instead of
writing database.sql into the archive; the Clone, Migrate and Delete copies
are still recorded in the panel's backup list (Pre-clone backup,
Pre-migration backup, Pre-delete backup), so a listed archive is not by
itself proof that a database is inside it. And archives taken before the
per-mode contract landed carry no dump either.
Restoring one of those dump-less archives does not complete: the files half is
replaced first, then the task log states that the archive carried no database dump and
the restore fails at the database step, leaving the live database as it was. So a
Restore that ends in error on such an archive with the database unchanged is the
expected reading of that archive, not a box fault. To wind a database back to that archive's moment, import the matching
dated split dumps under ~/backups/ by hand via myloader (see
Dumps & mydumper); the restore's own
pre-restore safety copy is a classic single-file archive and is listed in the panel
like any other backup.
A related task-log error is protective, not fatal: Refusing the fast database import: the shared dump store does not provably hold this site's database.
The shared mydumper store is not bound to a database name (Clone and Migrate
rename databases mid-flow), so before any fast import the store must prove it
holds the right dump — a leftover export from an earlier broken task is
refused instead of being imported into the wrong site's database. The store is
left intact for inspection; re-running the task takes a fresh dump and
proceeds normally.
Cause 6 — mysqldump: Permission denied
Distinct from Cause 1 (site-tree ownership). Here the dump or transfer binary
itself is non-executable for the unprivileged aegir user, with a task-log
line like:
mysqldump: Permission denied
(or rsync permission-denied during Clone/Migrate). On affected installs
/usr/bin/mysqldump and /usr/bin/rsync were left mode 750 root:root, so
aegir could not execute them and every Backup/Clone/Migrate failed. 5.10.1
restored mode 755.
The re-apply lives in autoupboa, not in barracuda:
autoupboa runs chmod 755 on both binaries whenever it executes, on the
weekly auto-update cron and from periodic maintenance — and autoupboa is the
orchestrator that drives barracuda as a leaf process, never the reverse.
Fix
chmod 755 /usr/bin/mysqldump /usr/bin/rsync
This is the immediate fix; re-run the failed task. On an auto-updating box the
correction re-applies on the next autoupboa pass with no operator action.
autoupboa reads /root/.restrict_this_vm.cnf (a restricted-VM marker that
would otherwise lock both binaries to 700), but it removes that marker
first and then always takes the 755 branch — so on a box that auto-updates
the binaries end up 755 regardless of whether the marker was present, and a
700 lock is never something an autoupboa run leaves behind.
A bare
barracuda up-<tier> system run by hand does not restore the mode —
barracuda has no such logic and never calls autoupboa. If you set the mode
by hand on a box that does not auto-update, re-apply the chmod 755 after any
maintenance that may reset it.
Diagnostic checklist
- Disk space (
df -h). - Permissions/ownership fix scripts.
-
sites/<domain>.restoredirectory. - Stray symlinks in
files/. -
/usr/bin/mysqldumpand/usr/bin/rsyncare mode755(Cause 6). - Site Verify passes.
- Re-attempt the failing task.
If all of the above check out and the task still fails, the cause is deeper (DB connectivity, an Ægir code-level bug) — escalate with the task log attached.
Related
- Ægir task failures — general task-failure patterns and
the
.restoredirectory. - Cross-host migration — Migrate (same backup mechanism).
- Site cloning — Clone (same backup mechanism), MyQuick mode.
- migratefs storage relocation
— relocating files stores and
/data/disk/archonto an attached disk (Cause 2 headroom). - Backups internals & operations — the off-site backup story.
- Dumps & mydumper —
mydumper/myloaderrestore of MyQuick split dumps.