Legacy backboa & duobackboa (AWS S3)

backboa is BOA's original off-site backup tool: AWS-S3-only, single bucket, symmetric-passphrase Duplicity encryption, configured inline through _AWS_* variables. duobackboa is its twin pointed at a second, independent bucket. Both ship and run on every host (no licence gate) and remain on the static crontab; the modern path is multiback/dcysetup. This page is the operating reference for hosts still running the legacy tools.

Configuration — _AWS_*

backboa sources /root/.barracuda.cnf; duobackboa sources /root/.duobackboa.cnf. Same variable set:

# Required
_AWS_KEY='AWS Access Key ID'
_AWS_SEC='AWS Secret Access Key'
_AWS_PWD='encryption passphrase'   # openssl rand -base64 32 — store it safely
_AWS_REG='us-east-1'               # AWS region (default us-east-1 if empty)

# Optional (defaults shown)
_AWS_TTL='30D'                     # retention window (remove-older-than)
_AWS_FLC='7D'                      # full-cycle frequency (--full-if-older-than)
_AWS_VLV='warning'                 # Duplicity verbosity: error/warning/notice/info/debug
_AWS_EXB='YES'                     # exclude Aegir Backup tarballs (data/disk/*/backups)

Missing any of the three credential values (_AWS_KEY, _AWS_SEC, _AWS_PWD) prints the configuration block (with the full supported _AWS_REG list) and exits. _AWS_REG is the fourth nominally-required value but does not gate that check — an empty _AWS_REG silently defaults to us-east-1. When set, _AWS_REG is compared against a hard-coded list of those same 31 region IDs, but that check is inert: the match flag it sets is never read, so the value is used exactly as given — passed verbatim into the S3 endpoint URL (s3.dualstack.<region>.amazonaws.com) and --s3-region-name whether or not it matched the list. backboa itself never rejects an out-of-list or misspelled region; a wrong one is not caught up front and only surfaces later as a connection failure at backboa test/backup time. _AWS_TTL defaults to 30D and _AWS_FLC to 7D when unset. _AWS_VLV (accepted: error/warning/notice/info/debug, or the first letter of each) sets the Duplicity verbosity for backup runs — but the two legs that delete remote data (the weekly remove-older-than pass and the monthly cleanup --force) never run below notice: Duplicity lists what it deletes only at notice, so with the warning default those deletions would happen without a trace in the log. info/debug raise the deletion legs too; error/warning apply only to the backup runs. /data/conf/arch is excluded regardless of _AWS_EXB; the variable only toggles the per-tenant tarball exclusion. The default YES adds --exclude-regexp '^/data/disk/.*/backups' (the Ægir Backup tarballs); _AWS_EXB=NO drops that one regexp so the tarballs are kept.

These are the legacy equivalents of multiback's KEEP_WITHIN / FULL_BACKUP_FREQUENCY — see Retention.

Supported _AWS_REG values

The configuration block backboa prints when the credentials are missing lists its 31 supported region IDs for _AWS_REG — 29 standard commercial regions plus the two AWS GovCloud regions. Set it to one of the codes below exactly; an empty _AWS_REG defaults to us-east-1.

Create the S3 bucket in this same region: the region is part of the S3 endpoint the backup is written to, so it must match the bucket's region (see Bucket names below).

What gets backed up

backboa and duobackboa both use the same _SOURCE:

/data /etc /home /opt/solr4 /var/aegir /var/solr7 /var/solr9 /var/www /var/xdrago

That is, the full nine-path set — including the Solr index dirs and /var/xdrago (BOA's own tooling/logs), which a shorter five-path list would wrongly imply are excluded. The backup is sent to a single S3 bucket with a --volsize 300, --s3-use-ia Duplicity invocation under symmetric (_AWS_PWD) encryption.

Bucket names

<host> is /etc/hostname lowercased with dots replaced by dashes (verify with cat /etc/hostname). Duplicity can create the bucket on demand but in practice AWS region propagation delays make that unreliable — create the bucket in the AWS S3 console first, in the _AWS_REG region, with the exact name above.

Install & first run

backboa install        # install/upgrade Duplicity + Python + par2 + awscli/boto3
backboa test           # verify connection to the bucket
backboa backup         # first run (a full)

install performs dependency setup only — it does not validate config. _AWS_* is sourced on every invocation; test is the connectivity check: it runs duplicity cleanup --dry-run against the bucket and declares failure on any non-zero duplicity exit — credential, endpoint and bucket problems alike — printing the exit code and the full duplicity output to the terminal. An empty, never-backed-up bucket still passes (a fresh first run is not a failure). The same applies to duobackboa test against its own bucket.

Scheduled runs — the static crontab

The legacy tools run from the BOA-managed root user crontab (the fixed crontab -l schedule), not the interval /etc/cron.d/duplicity_backup drop-in that the modern path uses:

15 3 * * *   ...  /opt/local/bin/backboa backup
15 5 * * *   ...  /opt/local/bin/duobackboa backup

So backboa runs at 03:15 and duobackboa at 05:15 daily. (For reference, the local SQL dump mysql_backup.sh runs at 01:15 and graceful.sh at 03:01 — neither is a remote backup.) Full-vs-incremental is decided inside the tool (_set_mode/_randomize_full), not by a separate weekly cron line.

Why the two runs do not collide

Each tool holds its own pid file for the length of a backup run: backboa writes its PID to /run/<host>_backboa.pid and duobackboa to /run/<host>_duobackboa.pid, removing it when the run completes. The gate is a liveness check (kill -0) on the PID inside the file, so a file left behind by a crashed run holds nothing back — it is overwritten and the run proceeds. When the PID is live, the second invocation touches /var/log/boa/wait_<host>_backup.log, prints Live run holds <pid file> (pid N)... and exits 1. A completed run touches /var/log/boa/run_<host>_backup.log.

The two tools gate differently, and deliberately so:

• backboa never waits. A positive pgrep -fc duplicity aborts it immediately, for every verb, with a line in /var/log/mybackup_waiting_queue.log.
• duobackboa backup waits instead, bounded at two hours, polling every five minutes. It treats as busy any live duplicity process and a live PID in backboa's pid file — so it also waits through the gaps between backboa's sequential Duplicity invocations. Every other duobackboa verb keeps the immediate refusal, so an operator restore or list never blocks for hours.

duobackboa reads backboa's pid file; backboa does not read duobackboa's.

Three lines in /var/log/mybackup_waiting_queue.log tell you which path a run took:

Active duplicity/backboa run detected, waiting for it to finish...
Backup slot freed after <n>s, proceeding...
Still busy after <n>s, giving up until the next run...

AWS CLI missing

On the backup and status verbs both tools check for /usr/local/bin/aws. If it is absent they print

NOTE: AWS CLI missing; S3-dependent legs will fail until '<tool> install' is run

and carry on — there is no unattended reinstall mid-backup. Run backboa install or duobackboa install to restore the dependency.

Randomised full-backup day

By default _RDW=7 (Sunday) is the full-backup weekday when _AWS_FLC=7D. Creating /root/.randomize_duplicity_full_backup_day.cnf switches to a randomised, persisted weekday (stored locally in /var/log/boa/<bucket>.randomize.full.log) and staggers each step with a random short sleep — useful to spread fleet load off a fixed Sunday peak. duobackboa defaults to _RDW=6 in the non-randomised branch.

On an omega8.cc-hosted box this marker does not survive. There (/root/.host8.cnf present, or a hostname ending .aegir.cc, plus the hosted layout — the Master Drush alias, /data/u and /var/xdrago) BOA's own normalisation pass, _update_agents in BOA.sh.txt, deletes it on every run — roughly every five minutes, because clear.sh pulls and runs that script — so the fixed default weekday always applies. The toggle is effective on self-hosted boxes only.

Monthly cleanup toggle

_monthly_cleanup runs cleanup --force once a month on a per-bucket random day-of-month 1–5 (persisted in /var/log/boa/<bucket>.randomize.cleanup.log), unless /root/.skip_duplicity_monthly_cleanup.cnf exists. Age retention itself runs via remove-older-than ${_AWS_TTL} --force after a full-day run.

Both legs run at a notice verbosity floor regardless of _AWS_VLV, and both write their output into the per-bucket log (and so into the emailed report): the retention pass lists every backup and signature chain it deletes, and the monthly cleanup lists the extraneous files it removes. This also covers a Duplicity 3.2.0.2 behaviour: remove-older-than --force first deletes the newest backup set when an interrupted run left it partial or incomplete — a damaged, non-restorable set (the active chain itself is never deleted) — and announces that only at notice. See Retention for the mechanism.

Like the randomised-day marker, this one is deleted on every _update_agents pass on an omega8.cc-hosted box, so the monthly cleanup --force always runs there; the toggle is effective on self-hosted boxes only.

Custom include / exclude

Beyond _AWS_EXB, fine-tune with singular-named files (a trailing s would be read by nothing):

/root/.backboa.include       (--include-filelist)
/root/.backboa.exclude       (--exclude-filelist)

duobackboa reads /root/.duobackboa.include / /root/.duobackboa.exclude. One path per line.

Restore & retrieve

restore is positional; the optional time is the second argument (there is no --time CLI flag — the --time is internal to the Duplicity call). The path must not start with a slash:

# latest version of a path to tmp/
backboa restore data/disk/o1/backups/foo.tar.gz tmp/foo.tar.gz

# an older version — interval or full date as the middle positional
backboa restore data/disk/o1/backups 7D8h8s tmp/backups
backboa restore data/disk/o1/backups 2014/11/11 tmp/backups

retrieve restores another host's bucket — it appends the source hostname so the bucket name resolves to that host, while reading the current host's _AWS_* credentials (set them to the source host's keys temporarily):

backboa retrieve data/disk/o1/backups/foo.tar.gz tmp/foo.tar.gz srv.foo.bar
backboa retrieve data/disk/o1/backups 2014/11/11 tmp/backups srv.foo.bar

Restored content goes to the target you name; it does not overwrite source files. Move it back with mv/rsync afterwards.

Logs

/var/log/boa/daily-boa-<host>.log        (backboa, + .archive.log)
/var/log/boa/daily-remote-<host>.log     (duobackboa, + .archive.log)

There is no file named backboa.log. Email summaries go to _MY_EMAIL when set and _INCIDENT_REPORT is not OFF.

backboa vs. multiback

To migrate: stand up multiback (dcysetup install + setup), prove it for several days against at least one provider, then drop the legacy cron line (crontab -l | grep -v backboa | crontab -) while keeping the tool available to restore from the old daily-boa-<host> bucket.

Related

• CLI reference — backboa / duobackboa verb shape.
• Retention — _AWS_TTL / _AWS_FLC semantics.
• Regions & buckets — how multiback constructs bucket names and per-backend target URLs.
• Control-file reference — the .backboa.* include/exclude lists and the duplicity run markers.