XDR9000 — permanent attack & health archive

A BOA box detects attacks and heals its own services around the clock — and then routinely destroys the records: ban ledgers are reaped within hours, incident logs rotate away, and the major-OS-upgrade path wipes /var/log/boa/*.log entirely. XDR9000 closes that gap. A read-only harvester copies those records into a permanent, append-only archive before their reapers run, and a root-only CLI renders the whole history with one command.

It observes only: nothing is banned, killed, restarted or reconfigured by it. Recording is on by default — no control-file entry and no operator step is needed.

Reading the record

Everything is rendered by the root-only xdr9000 command:

xdr9000 status            # archive health, first record, size, engine state
xdr9000 today             # today's events and samples
xdr9000 attacks --days 30 # per-vector totals, unique IPs, top offenders
xdr9000 heals --days 30   # incident counts per subsystem
xdr9000 report            # full report incl. since-first-record totals
xdr9000 json events       # raw JSONL feed (windowed; json rollup = whole spine)

Windows default to 7 days for attacks/heals and 30 for report, and past months are read through their gzip compression transparently. xdr9000 report totals the attacks blocked, SYN-flood packets absorbed and repairs survived since recording began — computed from the daily rollup, so the totals become meaningful from the second day of recording onward. A box starts recording at deployment and grows its history from there; the attack and incident streams do not backfill.

What is recorded

Attack events from the web, SSH and FTP ban ledgers — offending IP, hit count, timestamp — harvested from the archive ledgers rather than the live ban queues, so the harvester never races the ban-log reapers.
Auto-heal and incident events from eleven subsystem incident logs (high.load, oom, nginx, mysql, php, valkey, redis, unbound, java, system, batch_guard) plus i18n flood alerts, segfault alerts, SQL-watch kills and convert kills.
Backup outcomes, preserved past the upgrade-path log wipe.
Metric samples, roughly every 5 minutes: load averages, available memory, swap, disk use for / and /data, the current load tier, network and SYNPROXY counter deltas (reset-floored, so a csf reload records a 0 delta rather than a bogus spike), and csf deny/tempban counts.
Box facts, hourly: kernel, CPU count, RAM, swap, per-mount disk, uptime, and a service up/down verdict list matching boa info exactly.
The richest metric corpora, preserved verbatim: the fpm-tune, sqlprobe and load-profile per-day streams are copied into the archive, so their data outlives those corpora's own 30/30/14-day retention windows.

Where it lives, and for how long

The archive is JSONL under /var/log/boa/xdr9000/: per-day event and sample files filed as YYYY/MM/…, and a one-line-per-day rollup spine. The directory is mode 700, root-only. It sits in a subdirectory deliberately, so it survives the major-OS-upgrade log wipe and matches no logrotate glob.

The archive is permanent by construction: the engine only ever appends, past months compress with gzip -9, and the rollup is never pruned. Nothing is ever deleted, so budget the path into disk-space expectations on long-lived boxes.

Cost and control

The engine launches from the monitor fan-out, renices itself to background priority, reads at most 1 MiB per source per pass (carrying the remainder forward), holds a single-instance lock, throttles samples to ~5 minutes and box facts to ~1 hour, and performs the rollup plus month-end compression once per date change. Position is tracked by inode plus byte offset, so a rotated or truncated log resets cleanly and a half-written line waits for the next pass.

Set _XDR9000=NO in /root/.barracuda.cnf to stop recording on a box; everything already archived is kept, and xdr9000 status reports the engine OFF. For validation, bash /var/xdrago/monitor/check/xdr9000.sh --force runs all three passes immediately, bypassing the throttles.

Delivery scope

The recording engine reaches installed hosting boxes running the monitor stack. The xdr9000 CLI lands on every class of box at /usr/local/bin/xdr9000; on a box the engine has never run on, it exits with a clear message saying so.