The ban pipeline

The scan_nginx scoring engine decides who should be banned and writes two log files. This page documents what turns those files into an actual 444 on the next request. Three separately-scheduled BOA monitor scripts carry an offender from a log line, through the firewall, and back into the Nginx $is_banned geo:

scan_nginx.sh
   │  writes offenders
   ▼
web.log  +  scan_nginx.archive.log
   │                      │
   ▼                      ▼
guest-fire.sh        guest-water.sh
web.log → csf -td     archive ≥12 → csf.deny
900 (80/443)          "Brute force Web Server"
(temporary, 15 min)   (persistent; ≥24 = do-not-delete)
   │                      │
   └──────────┬───────────┘
              ▼
   /etc/csf  (csf.tempban 80/443  +  csf.deny)
              │
              ▼
   nginx_deny.sh  — regenerate the geo from current csf state
              │
              ▼
   /data/conf/nginx_banned_ips.conf
              │
              ▼
   geo $remote_addr $is_banned  (Nginx)
              │
              ▼
   return 444   (silent drop, zero backend cost)

The defining property of this stage is the same split that defines the whole Abuse Guard: detection is post-hoc, enforcement is real-time.

scan_nginx analyses log lines that already happened; the pipeline turns its verdict into a CSF ban, and the Nginx geo — a derived mirror of CSF, fully regenerated every pass — makes that ban bite on every subsequent request and self-expire the moment CSF drops it.

Stage 1 — temporary ban (guest-fire.sh)

guest-fire.sh is the fast path: a 15-minute temporary CSF ban for every fresh offender scan_nginx wrote to web.log. For the web tier it reads /var/xdrago/monitor/log/web.log, takes the IP field (cut -d '#' -f1 | sort | uniq), and for each unique IP:

1. Looks the IP up in CSF (csf -g) and checks csf.allow for an explicit tcp|in|d=80|s=<ip> allow entry.
2. If allowed (in csf.allow, or showing an ALLOW … ACCEPT … dpt:80 rule), it is cleared — csf -dr and csf -tr remove any stray block — and never banned. The allow list always wins.
3. If already denied on 80 or 443, nothing is done.

4. Otherwise it issues a 15-minute temporary ban on both web ports:

   BASHCopy

   csf -td ${_IP} 900 -p 80
   csf -td ${_IP} 900 -p 443
   

The same routine runs over three log files with identical allow-first / already-denied / temp-ban logic:

• Web — the web block above (web.log, ports 80/443).
• SSH twin — ssh.log, port 22.
• FTP twin — ftp.log, port 21.

Looping and the water interlock

The guard runs five times per invocation with a 10 s pause between passes, so an IP that appears mid-run is still caught within the same tick:

for _iteration in {1..5}; do
  [ ! -e "/run/water.pid" ] && _guest_guard
  sleep 10
done

The [ ! -e "/run/water.pid" ] test is the interlock with Stage 2: while guest-water.sh runs (it touches /run/water.pid on start and removes it on exit), guest-fire.sh skips its guard passes, so the two never fight over CSF at the same time.

Self-healing watchdog

Under a real flood web.log can hold thousands of IPs and a single run can stretch from its normal ~50 s (5 × 10 s) into minutes, blocking the following cron slots.

A watchdog records the PID in /run/fire.pid, and on the next invocation kills any predecessor still alive past _FIRE_TIMEOUT (180 s), logging to /var/log/boa/fire_stuck.log. A normal overlap (within the timeout) is left to the single-instance lock, which exits early if more than two copies run.

The full watchdog stack is on Operations + tuning.

Stage 2 — persistent escalation (guest-water.sh)

guest-fire.sh only ever applies temporary bans. Repeat offenders are escalated to a persistent CSF deny by guest-water.sh, which runs on its own (slower) schedule and works from the cumulative archive /var/xdrago/monitor/log/scan_nginx.archive.log (_WA) rather than the per-tick web.log. For each unique IP it counts how many times that IP appears across the whole archive:

_NR_TEST=$(tr -s ' ' '\n' < ${_WA} | grep -cF "${_IP}")

That count drives a two-tier escalation (after the csf.allow / /root/.local.IP.list exemptions):

The 12-hit deny lasts until the next CSF limits rotation (routine csf.deny trimming reclaims it). The 24-hit deny carries the literal do not delete tag, which CSF's routine trimming respects — so a heavy repeat offender stays banned across the housekeeping that would otherwise expire it.

The one thing that does clear it is the operator-triggered full cleanup: when /etc/boa/.full.csf.cleanup.cnf exists, guest-water.sh strips every do not delete line from csf.deny (sed -i "s/.*do not delete.*//g") — the explicit "wipe even the permanent bans" override.

guest-water.sh runs the same escalation over three archives in parallel structure:

For the whole run guest-water.sh holds the Stage 1 interlock — it touches /run/water.pid early (after its initial /root/.local.IP.list unblock pass, not first) and rms it at the very end — and once escalation is done it clears the per-tick web.log / ssh.log / ftp.log so the next scan_nginx window starts clean.

Under the hood. guest-water.sh also refreshes the csf.allow provider ranges (Cloudflare, Googlebot, Google's special-case crawlers — the AdsBot / Mediapartners / SERP-favicon-fetcher family, published separately from googlebot.json — Bingbot, Pingdom, and — behind /root/.extended.firewall.exceptions.cnf, which is file-only by design, with no cnf variable for it or for the Site24x7 marker it gates — Imperva, Sucuri, Auth0, Site24x7), with a diff-guard that reverts an unexpected csf.allow change and per-provider backups under /var/backups/csf/water/. The same pass mirrors the crawler ipv6Prefix ranges into the nginx-native IPv6 allow store /var/xdrago/monitor/log/web6.allow (CSF cannot hold them); there an empty provider fetch — endpoint down, format change — keeps the existing entries rather than stripping protection, and untagged manual operator lines are never touched. The two Google refreshes carry that same fail-safe on the csf.allow side: each fetches before it clears its own tagged lines and skips the clear entirely when the list comes back empty, while its static csf.deny healing runs either way. The other provider refreshes still clear their tagged lines first, so for them a failed fetch leaves the ranges absent until the next successful pass. That allow-list maintenance is what makes the keystone _is_whitelisted_ip guard reliable across the whole pipeline, for both address families.

/root/.local.IP.list is generated, not operator-maintained

The exemption list the pipeline keeps consulting is BOA's own, not a hand-written whitelist. When /root/.local.IP.list is missing, guest-water.sh deletes the whole /root/.local.IP.list* family and rebuilds the file from hostname -I; the master and satellite Nginx config passes carry the identical regeneration. When it is present, each pass appends any hostname -I address not already listed, so the box's own addresses are always in it.

Operator-added addresses are honoured — nothing removes individual lines — but they are only as durable as the file. Any pass that finds it missing wipes and regenerates the family, taking manual entries with it, so for a durable exemption prefer csf.allow.

Beyond the skip-and-unblock behaviour above, each pass also clears every listed address out of the CSF ban tables (csf -ar / -dr / -tr), and once per box — guarded by the /root/.local.IP.csf.listed done-flag — writes each one into csf.ignore and csf.allow. The same list is read by guest-fire.sh (_is_local_ip, never temp-banned), by scan_nginx (_ALLOWED_IPS, never scored) and by batch_guard as part of the box's own-IP set.

/root/.local.IP.list.allow is a second, generated file derived from it. A barracuda or octopus Nginx config pass rewrites it from scratch — one allow <ip>; line per entry, plus allow 127.0.0.1; and a closing deny all; — and splices it into the Ægir vhost templates in place of their deny all; line, so a multi-IP host can still reach its own cron-protected locations. Never hand-edit it: the next config pass rebuilds it, and the family wipe above deletes it whenever .local.IP.list goes missing.

Stage 3 — geo regeneration (nginx_deny.sh)

A CSF deny stops traffic at the origin firewall — but on a Cloudflare-proxied vhost the origin only ever sees the Cloudflare edge IP, so an iptables/CSF ban on the real client never bites.

nginx_deny.sh closes that gap: it mirrors the web bans into an Nginx realip-keyed geo deny set, so the banned real client is dropped with a 444 at Nginx even when it arrives via Cloudflare.

It rebuilds /data/conf/nginx_banned_ips.conf from current CSF state on every run, collecting web bans only from two sources:

# Active web temp bans: csf.tempban rows whose port field is 80 or 443
awk -F'|' '($3 == "80" || $3 == "443") { print $2 }' /var/lib/csf/csf.tempban
# Persistent web offenders: csf.deny lines tagged by guest-water
grep -F "Brute force Web Server" /etc/csf/csf.deny | awk '{ print $1 }'

This is deliberately web-scoped: it picks up the 80/443 temp bans from Stage 1 and the Brute force Web Server denies from Stage 2, and it excludes the SSH/FTP csf.deny entries and the broad CIDR blocklists — those work at the network layer (they are not Cloudflare-proxied) and have no place in the Nginx geo.

Each collected token is then validated as a bare IPv4 address or an IPv4 CIDR — every octet 0-255, the prefix 0-32:

_ipv4_octet="(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])"
[[ "$1" =~ ^(${_ipv4_octet}\.){3}${_ipv4_octet}(/(3[0-2]|[12]?[0-9]))?$ ]]

A loose pattern would let a malformed token (999.1.1.1, /99, IPv6, junk) reach the geo map and fail the Nginx configtest for the whole box. Only IPv4/CIDR tokens become IP 1; lines here — nginx_deny.sh stays IPv4-only because its source (csf) is IPv4-only; IPv6 offenders travel a separate path.

Stage 3b — IPv6 geo regeneration (nginx_deny6.sh)

csf cannot hold an IPv6 ban, so an IPv6 offender never reaches nginx_deny.sh. Instead scan_nginx's _block_ip writes it to an nginx-native store (/var/xdrago/monitor/log/web6.tempban, lines <ip6>|<expiry-epoch>), and nginx_deny6.sh (*/2, same shared lock) prunes the expired entries, collapses the refreshed duplicates to the max expiry, validates each as a strict IPv6 address, and emits the survivors as <ip6> 1; into /data/conf/nginx_banned_ips.conf6.

That file is pulled by the same geo $remote_addr $is_banned set (its wildcard nginx_banned_ips.c* include already covers it), so an IPv6 attacker gets the same 444. The store self-expires (default 900s, _NGINX_V6_BAN_TTL) — the nginx equivalent of csf's 15-minute web temp ban — and a re-offence refreshes it.

IPv6 can only arrive via the trusted realip proxy, so a banned v6 is always the real client. The whole IPv6 arm is gated by _NGINX_V6_BAN_DETECT (default YES).

Safe, idempotent reload

The rebuild only disturbs Nginx when something actually changed:

• Change-gate. The freshly-built file is compared to the live one with cmp -s; if identical, the run exits with no reload.
• Atomic install + validate. The new file is written to a leading-dot temp in the same directory (so the .c* include glob never sees it) and mv-d into place; the current file is backed up to .nginx_banned_ips.last_good.conf first.
• Revert on failure. After install it runs service nginx configtest and service nginx reload; on any failure it restores the last-good file (and reloads), so a bad ban set can never take Nginx down.
• Shared lock. The whole run holds the advisory lock /run/boa_nginx_config.lock (flock -w 30), shared with the other BOA Nginx-config writers (ip_access, user_admin_access, cloudflare_realip, ai_policy, nginx_deny6, migration_proxy_realip) so their configtest+reload cycles never overlap. If the lock can't be taken within 30 s the run skips and retries next tick.

The Provision templates own the consuming side — the geo $remote_addr $is_banned map, the if ($is_banned) return 444; guard, and the wildcard include of the generated file — documented in request guards.

Why the geo self-expires

The geo file holds no state of its own. Each nginx_deny.sh run rebuilds it from scratch out of the current CSF tables, so the lifecycle is entirely CSF's:

• A Stage 1 temp ban lives in csf.tempban for 15 minutes. While it is there it appears in the geo; once CSF drops the expired row, the next nginx_deny.sh pass simply doesn't emit it and the geo reload removes the 444.
• A Stage 2 persistent deny stays in the geo until CSF removes the csf.deny line — at limits rotation for a ≥12 ban, or never (until manual action) for a do not delete ≥24 ban.

So bans self-expire without any manual cleanup: the operator manages CSF, and the Nginx geo follows automatically. To lift a ban early, clear it in CSF (csf -dr / csf -tr) and the next geo regeneration drops it — see Operations + tuning.

Release valve: clearwebbans

Everything this page documents can be unwound in one command — clearwebbans:

1. removes the Stage 2 csf.deny Brute force Web Server escalations (csf -dr);
2. then the temporary 80/443 bans (csf -tr);
3. clears the per-tick web.log plus both archive generations (scan_nginx.archive.log and the guest-water-rotated scan_nginx.archive.x3.log — so guest-fire.sh cannot re-apply and guest-water.sh cannot re-escalate from history);
4. resets the /var/log/scan_nginx_lastpos byte offset to the current end of access.log;
5. and finally regenerates the geo via /var/xdrago/nginx_deny.sh.

It is strictly WEB-scoped: SSH/FTP bans and their archives are untouched. The run is idempotent, supports --dry-run, and holds a lock at /run/clearwebbans.lock. The full command reference — flags, output, when to run it — lives on Operations + tuning.

Allow list wins at every stage

None of the three stages trusts the previous stage's output blindly — each one re-checks the CSF allow list before it acts:

• guest-fire.sh clears (never bans) any IP in csf.allow / showing an ALLOW rule.
• guest-water.sh skips any IP listed in /root/.local.IP.list, never re-denies an IP already shown denied or allowed in the live csf -g output, and unblocks (csf -dr/csf -tr) only the IPs it finds in csf.allow.
• nginx_deny.sh only ever emits IPs that are currently banned in CSF, so an allow-listed IP that was never banned never reaches the geo.

This is the same keystone guarantee scan_nginx enforces with _is_whitelisted_ip: to exempt a service fleet-wide, put its IPs on the CSF allow list and every stage of the pipeline honours it. For IPv6 sources — which CSF cannot hold — the same guarantee rides the nginx-native allow store /var/xdrago/monitor/log/web6.allow, maintained by the same guest-water.sh pass and consulted by _is_whitelisted_ip for every IPv6 client. See scan_nginx scoring engine — Whitelisting and Operations + tuning.

Related

• scan_nginx scoring engine — the detection half that feeds web.log and scan_nginx.archive.log.
• Request guards — the $is_banned geo and the real-time 444 this pipeline ultimately drives.
• Configuration reference — the _NGINX_DOS_* knobs and the .instant.csf.block / .full.csf.cleanup markers.
• Operations + tuning — read live ban state, lift or cycle a ban, whitelist an IP.
• Security & isolation — CSF + LFD firewall — the firewall lifecycle that consumes this pipeline's output.