Compare commits
1 Commits
pia-gatewa
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 02804118d4 |
@ -189,76 +189,6 @@ real.
|
|||||||
(`forward_jump_first`) was working as intended when it refused to
|
(`forward_jump_first`) was working as intended when it refused to
|
||||||
register against the drifted state — this is a fix to the thing it
|
register against the drifted state — this is a fix to the thing it
|
||||||
checks, not a loosening of the check itself.
|
checks, not a loosening of the check itself.
|
||||||
|
|
||||||
**This recurs far more often than "boot-scale," and the fix above only
|
|
||||||
helped once.** The reposition logic is correct, but it only *ran* on
|
|
||||||
the existing triggers (boot, a UFW reload, an Ansible-detected content
|
|
||||||
change to the script). Confirmed live: after that fix landed and got
|
|
||||||
one `pia-gateway register` attempt past this check, the very next
|
|
||||||
attempt — roughly an hour later, no reboot, no UFW reload, no script
|
|
||||||
change in between — hit the identical failure again. kube-router
|
|
||||||
resyncs on ordinary Kubernetes API events (this cluster's CronJobs —
|
|
||||||
`ca-sync`, `porkbun-ddns`, `gitea-backup` — each create and delete a
|
|
||||||
Pod every few minutes, each a plausible trigger), not just at
|
|
||||||
boot/restart scale as originally assumed. `pia-killswitch-reassert
|
|
||||||
.timer` (every `pia_healthcheck_interval_sec`, unconditional, not
|
|
||||||
gated by `pia_auto_recovery_enabled`) exists specifically to close this
|
|
||||||
gap by giving the already-correct, already-idempotent script regular
|
|
||||||
chances to run instead of only sporadic ones.
|
|
||||||
|
|
||||||
**The reassert timer alone wasn't the full fix either — it created a
|
|
||||||
new failure mode.** Confirmed live, 2026-09-22, minutes after enabling
|
|
||||||
`pia-gateway-recover.timer` for the first time (rollout step 3):
|
|
||||||
`pia-gateway-recover` and `pia-killswitch-reassert` are two
|
|
||||||
independently-scheduled 60s timers with no coordination between them.
|
|
||||||
kube-router turned out to resync (and re-push our jump down) roughly
|
|
||||||
every 60-90s on this cluster — far more often than "rare, boot-scale"
|
|
||||||
— so a `recover` pass would periodically sample `iptables -S FORWARD`
|
|
||||||
in the window between a kube-router push-down and the next reassert
|
|
||||||
fix, correctly detect real (if momentary) drift, and call
|
|
||||||
`emergency_close()` — which unconditionally forces table `pia`'s route
|
|
||||||
to `unreachable` and resets `consecutive_failures` to 0 in the same
|
|
||||||
step. The result was a self-reinforcing outage loop, observed directly
|
|
||||||
in the journal:
|
|
||||||
|
|
||||||
```
|
|
||||||
19:38:58 3rd consecutive unhealthy check → recovery_cycle → restart
|
|
||||||
19:39:09 recovered via restart (10s) — route open, handshake fresh
|
|
||||||
19:40:03 drift detected again ("FORWARD jump ... not first")
|
|
||||||
→ emergency close → route forced closed, counter reset to 0
|
|
||||||
19:41:08 drift again → emergency close again → route stays closed
|
|
||||||
```
|
|
||||||
|
|
||||||
VLAN 50 was open for roughly 10 seconds out of every 60-90 second
|
|
||||||
cycle. Three consecutive real failures were needed to trigger the
|
|
||||||
restart that reopens the route, but a fresh drift-and-emergency-close
|
|
||||||
kept arriving before three could accumulate — the counter reset every
|
|
||||||
time. This is not a bug in the detection (the drift was real, if
|
|
||||||
transient) — it's that treating *every* momentary reordering as a full
|
|
||||||
safety emergency was stricter than the actual invariant required,
|
|
||||||
given kube-router's own resync cadence on this specific host.
|
|
||||||
|
|
||||||
**Fix: `pia_forward_bypass_chains` (`Classifier
|
|
||||||
._forward_jump_effectively_first` in `files/pia_gateway.py`).** The
|
|
||||||
safety check now tolerates a configured, named FORWARD chain sitting
|
|
||||||
ahead of our jump — but only for as long as that chain's *own current
|
|
||||||
content* re-verifies, live, on every single check, as structurally
|
|
||||||
unable to match `10.10.50.0/24` (every `-s`/`-d` value on every rule
|
|
||||||
disjoint from our subnet; a bare/unconditional rule, a negated match,
|
|
||||||
or any construct the checker doesn't explicitly recognize fails
|
|
||||||
closed, not open). This is not a one-time trust of kube-router — if
|
|
||||||
its rules ever changed to include something that could actually
|
|
||||||
intercept VLAN 50 traffic, or an unrecognized chain/rule appeared
|
|
||||||
ahead of ours instead, it still fails closed exactly as before. Scoped
|
|
||||||
to `KUBE-ROUTER-FORWARD` only (not `DOCKER-USER`/`DOCKER-FORWARD` —
|
|
||||||
see `pia_forward_bypass_chains`'s own comment in `defaults/main.yaml`
|
|
||||||
for why only kube-router earned the exception). An emergency close and
|
|
||||||
a hard stop remain the outcome for anything not on this narrow,
|
|
||||||
content-reverified allowlist. Immediate live mitigation while this fix
|
|
||||||
was being built: `sudo systemctl stop pia-gateway-recover.timer`, then
|
|
||||||
a manual `sudo systemctl restart wg-quick@pia-wg` to reopen the route
|
|
||||||
(`pia-killswitch-reassert.timer` was left running throughout — it was
|
|
||||||
never the problem).
|
|
||||||
- `pia-wg.conf`'s own `PostUp`/`PreDown` open/close the narrower path on
|
- `pia-wg.conf`'s own `PostUp`/`PreDown` open/close the narrower path on
|
||||||
top of that closed baseline: default route via `pia-wg` in the `pia`
|
top of that closed baseline: default route via `pia-wg` in the `pia`
|
||||||
table, ACCEPT + established/related return inserted into the
|
table, ACCEPT + established/related return inserted into the
|
||||||
@ -338,14 +268,10 @@ handled automatically — see the next section.
|
|||||||
## Automatic recovery
|
## Automatic recovery
|
||||||
|
|
||||||
`files/pia_gateway.py` is installed as `/usr/local/sbin/pia-gateway`
|
`files/pia_gateway.py` is installed as `/usr/local/sbin/pia-gateway`
|
||||||
(root, `0700`, runs on the host's `python3.13`). Almost entirely standard
|
(root, `0700`, standard library only — runs on the host's `python3.13`).
|
||||||
library — the one exception is `addKey` specifically, which shells out to
|
It is **one** implementation of PIA registration shared by the unattended
|
||||||
`curl` (now an explicit `apt` dependency of this role); see
|
timers and by this role's own registration step, so the two can never
|
||||||
`HttpClient.get_pinned_curl`'s own docstring and "PIA's CA certificate and
|
drift. Layout on the host:
|
||||||
why addKey uses curl, not Python's `ssl`" below for why. It is **one**
|
|
||||||
implementation of PIA registration shared by the unattended timers and by
|
|
||||||
this role's own registration step, so the two can never drift. Layout on
|
|
||||||
the host:
|
|
||||||
|
|
||||||
| Path | Purpose |
|
| Path | Purpose |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
@ -358,65 +284,6 @@ the host:
|
|||||||
| `pia-gateway-renew.timer` → `.service` | daily 03:30 ±30 min, proactive same-key renewal (`renew`); **disabled by default** (`pia_renewal_enabled`) |
|
| `pia-gateway-renew.timer` → `.service` | daily 03:30 ±30 min, proactive same-key renewal (`renew`); **disabled by default** (`pia_renewal_enabled`) |
|
||||||
| `pia-gateway-onfailure.service` | `OnFailure=` of the two above: posts an error alert if a unit crashes or refuses to run |
|
| `pia-gateway-onfailure.service` | `OnFailure=` of the two above: posts an error alert if a unit crashes or refuses to run |
|
||||||
| `pia-gateway-healthcheck.timer` | unchanged cadence, now runs `pia-gateway check` |
|
| `pia-gateway-healthcheck.timer` | unchanged cadence, now runs `pia-gateway check` |
|
||||||
| `pia-killswitch-reassert.timer` → `.service` | every `pia_healthcheck_interval_sec`, re-runs `pia-killswitch.sh` (idempotent). **Not** gated by `pia_auto_recovery_enabled` — see the "FORWARD jump's position" note above and the service unit's own header for why |
|
|
||||||
|
|
||||||
### PIA's CA certificate and why addKey uses curl, not Python's `ssl`
|
|
||||||
|
|
||||||
Found live, 2026-09-22, on the very first `pia-gateway register` attempt
|
|
||||||
after the FORWARD-jump fix above got past the kill-switch check:
|
|
||||||
`addKey` failed against every candidate with `kind=tls detail=certificate
|
|
||||||
verification failed`. Root-caused directly on minisforum:
|
|
||||||
|
|
||||||
```
|
|
||||||
$ sudo python3 -c '...ctx.wrap_socket(sock, server_hostname="Server-11669-10a")...'
|
|
||||||
SSLCertVerificationError verify_message= Basic Constraints of CA cert not marked critical
|
|
||||||
```
|
|
||||||
|
|
||||||
PIA's own CA certificate (`files/pia-ca.crt`) has a `CA:TRUE` Basic
|
|
||||||
Constraints extension that isn't flagged `critical` — a real RFC 5280
|
|
||||||
non-conformance, confirmed **not** a stale vendored copy: a fresh fetch
|
|
||||||
from `pia-foss/manual-connections` on GitHub is byte-identical (same
|
|
||||||
`32e9b1d1...` SHA-256 already recorded in the file's own header) and has
|
|
||||||
the identical defect. This is PIA's problem, not ours, and not fixable by
|
|
||||||
re-vendoring. Recent OpenSSL (3.2+; confirmed live against minisforum's
|
|
||||||
`3.5.7`) rejects this by default — a check that used to be opt-in only
|
|
||||||
under `X509_V_FLAG_X509_STRICT` and has since been promoted into the
|
|
||||||
default chain-verification path. `openssl s_client` on this repo's own
|
|
||||||
dev machine (LibreSSL) and this same host's `curl` both accept the exact
|
|
||||||
same certificate/chain/CA file without complaint — only Python's `ssl`
|
|
||||||
module, using this host's system OpenSSL, rejects it. That's also why
|
|
||||||
this went unnoticed until the auto-recovery work: the legacy Ansible
|
|
||||||
path's three successful registrations (2026-08-24, 08-25, and this
|
|
||||||
morning) all went through `curl --cacert` (`addkey-attempt.yaml`), never
|
|
||||||
through Python's `ssl` module — this was the *first* time that code path
|
|
||||||
ever ran for real.
|
|
||||||
|
|
||||||
Python's stdlib `ssl` module has no custom-verify-callback hook (a
|
|
||||||
long-standing, deliberate upstream gap — unlike Go's
|
|
||||||
`tls.Config.VerifyPeerCertificate` or OpenSSL's own C API), so there is
|
|
||||||
no clean way to tolerate just this one known, third-party-controlled
|
|
||||||
defect while still fully validating everything else (chain, expiry,
|
|
||||||
hostname/SAN). Rather than weakening `verify_mode` for the whole
|
|
||||||
connection or adding a new dependency (`pyOpenSSL`, for a real verify
|
|
||||||
callback — considered, rejected as a bigger lift than necessary) to work
|
|
||||||
around a gap in a library we don't control, `addKey` specifically now
|
|
||||||
shells out to `curl` (`HttpClient.get_pinned_curl`, `curl` added to this
|
|
||||||
role's `apt` install list) — the same tool PIA's own reference client
|
|
||||||
(`connect_to_wireguard_with_token.sh`) already uses for this exact
|
|
||||||
endpoint, with `--cacert` and never `-k`/`--insecure`, so this is a
|
|
||||||
different, already-proven-correct TLS backend for the one endpoint that
|
|
||||||
needs it, not a security downgrade. `serverlist` and the token endpoint
|
|
||||||
are unaffected (they present public WebPKI certs, which don't have this
|
|
||||||
problem) and still use the stdlib `ssl` path. Verified: a real subprocess
|
|
||||||
`curl` call from this repo's dev machine against the actual failing
|
|
||||||
candidate (`Server-11669-10a` / `69.33.198.97`) gets a genuine HTTP
|
|
||||||
response from PIA (401, expected — a dummy token was used), proving TLS
|
|
||||||
validation succeeds via this path where it failed via Python's `ssl`.
|
|
||||||
curl's own exit codes for a `--fail`-triggered rejection aren't fully
|
|
||||||
portable either — confirmed live: a curl built against SecureTransport
|
|
||||||
(this dev machine) reports exit 56, not the "usual" 22 GNU/OpenSSL curl
|
|
||||||
uses for the identical case — so classification checks curl's own error
|
|
||||||
text as a second signal alongside the exit code, not the exit code alone.
|
|
||||||
|
|
||||||
### Status classes
|
### Status classes
|
||||||
|
|
||||||
@ -599,83 +466,18 @@ external secret broker — not a 24-hour token.
|
|||||||
- `unknown` (a read command failing) is logged/alerted but never acted
|
- `unknown` (a read command failing) is logged/alerted but never acted
|
||||||
on, by design.
|
on, by design.
|
||||||
|
|
||||||
### Rollout status (2026-09-23)
|
### Rollout status (2026-09-22)
|
||||||
|
|
||||||
Steps 1-3 are done and verified live. Step 3 required two attempts —
|
Step 1 (baseline) and the first attempt at step 2 have been run live.
|
||||||
the first caused a real (safely fail-closed) outage, root-caused and
|
Step 2 failed on the first try: `pia-gateway register` correctly refused
|
||||||
fixed same-day; the second was verified against a full day of real
|
with `safety invariants failing ... FORWARD jump to kill-switch chain is
|
||||||
production conditions, not just a short window.
|
not first` — a real, pre-existing drift (see the "FORWARD jump's
|
||||||
|
position is actively re-asserted" note above), not a false alarm from
|
||||||
- **Step 2, attempt 1** failed: `pia-gateway register` correctly refused
|
this session's work, and not the "session dies" handshake issue either.
|
||||||
with `safety invariants failing ... FORWARD jump to kill-switch chain
|
Root-caused and fixed in `pia-killswitch.sh.j2`; not yet re-applied or
|
||||||
is not first` — a real, pre-existing drift (see the "FORWARD jump's
|
re-verified live. **Use `iptables -S FORWARD` (no `head`) for the
|
||||||
position is actively re-asserted" note above), not a false alarm from
|
baseline capture below** — the original `head -3` suggestion is what
|
||||||
this session's work, and not the "session dies" handshake issue
|
hid this the first time; the drift was 4 rules in, not at the top.
|
||||||
either. Root-caused and fixed in `pia-killswitch.sh.j2`.
|
|
||||||
- **Attempt 2**, after that fix, got past the kill-switch check and
|
|
||||||
failed differently: `addKey` rejected every candidate with
|
|
||||||
`kind=tls detail=certificate verification failed`. Root-caused and
|
|
||||||
fixed — see "PIA's CA certificate and why addKey uses curl, not
|
|
||||||
Python's `ssl`" above.
|
|
||||||
- **Attempt 3**, roughly an hour after attempt 1's fix with no reboot,
|
|
||||||
UFW reload, or script change in between, hit the *identical* FORWARD-
|
|
||||||
jump failure again — proving the drift recurs on a timescale of
|
|
||||||
minutes under this cluster's normal CronJob/pod churn, not just at
|
|
||||||
boot-scale events as originally assumed. Unblocked live with a manual
|
|
||||||
`sudo systemctl restart pia-killswitch.service`; `pia-killswitch
|
|
||||||
-reassert.timer` added as the durable fix.
|
|
||||||
- **Attempt 4** succeeded cleanly end to end: kill-switch check passed,
|
|
||||||
`addKey` accepted (`hk`/`Server-11669-10a`), and a **fresh handshake
|
|
||||||
was verified** (age 10s) — the first-ever successful shared-path
|
|
||||||
registration. `pia-gateway status` confirmed `healthy`/`safety_ok:
|
|
||||||
True`, `iptables -S FORWARD` confirmed the jump first, and
|
|
||||||
`pia-killswitch-reassert.timer` confirmed cycling correctly (7
|
|
||||||
successful runs, ~60s apart, in the journal). **Step 2 is done.**
|
|
||||||
- **Step 3** (`ansible-playbook ... -K -J`, no override — enables
|
|
||||||
`pia-gateway-recover.timer` for the first time) deployed cleanly
|
|
||||||
(`Run the shared PIA registration transaction` correctly skipped, a
|
|
||||||
verified baseline already existed) but caused a **real live outage**
|
|
||||||
within minutes: `pia-gateway-recover` and `pia-killswitch-reassert`,
|
|
||||||
two independent 60s timers, raced — kube-router's actual resync
|
|
||||||
cadence (~60-90s) turned out to be far more frequent than assumed, and
|
|
||||||
every momentary drift-then-self-correct was treated as a full safety
|
|
||||||
emergency, forcibly closing the route and resetting the 3-strikes
|
|
||||||
counter before it could accumulate. VLAN 50 was open for ~10s out of
|
|
||||||
every 60-90s cycle for several minutes, confirmed via Discord alerts
|
|
||||||
from `alert-bridge` (working end to end, first real trigger) and the
|
|
||||||
full `pia-gateway-recover`/`pia-killswitch-reassert` journals.
|
|
||||||
Mitigated live: `sudo systemctl stop pia-gateway-recover.timer` +
|
|
||||||
manual `sudo systemctl restart wg-quick@pia-wg`. Root-caused and fixed
|
|
||||||
same-day — see the "reassert timer alone wasn't the full fix" addendum
|
|
||||||
above (`pia_forward_bypass_chains`, content-verified live on every
|
|
||||||
check).
|
|
||||||
- **Step 3, re-attempt**, with the bypass fix deployed (`ansible-playbook
|
|
||||||
... -K -J`; config re-rendered with `forward_bypass_chains:
|
|
||||||
["KUBE-ROUTER-FORWARD"]`, `rewrite-config` correctly re-rendered
|
|
||||||
`pia-wg.conf` from the config-only change without a PIA call since a
|
|
||||||
baseline already existed, recovery timer re-enabled). Left running
|
|
||||||
unattended for a full **~25.5 hours** before re-checking, deliberately
|
|
||||||
— the original incident took several minutes and multiple cycles to
|
|
||||||
manifest, so a short recheck window would have proven nothing.
|
|
||||||
Confirmed via exact counts, not inference: `journalctl -u
|
|
||||||
pia-gateway-recover --since "2026-09-22 20:37:00"` showed **1528**
|
|
||||||
`class=healthy` passes (matches the expected ~1530 at 60s cadence —
|
|
||||||
the timer ran continuously the whole window), **0** `SAFETY DRIFT`
|
|
||||||
lines, **0** `class=safety_drift` classifications. `pia-gateway
|
|
||||||
status` at recheck time: `healthy`, `safety_ok: True`,
|
|
||||||
`emergency_close_active: False`, `consecutive_failures: 0`,
|
|
||||||
`last recovery: 1d ago restart` — the last restart was the one from
|
|
||||||
*before* this fix; none has been needed since. **Step 3 is done.**
|
|
||||||
kube-router did keep resyncing during this window (that's the whole
|
|
||||||
premise of the fix); the bypass tolerance handled it silently every
|
|
||||||
time, with no observable effect on the tunnel.
|
|
||||||
|
|
||||||
**Use `iptables -S FORWARD` (no `head`) for the baseline capture below**
|
|
||||||
— the original `head -3` suggestion is what hid the first issue; the
|
|
||||||
drift was 4 rules in, not at the top. **Use `head -2`, not `head -1`**,
|
|
||||||
for any quick re-check — `-S FORWARD`'s first line is always the `-P`
|
|
||||||
policy line, never a real rule (this cost a round-trip during the
|
|
||||||
investigation above).
|
|
||||||
|
|
||||||
1. Baseline on minisforum: `wg show pia-wg`, `ip rule`, `ip route show
|
1. Baseline on minisforum: `wg show pia-wg`, `ip rule`, `ip route show
|
||||||
table pia`, `iptables -S FORWARD`, `iptables -S PIA-VLAN50`,
|
table pia`, `iptables -S FORWARD`, `iptables -S PIA-VLAN50`,
|
||||||
@ -697,20 +499,8 @@ investigation above).
|
|||||||
should then get past registration. Confirm with `iptables -S FORWARD`
|
should then get past registration. Confirm with `iptables -S FORWARD`
|
||||||
afterwards that the jump is genuinely first, not just that the retry
|
afterwards that the jump is genuinely first, not just that the retry
|
||||||
succeeded.
|
succeeded.
|
||||||
3. Enable: `ansible-playbook ... -K -J` (recovery on; also redeploys
|
3. Enable: `ansible-playbook ... -K -J` (recovery on). Confirm one healthy
|
||||||
`pia_forward_bypass_chains` into `config.json` if not already applied
|
pass makes no changes (`journalctl -u pia-gateway-recover -n 5`).
|
||||||
— confirm with `sudo pia-gateway --config /etc/pia-gateway/config.json
|
|
||||||
status` or just `cat /etc/pia-gateway/config.json`). **Watch longer
|
|
||||||
than one pass** — the live incident here took 2-3 cycles (several
|
|
||||||
minutes) to manifest, not the first check. `journalctl -u
|
|
||||||
pia-gateway-recover -f` for at least 5-10 minutes; expect `class=
|
|
||||||
healthy` throughout, with **zero** `SAFETY DRIFT` lines, even while
|
|
||||||
`pia-killswitch-reassert` keeps cycling in parallel
|
|
||||||
(`journalctl -u pia-killswitch-reassert -f` alongside it). If a
|
|
||||||
`bypass chain KUBE-ROUTER-FORWARD content no longer verified safe`
|
|
||||||
reason ever appears, treat it as real — it means kube-router's rules
|
|
||||||
changed to include something that could actually match VLAN 50
|
|
||||||
traffic, not a false positive to relax further.
|
|
||||||
4. Drill the silent death. First get the **live WireGuard UDP endpoint**
|
4. Drill the silent death. First get the **live WireGuard UDP endpoint**
|
||||||
from the interface itself — **not** port 1337, which is only the
|
from the interface itself — **not** port 1337, which is only the
|
||||||
`addKey` registration API port and has nothing to do with the tunnel's
|
`addKey` registration API port and has nothing to do with the tunnel's
|
||||||
@ -785,10 +575,8 @@ touching `wg0`, k3s, or the host default route):
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
sudo systemctl disable --now pia-gateway-recover.timer pia-gateway-renew.timer # controller first, so it cannot react to the teardown
|
sudo systemctl disable --now pia-gateway-recover.timer pia-gateway-renew.timer # controller first, so it cannot react to the teardown
|
||||||
sudo systemctl disable --now wg-quick@pia-wg pia-killswitch.service pia-killswitch-reassert.timer pia-gateway-healthcheck.timer
|
sudo systemctl disable --now wg-quick@pia-wg pia-killswitch.service pia-gateway-healthcheck.timer
|
||||||
sudo rm -f /etc/systemd/system/pia-killswitch.service \
|
sudo rm -f /etc/systemd/system/pia-killswitch.service \
|
||||||
/etc/systemd/system/pia-killswitch-reassert.service \
|
|
||||||
/etc/systemd/system/pia-killswitch-reassert.timer \
|
|
||||||
/etc/systemd/system/pia-gateway-healthcheck.service \
|
/etc/systemd/system/pia-gateway-healthcheck.service \
|
||||||
/etc/systemd/system/pia-gateway-healthcheck.timer \
|
/etc/systemd/system/pia-gateway-healthcheck.timer \
|
||||||
/etc/systemd/system/pia-gateway-recover.service \
|
/etc/systemd/system/pia-gateway-recover.service \
|
||||||
|
|||||||
@ -94,35 +94,6 @@ pia_iptables_chain: PIA-VLAN50
|
|||||||
# {{ pia_wg_interface }} rather than enumerating the normal interface.
|
# {{ pia_wg_interface }} rather than enumerating the normal interface.
|
||||||
pia_lan_interface: enp1s0
|
pia_lan_interface: enp1s0
|
||||||
|
|
||||||
# FORWARD chains the controller may tolerate sitting AHEAD of our own
|
|
||||||
# kill-switch jump, on the condition that their OWN current content is
|
|
||||||
# re-verified — live, on every check, never a one-time trust — to be
|
|
||||||
# structurally unable to match {{ pia_vlan50_subnet }} traffic. Empty (the
|
|
||||||
# safe default for any other host) means the literal, original "must be
|
|
||||||
# absolutely first" check. Set to ["KUBE-ROUTER-FORWARD"] here because
|
|
||||||
# k3s's kube-router network-policy controller and dockerd both also
|
|
||||||
# insert unconditional jumps near the top of FORWARD, independently of
|
|
||||||
# this role, and — confirmed live, 2026-09-22 — kube-router resyncs (and
|
|
||||||
# re-asserts its own top-of-chain position) roughly every 60-90s on this
|
|
||||||
# cluster's normal CronJob/pod churn, not just at boot/restart scale as
|
|
||||||
# first assumed. Treating every one of those transient reorderings as a
|
|
||||||
# full safety-drift emergency (which unconditionally closes table pia's
|
|
||||||
# route) caused a real live outage: a repeating cycle of recover-succeeds
|
|
||||||
# -> drift-detected-within-a-minute -> emergency-close -> counter reset
|
|
||||||
# -> repeat, which kept VLAN 50 open for only seconds at a time. Only
|
|
||||||
# kube-router is listed — NOT dockerd's DOCKER-USER/DOCKER-FORWARD —
|
|
||||||
# because kube-router was the only one observed reasserting position
|
|
||||||
# repeatedly (dockerd sets its hooks up once at daemon start, not on a
|
|
||||||
# recurring cycle), and DOCKER-FORWARD itself jumps to three further
|
|
||||||
# nested chains (DOCKER-CT, DOCKER-INTERNAL, DOCKER-BRIDGE) never fully
|
|
||||||
# content-verified — safer to leave those as a hard drift trigger than
|
|
||||||
# assume they're equally inert. See
|
|
||||||
# Classifier._forward_jump_effectively_first's own docstring in
|
|
||||||
# files/pia_gateway.py for exactly what "re-verified safe" requires of a
|
|
||||||
# bypassed chain's content.
|
|
||||||
pia_forward_bypass_chains:
|
|
||||||
- KUBE-ROUTER-FORWARD
|
|
||||||
|
|
||||||
# Reverse-path filtering, applied to {{ pia_wg_interface }} only. The
|
# Reverse-path filtering, applied to {{ pia_wg_interface }} only. The
|
||||||
# kernel default on minisforum (conf.default.rp_filter) is already loose
|
# kernel default on minisforum (conf.default.rp_filter) is already loose
|
||||||
# (2) at time of writing, so new interfaces inherit loose mode — this task
|
# (2) at time of writing, so new interfaces inherit loose mode — this task
|
||||||
|
|||||||
@ -35,7 +35,6 @@ import datetime as _dt
|
|||||||
import fcntl
|
import fcntl
|
||||||
import http.client
|
import http.client
|
||||||
import io
|
import io
|
||||||
import ipaddress
|
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import random
|
import random
|
||||||
@ -120,7 +119,6 @@ class Config:
|
|||||||
vlan50_subnet: str = "10.10.50.0/24"
|
vlan50_subnet: str = "10.10.50.0/24"
|
||||||
iptables_chain: str = "PIA-VLAN50"
|
iptables_chain: str = "PIA-VLAN50"
|
||||||
lan_interface: str = "enp1s0"
|
lan_interface: str = "enp1s0"
|
||||||
forward_bypass_chains: list = dataclasses.field(default_factory=list)
|
|
||||||
mss_clamp_enabled: bool = True
|
mss_clamp_enabled: bool = True
|
||||||
region_candidates: list = dataclasses.field(default_factory=lambda: ["hk", "taiwan", "japan"])
|
region_candidates: list = dataclasses.field(default_factory=lambda: ["hk", "taiwan", "japan"])
|
||||||
handshake_max_age_sec: int = 180
|
handshake_max_age_sec: int = 180
|
||||||
@ -511,59 +509,6 @@ def _is_emergency_rule(rule: str) -> bool:
|
|||||||
return f"--comment {EMERGENCY_COMMENT}" in rule or f'--comment "{EMERGENCY_COMMENT}"' in rule
|
return f"--comment {EMERGENCY_COMMENT}" in rule or f'--comment "{EMERGENCY_COMMENT}"' in rule
|
||||||
|
|
||||||
|
|
||||||
_COMMENT_ONLY_RE = re.compile(r'-m comment --comment "[^"]*"')
|
|
||||||
|
|
||||||
|
|
||||||
def _forward_bypass_target(rule: str) -> Optional[str]:
|
|
||||||
"""If `rule` is an UNCONDITIONAL FORWARD-level jump to another chain
|
|
||||||
(optionally with only a cosmetic -m comment match, nothing that
|
|
||||||
actually narrows which packets it applies to), return that chain's
|
|
||||||
name; otherwise None. A rule with any real matcher (-s/-d/-p/-i/-o/
|
|
||||||
any -m other than comment) is deliberately never treated as a bypass
|
|
||||||
candidate here, however harmless it might look — that judgment is
|
|
||||||
made once, by a human, for the caller's own forward_bypass_chains
|
|
||||||
list, not inferred from the FORWARD-level rule shape.
|
|
||||||
"""
|
|
||||||
stripped = _COMMENT_ONLY_RE.sub("", rule).split()
|
|
||||||
if len(stripped) == 4 and stripped[0] == "-A" and stripped[1] == "FORWARD" and stripped[2] == "-j":
|
|
||||||
return stripped[3]
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _cidr_overlaps(value: str, subnet: str) -> bool:
|
|
||||||
"""True if `value` could overlap `subnet`; conservative (True) on
|
|
||||||
anything unparseable rather than silently treating it as disjoint."""
|
|
||||||
try:
|
|
||||||
return ipaddress.ip_network(value, strict=False).overlaps(ipaddress.ip_network(subnet))
|
|
||||||
except ValueError:
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
def _bypass_chain_rule_is_safe(rule: str, vlan50_subnet: str) -> bool:
|
|
||||||
"""Conservative check for one rule inside a configured bypass chain:
|
|
||||||
True only if this rule is PROVABLY unable to match traffic sourced
|
|
||||||
from vlan50_subnet — every -s/-d value on it is a CIDR disjoint from
|
|
||||||
vlan50_subnet, there is at least one such value (a bare/unconditional
|
|
||||||
rule is never safe), and there is no negation or other construct this
|
|
||||||
function doesn't explicitly recognize (fails closed, not open, on
|
|
||||||
anything it can't prove).
|
|
||||||
"""
|
|
||||||
if "!" in rule.split():
|
|
||||||
return False
|
|
||||||
tokens = rule.split()
|
|
||||||
addrs: list[str] = []
|
|
||||||
i = 0
|
|
||||||
while i < len(tokens):
|
|
||||||
if tokens[i] in ("-s", "-d") and i + 1 < len(tokens):
|
|
||||||
addrs.append(tokens[i + 1])
|
|
||||||
i += 2
|
|
||||||
continue
|
|
||||||
i += 1
|
|
||||||
if not addrs:
|
|
||||||
return False
|
|
||||||
return all(not _cidr_overlaps(a, vlan50_subnet) for a in addrs)
|
|
||||||
|
|
||||||
|
|
||||||
class Classifier:
|
class Classifier:
|
||||||
def __init__(self, cfg: Config, system: System):
|
def __init__(self, cfg: Config, system: System):
|
||||||
self.cfg = cfg
|
self.cfg = cfg
|
||||||
@ -572,34 +517,6 @@ class Classifier:
|
|||||||
def _rc(self, argv: list[str]) -> CmdResult:
|
def _rc(self, argv: list[str]) -> CmdResult:
|
||||||
return self.sys.run(argv, timeout=20)
|
return self.sys.run(argv, timeout=20)
|
||||||
|
|
||||||
def _forward_jump_effectively_first(self, effective_rules: list[str]) -> tuple[bool, list[str]]:
|
|
||||||
"""Walk FORWARD from the top. Tolerate a configured bypass chain
|
|
||||||
(cfg.forward_bypass_chains) sitting ahead of our own jump ONLY
|
|
||||||
while that chain's OWN current content is re-verified, live, on
|
|
||||||
THIS call to be unable to match VLAN 50 traffic — never a one-time
|
|
||||||
trust. Anything else ahead of our jump (an unrecognized rule, or a
|
|
||||||
bypass-listed chain whose content no longer verifies safe) fails
|
|
||||||
closed. Returns (ok, extra_reasons).
|
|
||||||
"""
|
|
||||||
cfg = self.cfg
|
|
||||||
expected_jump = f"-A FORWARD -s {cfg.vlan50_subnet} -j {cfg.iptables_chain}"
|
|
||||||
extra: list[str] = []
|
|
||||||
for r in effective_rules:
|
|
||||||
if r.strip() == expected_jump:
|
|
||||||
return True, extra
|
|
||||||
target = _forward_bypass_target(r)
|
|
||||||
if target is None or target not in cfg.forward_bypass_chains:
|
|
||||||
return False, extra
|
|
||||||
chain = self._rc(["iptables", "-S", target])
|
|
||||||
if chain.rc != 0:
|
|
||||||
extra.append(f"bypass chain {target} unreadable")
|
|
||||||
return False, extra
|
|
||||||
chain_rules = [l for l in chain.out.splitlines() if l.startswith(f"-A {target}")]
|
|
||||||
if not chain_rules or not all(_bypass_chain_rule_is_safe(cr, cfg.vlan50_subnet) for cr in chain_rules):
|
|
||||||
extra.append(f"bypass chain {target} content no longer verified safe")
|
|
||||||
return False, extra
|
|
||||||
return False, extra
|
|
||||||
|
|
||||||
def classify(self, state: dict) -> Status:
|
def classify(self, state: dict) -> Status:
|
||||||
cfg = self.cfg
|
cfg = self.cfg
|
||||||
now = self.sys.now()
|
now = self.sys.now()
|
||||||
@ -719,7 +636,6 @@ class Classifier:
|
|||||||
fwd = self._rc(["iptables", "-S", "FORWARD"])
|
fwd = self._rc(["iptables", "-S", "FORWARD"])
|
||||||
jump_first = False
|
jump_first = False
|
||||||
emergency_present = False
|
emergency_present = False
|
||||||
jump_bypass_reasons: list[str] = []
|
|
||||||
if fwd.rc == 0:
|
if fwd.rc == 0:
|
||||||
rules = [l for l in fwd.out.splitlines() if l.startswith("-A FORWARD")]
|
rules = [l for l in fwd.out.splitlines() if l.startswith("-A FORWARD")]
|
||||||
effective = []
|
effective = []
|
||||||
@ -728,15 +644,15 @@ class Classifier:
|
|||||||
emergency_present = True
|
emergency_present = True
|
||||||
continue
|
continue
|
||||||
effective.append(r)
|
effective.append(r)
|
||||||
jump_first, jump_bypass_reasons = self._forward_jump_effectively_first(effective)
|
expected_jump = f"-A FORWARD -s {cfg.vlan50_subnet} -j {cfg.iptables_chain}"
|
||||||
|
jump_first = bool(effective) and effective[0].strip() == expected_jump
|
||||||
else:
|
else:
|
||||||
unknown = True
|
unknown = True
|
||||||
checks["forward_jump_first"] = jump_first
|
checks["forward_jump_first"] = jump_first
|
||||||
checks["emergency_rule_present"] = emergency_present
|
checks["emergency_rule_present"] = emergency_present
|
||||||
if not jump_first:
|
if not jump_first:
|
||||||
safety_ok = False
|
safety_ok = False
|
||||||
detail = f" ({'; '.join(jump_bypass_reasons)})" if jump_bypass_reasons else ""
|
reasons.append("FORWARD jump to kill-switch chain is not first")
|
||||||
reasons.append(f"FORWARD jump to kill-switch chain is not first{detail}")
|
|
||||||
|
|
||||||
# -- chain contents
|
# -- chain contents
|
||||||
chain = self._rc(["iptables", "-S", cfg.iptables_chain])
|
chain = self._rc(["iptables", "-S", cfg.iptables_chain])
|
||||||
@ -835,14 +751,33 @@ class Classifier:
|
|||||||
# --------------------------------------------------------------------------
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class _PinnedHTTPSConnection(http.client.HTTPSConnection):
|
||||||
|
"""Connect to a fixed IP while verifying TLS against the hostname (SNI)."""
|
||||||
|
|
||||||
|
def __init__(self, host: str, ip: str, port: int, context: ssl.SSLContext, connect_timeout: float, total_timeout: float):
|
||||||
|
super().__init__(host, port, timeout=connect_timeout, context=context)
|
||||||
|
self._pinned_ip = ip
|
||||||
|
self._total_timeout = total_timeout
|
||||||
|
|
||||||
|
def connect(self) -> None: # noqa: D401
|
||||||
|
sock = socket.create_connection((self._pinned_ip, self.port), self.timeout)
|
||||||
|
sock.settimeout(self._total_timeout)
|
||||||
|
self.sock = self._context.wrap_socket(sock, server_hostname=self.host)
|
||||||
|
|
||||||
|
|
||||||
class HttpClient:
|
class HttpClient:
|
||||||
def __init__(self, cfg: Config, system: Optional[System] = None):
|
def __init__(self, cfg: Config):
|
||||||
self.cfg = cfg
|
self.cfg = cfg
|
||||||
self.system = system or System()
|
|
||||||
|
|
||||||
def _system_ctx(self) -> ssl.SSLContext:
|
def _system_ctx(self) -> ssl.SSLContext:
|
||||||
return ssl.create_default_context()
|
return ssl.create_default_context()
|
||||||
|
|
||||||
|
def _pia_ctx(self) -> ssl.SSLContext:
|
||||||
|
ctx = ssl.create_default_context(cafile=self.cfg.ca_file)
|
||||||
|
ctx.check_hostname = True
|
||||||
|
ctx.verify_mode = ssl.CERT_REQUIRED
|
||||||
|
return ctx
|
||||||
|
|
||||||
def _request(self, conn: http.client.HTTPSConnection, method: str, path: str, body: Optional[bytes], headers: dict) -> tuple[int, bytes]:
|
def _request(self, conn: http.client.HTTPSConnection, method: str, path: str, body: Optional[bytes], headers: dict) -> tuple[int, bytes]:
|
||||||
try:
|
try:
|
||||||
conn.request(method, path, body=body, headers=headers)
|
conn.request(method, path, body=body, headers=headers)
|
||||||
@ -850,22 +785,13 @@ class HttpClient:
|
|||||||
data = resp.read(4_000_000)
|
data = resp.read(4_000_000)
|
||||||
return resp.status, data
|
return resp.status, data
|
||||||
except ssl.SSLCertVerificationError as e:
|
except ssl.SSLCertVerificationError as e:
|
||||||
# verify_message/reason are standard OpenSSL diagnostic text
|
raise ApiError("tls", "certificate verification failed") from e
|
||||||
# (e.g. "unable to get local issuer certificate", "certificate
|
|
||||||
# has expired", a hostname mismatch) — never a secret, and
|
|
||||||
# exactly what's needed to tell "wrong CA", "expired cert",
|
|
||||||
# "clock skew", and "hostname mismatch" apart instead of
|
|
||||||
# collapsing all of them into one opaque string.
|
|
||||||
detail = e.verify_message or e.reason or str(e) or "certificate verification failed"
|
|
||||||
raise ApiError("tls", detail) from e
|
|
||||||
except ssl.SSLError as e:
|
except ssl.SSLError as e:
|
||||||
detail = getattr(e, "reason", None) or str(e) or e.__class__.__name__
|
raise ApiError("tls", e.__class__.__name__) from e
|
||||||
raise ApiError("tls", f"{e.__class__.__name__}: {detail}") from e
|
|
||||||
except socket.timeout as e:
|
except socket.timeout as e:
|
||||||
raise ApiError("timeout", "request timed out") from e
|
raise ApiError("timeout", "request timed out") from e
|
||||||
except (OSError, http.client.HTTPException) as e:
|
except (OSError, http.client.HTTPException) as e:
|
||||||
detail = str(e) or e.__class__.__name__
|
raise ApiError("network", e.__class__.__name__) from e
|
||||||
raise ApiError("network", f"{e.__class__.__name__}: {detail}") from e
|
|
||||||
finally:
|
finally:
|
||||||
with contextlib.suppress(Exception):
|
with contextlib.suppress(Exception):
|
||||||
conn.close()
|
conn.close()
|
||||||
@ -890,68 +816,12 @@ class HttpClient:
|
|||||||
}
|
}
|
||||||
return self._request(conn, "POST", u.path, body, headers)
|
return self._request(conn, "POST", u.path, body, headers)
|
||||||
|
|
||||||
def get_pinned_curl(self, hostname: str, ip: str, port: int, path: str, token: str, pubkey: str) -> tuple[int, bytes]:
|
def get_pinned(self, hostname: str, ip: str, port: int, path: str, query: dict[str, str]) -> tuple[int, bytes]:
|
||||||
"""Pinned-IP, SNI-validated, CA-pinned GET — for PIA's addKey only.
|
conn = _PinnedHTTPSConnection(
|
||||||
|
hostname, ip, port, self._pia_ctx(), self.cfg.http_connect_timeout_sec, self.cfg.http_total_timeout_sec
|
||||||
This shells out to curl instead of using the stdlib `ssl` module.
|
)
|
||||||
Confirmed live (2026-09-22): PIA's own CA certificate (byte-
|
full = path + "?" + urllib.parse.urlencode(query)
|
||||||
identical to a fresh fetch from PIA's own repo — not a stale
|
return self._request(conn, "GET", full, None, {"User-Agent": "pia-gateway"})
|
||||||
vendored copy) has a non-critical Basic Constraints extension on
|
|
||||||
its CA cert, a real RFC 5280 non-conformance in a CA we don't
|
|
||||||
control. Recent OpenSSL (3.2+; confirmed on minisforum's 3.5.7 via
|
|
||||||
a direct repro — SSLCertVerificationError, verify_message
|
|
||||||
"Basic Constraints of CA cert not marked critical") rejects that
|
|
||||||
in its default chain verification, a check promoted from
|
|
||||||
X509_STRICT-only to the default path in recent OpenSSL releases.
|
|
||||||
Python's stdlib `ssl` module has no custom verify-callback hook to
|
|
||||||
selectively tolerate just this one known, third-party-controlled
|
|
||||||
defect while still fully validating everything else (chain,
|
|
||||||
expiry, hostname/SAN) — that gap is long-standing and
|
|
||||||
deliberate upstream, not something to route around by weakening
|
|
||||||
verify_mode. curl's TLS stack on this host does not enforce this
|
|
||||||
specific check and is what PIA's own reference implementation
|
|
||||||
(connect_to_wireguard_with_token.sh) already uses for this exact
|
|
||||||
endpoint — reusing it here with --cacert (never -k/--insecure) is
|
|
||||||
not a security downgrade, just a different, already-proven-correct
|
|
||||||
TLS backend for the one endpoint that needs it. serverlist and
|
|
||||||
token still use the stdlib ssl path above against public WebPKI
|
|
||||||
certs, which do not have this problem.
|
|
||||||
"""
|
|
||||||
argv = [
|
|
||||||
"curl", "--silent", "--show-error", "--fail", "--location", "-G",
|
|
||||||
"--connect-timeout", str(self.cfg.http_connect_timeout_sec),
|
|
||||||
"--max-time", str(self.cfg.http_total_timeout_sec),
|
|
||||||
"--cacert", self.cfg.ca_file,
|
|
||||||
"--connect-to", f"{hostname}::{ip}:",
|
|
||||||
"--data-urlencode", "pt@-",
|
|
||||||
"--data-urlencode", f"pubkey={pubkey}",
|
|
||||||
f"https://{hostname}:{port}{path}",
|
|
||||||
]
|
|
||||||
r = self.system.run(argv, stdin=token, timeout=self.cfg.http_total_timeout_sec + 10)
|
|
||||||
if r.rc == 0:
|
|
||||||
return 200, r.out.encode("utf-8")
|
|
||||||
err = (r.err or "").strip()
|
|
||||||
# Exit codes are the primary signal, but not fully portable across
|
|
||||||
# curl builds/TLS backends — confirmed live: a curl built against
|
|
||||||
# SecureTransport reported exit 56 (CURLE_RECV_ERROR) for the exact
|
|
||||||
# same --fail-triggered HTTP rejection GNU/OpenSSL curl reports as
|
|
||||||
# 22. curl's own error text ("returned error: NNN" for --fail,
|
|
||||||
# "timed out" for CURLE_OPERATION_TIMEDOUT, TLS-related wording for
|
|
||||||
# cert/handshake problems) is a second, more portable signal — used
|
|
||||||
# here as a fallback classifier alongside the exit code, not a
|
|
||||||
# replacement for it.
|
|
||||||
if r.rc == 127:
|
|
||||||
raise ApiError("network", "curl not found on PATH")
|
|
||||||
if r.rc == 28 or "timed out" in err.lower():
|
|
||||||
raise ApiError("timeout", f"curl: connection or transfer timed out: {err}")
|
|
||||||
if r.rc in (35, 53, 54, 58, 59, 60, 66, 77, 80, 82, 83) or "ssl" in err.lower() or "certificate" in err.lower():
|
|
||||||
raise ApiError("tls", f"curl TLS error (exit {r.rc}): {err}")
|
|
||||||
if r.rc == 22 or "returned error:" in err:
|
|
||||||
# --fail suppresses the body on a non-2xx response, so there is
|
|
||||||
# no status code or body to parse here — same limitation the
|
|
||||||
# legacy curl-based Ansible path already had.
|
|
||||||
raise ApiError("http", f"curl: non-2xx HTTP response (exit {r.rc}): {err}")
|
|
||||||
raise ApiError("network", f"curl exit {r.rc}: {err}")
|
|
||||||
|
|
||||||
|
|
||||||
@dataclasses.dataclass
|
@dataclasses.dataclass
|
||||||
@ -1063,7 +933,7 @@ class PiaApi:
|
|||||||
raise ApiError("http", f"token HTTP {last_status}")
|
raise ApiError("http", f"token HTTP {last_status}")
|
||||||
|
|
||||||
def add_key(self, cand: Candidate, token: str, pubkey: str) -> PeerInfo:
|
def add_key(self, cand: Candidate, token: str, pubkey: str) -> PeerInfo:
|
||||||
status, data = self.http.get_pinned_curl(cand.cn, cand.ip, ADDKEY_PORT, "/addKey", token, pubkey)
|
status, data = self.http.get_pinned(cand.cn, cand.ip, ADDKEY_PORT, "/addKey", {"pt": token, "pubkey": pubkey})
|
||||||
if status != 200:
|
if status != 200:
|
||||||
raise ApiError("http", f"addKey HTTP {status}")
|
raise ApiError("http", f"addKey HTTP {status}")
|
||||||
if not data.strip():
|
if not data.strip():
|
||||||
@ -1254,7 +1124,7 @@ class Controller:
|
|||||||
self.cfg = cfg
|
self.cfg = cfg
|
||||||
self.sys = system
|
self.sys = system
|
||||||
self.log = log
|
self.log = log
|
||||||
self.http = http or HttpClient(cfg, system)
|
self.http = http or HttpClient(cfg)
|
||||||
self._creds = creds
|
self._creds = creds
|
||||||
self.store = StateStore(cfg, system)
|
self.store = StateStore(cfg, system)
|
||||||
self.classifier = Classifier(cfg, system)
|
self.classifier = Classifier(cfg, system)
|
||||||
|
|||||||
@ -21,9 +21,6 @@
|
|||||||
# killswitch script changed in the same run), pia-wg's own PostUp
|
# killswitch script changed in the same run), pia-wg's own PostUp
|
||||||
# needs to run last so its "open" route is the final word, not
|
# needs to run last so its "open" route is the final word, not
|
||||||
# killswitch's route-table seeding.
|
# killswitch's route-table seeding.
|
||||||
# 4. "Restart pia-killswitch reassertion timer" only restarts the
|
|
||||||
# *timer* (a scheduling change, e.g. its interval), never runs the
|
|
||||||
# script itself — no ordering constraint against the others.
|
|
||||||
|
|
||||||
- name: Reload systemd daemon
|
- name: Reload systemd daemon
|
||||||
ansible.builtin.systemd:
|
ansible.builtin.systemd:
|
||||||
@ -38,11 +35,6 @@
|
|||||||
name: pia-killswitch.service
|
name: pia-killswitch.service
|
||||||
state: restarted
|
state: restarted
|
||||||
|
|
||||||
- name: Restart pia-killswitch reassertion timer
|
|
||||||
ansible.builtin.systemd:
|
|
||||||
name: pia-killswitch-reassert.timer
|
|
||||||
state: restarted
|
|
||||||
|
|
||||||
- name: Restart pia-wg
|
- name: Restart pia-wg
|
||||||
ansible.builtin.systemd:
|
ansible.builtin.systemd:
|
||||||
name: "wg-quick@{{ pia_wg_interface }}"
|
name: "wg-quick@{{ pia_wg_interface }}"
|
||||||
|
|||||||
@ -27,16 +27,10 @@
|
|||||||
- wireguard-tools
|
- wireguard-tools
|
||||||
- iproute2
|
- iproute2
|
||||||
- python3
|
- python3
|
||||||
- curl
|
|
||||||
state: present
|
state: present
|
||||||
update_cache: true
|
update_cache: true
|
||||||
# python3-requests is no longer needed: the controller uses only the
|
# python3-requests is no longer needed: the controller uses only the
|
||||||
# standard library, EXCEPT for the addKey endpoint specifically, which
|
# standard library.
|
||||||
# shells out to curl — see HttpClient.get_pinned_curl's own docstring
|
|
||||||
# for why (a non-critical Basic Constraints extension on PIA's own CA
|
|
||||||
# cert that recent OpenSSL rejects by default and Python's ssl module
|
|
||||||
# has no way to selectively tolerate). curl is virtually always already
|
|
||||||
# present on Debian, but declared explicitly rather than assumed.
|
|
||||||
|
|
||||||
- name: Pin reverse-path filtering for {{ pia_wg_interface }}
|
- name: Pin reverse-path filtering for {{ pia_wg_interface }}
|
||||||
ansible.posix.sysctl:
|
ansible.posix.sysctl:
|
||||||
|
|||||||
@ -40,35 +40,3 @@
|
|||||||
name: pia-killswitch.service
|
name: pia-killswitch.service
|
||||||
enabled: true
|
enabled: true
|
||||||
state: started
|
state: started
|
||||||
|
|
||||||
- name: Deploy the kill-switch periodic reassertion systemd service
|
|
||||||
ansible.builtin.template:
|
|
||||||
src: pia-killswitch-reassert.service.j2
|
|
||||||
dest: /etc/systemd/system/pia-killswitch-reassert.service
|
|
||||||
mode: "0644"
|
|
||||||
owner: root
|
|
||||||
group: root
|
|
||||||
notify:
|
|
||||||
- Reload systemd daemon
|
|
||||||
- Restart pia-killswitch reassertion timer
|
|
||||||
|
|
||||||
- name: Deploy the kill-switch periodic reassertion systemd timer
|
|
||||||
ansible.builtin.template:
|
|
||||||
src: pia-killswitch-reassert.timer.j2
|
|
||||||
dest: /etc/systemd/system/pia-killswitch-reassert.timer
|
|
||||||
mode: "0644"
|
|
||||||
owner: root
|
|
||||||
group: root
|
|
||||||
notify:
|
|
||||||
- Reload systemd daemon
|
|
||||||
- Restart pia-killswitch reassertion timer
|
|
||||||
|
|
||||||
- name: Enable and start the kill-switch reassertion timer
|
|
||||||
ansible.builtin.systemd:
|
|
||||||
name: pia-killswitch-reassert.timer
|
|
||||||
enabled: true
|
|
||||||
state: started
|
|
||||||
daemon_reload: true
|
|
||||||
# Unconditional — not gated by pia_auto_recovery_enabled. See the
|
|
||||||
# service unit's own header for why this is scoped differently from
|
|
||||||
# the automatic recovery/registration timers.
|
|
||||||
|
|||||||
@ -10,7 +10,6 @@
|
|||||||
"vlan50_subnet": {{ pia_vlan50_subnet | to_json }},
|
"vlan50_subnet": {{ pia_vlan50_subnet | to_json }},
|
||||||
"iptables_chain": {{ pia_iptables_chain | to_json }},
|
"iptables_chain": {{ pia_iptables_chain | to_json }},
|
||||||
"lan_interface": {{ pia_lan_interface | to_json }},
|
"lan_interface": {{ pia_lan_interface | to_json }},
|
||||||
"forward_bypass_chains": {{ pia_forward_bypass_chains | to_json }},
|
|
||||||
"mss_clamp_enabled": {{ pia_mss_clamp_enabled | bool | to_json }},
|
"mss_clamp_enabled": {{ pia_mss_clamp_enabled | bool | to_json }},
|
||||||
"region_candidates": {{ pia_region_candidates | to_json }},
|
"region_candidates": {{ pia_region_candidates | to_json }},
|
||||||
"handshake_max_age_sec": {{ pia_healthcheck_handshake_max_age_sec | int }},
|
"handshake_max_age_sec": {{ pia_healthcheck_handshake_max_age_sec | int }},
|
||||||
|
|||||||
@ -1,30 +0,0 @@
|
|||||||
# Managed by Ansible (role: pia-gateway). Do not edit by hand.
|
|
||||||
#
|
|
||||||
# Periodically re-runs the kill-switch seed script (pia-killswitch.sh) so
|
|
||||||
# its invariants — source rule, table default, and specifically the
|
|
||||||
# FORWARD jump's position — are re-asserted on a schedule, not only at
|
|
||||||
# the external trigger points that used to be the only times it ran
|
|
||||||
# (boot, a UFW reload, or an Ansible-detected content change to the
|
|
||||||
# script itself). Needed because other host components that also manage
|
|
||||||
# iptables — kube-router and dockerd — reassert their OWN position-1
|
|
||||||
# jumps independently, and confirmed live (2026-09-22) to do so more
|
|
||||||
# often than originally assumed: the FORWARD jump drifted twice within
|
|
||||||
# about an hour under this cluster's ordinary CronJob/pod churn (ca-sync,
|
|
||||||
# porkbun-ddns, gitea-backup, ...), not just at boot-scale events.
|
|
||||||
#
|
|
||||||
# The script itself is already fully idempotent (safe to run repeatedly
|
|
||||||
# — see its own header comment); this unit just gives it more chances to
|
|
||||||
# run. Deliberately separate from pia-gateway-recover.timer and NOT
|
|
||||||
# gated by pia_auto_recovery_enabled: this is the kill switch continuing
|
|
||||||
# to assert its own already-established, narrowly-scoped state — the
|
|
||||||
# same kind of thing it already does unconditionally at boot and after a
|
|
||||||
# UFW reload, just on a schedule too — not the higher-level automatic
|
|
||||||
# recovery/registration logic the plan was deliberately cautious about
|
|
||||||
# running unattended.
|
|
||||||
[Unit]
|
|
||||||
Description=Periodically re-assert the PIA gateway kill switch's FORWARD jump position
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=oneshot
|
|
||||||
ExecStart=/usr/local/sbin/pia-killswitch.sh
|
|
||||||
LimitCORE=0
|
|
||||||
@ -1,11 +0,0 @@
|
|||||||
# Managed by Ansible (role: pia-gateway). Do not edit by hand.
|
|
||||||
[Unit]
|
|
||||||
Description=Run the PIA gateway kill-switch reassertion every {{ pia_healthcheck_interval_sec }}s
|
|
||||||
|
|
||||||
[Timer]
|
|
||||||
OnBootSec=30
|
|
||||||
OnUnitActiveSec={{ pia_healthcheck_interval_sec }}
|
|
||||||
AccuracySec=5s
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=timers.target
|
|
||||||
@ -1,16 +1,12 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# Managed by Ansible (role: pia-gateway). Do not edit by hand.
|
# Managed by Ansible (role: pia-gateway). Do not edit by hand.
|
||||||
#
|
#
|
||||||
# Runs at boot, before wg-quick@{{ pia_wg_interface }}.service (enforced
|
# Runs once at boot, before wg-quick@{{ pia_wg_interface }}.service
|
||||||
# by the systemd drop-in in pia-killswitch.service's Before=/wg-quick's
|
# (enforced by the systemd drop-in in pia-killswitch.service's
|
||||||
# Requires=After=); any time UFW is reloaded (see handlers/main.yaml —
|
# Before=/wg-quick's Requires=After=), and again any time UFW is
|
||||||
# "Reload ufw" fires before "Restart pia-killswitch" so a reload can
|
# reloaded (see handlers/main.yaml — "Reload ufw" fires before "Restart
|
||||||
# never leave a permanent gap); and — since 2026-09-22, after the FORWARD
|
# pia-killswitch" so a reload can never leave a permanent gap). Seeds the
|
||||||
# jump was found drifting twice within about an hour under this
|
# closed state that PIA gateway's kill switch relies on:
|
||||||
# cluster's ordinary CronJob/pod churn, far more often than a boot/reload
|
|
||||||
# -only cadence catches — every {{ pia_healthcheck_interval_sec }}s via
|
|
||||||
# pia-killswitch-reassert.timer. Seeds the closed state that PIA
|
|
||||||
# gateway's kill switch relies on:
|
|
||||||
#
|
#
|
||||||
# - a source rule sending {{ pia_vlan50_subnet }} to the dedicated
|
# - a source rule sending {{ pia_vlan50_subnet }} to the dedicated
|
||||||
# "{{ pia_route_table_name }}" table, evaluated after the kernel's
|
# "{{ pia_route_table_name }}" table, evaluated after the kernel's
|
||||||
|
|||||||
@ -88,7 +88,6 @@ class FakeSystem(pg.System):
|
|||||||
self.stdins: list[str] = []
|
self.stdins: list[str] = []
|
||||||
self.genkey_queue = [PRIV_B]
|
self.genkey_queue = [PRIV_B]
|
||||||
self.jitter = 0.5
|
self.jitter = 0.5
|
||||||
self.extra_chains: dict = {} # name -> list[str] | None ("-A <name> ..." lines, or None = missing)
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def open_chain():
|
def open_chain():
|
||||||
@ -207,12 +206,6 @@ class FakeSystem(pg.System):
|
|||||||
if self.chain_rules is None:
|
if self.chain_rules is None:
|
||||||
return pg.CmdResult(1, "", "iptables: No chain/target/match by that name.")
|
return pg.CmdResult(1, "", "iptables: No chain/target/match by that name.")
|
||||||
return pg.CmdResult(0, f"-N {CHAIN}\n" + "\n".join(self.chain_rules) + "\n", "")
|
return pg.CmdResult(0, f"-N {CHAIN}\n" + "\n".join(self.chain_rules) + "\n", "")
|
||||||
if a[2] in self.extra_chains:
|
|
||||||
rules = self.extra_chains[a[2]]
|
|
||||||
if rules is None:
|
|
||||||
return pg.CmdResult(1, "", "iptables: No chain/target/match by that name.")
|
|
||||||
return pg.CmdResult(0, f"-N {a[2]}\n" + "\n".join(rules) + "\n", "")
|
|
||||||
return pg.CmdResult(1, "", "iptables: No chain/target/match by that name.")
|
|
||||||
if a[:2] == ["iptables", "-C"]:
|
if a[:2] == ["iptables", "-C"]:
|
||||||
rule = "-A " + " ".join(a[2:])
|
rule = "-A " + " ".join(a[2:])
|
||||||
return pg.CmdResult(0 if rule in self.forward_rules else 1, "", "")
|
return pg.CmdResult(0 if rule in self.forward_rules else 1, "", "")
|
||||||
@ -262,8 +255,8 @@ class FakeHttp:
|
|||||||
return 200, val
|
return 200, val
|
||||||
return 200, json.dumps({"token": val}).encode()
|
return 200, json.dumps({"token": val}).encode()
|
||||||
|
|
||||||
def get_pinned_curl(self, hostname, ip, port, path, token, pubkey):
|
def get_pinned(self, hostname, ip, port, path, query):
|
||||||
self.calls.append(("addkey", hostname, ip, port, path, {"pt": token, "pubkey": pubkey}))
|
self.calls.append(("addkey", hostname, ip, port, path, dict(query)))
|
||||||
kind, val = self.addkey.get(hostname, ("http", 502))
|
kind, val = self.addkey.get(hostname, ("http", 502))
|
||||||
if kind == "err":
|
if kind == "err":
|
||||||
raise pg.ApiError(val, "fake")
|
raise pg.ApiError(val, "fake")
|
||||||
@ -502,139 +495,6 @@ class TestSafetyDrift(Base):
|
|||||||
self.assertEqual(self.http.calls, [])
|
self.assertEqual(self.http.calls, [])
|
||||||
|
|
||||||
|
|
||||||
KUBE_ROUTER_FORWARD_JUMP = '-A FORWARD -m comment --comment "kube-router netpol - TEMCG2JMHZYE7H7T" -j KUBE-ROUTER-FORWARD'
|
|
||||||
KUBE_ROUTER_SAFE_RULES = [
|
|
||||||
'-A KUBE-ROUTER-FORWARD -d 10.42.0.39/32 -m comment --comment "pod X" -j KUBE-POD-FW-AAAA',
|
|
||||||
'-A KUBE-ROUTER-FORWARD -s 10.42.0.39/32 -m comment --comment "pod X" -j KUBE-POD-FW-AAAA',
|
|
||||||
'-A KUBE-ROUTER-FORWARD -d 10.42.1.5/32 -m physdev --physdev-is-bridged -m comment --comment "pod Y" -j KUBE-POD-FW-BBBB',
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
class TestForwardBypassChains(Base):
|
|
||||||
"""The content-verified bypass mechanism for kube-router's own FORWARD
|
|
||||||
jump — found necessary live, 2026-09-22, after the plain reposition
|
|
||||||
fix alone caused a repeating outage loop: kube-router resyncs (and
|
|
||||||
re-pushes our jump down) roughly every 60-90s on this cluster, far
|
|
||||||
more often than assumed, and treating every transient reordering as a
|
|
||||||
full safety-drift emergency (closing the route each time) kept the
|
|
||||||
tunnel down far more than it was up.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def classify_with_forward(self, forward_rules, bypass_chains=(), extra_chains=None):
|
|
||||||
self.cfg.forward_bypass_chains = list(bypass_chains)
|
|
||||||
self.sysm.forward_rules = list(forward_rules)
|
|
||||||
if extra_chains:
|
|
||||||
self.sysm.extra_chains.update(extra_chains)
|
|
||||||
return self.ctl.classifier.classify(pg.default_state())
|
|
||||||
|
|
||||||
def test_disabled_by_default_kube_router_ahead_still_blocks(self):
|
|
||||||
st = self.classify_with_forward(
|
|
||||||
[KUBE_ROUTER_FORWARD_JUMP, f"-A FORWARD -s {SUBNET} -j {CHAIN}"],
|
|
||||||
bypass_chains=(), # default: nothing configured
|
|
||||||
)
|
|
||||||
self.assertFalse(st.safety_ok)
|
|
||||||
self.assertIn("FORWARD jump to kill-switch chain is not first", " ".join(st.reasons))
|
|
||||||
|
|
||||||
def test_configured_bypass_with_verified_safe_content_tolerated(self):
|
|
||||||
st = self.classify_with_forward(
|
|
||||||
[KUBE_ROUTER_FORWARD_JUMP, f"-A FORWARD -s {SUBNET} -j {CHAIN}"],
|
|
||||||
bypass_chains=["KUBE-ROUTER-FORWARD"],
|
|
||||||
extra_chains={"KUBE-ROUTER-FORWARD": KUBE_ROUTER_SAFE_RULES},
|
|
||||||
)
|
|
||||||
self.assertTrue(st.safety_ok, st.reasons)
|
|
||||||
|
|
||||||
def test_duplicate_bypass_jump_tolerated(self):
|
|
||||||
# The exact live anomaly: kube-router left two copies of its own
|
|
||||||
# jump behind after a resync.
|
|
||||||
st = self.classify_with_forward(
|
|
||||||
[KUBE_ROUTER_FORWARD_JUMP, KUBE_ROUTER_FORWARD_JUMP, f"-A FORWARD -s {SUBNET} -j {CHAIN}"],
|
|
||||||
bypass_chains=["KUBE-ROUTER-FORWARD"],
|
|
||||||
extra_chains={"KUBE-ROUTER-FORWARD": KUBE_ROUTER_SAFE_RULES},
|
|
||||||
)
|
|
||||||
self.assertTrue(st.safety_ok, st.reasons)
|
|
||||||
|
|
||||||
def test_bypass_chain_picking_up_a_matching_source_still_blocks(self):
|
|
||||||
dangerous = KUBE_ROUTER_SAFE_RULES + [f"-A KUBE-ROUTER-FORWARD -s {SUBNET} -j ACCEPT"]
|
|
||||||
st = self.classify_with_forward(
|
|
||||||
[KUBE_ROUTER_FORWARD_JUMP, f"-A FORWARD -s {SUBNET} -j {CHAIN}"],
|
|
||||||
bypass_chains=["KUBE-ROUTER-FORWARD"],
|
|
||||||
extra_chains={"KUBE-ROUTER-FORWARD": dangerous},
|
|
||||||
)
|
|
||||||
self.assertFalse(st.safety_ok)
|
|
||||||
self.assertIn("no longer verified safe", " ".join(st.reasons))
|
|
||||||
|
|
||||||
def test_bypass_chain_with_bare_unconditional_rule_blocks(self):
|
|
||||||
dangerous = KUBE_ROUTER_SAFE_RULES + ["-A KUBE-ROUTER-FORWARD -j ACCEPT"]
|
|
||||||
st = self.classify_with_forward(
|
|
||||||
[KUBE_ROUTER_FORWARD_JUMP, f"-A FORWARD -s {SUBNET} -j {CHAIN}"],
|
|
||||||
bypass_chains=["KUBE-ROUTER-FORWARD"],
|
|
||||||
extra_chains={"KUBE-ROUTER-FORWARD": dangerous},
|
|
||||||
)
|
|
||||||
self.assertFalse(st.safety_ok)
|
|
||||||
|
|
||||||
def test_bypass_chain_with_negated_match_blocks(self):
|
|
||||||
dangerous = [f'-A KUBE-ROUTER-FORWARD ! -s {SUBNET} -j ACCEPT']
|
|
||||||
st = self.classify_with_forward(
|
|
||||||
[KUBE_ROUTER_FORWARD_JUMP, f"-A FORWARD -s {SUBNET} -j {CHAIN}"],
|
|
||||||
bypass_chains=["KUBE-ROUTER-FORWARD"],
|
|
||||||
extra_chains={"KUBE-ROUTER-FORWARD": dangerous},
|
|
||||||
)
|
|
||||||
self.assertFalse(st.safety_ok)
|
|
||||||
|
|
||||||
def test_unrecognized_chain_ahead_still_blocks(self):
|
|
||||||
# DOCKER-USER is not on the allowlist even when KUBE-ROUTER-FORWARD is.
|
|
||||||
st = self.classify_with_forward(
|
|
||||||
["-A FORWARD -j DOCKER-USER", f"-A FORWARD -s {SUBNET} -j {CHAIN}"],
|
|
||||||
bypass_chains=["KUBE-ROUTER-FORWARD"],
|
|
||||||
extra_chains={"DOCKER-USER": []},
|
|
||||||
)
|
|
||||||
self.assertFalse(st.safety_ok)
|
|
||||||
|
|
||||||
def test_forward_level_rule_with_real_matcher_not_treated_as_bypass(self):
|
|
||||||
# Even though KUBE-ROUTER-FORWARD is allowlisted and its content is
|
|
||||||
# safe, a FORWARD-level rule that itself carries a real matcher
|
|
||||||
# (not just a cosmetic comment) is never treated as a bypass jump.
|
|
||||||
st = self.classify_with_forward(
|
|
||||||
[f"-A FORWARD -s 1.2.3.0/24 -j KUBE-ROUTER-FORWARD", f"-A FORWARD -s {SUBNET} -j {CHAIN}"],
|
|
||||||
bypass_chains=["KUBE-ROUTER-FORWARD"],
|
|
||||||
extra_chains={"KUBE-ROUTER-FORWARD": KUBE_ROUTER_SAFE_RULES},
|
|
||||||
)
|
|
||||||
self.assertFalse(st.safety_ok)
|
|
||||||
|
|
||||||
def test_missing_bypass_chain_blocks(self):
|
|
||||||
st = self.classify_with_forward(
|
|
||||||
[KUBE_ROUTER_FORWARD_JUMP, f"-A FORWARD -s {SUBNET} -j {CHAIN}"],
|
|
||||||
bypass_chains=["KUBE-ROUTER-FORWARD"],
|
|
||||||
extra_chains={"KUBE-ROUTER-FORWARD": None},
|
|
||||||
)
|
|
||||||
self.assertFalse(st.safety_ok)
|
|
||||||
self.assertIn("unreadable", " ".join(st.reasons))
|
|
||||||
|
|
||||||
def test_end_to_end_recover_does_not_emergency_close_on_tolerated_drift(self):
|
|
||||||
# The actual live incident, reproduced end to end: kube-router's
|
|
||||||
# jump sits ahead of ours, content-verified-safe, and the recovery
|
|
||||||
# timer must NOT declare safety drift or touch the route.
|
|
||||||
self.cfg.forward_bypass_chains = ["KUBE-ROUTER-FORWARD"]
|
|
||||||
self.sysm.forward_rules = [KUBE_ROUTER_FORWARD_JUMP, f"-A FORWARD -s {SUBNET} -j {CHAIN}"]
|
|
||||||
self.sysm.extra_chains["KUBE-ROUTER-FORWARD"] = KUBE_ROUTER_SAFE_RULES
|
|
||||||
rc = self.ctl.cmd_recover()
|
|
||||||
self.assertEqual(rc, pg.EXIT_OK)
|
|
||||||
self.assertFalse(self.state()["emergency_close_active"])
|
|
||||||
self.assertEqual(self.sysm.table_default, "open")
|
|
||||||
self.assertEqual(self.sysm.restart_count, 0)
|
|
||||||
|
|
||||||
def test_end_to_end_recover_still_emergency_closes_on_real_drift(self):
|
|
||||||
# Regression guard: an ACTUALLY unrecognized rule ahead of ours
|
|
||||||
# must still trigger the full emergency-close path, even with a
|
|
||||||
# bypass list configured for something else.
|
|
||||||
self.cfg.forward_bypass_chains = ["KUBE-ROUTER-FORWARD"]
|
|
||||||
self.sysm.forward_rules = ["-A FORWARD -j DOCKER-USER", f"-A FORWARD -s {SUBNET} -j {CHAIN}"]
|
|
||||||
self.sysm.extra_chains["DOCKER-USER"] = []
|
|
||||||
rc = self.ctl.cmd_recover()
|
|
||||||
self.assertEqual(rc, pg.EXIT_SAFETY)
|
|
||||||
self.assertTrue(self.state()["emergency_close_active"])
|
|
||||||
|
|
||||||
|
|
||||||
class TestRecoveryStateMachine(Base):
|
class TestRecoveryStateMachine(Base):
|
||||||
def test_healthy_runs_are_idempotent_and_do_not_mutate(self):
|
def test_healthy_runs_are_idempotent_and_do_not_mutate(self):
|
||||||
self.run_recover_n(3)
|
self.run_recover_n(3)
|
||||||
@ -1367,106 +1227,6 @@ def dataclass_dict(cfg):
|
|||||||
return dataclasses.asdict(cfg)
|
return dataclasses.asdict(cfg)
|
||||||
|
|
||||||
|
|
||||||
class FakeCurlSystem(pg.System):
|
|
||||||
"""Fakes just enough of System.run to exercise get_pinned_curl's argv
|
|
||||||
construction, stdin handling, and exit-code classification, without a
|
|
||||||
real curl binary or network access."""
|
|
||||||
|
|
||||||
def __init__(self, result=None):
|
|
||||||
self.result = result or pg.CmdResult(0, '{"status":"OK"}', "")
|
|
||||||
self.calls = []
|
|
||||||
self.stdins = []
|
|
||||||
|
|
||||||
def run(self, argv, stdin=None, timeout=60):
|
|
||||||
self.calls.append(list(argv))
|
|
||||||
if stdin is not None:
|
|
||||||
self.stdins.append(stdin)
|
|
||||||
return self.result
|
|
||||||
|
|
||||||
|
|
||||||
class TestPinnedCurlAddKey(unittest.TestCase):
|
|
||||||
def setUp(self):
|
|
||||||
self.cfg = pg.Config(ca_file="/etc/wireguard/pia-ca.crt", http_connect_timeout_sec=10, http_total_timeout_sec=30)
|
|
||||||
|
|
||||||
def test_argv_shape_matches_the_proven_legacy_curl_invocation(self):
|
|
||||||
sysm = FakeCurlSystem()
|
|
||||||
http_client = pg.HttpClient(self.cfg, sysm)
|
|
||||||
status, data = http_client.get_pinned_curl("Server-11669-10a", "69.33.198.97", 1337, "/addKey", SECRET_TOKEN, PUB_A)
|
|
||||||
self.assertEqual(status, 200)
|
|
||||||
self.assertEqual(data, b'{"status":"OK"}')
|
|
||||||
argv = sysm.calls[0]
|
|
||||||
self.assertEqual(argv[0], "curl")
|
|
||||||
self.assertIn("--fail", argv)
|
|
||||||
self.assertIn("-G", argv)
|
|
||||||
self.assertIn("--cacert", argv)
|
|
||||||
self.assertEqual(argv[argv.index("--cacert") + 1], "/etc/wireguard/pia-ca.crt")
|
|
||||||
self.assertIn("--connect-to", argv)
|
|
||||||
self.assertEqual(argv[argv.index("--connect-to") + 1], "Server-11669-10a::69.33.198.97:")
|
|
||||||
self.assertIn("pt@-", argv)
|
|
||||||
self.assertIn(f"pubkey={PUB_A}", argv)
|
|
||||||
self.assertEqual(argv[-1], "https://Server-11669-10a:1337/addKey")
|
|
||||||
# never -k/--insecure — this must still be a real, validating request
|
|
||||||
self.assertNotIn("-k", argv)
|
|
||||||
self.assertNotIn("--insecure", argv)
|
|
||||||
|
|
||||||
def test_token_passed_via_stdin_with_no_trailing_newline_and_never_in_argv(self):
|
|
||||||
sysm = FakeCurlSystem()
|
|
||||||
http_client = pg.HttpClient(self.cfg, sysm)
|
|
||||||
http_client.get_pinned_curl("Server-11669-10a", "69.33.198.97", 1337, "/addKey", SECRET_TOKEN, PUB_A)
|
|
||||||
self.assertEqual(sysm.stdins, [SECRET_TOKEN]) # exact match: no appended "\n"
|
|
||||||
for arg in sysm.calls[0]:
|
|
||||||
self.assertNotIn(SECRET_TOKEN, arg)
|
|
||||||
|
|
||||||
def test_timeout_exit_code_classified(self):
|
|
||||||
sysm = FakeCurlSystem(pg.CmdResult(28, "", "curl: (28) Connection timed out"))
|
|
||||||
http_client = pg.HttpClient(self.cfg, sysm)
|
|
||||||
with self.assertRaises(pg.ApiError) as cm:
|
|
||||||
http_client.get_pinned_curl("h", "1.2.3.4", 1337, "/addKey", "t", "p")
|
|
||||||
self.assertEqual(cm.exception.kind, "timeout")
|
|
||||||
|
|
||||||
def test_tls_exit_codes_classified_with_stderr_detail(self):
|
|
||||||
for rc in (35, 53, 54, 58, 59, 60, 66, 77, 80, 82, 83):
|
|
||||||
with self.subTest(rc=rc):
|
|
||||||
sysm = FakeCurlSystem(pg.CmdResult(rc, "", "curl: (60) SSL certificate problem"))
|
|
||||||
http_client = pg.HttpClient(self.cfg, sysm)
|
|
||||||
with self.assertRaises(pg.ApiError) as cm:
|
|
||||||
http_client.get_pinned_curl("h", "1.2.3.4", 1337, "/addKey", "t", "p")
|
|
||||||
self.assertEqual(cm.exception.kind, "tls")
|
|
||||||
self.assertIn("SSL certificate problem", cm.exception.detail)
|
|
||||||
|
|
||||||
def test_http_failure_exit_code_classified(self):
|
|
||||||
sysm = FakeCurlSystem(pg.CmdResult(22, "", "curl: (22) The requested URL returned error: 401"))
|
|
||||||
http_client = pg.HttpClient(self.cfg, sysm)
|
|
||||||
with self.assertRaises(pg.ApiError) as cm:
|
|
||||||
http_client.get_pinned_curl("h", "1.2.3.4", 1337, "/addKey", "t", "p")
|
|
||||||
self.assertEqual(cm.exception.kind, "http")
|
|
||||||
|
|
||||||
def test_nonstandard_exit_code_with_fail_text_still_classified_as_http(self):
|
|
||||||
# Observed live: curl built against SecureTransport reports exit 56
|
|
||||||
# (CURLE_RECV_ERROR), not the "usual" 22, for the same --fail
|
|
||||||
# -triggered HTTP rejection — the error text is the portable signal.
|
|
||||||
sysm = FakeCurlSystem(pg.CmdResult(56, "", "curl: (56) The requested URL returned error: 401"))
|
|
||||||
http_client = pg.HttpClient(self.cfg, sysm)
|
|
||||||
with self.assertRaises(pg.ApiError) as cm:
|
|
||||||
http_client.get_pinned_curl("h", "1.2.3.4", 1337, "/addKey", "t", "p")
|
|
||||||
self.assertEqual(cm.exception.kind, "http")
|
|
||||||
|
|
||||||
def test_curl_missing_classified_as_network_not_a_crash(self):
|
|
||||||
sysm = FakeCurlSystem(pg.CmdResult(127, "", "curl: not found"))
|
|
||||||
http_client = pg.HttpClient(self.cfg, sysm)
|
|
||||||
with self.assertRaises(pg.ApiError) as cm:
|
|
||||||
http_client.get_pinned_curl("h", "1.2.3.4", 1337, "/addKey", "t", "p")
|
|
||||||
self.assertEqual(cm.exception.kind, "network")
|
|
||||||
|
|
||||||
def test_unclassified_exit_code_falls_back_to_network_with_stderr(self):
|
|
||||||
sysm = FakeCurlSystem(pg.CmdResult(7, "", "curl: (7) Couldn't connect to server"))
|
|
||||||
http_client = pg.HttpClient(self.cfg, sysm)
|
|
||||||
with self.assertRaises(pg.ApiError) as cm:
|
|
||||||
http_client.get_pinned_curl("h", "1.2.3.4", 1337, "/addKey", "t", "p")
|
|
||||||
self.assertEqual(cm.exception.kind, "network")
|
|
||||||
self.assertIn("Couldn't connect", cm.exception.detail)
|
|
||||||
|
|
||||||
|
|
||||||
class TestHttpHelpers(unittest.TestCase):
|
class TestHttpHelpers(unittest.TestCase):
|
||||||
def test_key_and_ip_validators(self):
|
def test_key_and_ip_validators(self):
|
||||||
self.assertTrue(pg.valid_wg_key(PUB_A))
|
self.assertTrue(pg.valid_wg_key(PUB_A))
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user