diff --git a/ansible/README.md b/ansible/README.md index e3b898c..a5090d9 100644 --- a/ansible/README.md +++ b/ansible/README.md @@ -49,6 +49,8 @@ ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-ollama.yaml - ansible-playbook -i ansible/inventory.yaml ansible/playbooks/deploy-watch-party.yaml ansible-playbook -i ansible/inventory.yaml ansible/playbooks/wireguard.yaml -K ansible-playbook -i ansible/inventory.yaml ansible/playbooks/homeassistant.yaml -K +ansible-playbook -i ansible/inventory.yaml ansible/playbooks/pia-gateway.yaml -K +ansible-playbook -i ansible/inventory.yaml ansible/playbooks/vlan50-parent.yaml -K ``` ## Roles @@ -66,7 +68,9 @@ ansible-playbook -i ansible/inventory.yaml ansible/playbooks/homeassistant.yaml | `glances` | Host-level Glances service | | `ollama` | Ollama service on the Mac Mini and GPU node (branches on OS) | | `watch-party` | Watch Party Docker Compose deployment on the Mac Mini | -| `wireguard` | WireGuard server configuration | +| `wireguard` | WireGuard server configuration (inbound home-VPN access — phone/Mac clients) | +| `pia-gateway` | Minisforum's PIA WireGuard *egress* gateway for VPN VLAN 50 (policy routing, kill switch, health checks) — see its own README | +| `vlan50-parent` | nik-debian's tagged VLAN 50 parent interface (`enp1s0.50`) for Multus — see its own README | | `homeassistant` | Standalone Home Assistant deployment (Docker Compose + systemd on `minisforum`) — this is the **only** thing serving `ha.home.arpa`, not legacy/dead | ## Notes diff --git a/ansible/host_vars/minisforum.yaml b/ansible/host_vars/minisforum.yaml index 9d000be..db6af7d 100644 --- a/ansible/host_vars/minisforum.yaml +++ b/ansible/host_vars/minisforum.yaml @@ -1,2 +1,14 @@ # Host vars for: minisforum (k3s server) ansible_python_interpreter: /usr/bin/python3.13 + +# ── pia-gateway ──────────────────────────────────────────────────────────────── +# Read directly from the repo-root .env (same PIA_USER/PIA_PASSWORD keys +# manifests/media/pia-secret.sh already uses) rather than ansible-vault — +# .env is already the established, gitignored, plaintext-at-rest secret +# source for this account elsewhere in the repo, and lookups run on the +# control node, so this never touches minisforum or gets committed. +# playbook_dir is always ansible/playbooks for every playbook in this +# repo, so ../../.env resolves to the repo root regardless of which +# playbook triggers evaluation of these two vars. +pia_user: "{{ lookup('ansible.builtin.file', playbook_dir + '/../../.env') | regex_search('^PIA_USER=(.*)$', '\\1', multiline=True) | first }}" +pia_password: "{{ lookup('ansible.builtin.file', playbook_dir + '/../../.env') | regex_search('^PIA_PASSWORD=(.*)$', '\\1', multiline=True) | first }}" diff --git a/ansible/playbooks/pia-gateway.yaml b/ansible/playbooks/pia-gateway.yaml new file mode 100644 index 0000000..92ad936 --- /dev/null +++ b/ansible/playbooks/pia-gateway.yaml @@ -0,0 +1,6 @@ +--- +- name: PIA WireGuard egress gateway for VPN VLAN 50 + hosts: minisforum + become: true + roles: + - pia-gateway diff --git a/ansible/roles/pia-gateway/README.md b/ansible/roles/pia-gateway/README.md new file mode 100644 index 0000000..50afa3a --- /dev/null +++ b/ansible/roles/pia-gateway/README.md @@ -0,0 +1,217 @@ +# pia-gateway + +Makes `minisforum` a PIA WireGuard egress gateway for VPN VLAN 50 +(`10.10.50.0/24`), without changing its own default route. Implements +Phase 2 of `~/repo/homelab/plan.md`. + +## Before the first real run + +1. **The PIA API calls in `tasks/register.yaml` have gone through three + rounds of correction against real live testing (2026-08-24):** + - `serverlist.piaservers.net/vpninfo/servers/v6`: works. Region values + everywhere in this role are the API's **id** field (e.g. `japan`, + `hk`, `taiwan`), never a display name ("JP Tokyo", "Hong Kong", + "Taiwan" — those are Gluetun's labels, not something this role or + PIA's API accepts). + - Token acquisition originally guessed a regional-meta-server + endpoint (`/authv3/generateToken`) that turned out not to be the + real flow at all — it was rebuilt from PIA's actual + `pia-foss/manual-connections` `get_token.sh` (fetched and read in + full): a single fixed `POST + https://www.privateinternetaccess.com/api/client/v2/token`, + multipart `username`/`password`, **normal system CA validation** + (no `--cacert`, no `validate_certs: false`), independent of region + entirely — only WireGuard server *selection* is regional. Not yet + exercised with real credentials end to end. + - `{wg}:1337/addKey` (WireGuard key registration) — confirmed against + `connect_to_wireguard_with_token.sh` (fetched and read in full): + this one genuinely does need PIA's own CA bundle + (`files/pia-ca.crt`, `--cacert`) and `--connect-to`. Originally + failed over only across servers *within* a single fixed region + (`pia_region: japan`) — in practice all three JP Tokyo servers + failed the same way, one after another, so failover now also + crosses regions: every server in Hong Kong, then every server in + Taiwan, then every server in JP Tokyo (`pia_region_candidates` in + `defaults/main.yaml`), stopping at the first `status: OK`. Token + passed via stdin rather than argv. See `tasks/addkey-attempt.yaml` + and "Region fallback" below. +2. Make sure the repo-root `.env` has `PIA_USER`/`PIA_PASSWORD` set (same + keys `manifests/media/pia-secret.sh` already reads) — `ansible/ + host_vars/minisforum.yaml` reads them directly from there via a + `lookup('ansible.builtin.file', ...)` + regex, deliberately not + ansible-vault, so this needs no vault password at all. The lookup runs + on the control node, so `.env` never touches minisforum and is never + committed (already gitignored). +3. Confirm console/recovery access to `minisforum` (IPMI/physical/other + out-of-band path) before applying — this role changes host routing and + firewall policy. If SSH becomes unreachable, use that path. +4. `ufw status verbose` and `wg show` (existing `wg0`) were not inspected + live before writing this role (no passwordless sudo in discovery) — + spot-check that `wg0` (the separate home-VPN role) is unaffected after + applying. + +## Region fallback + +Registration tries every WireGuard server in every configured region, in +order, stopping at the first `status: OK`: + +```text +hk (Hong Kong): + wg server 1 + wg server 2 +taiwan (Taiwan): + wg server 1 + wg server 2 +japan (JP Tokyo): + wg server 1 + wg server 2 + wg server 3 +``` + +The actual number and order of servers within each region always comes +from the live serverlist response — never hardcoded here. + +**Automatic fallback** (default — tries `pia_region_candidates` in +order): + +```bash +ansible-playbook -i ansible/inventory.yaml \ + ansible/playbooks/pia-gateway.yaml -K -J +``` + +**Forced single region**, for troubleshooting one region in isolation — +bypasses `pia_region_candidates` entirely, does not fall back to the +others: + +```bash +ansible-playbook -i ansible/inventory.yaml \ + ansible/playbooks/pia-gateway.yaml -K -J \ + -e pia_region=hk +``` + +Both `pia_region` and every entry in `pia_region_candidates` are PIA API +**ids** — `hk`, `taiwan`, `japan` — not the display names Gluetun's +`SERVER_REGIONS` config uses ("Hong Kong", "Taiwan", "JP Tokyo"). Passing +a display name here matches nothing and fails the "region exists" +assertion before any network call is even made. + +Which region/server actually ended up registered is recorded as +`pia_region_used`/`pia_region_name_used`/`pia_wg_server_used`, written +into `pia-wg.conf`'s own `[Peer]` comment, and surfaced by the health +check (`INFO configured-peer: ...`) — it is whichever one answered first, +not necessarily `pia_region_candidates[0]`. + +If every region fails: `journalctl` (or the play's own output) has one +`debug` line per failed attempt, each showing region name+id, server +hostname+IP, curl exit code, a classified reason (connection timeout / +TLS-SSL failure / HTTP failure / empty response / malformed JSON / +parsed-but-not-OK status, never a generic catch-all), and sanitized +stderr — never the token, credentials, the full curl invocation, request +stdin, or a complete response body. If Hong Kong and Taiwan fail exactly +the same way Tokyo did, that's a strong signal the problem is shared +across all three (the request shape, the account credentials, or +minisforum's own network path) rather than one region being down — see +`tasks/register.yaml`'s header comment. + +**Worked example (2026-08-24):** exactly that happened — all 7 servers +across all 3 regions came back `HTTP failure (non-2xx response)`, curl +exit 22, `The requested URL returned error: 401`, uniformly. The cause +was in `addkey-attempt.yaml`'s own curl task, not PIA or any region: +`ansible.builtin.command`'s `stdin` argument appends a trailing newline +by default (`stdin_add_newline` defaults to `true`), and +`--data-urlencode pt@-` does not strip it — every server was receiving +`"\n"` (URL-encoded, so a trailing `%0A`) as `pt`, a +different and invalid value, and correctly rejecting it. Fixed with +`stdin_add_newline: false` on that task. `connect_to_wireguard_with_token +.sh` never hits this because it passes the token as a literal shell +variable, not via stdin — the stdin delivery here is this repo's own +addition (to keep the token out of `ps`), so it needed the flag PIA's +reference script never had to think about. Left as a worked example +because "every candidate failed identically" pointing at one shared bug +in *this* code, not PIA, is exactly the diagnostic story this section +promises — and it happened to be true the first time it was tested for +real. + +## What it does + +- Registers minisforum as a PIA WireGuard peer — see "Region fallback" + above — and writes `/etc/wireguard/pia-wg.conf` (`Table = off` — this + role owns all routing for the interface, not wg-quick). +- Installs `pia-killswitch.service`, ordered `Before= + wg-quick@pia-wg.service`, that seeds a closed state at every boot (and + after any UFW reload — see handlers/main.yaml): the + `from 10.10.50.0/24 lookup pia` rule, a terminal `unreachable default` + route in the `pia` table (only when `pia-wg` isn't already up — see + the script's own comment for why), and a dedicated `PIA-VLAN50` + iptables chain jumped into by a single rule at the very top of + `FORWARD`, matching only source `10.10.50.0/24`, ending in an + unconditional rate-limited log+drop. This is deliberately **not** a + global `FORWARD` default-policy change — minisforum runs Flannel, + which needs its own broad `FORWARD` ACCEPTs for pod traffic (source + `10.42.0.0/16`, disjoint from VLAN 50) — so the kill switch is scoped + to a chain Flannel/k3s traffic can never enter, rather than risking a + chain-wide policy that was never actually proven safe against it. + Technitium (`10.10.40.53`, minisforum's own address) gets a normal UFW + **input** allow for 53/tcp+udp — it's locally-terminated traffic, not + routed — see `tasks/firewall.yaml`. +- `pia-wg.conf`'s own `PostUp`/`PreDown` open/close the narrower path on + top of that closed baseline: default route via `pia-wg` in the `pia` + table, ACCEPT + established/related return inserted into the + `PIA-VLAN50` chain (not `FORWARD` directly), and source-NAT/MASQUERADE + scoped to `10.10.50.0/24` on `pia-wg` only. +- Installs an observability-only health check (`pia-gateway-healthcheck + .timer`, every `pia_healthcheck_interval_sec`) that logs interface, + handshake age, rule/route, and firewall-policy state to the journal. It + never remediates — see the script's header comment for why, and for what + its "route-decision" check does and does not prove. + +## What it deliberately does not do + +- Does not touch minisforum's own default route (asserted at the start of + every run — `tasks/assert-baseline.yaml` fails loudly if that's already + wrong). +- Does not enable IPv6 forwarding or any IPv6 handling for VLAN 50. +- Does not enable MSS clamping (`pia_mss_clamp_enabled: false` by + default) — flip only after Phase 5 canary MTU testing shows it's + actually needed. +- Does not modify `ansible/roles/wireguard` (the separate `wg0` home-VPN + server role) or its interface. + +## Rollback + +No tag-based rollback is defined — this role doesn't have an "absent" +mode. Roll back manually on minisforum (reverses this role without +touching `wg0`, k3s, or the host default route): + +```bash +sudo systemctl disable --now wg-quick@pia-wg pia-killswitch.service pia-gateway-healthcheck.timer +sudo rm -f /etc/systemd/system/pia-killswitch.service \ + /etc/systemd/system/pia-gateway-healthcheck.service \ + /etc/systemd/system/pia-gateway-healthcheck.timer +sudo systemctl daemon-reload +sudo ip rule del from 10.10.50.0/24 table pia priority 100 +sudo ip route flush table pia +sudo sed -i '/^[0-9]\+\s\+pia$/d' /etc/iproute2/rt_tables +sudo iptables -D FORWARD -s 10.10.50.0/24 -j PIA-VLAN50 +sudo iptables -F PIA-VLAN50 +sudo iptables -X PIA-VLAN50 +sudo ufw delete allow from 10.10.50.0/24 to 10.10.40.53 port 53 proto tcp +sudo ufw delete allow from 10.10.50.0/24 to 10.10.40.53 port 53 proto udp +sudo rm -f /etc/wireguard/pia-wg.conf /etc/wireguard/pia-wg.key /etc/wireguard/pia-wg.key.new /etc/wireguard/pia-ca.crt +``` + +Inspection commands to confirm rollback actually took (read-only): + +```bash +sudo iptables -S FORWARD | grep PIA-VLAN50 # expect: no output +sudo iptables -L PIA-VLAN50 # expect: "iptables: No chain/target/match by that name" +sudo ufw status verbose | grep 10.10.50 # expect: no output +ip rule show | grep pia # expect: no output +ip route show table pia # expect: empty/error (table gone) +``` + +Does not remove `pia-credentials`/`pia-credentials-sealed.yaml` (the K8s +Secret used by the Gluetun sidecars) — that's a separate, unrelated +secret and rollback path. Does not touch `ansible/roles/common`'s own +UFW rules (Flannel pod-to-pod, pod-to-Technitium) — this role never +modified those. diff --git a/ansible/roles/pia-gateway/defaults/main.yaml b/ansible/roles/pia-gateway/defaults/main.yaml new file mode 100644 index 0000000..3c377a9 --- /dev/null +++ b/ansible/roles/pia-gateway/defaults/main.yaml @@ -0,0 +1,116 @@ +--- +# Part of role: pia-gateway +# Called by: ansible/playbooks/pia-gateway.yaml +# Description: Default variables for minisforum's PIA WireGuard egress +# gateway for VPN VLAN 50 — policy routing, kill switch, and health +# checks. See ~/repo/homelab/plan.md for the cross-repo contract this +# role implements (fixed values below match that contract). + +# PIA regions to try, in order, for WireGuard addKey registration — +# first server (in the first region that has any) to return +# `status: OK` wins. These are API region **ids** from PIA's serverlist +# (https://serverlist.piaservers.net/vpninfo/servers/v6), e.g. +# {"id": "japan", "name": "JP Tokyo", ...} — NOT Gluetun's display names +# ("JP Tokyo", "Hong Kong", "Taiwan") and not the slug you might guess +# from one (confirmed live 2026-08-24: Tokyo's id is "japan", not +# "jp_tokyo" — that matched zero regions on a first pass). Order matches +# the region set already used by the Gluetun sidecars, Hong Kong/Taiwan +# moved ahead of Tokyo after all three Tokyo servers were observed +# failing in practice — see tasks/register.yaml's header for what that +# means and doesn't mean. +pia_region_candidates: + - hk + - taiwan + - japan + +# Force a SINGLE region for troubleshooting, bypassing +# pia_region_candidates entirely — e.g. `-e pia_region=hk`. This is also +# an API id, not a display name; same rule as above. Leave null (the +# default) to use the ordered candidate list. tasks/register.yaml +# computes the actual list to try (pia_effective_regions) from whichever +# of these two is set — see that file's own comment for the exact logic, +# not duplicated here to avoid the two drifting out of sync. +pia_region: null + +# WireGuard interface identity. Deliberately not "wg0" — that name is +# already used by ansible/roles/wireguard, minisforum's separate *inbound* +# home-VPN server role. Colliding with it would tear down remote access. +pia_wg_interface: pia-wg +pia_wg_config_dir: /etc/wireguard + +# Re-registration behaviour — two independent, unambiguous knobs (an +# earlier single pia_force_reregister conflated "re-run the API calls" +# with "get a new keypair", which isn't the same thing and isn't safe to +# conflate — the key-generation task uses `creates:`, so a naive +# "force" only re-ran the PIA API calls with the SAME existing key, +# silently, unless you also knew to delete the key file yourself): +# +# - pia_force_reregister: re-run PIA registration (serverlist lookup, +# token, addKey) even if {{ pia_wg_interface }}.conf already exists. +# Reuses whatever keypair is already on disk (generates one only if +# none exists yet). Use this to pick up a new PIA-assigned peer/server +# without changing minisforum's own WireGuard identity. +# - pia_force_key_rotation: additionally generate a NEW keypair and +# register THAT with PIA (implies pia_force_reregister). Rotation is +# safe-by-construction: the new key is generated to a *.key.new path +# and only promoted (with the old key backed up alongside a timestamp) +# after PIA has actually accepted the new key and the new config has +# been written — see tasks/register.yaml. If registration fails +# partway, the previous key/config are left completely untouched +# rather than ending up in a broken half-state. +pia_force_reregister: false +pia_force_key_rotation: false + +# Dedicated policy-routing table for VLAN 50 egress. 0/253/254/255 are +# kernel-reserved (unspec/default/main/local) — this must stay clear of +# those and of any other custom table already registered on the host. +pia_route_table_id: 52 +pia_route_table_name: pia + +# Rule priority for "from 10.10.50.0/24 lookup pia". Must sort after the +# kernel's built-in local table (priority 0, always evaluated first, keeps +# traffic addressed to the host's own local/Technitium address delivered +# locally per plan.md) and before main (priority 32766). +pia_rule_priority: 100 + +# Fixed cross-repo contract (see plan.md "Fixed cross-repository +# contract") — VLAN 50 subnet and the Flint gateway that owns the return +# route back to VLAN 50. Not derived from live discovery; change only if +# the contract itself changes. +pia_vlan50_subnet: 10.10.50.0/24 +pia_flint_gateway: 10.10.50.1 + +# Dedicated iptables chain the kill switch governs, jumped into by a +# single rule at FORWARD position 1 matching only +# {{ pia_vlan50_subnet }} — deliberately not a global FORWARD default- +# policy change, so k3s/Flannel's own FORWARD-chain traffic (source +# 10.42.0.0/16, disjoint from VLAN 50) is never affected by this role at +# all. See templates/pia-killswitch.sh.j2 for the full reasoning. +pia_iptables_chain: PIA-VLAN50 + +# Real parent LAN interface on minisforum — verified live 2026-08-23 via +# `ip -brief addr`; it is enp1s0, not "eth0". Used for MASQUERADE-exclusion +# bookkeeping and documentation only, since NAT/forward rules below key off +# {{ pia_wg_interface }} rather than enumerating the normal interface. +pia_lan_interface: enp1s0 + +# Reverse-path filtering, applied to {{ pia_wg_interface }} only. The +# kernel default on minisforum (conf.default.rp_filter) is already loose +# (2) at time of writing, so new interfaces inherit loose mode — this task +# pins it explicitly rather than relying on that inherited default holding +# across future changes. Loose mode is required because PIA's return +# traffic can legitimately arrive with routing asymmetry relative to how +# minisforum would route toward that source itself. +pia_rp_filter_mode: 2 + +# MSS clamping on the pia-wg forward path — OFF by default. plan.md is +# explicit: do not guess this. Flip to true only after Phase 5 canary +# MTU/path-MTU testing (large-packet ping with DF set, across the +# VLAN 50 -> Flint -> minisforum -> pia-wg path) shows fragmentation or +# black-holing that clamping fixes. +pia_mss_clamp_enabled: false + +# Health check cadence and thresholds. The check only observes and logs — +# see tasks/healthcheck.yaml — it must never itself open the kill switch. +pia_healthcheck_interval_sec: 60 +pia_healthcheck_handshake_max_age_sec: 180 diff --git a/ansible/roles/pia-gateway/files/pia-ca.crt b/ansible/roles/pia-gateway/files/pia-ca.crt new file mode 100644 index 0000000..82dec69 --- /dev/null +++ b/ansible/roles/pia-gateway/files/pia-ca.crt @@ -0,0 +1,43 @@ +-----BEGIN CERTIFICATE----- +MIIHqzCCBZOgAwIBAgIJAJ0u+vODZJntMA0GCSqGSIb3DQEBDQUAMIHoMQswCQYD +VQQGEwJVUzELMAkGA1UECBMCQ0ExEzARBgNVBAcTCkxvc0FuZ2VsZXMxIDAeBgNV +BAoTF1ByaXZhdGUgSW50ZXJuZXQgQWNjZXNzMSAwHgYDVQQLExdQcml2YXRlIElu +dGVybmV0IEFjY2VzczEgMB4GA1UEAxMXUHJpdmF0ZSBJbnRlcm5ldCBBY2Nlc3Mx +IDAeBgNVBCkTF1ByaXZhdGUgSW50ZXJuZXQgQWNjZXNzMS8wLQYJKoZIhvcNAQkB +FiBzZWN1cmVAcHJpdmF0ZWludGVybmV0YWNjZXNzLmNvbTAeFw0xNDA0MTcxNzQw +MzNaFw0zNDA0MTIxNzQwMzNaMIHoMQswCQYDVQQGEwJVUzELMAkGA1UECBMCQ0Ex +EzARBgNVBAcTCkxvc0FuZ2VsZXMxIDAeBgNVBAoTF1ByaXZhdGUgSW50ZXJuZXQg +QWNjZXNzMSAwHgYDVQQLExdQcml2YXRlIEludGVybmV0IEFjY2VzczEgMB4GA1UE +AxMXUHJpdmF0ZSBJbnRlcm5ldCBBY2Nlc3MxIDAeBgNVBCkTF1ByaXZhdGUgSW50 +ZXJuZXQgQWNjZXNzMS8wLQYJKoZIhvcNAQkBFiBzZWN1cmVAcHJpdmF0ZWludGVy +bmV0YWNjZXNzLmNvbTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBALVk +hjumaqBbL8aSgj6xbX1QPTfTd1qHsAZd2B97m8Vw31c/2yQgZNf5qZY0+jOIHULN +De4R9TIvyBEbvnAg/OkPw8n/+ScgYOeH876VUXzjLDBnDb8DLr/+w9oVsuDeFJ9K +V2UFM1OYX0SnkHnrYAN2QLF98ESK4NCSU01h5zkcgmQ+qKSfA9Ny0/UpsKPBFqsQ +25NvjDWFhCpeqCHKUJ4Be27CDbSl7lAkBuHMPHJs8f8xPgAbHRXZOxVCpayZ2SND +fCwsnGWpWFoMGvdMbygngCn6jA/W1VSFOlRlfLuuGe7QFfDwA0jaLCxuWt/BgZyl +p7tAzYKR8lnWmtUCPm4+BtjyVDYtDCiGBD9Z4P13RFWvJHw5aapx/5W/CuvVyI7p +Kwvc2IT+KPxCUhH1XI8ca5RN3C9NoPJJf6qpg4g0rJH3aaWkoMRrYvQ+5PXXYUzj +tRHImghRGd/ydERYoAZXuGSbPkm9Y/p2X8unLcW+F0xpJD98+ZI+tzSsI99Zs5wi +jSUGYr9/j18KHFTMQ8n+1jauc5bCCegN27dPeKXNSZ5riXFL2XX6BkY68y58UaNz +meGMiUL9BOV1iV+PMb7B7PYs7oFLjAhh0EdyvfHkrh/ZV9BEhtFa7yXp8XR0J6vz +1YV9R6DYJmLjOEbhU8N0gc3tZm4Qz39lIIG6w3FDAgMBAAGjggFUMIIBUDAdBgNV +HQ4EFgQUrsRtyWJftjpdRM0+925Y6Cl08SUwggEfBgNVHSMEggEWMIIBEoAUrsRt +yWJftjpdRM0+925Y6Cl08SWhge6kgeswgegxCzAJBgNVBAYTAlVTMQswCQYDVQQI +EwJDQTETMBEGA1UEBxMKTG9zQW5nZWxlczEgMB4GA1UEChMXUHJpdmF0ZSBJbnRl +cm5ldCBBY2Nlc3MxIDAeBgNVBAsTF1ByaXZhdGUgSW50ZXJuZXQgQWNjZXNzMSAw +HgYDVQQDExdQcml2YXRlIEludGVybmV0IEFjY2VzczEgMB4GA1UEKRMXUHJpdmF0 +ZSBJbnRlcm5ldCBBY2Nlc3MxLzAtBgkqhkiG9w0BCQEWIHNlY3VyZUBwcml2YXRl +aW50ZXJuZXRhY2Nlc3MuY29tggkAnS7684Nkme0wDAYDVR0TBAUwAwEB/zANBgkq +hkiG9w0BAQ0FAAOCAgEAJsfhsPk3r8kLXLxY+v+vHzbr4ufNtqnL9/1Uuf8NrsCt +pXAoyZ0YqfbkWx3NHTZ7OE9ZRhdMP/RqHQE1p4N4Sa1nZKhTKasV6KhHDqSCt/dv +Em89xWm2MVA7nyzQxVlHa9AkcBaemcXEiyT19XdpiXOP4Vhs+J1R5m8zQOxZlV1G +tF9vsXmJqWZpOVPmZ8f35BCsYPvv4yMewnrtAC8PFEK/bOPeYcKN50bol22QYaZu +LfpkHfNiFTnfMh8sl/ablPyNY7DUNiP5DRcMdIwmfGQxR5WEQoHL3yPJ42LkB5zs +6jIm26DGNXfwura/mi105+ENH1CaROtRYwkiHb08U6qLXXJz80mWJkT90nr8Asj3 +5xN2cUppg74nG3YVav/38P48T56hG1NHbYF5uOCske19F6wi9maUoto/3vEr0rnX +JUp2KODmKdvBI7co245lHBABWikk8VfejQSlCtDBXn644ZMtAdoxKNfR2WTFVEwJ +iyd1Fzx0yujuiXDROLhISLQDRjVVAvawrAtLZWYK31bY7KlezPlQnl/D9Asxe85l +8jO5+0LdJ6VyOs/Hd4w52alDW/MFySDZSfQHMTIc30hLBJ8OnCEIvluVQQ2UQvoW ++no177N9L2Y+M9TcTA62ZyMXShHQGeh20rb4kK8f+iFX8NxtdHVSkxMEFSfDDyQ= +-----END CERTIFICATE----- diff --git a/ansible/roles/pia-gateway/handlers/main.yaml b/ansible/roles/pia-gateway/handlers/main.yaml new file mode 100644 index 0000000..f99c132 --- /dev/null +++ b/ansible/roles/pia-gateway/handlers/main.yaml @@ -0,0 +1,46 @@ +--- +# Part of role: pia-gateway +# Handlers fire in the order listed here (not the order notified): +# 1. systemd daemon reload must stay first so unit-file content changes +# are picked up before any restart handler below acts on them. +# 2. "Reload ufw" must come before "Restart pia-killswitch" — a UFW +# reload rebuilds ufw's own chains and, depending on how it's +# triggered, can in principle disturb rules inserted directly into +# the base FORWARD chain (our position-1 jump to +# {{ pia_iptables_chain }}) outside ufw's own bookkeeping. Restarting +# pia-killswitch immediately after re-establishes that jump and the +# chain's contents idempotently, closing the window rather than +# leaving it open until the next reboot. This only protects against +# ufw reloads that happen *through this Ansible run* (i.e., a task +# here notifies both) — a `ufw reload` run manually and separately, +# outside of Ansible entirely, is not something a handler can react +# to; that residual gap is closed only by every real boot re-running +# pia-killswitch.service fresh (WantedBy=multi-user.target). +# 3. "Restart pia-killswitch" must come before "Restart pia-wg" — if +# both fire together (e.g. pia-wg's config changed AND the +# killswitch script changed in the same run), pia-wg's own PostUp +# needs to run last so its "open" route is the final word, not +# killswitch's route-table seeding. + +- name: Reload systemd daemon + ansible.builtin.systemd: + daemon_reload: true + +- name: Reload ufw + community.general.ufw: + state: reloaded + +- name: Restart pia-killswitch + ansible.builtin.systemd: + name: pia-killswitch.service + state: restarted + +- name: Restart pia-wg + ansible.builtin.systemd: + name: "wg-quick@{{ pia_wg_interface }}" + state: restarted + +- name: Restart pia-gateway-healthcheck timer + ansible.builtin.systemd: + name: pia-gateway-healthcheck.timer + state: restarted diff --git a/ansible/roles/pia-gateway/tasks/addkey-attempt.yaml b/ansible/roles/pia-gateway/tasks/addkey-attempt.yaml new file mode 100644 index 0000000..9fae8cd --- /dev/null +++ b/ansible/roles/pia-gateway/tasks/addkey-attempt.yaml @@ -0,0 +1,179 @@ +--- +# Part of role: pia-gateway +# Included by: tasks/register.yaml, once per candidate in pia_candidates +# (loop_var: pia_candidate — a flattened {region_id, region_name, cn, ip} +# dict, one server, built by register.yaml via subelements over every +# requested region in order), in order, until one succeeds. +# +# Every meaningful task below is individually guarded with +# `when: pia_peer is not defined` — confirmed empirically (not assumed) +# that putting this guard only on the include statement in register.yaml +# does NOT stop later loop iterations from running; ansible re-evaluates +# a `when` on tasks inside an included file per-iteration, but does not +# re-evaluate a `when` on the include statement itself the same way. The +# guard has to live here, on every task, or a candidate that comes up +# after an earlier one already worked would still get hit. + +- name: "Reset per-attempt response state for {{ pia_candidate.cn }}" + ansible.builtin.set_fact: + pia_addkey_response: {} + pia_addkey_rc: -1 + pia_addkey_stderr: "" + pia_addkey_parsed: null + pia_addkey_diagnosis: "" + when: pia_peer is not defined + # Explicit reset, every iteration, before this candidate's own curl + # call runs — a prior version of this file only ever *conditionally* + # set pia_addkey_parsed (rc == 0 and non-empty stdout), so a candidate + # whose curl call failed outright (rc != 0) would skip that task and + # silently keep the PREVIOUS candidate's parsed response sitting in + # pia_addkey_parsed. That's not just cosmetic: it means a stale + # {"status": "..."} from an earlier failure could read as if it came + # from the current candidate in diagnostics, or — if the guard logic + # were ever refactored slightly differently — actually be accepted for + # the wrong server. Resetting to a known-empty/null state here, every + # time, makes each iteration self-contained regardless of what + # happened before it. + +- name: "Register WireGuard key with server {{ pia_candidate.cn }}" + ansible.builtin.command: + cmd: >- + curl --silent --show-error --fail --location -G + --connect-timeout 10 --max-time 30 + --cacert {{ pia_wg_config_dir }}/pia-ca.crt + --connect-to {{ pia_candidate.cn }}::{{ pia_candidate.ip }}: + --data-urlencode pt@- + --data-urlencode pubkey={{ pia_public_key.stdout }} + https://{{ pia_candidate.cn }}:1337/addKey + args: + stdin: "{{ pia_auth_token }}" + stdin_add_newline: false + register: pia_addkey_response + changed_when: false + failed_when: false + no_log: true + when: pia_peer is not defined + # Same request shape as PIA's own connect_to_wireguard_with_token.sh + # for every region — WireGuard hostname for TLS/SNI, its server IP via + # --connect-to, PIA's private CA (never -k/--insecure/validate_certs: + # false), port 1337, /addKey, generated pubkey. Token passed via stdin + # (--data-urlencode pt@-) rather than as a plain argv value — verified + # empirically (this session) that curl's argv, including a literal + # "pt@-", never the token itself, is what shows up in `ps`, so the + # token is never visible to another local user on the host for the + # process's lifetime. pubkey is not sensitive (public by design) and + # stays a normal argument. No curl --retry here — retries happen by + # moving to the next candidate in the loop instead, so one slow/dead + # server can't eat the whole attempt budget retrying itself before + # failover to the next server, or the next region, gets a chance. + # + # stdin_add_newline: false is load-bearing, not cosmetic. Ansible's + # command module appends a trailing newline to `stdin` by default + # (stdin_add_newline defaults to true) — confirmed live (this session) + # against a real run: every one of 7 servers across all 3 regions + # rejected the token with a uniform HTTP 401, and reproducing the exact + # same stdin mechanism against a plain echo endpoint showed why — + # `--data-urlencode pt@-` does not strip that trailing newline, so PIA + # received "\n" (URL-encoded, with a trailing %0A) as the + # pt value: a different, invalid string. This is what actually broke + # every region identically, not a credentials or connectivity problem + # in any of them. get_token.sh/connect_to_wireguard_with_token.sh never + # hit this because they pass the token as a literal shell variable + # (`"pt=${PIA_TOKEN}"`), not via stdin — the stdin delivery mechanism + # itself was introduced in this repo specifically to keep the token out + # of `ps` output, and needs this flag to behave the same as PIA's own + # reference flow. + +- name: "Capture safe diagnostics for {{ pia_candidate.cn }}" + ansible.builtin.set_fact: + pia_addkey_rc: "{{ pia_addkey_response.rc | default(-1) }}" + pia_addkey_stderr: "{{ pia_addkey_response.stderr | default('') }}" + when: pia_peer is not defined + # rc/stderr never contain the token or pubkey — same reasoning as the + # token request's diagnostics in register.yaml. Never derived from + # pia_addkey_response.cmd/.invocation (the literal argv, harmless here + # since the token isn't in it either, but still not something to print + # wholesale) — only the two specific safe fields are ever extracted. + +# --- Classify the outcome into one human-readable diagnosis. Each of +# these is mutually exclusive by its `when:`, and pia_addkey_diagnosis +# was reset to "" above, so exactly one (or zero, if this candidate was +# skipped because an earlier one already succeeded) ends up set. Curl +# exit codes confirmed against curl's own documented libcurl error list, +# not guessed — 28 is CURLE_OPERATION_TIMEDOUT, 22 is +# CURLE_HTTP_RETURNED_ERROR (only possible here because of --fail), and +# the TLS set is every CURLE_SSL_*/CURLE_PEER_FAILED_VERIFICATION code +# curl currently defines (35, 53, 54, 58, 59, 60, 66, 77, 80, 82, 83). + +- name: "Diagnose a connection timeout for {{ pia_candidate.cn }}" + ansible.builtin.set_fact: + pia_addkey_diagnosis: connection timeout + when: (pia_peer is not defined) and (pia_addkey_rc | int == 28) + +- name: "Diagnose a TLS validation failure for {{ pia_candidate.cn }}" + ansible.builtin.set_fact: + pia_addkey_diagnosis: TLS/SSL failure + when: (pia_peer is not defined) and (pia_addkey_rc | int in [35, 53, 54, 58, 59, 60, 66, 77, 80, 82, 83]) + +- name: "Diagnose an HTTP failure for {{ pia_candidate.cn }}" + ansible.builtin.set_fact: + pia_addkey_diagnosis: HTTP failure (non-2xx response) + when: (pia_peer is not defined) and (pia_addkey_rc | int == 22) + +- name: "Diagnose an unclassified curl error for {{ pia_candidate.cn }}" + ansible.builtin.set_fact: + pia_addkey_diagnosis: "curl error (exit {{ pia_addkey_rc }})" + when: (pia_peer is not defined) and (pia_addkey_rc | int not in [0, 22, 28, 35, 53, 54, 58, 59, 60, 66, 77, 80, 82, 83]) + +- name: "Parse the addKey response from {{ pia_candidate.cn }}" + ansible.builtin.set_fact: + pia_addkey_parsed: "{{ pia_addkey_response.stdout | from_json }}" + when: (pia_peer is not defined) and (pia_addkey_rc | int == 0) and (pia_addkey_response.stdout | trim | length > 0) + ignore_errors: true + # ignore_errors here on purpose: a genuinely malformed body from one + # misbehaving server must not abort the whole play — it should just + # leave pia_addkey_parsed at the null it was reset to above, so this + # candidate is diagnosed as malformed (below) and the loop moves on. + +- name: "Diagnose an empty response for {{ pia_candidate.cn }}" + ansible.builtin.set_fact: + pia_addkey_diagnosis: empty response body + when: (pia_peer is not defined) and (pia_addkey_rc | int == 0) and (pia_addkey_response.stdout | trim | length == 0) + +- name: "Diagnose a malformed JSON response for {{ pia_candidate.cn }}" + ansible.builtin.set_fact: + pia_addkey_diagnosis: malformed JSON response + when: (pia_peer is not defined) and (pia_addkey_rc | int == 0) and (pia_addkey_response.stdout | trim | length > 0) and (pia_addkey_parsed is none) + +- name: "Diagnose a non-OK status for {{ pia_candidate.cn }}" + ansible.builtin.set_fact: + pia_addkey_diagnosis: "parsed OK but status is '{{ pia_addkey_parsed.status | default('missing') }}', not OK" + when: (pia_peer is not defined) and (pia_addkey_parsed is not none) and ((pia_addkey_parsed.status | default('')) != 'OK') + +- name: "Log the attempt outcome for {{ pia_candidate.cn }}" + ansible.builtin.debug: + msg: >- + region={{ pia_candidate.region_name }} ({{ pia_candidate.region_id }}) + server={{ pia_candidate.cn }} ip={{ pia_candidate.ip }} + rc={{ pia_addkey_rc }} reason=[{{ pia_addkey_diagnosis }}] + stderr=[{{ pia_addkey_stderr }}] + status={{ pia_addkey_parsed.status | default('n/a') }} + when: (pia_peer is not defined) and (pia_addkey_diagnosis | length > 0) + # Deliberately never includes: the token, username/password, the full + # curl invocation/cmd array, request stdin, the complete response body + # (only its parsed .status field, when parsing succeeded), or any key + # material beyond what pubkey already is (public, safe by design). + +- name: "Accept the response from {{ pia_candidate.cn }}" + ansible.builtin.set_fact: + pia_peer: "{{ pia_addkey_parsed }}" + pia_region_used: "{{ pia_candidate.region_id }}" + pia_region_name_used: "{{ pia_candidate.region_name }}" + pia_wg_server_used: "{{ pia_candidate }}" + when: (pia_peer is not defined) and (pia_addkey_parsed is not none) and ((pia_addkey_parsed.status | default('')) == 'OK') + # Deliberately checks status == OK here, not just "did it parse" — a + # response that parses but isn't actually OK must not stop the loop + # from trying the remaining candidates. Once this fires, every + # subsequent iteration's tasks are no-ops (all guarded on + # `pia_peer is not defined`), so pia_region_used/pia_wg_server_used are + # never overwritten by a later candidate. diff --git a/ansible/roles/pia-gateway/tasks/assert-baseline.yaml b/ansible/roles/pia-gateway/tasks/assert-baseline.yaml new file mode 100644 index 0000000..aadb5a5 --- /dev/null +++ b/ansible/roles/pia-gateway/tasks/assert-baseline.yaml @@ -0,0 +1,34 @@ +--- +# Part of role: pia-gateway +# Included by: tasks/main.yaml +# Description: Enables IP forwarding persistently (required for minisforum +# to route VLAN 50 traffic at all) and asserts the host's own default +# route is still the normal LAN gateway before any PIA-specific routing +# is touched. This is the guard behind "never change minisforum's normal +# default route globally" — it fails the play loudly instead of layering +# PIA routing on top of an already-wrong baseline. + +- name: Enable IP forwarding persistently + ansible.posix.sysctl: + name: net.ipv4.ip_forward + value: "1" + sysctl_set: true + state: present + reload: true + +- name: Read current default route + ansible.builtin.command: ip -4 route show default + register: pia_current_default_route + changed_when: false + +- name: Assert the host default route is still the normal LAN interface + ansible.builtin.assert: + that: + - pia_current_default_route.stdout is search('dev ' + pia_lan_interface) + - not (pia_current_default_route.stdout is search('dev ' + pia_wg_interface)) + fail_msg: >- + minisforum's default route is not via {{ pia_lan_interface }} + ({{ pia_current_default_route.stdout }}). Refusing to continue — + this role must never run against a host whose own default route + has already been changed. + success_msg: "Host default route confirmed via {{ pia_lan_interface }}" diff --git a/ansible/roles/pia-gateway/tasks/firewall.yaml b/ansible/roles/pia-gateway/tasks/firewall.yaml new file mode 100644 index 0000000..7532d73 --- /dev/null +++ b/ansible/roles/pia-gateway/tasks/firewall.yaml @@ -0,0 +1,66 @@ +--- +# Part of role: pia-gateway +# Included by: tasks/main.yaml +# Description: The one thing this file does now: permit VLAN 50 clients +# to reach the local Technitium listener on minisforum. Everything +# else the kill switch needs (the dedicated FORWARD-chain scoping, the +# routing-table unreachable fallback) lives in tasks/routing.yaml / +# templates/pia-killswitch.sh.j2 — see those for the actual fail-closed +# layers. +# +# Two corrections from an earlier version of this file, both from code +# review against the live topology rather than assumption: +# +# 1. Technitium (10.10.40.53) IS minisforum's own address — traffic from +# a VLAN 50 client to it terminates LOCALLY on minisforum. That's +# INPUT traffic, not FORWARD/routed traffic, so it needs a normal UFW +# input allow, not `route: true` (which adds a FORWARD-chain rule for +# traffic passing *through* the host to some other destination — the +# wrong chain for a locally-terminated flow, and it would never +# actually have matched real Technitium queries). +# +# 2. This file used to also force UFW's DEFAULT_FORWARD_POLICY to DROP, +# intended as a second kill-switch layer. That was never actually +# proven safe: minisforum is the k3s server and runs Flannel, which +# depends on the FORWARD chain for pod-to-pod and pod-to-internet +# traffic (source 10.42.0.0/16) — changing the chain-wide default +# policy risked breaking that, for a benefit (fail-closed for VLAN 50 +# specifically) that's already fully covered by the dedicated +# {{ pia_iptables_chain }} chain in pia-killswitch.sh.j2, which is +# scoped to source {{ pia_vlan50_subnet }} only and can structurally +# never see Flannel's traffic (disjoint source CIDR). Removed rather +# than shipped unproven. +# +# Inspection commands (read-only): +# sudo iptables -S FORWARD | head -3 # confirm the single +# # jump to {{ pia_iptables_chain }} +# # at position 1, and that +# # the chain-wide policy +# # is still whatever it +# # was before this role +# # (ACCEPT, typically) +# sudo iptables -S {{ pia_iptables_chain }} # the kill-switch chain +# # itself: ACCEPT pair +# # when pia-wg is up, +# # LOG+DROP catch-all +# # always present +# sudo ufw status verbose | grep 53 # the Technitium INPUT +# # allow, both protocols +# +# Rollback (this file's part only — see routing.yaml/README.md for the +# rest of the role's rollback): +# sudo ufw delete allow from 10.10.50.0/24 to 10.10.40.53 port 53 proto tcp +# sudo ufw delete allow from 10.10.50.0/24 to 10.10.40.53 port 53 proto udp + +- name: Allow VLAN 50 clients to reach Technitium DNS (local INPUT traffic) + community.general.ufw: + rule: allow + direction: in + src: "{{ pia_vlan50_subnet }}" + dest: "{{ k3s_dns_resolver }}" + port: "53" + proto: "{{ item }}" + comment: VLAN 50 clients to Technitium (local, not routed) + loop: + - tcp + - udp diff --git a/ansible/roles/pia-gateway/tasks/healthcheck.yaml b/ansible/roles/pia-gateway/tasks/healthcheck.yaml new file mode 100644 index 0000000..8706c8a --- /dev/null +++ b/ansible/roles/pia-gateway/tasks/healthcheck.yaml @@ -0,0 +1,42 @@ +--- +# Part of role: pia-gateway +# Included by: tasks/main.yaml +# Description: Installs the observability-only health check (script, +# oneshot service, timer). See templates/pia-gateway-healthcheck.sh.j2 +# for exactly what it checks and why it never remediates. + +- name: Deploy the health check script + ansible.builtin.template: + src: pia-gateway-healthcheck.sh.j2 + dest: /usr/local/sbin/pia-gateway-healthcheck.sh + mode: "0700" + owner: root + group: root + +- name: Deploy the health check systemd service + ansible.builtin.template: + src: pia-gateway-healthcheck.service.j2 + dest: /etc/systemd/system/pia-gateway-healthcheck.service + mode: "0644" + owner: root + group: root + notify: + - Reload systemd daemon + - Restart pia-gateway-healthcheck timer + +- name: Deploy the health check systemd timer + ansible.builtin.template: + src: pia-gateway-healthcheck.timer.j2 + dest: /etc/systemd/system/pia-gateway-healthcheck.timer + mode: "0644" + owner: root + group: root + notify: + - Reload systemd daemon + - Restart pia-gateway-healthcheck timer + +- name: Enable and start the health check timer + ansible.builtin.systemd: + name: pia-gateway-healthcheck.timer + enabled: true + state: started diff --git a/ansible/roles/pia-gateway/tasks/interface.yaml b/ansible/roles/pia-gateway/tasks/interface.yaml new file mode 100644 index 0000000..af7ba83 --- /dev/null +++ b/ansible/roles/pia-gateway/tasks/interface.yaml @@ -0,0 +1,32 @@ +--- +# Part of role: pia-gateway +# Included by: tasks/main.yaml +# Description: Enables and starts the pia-wg WireGuard interface itself. +# The config file was already written by tasks/register.yaml (only on +# first run / forced re-registration); this task just makes sure the +# systemd unit is enabled and running, and that pia-killswitch.service +# (tasks/routing.yaml) has already run first — both are WantedBy +# multi-user.target with an explicit Before= from the kill switch, so a +# normal boot orders them correctly without help here, but a first +# Ansible run on an already-booted host needs the same ordering applied +# explicitly since neither unit was part of a boot transaction yet. + +- name: Confirm the kill switch is active before bringing up pia-wg + ansible.builtin.systemd: + name: pia-killswitch.service + register: pia_killswitch_status + failed_when: false + +- name: Assert the kill switch is active + ansible.builtin.assert: + that: + - pia_killswitch_status.status.ActiveState == "active" + fail_msg: >- + pia-killswitch.service is not active — refusing to start + {{ pia_wg_interface }} without the kill switch already seeded. + +- name: Enable and start wg-quick@{{ pia_wg_interface }} + ansible.builtin.systemd: + name: "wg-quick@{{ pia_wg_interface }}" + enabled: true + state: started diff --git a/ansible/roles/pia-gateway/tasks/main.yaml b/ansible/roles/pia-gateway/tasks/main.yaml new file mode 100644 index 0000000..b50f12a --- /dev/null +++ b/ansible/roles/pia-gateway/tasks/main.yaml @@ -0,0 +1,53 @@ +--- +# Part of role: pia-gateway +# Called by: ansible/playbooks/pia-gateway.yaml +# Description: Orchestrates minisforum's PIA WireGuard egress gateway for +# VPN VLAN 50 — registration, boot-ordered kill switch + policy routing, +# the WireGuard interface itself, and health checks. Split into included +# files because each stage has a distinct idempotency/ordering concern; +# see the comment at the top of each included file. + +- name: Install WireGuard tooling and iproute2 + ansible.builtin.apt: + name: + - wireguard + - wireguard-tools + - iproute2 + - python3-requests + state: present + update_cache: true + +- name: Pin reverse-path filtering for {{ pia_wg_interface }} + ansible.posix.sysctl: + name: "net.ipv4.conf.{{ pia_wg_interface }}.rp_filter" + value: "{{ pia_rp_filter_mode | string }}" + sysctl_file: /etc/sysctl.d/60-pia-gateway.conf + reload: false # interface does not exist yet on a first run; applied for real after the WG interface comes up (handlers/main.yaml) + register: pia_rp_filter_pinned + +- name: Confirm host default route and ip_forward are untouched + ansible.builtin.include_tasks: assert-baseline.yaml + +- name: Register minisforum with PIA and obtain WireGuard peer config + ansible.builtin.include_tasks: register.yaml + no_log: true # PIA credentials and the derived private key pass through this include + +- name: Configure boot-ordered kill switch and policy routing + ansible.builtin.include_tasks: routing.yaml + +- name: Configure firewall-layer kill switch and Technitium access + ansible.builtin.include_tasks: firewall.yaml + +- name: Write and enable the pia-wg WireGuard interface + ansible.builtin.include_tasks: interface.yaml + +- name: Install gateway health checks + ansible.builtin.include_tasks: healthcheck.yaml + +- name: Re-apply rp_filter now that {{ pia_wg_interface }} exists + ansible.posix.sysctl: + name: "net.ipv4.conf.{{ pia_wg_interface }}.rp_filter" + value: "{{ pia_rp_filter_mode | string }}" + sysctl_file: /etc/sysctl.d/60-pia-gateway.conf + reload: true + when: pia_rp_filter_pinned is succeeded diff --git a/ansible/roles/pia-gateway/tasks/register.yaml b/ansible/roles/pia-gateway/tasks/register.yaml new file mode 100644 index 0000000..44c699c --- /dev/null +++ b/ansible/roles/pia-gateway/tasks/register.yaml @@ -0,0 +1,383 @@ +--- +# Part of role: pia-gateway +# Included by: tasks/main.yaml (with no_log: true — this file handles the +# PIA account password, the derived WireGuard private key, and the +# short-lived PIA auth token) +# Description: Registers minisforum as a PIA WireGuard peer and writes +# {{ pia_wg_config_dir }}/{{ pia_wg_interface }}.conf. Skipped once that +# file exists, unless pia_force_reregister or pia_force_key_rotation is +# set — see defaults/main.yaml for exactly what each does. +# +# Status as of 2026-08-24 (third pass — multi-region fallback added after +# all three JP Tokyo servers were observed failing live, one after +# another, against the single-region version of this file): +# - Token acquisition (`Request a PIA auth token` below) was completely +# rebuilt after live testing against the *actual* PIA API showed the +# first version's endpoint (a regional meta server's +# /authv3/generateToken, with PIA's private CA) doesn't exist as +# documented — it's unreachable/wrong. The correct, current flow +# (confirmed against PIA's own pia-foss/manual-connections +# get_token.sh, fetched and read in full, not from memory) is a +# single fixed POST to www.privateinternetaccess.com — a normal +# public domain with a normal publicly-trusted cert — with +# username/password as multipart form fields. It does not depend on +# pia_region/pia_region_candidates at all; only WireGuard server +# SELECTION does. +# - addKey (WireGuard key registration) was already close to correct on +# the first pass and is confirmed against +# connect_to_wireguard_with_token.sh, fetched and read in full: a +# regional server IP with SNI pinned to its hostname via +# --connect-to, validated against PIA's own CA bundle (still needed +# here — only the token endpoint uses the system CA store), token via +# stdin not argv. Failing over across only the servers *within* one +# region turned out not to be enough in practice — all three JP Tokyo +# servers failed the same way, one after another. This version tries +# every server in every configured region, in order (pia_region_ +# candidates, or a single pia_region override — see defaults/ +# main.yaml), not just every server within a single fixed region. The +# underlying request shape sent to each server is unchanged from the +# previous pass (still connect_to_wireguard_with_token.sh's flow) — +# if Hong Kong and Taiwan fail identically to how Tokyo did, that +# points at something shared across every region (the request itself, +# credentials, or minisforum's own network path), not at Tokyo +# specifically, and the per-attempt diagnostics below are what +# distinguishes those cases from three-independent-regions-all-down. +# - serverlist fetch + parse: unchanged, exercised live and working. + +- name: Determine the effective list of PIA regions to try + ansible.builtin.set_fact: + pia_effective_regions: "{{ [pia_region] if (pia_region is not none and (pia_region | trim | length) > 0) else pia_region_candidates }}" + # A single -e pia_region= override takes over completely (for + # troubleshooting one region in isolation) rather than being prepended + # to the candidate list — mixing the two would make "did it use my + # override or fall through to the list" ambiguous. + +- name: Assert the effective region list is usable + ansible.builtin.assert: + that: + - pia_effective_regions is defined + - pia_effective_regions | length > 0 + - pia_effective_regions | select('none') | list | length == 0 + - pia_effective_regions | map('string') | map('trim') | select('equalto', '') | list | length == 0 + fail_msg: >- + pia_region_candidates (or a single -e pia_region override) resolved + to an empty or invalid region list: + {{ pia_effective_regions | default('undefined') }}. Set + pia_region_candidates in defaults/main.yaml to a nonempty list of + PIA region ids (not display names), or pass -e pia_region= for + a single region. + +- name: Check whether a PIA WireGuard config already exists + ansible.builtin.stat: + path: "{{ pia_wg_config_dir }}/{{ pia_wg_interface }}.conf" + register: pia_wg_conf_stat + +- name: PIA registration + when: pia_force_reregister or pia_force_key_rotation or not pia_wg_conf_stat.stat.exists + block: + - name: Ensure WireGuard config directory exists + ansible.builtin.file: + path: "{{ pia_wg_config_dir }}" + state: directory + mode: "0700" + owner: root + group: root + + - name: Determine which private key path this run will use + ansible.builtin.set_fact: + pia_active_key_path: "{{ (pia_wg_config_dir + '/' + pia_wg_interface + '.key.new') if pia_force_key_rotation else (pia_wg_config_dir + '/' + pia_wg_interface + '.key') }}" + # Rotation stages the new key at a separate *.key.new path and only + # promotes it (see the "Promote the rotated key" block at the end + # of this file) after PIA has accepted it AND the new config has + # been written successfully — the previous key/config are never + # touched until both of those have actually succeeded, so a failed + # rotation leaves the working gateway exactly as it was rather + # than half-migrated. + + - name: Generate minisforum's PIA WireGuard private key for this run + ansible.builtin.shell: | + set -euo pipefail + umask 077 + wg genkey > {{ pia_active_key_path }} + args: + creates: "{{ pia_active_key_path }}" + + - name: Set permissions on the active private key + ansible.builtin.file: + path: "{{ pia_active_key_path }}" + mode: "0600" + owner: root + group: root + + - name: Read the active private key + ansible.builtin.slurp: + src: "{{ pia_active_key_path }}" + register: pia_private_key_raw + + - name: Derive the public key + ansible.builtin.command: wg pubkey + args: + stdin: "{{ pia_private_key_raw.content | b64decode | trim }}" + register: pia_public_key + changed_when: false + + - name: Fetch PIA's region/server list + ansible.builtin.uri: + url: https://serverlist.piaservers.net/vpninfo/servers/v6 + return_content: true + register: pia_serverlist_raw + # The response body is one line of minified JSON followed by a + # detached signature block. Take just the first line (matching the + # standard `head -1` extraction used elsewhere for this endpoint) — + # do NOT split on a blank-line separator: that assumption was wrong + # in practice (observed failing against the live response) and + # silently handed the whole blob, signature included, to from_json. + + - name: Parse the full region list + # Deliberately one physical line, not a folded (>-) block: an + # earlier version split a similar expression across two lines with + # the second more indented than the first, which YAML's folding + # rule ("more-indented lines are not folded") turns into a literal + # embedded newline instead of a space — that corrupted from_json's + # view of the string in practice (confirmed against the live + # endpoint) even though the newline sits outside any bracket. Keep + # this on one line if it's ever touched again. + ansible.builtin.set_fact: + pia_all_regions: "{{ (pia_serverlist_raw.content.split('\n')[0] | trim | from_json)['regions'] }}" + + - name: Look up each requested region's metadata, in order + ansible.builtin.set_fact: + pia_region_lookup: "{{ (pia_region_lookup | default([])) + [pia_match] }}" + loop: "{{ pia_effective_regions }}" + loop_control: + loop_var: pia_region_id + vars: + pia_match_list: "{{ pia_all_regions | selectattr('id', 'equalto', pia_region_id) | list }}" + pia_match: "{{ pia_match_list[0] if (pia_match_list | length > 0) else {'id': pia_region_id, 'name': None, 'servers': {'wg': []}} }}" + # Builds pia_region_lookup as one entry per requested region, in + # the SAME order as pia_effective_regions (not the API's own region + # order) — that order is what makes "try hk, then taiwan, then + # japan" actually mean that. A region that doesn't exist in the + # live serverlist gets a sentinel entry (name: null, no wg + # servers) rather than being silently skipped, so it can be + # reported precisely below instead of just quietly trying one + # fewer region than requested. + + - name: Identify any requested regions that don't exist in the current serverlist + ansible.builtin.set_fact: + pia_missing_regions: "{{ pia_region_lookup | selectattr('name', 'none') | map(attribute='id') | list }}" + + - name: Identify any requested regions with no WireGuard servers + ansible.builtin.set_fact: + pia_empty_wg_regions: "{{ pia_region_lookup | rejectattr('name', 'none') | rejectattr('servers.wg') | map(attribute='id') | list }}" + + - name: Assert every requested region exists and offers WireGuard + ansible.builtin.assert: + that: + - pia_missing_regions | length == 0 + - pia_empty_wg_regions | length == 0 + fail_msg: >- + One or more requested PIA regions are unusable — check each id + against the 'id' field of + https://serverlist.piaservers.net/vpninfo/servers/v6 (these are + API ids, not Gluetun's display names like "JP Tokyo"). Not + found in the current serverlist at all: + {{ pia_missing_regions | default([]) }}. Found but offer no + WireGuard servers right now: + {{ pia_empty_wg_regions | default([]) }}. Requested (in order): + {{ pia_effective_regions }}. Not silently substituting a + different region — fix pia_region_candidates/pia_region and + rerun. + + - name: Build the ordered, flattened list of region+server candidates + ansible.builtin.set_fact: + pia_candidates: "{{ (pia_candidates | default([])) + [{'region_id': pia_pair.0.id, 'region_name': pia_pair.0.name, 'cn': pia_pair.1.cn, 'ip': pia_pair.1.ip}] }}" + loop: "{{ pia_region_lookup | subelements('servers.wg') }}" + loop_control: + loop_var: pia_pair + # subelements preserves both list orders: outer (region, matching + # pia_effective_regions) and inner (server, matching whatever order + # the live API returned for that region — never hardcoded here). + # Each flattened candidate keeps region id, region display name, + # WireGuard hostname, and WireGuard IP together, so + # addkey-attempt.yaml and diagnostics never need to re-look-up the + # parent region for a given server. + + - name: Assert at least one candidate server was found + ansible.builtin.assert: + that: + - pia_candidates | length > 0 + fail_msg: >- + No WireGuard servers found across any requested region + ({{ pia_effective_regions }}) — this should be unreachable given + the region-level assert above; if you see this, the serverlist + response shape has likely changed. + + - name: Deploy PIA's CA bundle (WireGuard addKey endpoint only) + ansible.builtin.copy: + src: pia-ca.crt + dest: "{{ pia_wg_config_dir }}/pia-ca.crt" + mode: "0644" + owner: root + group: root + # Only the regional addKey endpoint (1337/addKey, below) needs + # this — it presents a cert chain that does not validate against a + # normal system CA store (confirmed live: curl exit 60, "unable to + # get local issuer certificate"), and PIA's own tooling pins this + # exact bundle rather than trusting the system store. The token + # endpoint (www.privateinternetaccess.com, below) is a normal + # public domain with a normal publicly-trusted cert and uses + # validate_certs: true / the system store — do not add --cacert or + # validate_certs: false there. + # + # Source: https://raw.githubusercontent.com/pia-foss/manual-connections/master/ca.rsa.4096.crt + # Re-fetched fresh and diffed byte-for-byte identical against the + # vendored copy on 2026-08-24. + # File SHA-256 (sha256sum of the .crt file's bytes — this is what + # to compare against a fresh `curl ... | sha256sum` to verify the + # vendored copy, NOT the same thing as the X.509 certificate + # fingerprint below): + # 32e9b1d1433ea97614f2a14c6e358e3f57c0570cc9f6b2ee812699ba696c66ab + # X.509 certificate fingerprint (openssl x509 -noout -fingerprint + # -sha256 — a hash of the DER-encoded certificate structure, a + # different value from the file SHA-256 above; useful for + # comparing against a cert viewed some other way, e.g. in a + # browser or `openssl s_client`): + # SHA256 Fingerprint=1F:D2:56:58:45:6E:AB:30:41:FB:A7:7C:CD:39:8A:B8:12:4E:DC:C1:B8:B2:FC:1D:55:FD:F6:B1:BB:FC:9D:70 + + - name: Request a PIA auth token + ansible.builtin.uri: + url: https://www.privateinternetaccess.com/api/client/v2/token + method: POST + body_format: form-multipart + body: + username: "{{ pia_user }}" + password: "{{ pia_password }}" + validate_certs: true + return_content: true + timeout: 15 + register: pia_token_response + until: pia_token_response.status | default(0) == 200 + retries: 3 + delay: 5 + failed_when: false + no_log: true + # Fixed public endpoint, system CA validation — see get_token.sh. + # Independent of pia_region entirely. + + - name: Capture safe (non-credential) diagnostics from the token request + ansible.builtin.set_fact: + pia_token_status: "{{ pia_token_response.status | default(-1) }}" + pia_token_msg: "{{ pia_token_response.msg | default('') }}" + # Deliberately NOT no_log. .status (an HTTP code) and .msg (Ansible's + # own generated "Status code was N and not [200]: ..." text) never + # contain the credential, the token, or the request body — verified + # empirically before use here, not assumed. What DOES contain + # secrets and stays behind no_log: pia_token_response itself (its + # .json/.content on a 200 IS the token; its .invocation.module_args + # is the literal username/password), and pia_user/pia_password. + + - name: Fail clearly if the auth token request failed + ansible.builtin.fail: + msg: >- + PIA auth token request failed: HTTP {{ pia_token_status }} + ({{ pia_token_msg }}) from + www.privateinternetaccess.com/api/client/v2/token + when: pia_token_status | int != 200 + + - name: Extract the auth token + ansible.builtin.set_fact: + pia_auth_token: "{{ pia_token_response.json.token | default('') }}" + no_log: true + + - name: Assert the token is non-empty + ansible.builtin.assert: + that: + - pia_auth_token | length > 0 + fail_msg: "PIA token response (HTTP 200) did not contain a non-empty token" + + - name: Attempt WireGuard key registration against each region/server candidate, in order + ansible.builtin.include_tasks: addkey-attempt.yaml + loop: "{{ pia_candidates }}" + loop_control: + loop_var: pia_candidate + # pia_candidates is already flattened and ordered: every server in + # the first requested region, then every server in the second, and + # so on (built above via subelements over pia_region_lookup — never + # re-sorted or grouped differently here). Every meaningful task in + # addkey-attempt.yaml is individually guarded with + # `when: pia_peer is not defined` (re-evaluated fresh for each loop + # iteration) — confirmed empirically that a guard on this include + # statement itself does NOT re-evaluate per iteration and would NOT + # actually stop later candidates from being attempted after an + # earlier one already succeeded; the guard has to live on the inner + # tasks, which is also where addkey-attempt.yaml resets its + # per-attempt state so a stale parsed response from one candidate + # can never be mistaken for another's. + + - name: Assert PIA accepted the registration against at least one candidate + ansible.builtin.assert: + that: + - pia_peer is defined + - pia_peer.status == "OK" + - pia_region_used is defined + - pia_wg_server_used is defined + fail_msg: >- + PIA addKey failed against every candidate: {{ pia_candidates | + length }} server(s) across {{ pia_effective_regions | length }} + region(s) ({{ pia_effective_regions }}) all failed. See the + per-attempt diagnostics logged above for exactly why each one + failed (timeout, TLS, HTTP error, empty/malformed response, or + a parsed non-OK status). If every region failed the same way, + that points at something shared across all of them — the + request itself, credentials, or minisforum's own network path — + not at any one region being down. + + - name: Write the pia-wg WireGuard interface config + ansible.builtin.template: + src: pia-wg.conf.j2 + dest: "{{ pia_wg_config_dir }}/{{ pia_wg_interface }}.conf" + mode: "0600" + owner: root + group: root + vars: + pia_private_key: "{{ pia_private_key_raw.content | b64decode | trim }}" + no_log: true + notify: Restart pia-wg + # ansible.builtin.template writes atomically (temp file + rename), + # so a failure partway through this specific task cannot leave a + # partially-written config — the previous file, if any, is left + # untouched. Everything that can fail on PIA's side (registration, + # both API calls) happens strictly before this point, so the + # previous working config is never at risk from those failures + # either. + + - name: Promote the rotated key now that registration and config write both succeeded + when: pia_force_key_rotation + block: + - name: Check whether a previous key exists to back up + ansible.builtin.stat: + path: "{{ pia_wg_config_dir }}/{{ pia_wg_interface }}.key" + register: pia_previous_key_stat + + - name: Back up the previous key + ansible.builtin.command: + cmd: >- + cp -p {{ pia_wg_config_dir }}/{{ pia_wg_interface }}.key + {{ pia_wg_config_dir }}/{{ pia_wg_interface }}.key.bak-{{ ansible_date_time.iso8601_basic_short }} + when: pia_previous_key_stat.stat.exists + changed_when: true + + - name: Promote the new key into place + ansible.builtin.command: + cmd: >- + mv {{ pia_wg_config_dir }}/{{ pia_wg_interface }}.key.new + {{ pia_wg_config_dir }}/{{ pia_wg_interface }}.key + changed_when: true + # Only reached after the config template task above has already + # succeeded (Ansible blocks/tasks run sequentially and stop on + # first failure) — by this point PIA has accepted the new key and + # the new config referencing it is already safely on disk, so + # promoting the key file itself (for next run's `creates:` check + # to find it under the normal, non-.new name) is the only + # remaining step, not a point where failure could strand anything. diff --git a/ansible/roles/pia-gateway/tasks/routing.yaml b/ansible/roles/pia-gateway/tasks/routing.yaml new file mode 100644 index 0000000..656e48b --- /dev/null +++ b/ansible/roles/pia-gateway/tasks/routing.yaml @@ -0,0 +1,42 @@ +--- +# Part of role: pia-gateway +# Included by: tasks/main.yaml +# Description: Registers the dedicated PIA policy-routing table and +# installs+enables the boot-time kill-switch service that seeds it +# (rule, terminal unreachable route, static drop rule) before +# wg-quick@{{ pia_wg_interface }} is allowed to start. See +# templates/pia-killswitch.sh.j2 for what the seeded state actually is. + +- name: Register the PIA routing table name + ansible.builtin.lineinfile: + path: /etc/iproute2/rt_tables + line: "{{ pia_route_table_id }}\t{{ pia_route_table_name }}" + regexp: "^[0-9]+\\s+{{ pia_route_table_name }}$" + create: true + mode: "0644" + +- name: Deploy the kill-switch seed script + ansible.builtin.template: + src: pia-killswitch.sh.j2 + dest: /usr/local/sbin/pia-killswitch.sh + mode: "0700" + owner: root + group: root + notify: Restart pia-killswitch + +- name: Deploy the pia-killswitch systemd unit + ansible.builtin.template: + src: pia-killswitch.service.j2 + dest: /etc/systemd/system/pia-killswitch.service + mode: "0644" + owner: root + group: root + notify: + - Reload systemd daemon + - Restart pia-killswitch + +- name: Enable and start the kill-switch service now + ansible.builtin.systemd: + name: pia-killswitch.service + enabled: true + state: started diff --git a/ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.service.j2 b/ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.service.j2 new file mode 100644 index 0000000..b9a848d --- /dev/null +++ b/ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.service.j2 @@ -0,0 +1,7 @@ +# Managed by Ansible (role: pia-gateway). Do not edit by hand. +[Unit] +Description=PIA gateway health check (observability only, no remediation) + +[Service] +Type=oneshot +ExecStart=/usr/local/sbin/pia-gateway-healthcheck.sh diff --git a/ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.sh.j2 b/ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.sh.j2 new file mode 100644 index 0000000..802e9c3 --- /dev/null +++ b/ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.sh.j2 @@ -0,0 +1,110 @@ +#!/bin/bash +# Managed by Ansible (role: pia-gateway). Do not edit by hand. +# +# Observability only. This script NEVER modifies routing, firewall, or +# WireGuard state — a failed check must close the path (already true by +# construction, via the kill switch in tasks/routing.yaml/firewall.yaml), +# never "recover" it. Output goes to the journal +# (journalctl -u pia-gateway-healthcheck); wire alerting off of that +# separately (see plan.md Phase 9) rather than adding remediation here. +# +# The "route decision" check below confirms what the kernel FIB *would* +# do for a VLAN 50 source — it does not send a packet and is not proof of +# actual PIA egress or identity. That proof requires the packet-capture +# test matrix in plan.md Phase 5; do not treat a clean run of this script +# as substituting for it. +set -uo pipefail + +WG_IF="{{ pia_wg_interface }}" +TABLE="{{ pia_route_table_name }}" +SUBNET="{{ pia_vlan50_subnet }}" +PRIORITY="{{ pia_rule_priority }}" +CHAIN="{{ pia_iptables_chain }}" +MAX_HANDSHAKE_AGE={{ pia_healthcheck_handshake_max_age_sec }} +PROBE_SRC="${SUBNET%.*}.2" # a representative VLAN 50 address for the route-decision probe only; never actually used as a source + +fail=0 + +# Informational only (not pass/fail) — which region/server this host is +# actually registered against right now, per the comment +# tasks/register.yaml writes into the generated config. Since PIA +# WireGuard registration now falls back across multiple regions +# (pia_region_candidates in defaults/main.yaml), "which one is live" +# isn't implied by config alone — it's whichever one answered OK last +# time the playbook ran, not necessarily the first-configured one. +configured_peer=$(grep -m1 '^# PIA WireGuard server actually used' "{{ pia_wg_config_dir }}/{{ pia_wg_interface }}.conf" 2>/dev/null || true) +if [[ -n "${configured_peer}" ]]; then + echo "INFO configured-peer: ${configured_peer#\# }" +fi + +if ip link show "${WG_IF}" up &>/dev/null; then + echo "PASS interface: ${WG_IF} is up" +else + echo "FAIL interface: ${WG_IF} is not up" + fail=1 +fi + +if command -v wg &>/dev/null && wg show "${WG_IF}" latest-handshakes &>/dev/null; then + handshake_epoch=$(wg show "${WG_IF}" latest-handshakes 2>/dev/null | awk '{print $2}') + now=$(date +%s) + if [[ -n "${handshake_epoch}" && "${handshake_epoch}" -gt 0 ]]; then + age=$((now - handshake_epoch)) + if (( age <= MAX_HANDSHAKE_AGE )); then + echo "PASS handshake: ${age}s old (max ${MAX_HANDSHAKE_AGE}s)" + else + echo "FAIL handshake: ${age}s old, exceeds max ${MAX_HANDSHAKE_AGE}s" + fail=1 + fi + else + echo "FAIL handshake: no handshake recorded yet" + fail=1 + fi +else + echo "WARN handshake: unable to query wg show (interface absent or wg missing)" + fail=1 +fi + +if ip rule show | grep -qE "^${PRIORITY}:[[:space:]]*from ${SUBNET} lookup ${TABLE}\$"; then + echo "PASS rule: from ${SUBNET} lookup ${TABLE} present at priority ${PRIORITY}" +else + echo "FAIL rule: from ${SUBNET} lookup ${TABLE} missing" + fail=1 +fi + +default_route=$(ip route show table "${TABLE}" 2>/dev/null | grep '^default' || true) +if [[ "${default_route}" == *"dev ${WG_IF}"* ]]; then + echo "PASS route: table ${TABLE} default is via ${WG_IF} (open)" +elif [[ "${default_route}" == *unreachable* ]] || ip route show table "${TABLE}" 2>/dev/null | grep -q '^unreachable default'; then + echo "PASS route: table ${TABLE} default is unreachable (closed, kill switch engaged)" +else + echo "FAIL route: table ${TABLE} has no default route at all (neither open via ${WG_IF} nor a terminal unreachable) — investigate immediately" + fail=1 +fi + +# Not a check on FORWARD's own default policy — that's deliberately left +# alone (see tasks/firewall.yaml for why: minisforum runs Flannel, which +# needs its own broad FORWARD ACCEPTs). What actually matters is that the +# dedicated chain exists, is jumped into first for this source, and ends +# in a real DROP. +if iptables -C FORWARD -s "${SUBNET}" -j "${CHAIN}" 2>/dev/null; then + echo "PASS firewall: FORWARD jumps to ${CHAIN} for ${SUBNET}" +else + echo "FAIL firewall: FORWARD does not jump to ${CHAIN} for ${SUBNET}" + fail=1 +fi + +if iptables -S "${CHAIN}" 2>/dev/null | grep -qE '^-A '"${CHAIN}"' -j DROP$'; then + echo "PASS firewall: ${CHAIN} ends in an unconditional DROP" +else + echo "FAIL firewall: ${CHAIN} has no unconditional DROP catch-all" + fail=1 +fi + +route_decision=$(ip route get 1.1.1.1 from "${PROBE_SRC}" 2>&1 || true) +if [[ "${route_decision}" == *"dev ${WG_IF}"* ]]; then + echo "PASS route-decision (not proof of live egress — see header comment): ${SUBNET} sourced traffic resolves via ${WG_IF}" +else + echo "WARN route-decision (not proof of live egress — see header comment): ${route_decision}" +fi + +exit "${fail}" diff --git a/ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.timer.j2 b/ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.timer.j2 new file mode 100644 index 0000000..d792711 --- /dev/null +++ b/ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.timer.j2 @@ -0,0 +1,11 @@ +# Managed by Ansible (role: pia-gateway). Do not edit by hand. +[Unit] +Description=Run the PIA gateway health check every {{ pia_healthcheck_interval_sec }}s + +[Timer] +OnBootSec={{ pia_healthcheck_interval_sec }} +OnUnitActiveSec={{ pia_healthcheck_interval_sec }} +AccuracySec=5s + +[Install] +WantedBy=timers.target diff --git a/ansible/roles/pia-gateway/templates/pia-killswitch.service.j2 b/ansible/roles/pia-gateway/templates/pia-killswitch.service.j2 new file mode 100644 index 0000000..ea197c6 --- /dev/null +++ b/ansible/roles/pia-gateway/templates/pia-killswitch.service.j2 @@ -0,0 +1,20 @@ +# Managed by Ansible (role: pia-gateway). Do not edit by hand. +# +# Seeds the VLAN 50 kill switch (policy-routing rule, terminal unreachable +# route, static drop rule) before wg-quick@{{ pia_wg_interface }}.service +# is allowed to start — see the drop-in this role installs on that unit +# (wg-quick-order.conf.j2) for the other half of the ordering. +[Unit] +Description=PIA gateway kill switch (VLAN 50 policy routing seed) +Before=wg-quick@{{ pia_wg_interface }}.service +DefaultDependencies=no +After=network-online.target +Wants=network-online.target + +[Service] +Type=oneshot +RemainAfterExit=yes +ExecStart=/usr/local/sbin/pia-killswitch.sh + +[Install] +WantedBy=multi-user.target diff --git a/ansible/roles/pia-gateway/templates/pia-killswitch.sh.j2 b/ansible/roles/pia-gateway/templates/pia-killswitch.sh.j2 new file mode 100644 index 0000000..ce60e1b --- /dev/null +++ b/ansible/roles/pia-gateway/templates/pia-killswitch.sh.j2 @@ -0,0 +1,92 @@ +#!/bin/bash +# Managed by Ansible (role: pia-gateway). Do not edit by hand. +# +# Runs once at boot, before wg-quick@{{ pia_wg_interface }}.service +# (enforced by the systemd drop-in in pia-killswitch.service's +# Before=/wg-quick's Requires=After=), and again any time UFW is +# reloaded (see handlers/main.yaml — "Reload ufw" fires before "Restart +# pia-killswitch" so a reload can never leave a permanent gap). Seeds the +# closed state that PIA gateway's kill switch relies on: +# +# - a source rule sending {{ pia_vlan50_subnet }} to the dedicated +# "{{ pia_route_table_name }}" table, evaluated after the kernel's +# built-in "local" rule (priority 0) and before "main" (32766) — this +# is what keeps Technitium (10.10.40.53, a local address) delivered +# via the local table rather than pulled into policy routing. +# - a terminal "unreachable" default route in that table — but only +# when {{ pia_wg_interface }} isn't already up — so VLAN 50 traffic +# fails closed instead of falling through to main if the interface is +# down, absent, or its PostUp hasn't run yet, without this script +# stomping an already-open, healthy route on a restart unrelated to +# the tunnel's own state (see the check below). +# - a DEDICATED "{{ pia_iptables_chain }}" chain, jumped into by exactly +# one rule at the very top of FORWARD (position 1), matching only +# source {{ pia_vlan50_subnet }}. This is deliberately NOT a global +# FORWARD default-policy change: minisforum is the k3s server and +# runs Flannel, which depends on its own broad FORWARD-chain +# ACCEPTs for pod-to-pod and pod-to-internet traffic (source +# 10.42.0.0/16, entirely disjoint from 10.10.50.0/24) — changing the +# chain-wide default policy was never actually proven safe against +# that, so this scopes the kill switch to a chain that Flannet/k3s +# traffic can never enter in the first place, rather than risking it. +# Being the FIRST rule in FORWARD also means no other, later rule +# (ufw's own, Docker's, anything) can accidentally pre-empt this +# source's fate with a broader ACCEPT — the jump is unconditional for +# this source and the chain itself ends in an unconditional DROP, not +# a RETURN, so nothing after it in FORWARD is ever consulted for +# this source either. +# +# Idempotent: safe to run more than once (e.g. `systemctl restart +# pia-killswitch`, or after this script re-runs following a UFW reload) +# without creating duplicate rules or duplicate chains. +set -euo pipefail + +TABLE="{{ pia_route_table_name }}" +SUBNET="{{ pia_vlan50_subnet }}" +WG_IF="{{ pia_wg_interface }}" +PRIORITY="{{ pia_rule_priority }}" +CHAIN="{{ pia_iptables_chain }}" + +if ! ip rule show | grep -qE "^${PRIORITY}:[[:space:]]*from ${SUBNET} lookup ${TABLE}\$"; then + ip rule add from "${SUBNET}" table "${TABLE}" priority "${PRIORITY}" +fi + +# Only seed the terminal-unreachable fallback if {{ pia_wg_interface }} +# isn't already up. This script re-runs any time it's restarted for +# reasons unrelated to the tunnel's own state (a UFW reload, a content +# fix to this script itself) — if the interface is already up and +# healthy, its own PostUp already set the real "default dev %i" route in +# this table, and blindly overwriting that back to "unreachable" here +# would break a working tunnel's routing for no reason until something +# re-runs PostUp. Only actual absence of the interface should close it. +if ! ip link show "${WG_IF}" up &>/dev/null; then + ip route replace unreachable default table "${TABLE}" +fi + +# Dedicated chain: create if missing (iptables -N fails harmlessly if it +# already exists, hence || true rather than a -C-style existence check — +# there is no direct "does this chain exist" check short of grepping -L). +iptables -N "${CHAIN}" 2>/dev/null || true + +# Exactly one jump from FORWARD into it, at absolute position 1. +if ! iptables -C FORWARD -s "${SUBNET}" -j "${CHAIN}" 2>/dev/null; then + iptables -I FORWARD 1 -s "${SUBNET}" -j "${CHAIN}" +fi + +# Static catch-all inside the chain: anything that reaches this point +# (i.e., wasn't already ACCEPTed by a more specific rule that +# pia-wg.conf.j2's PostUp inserts ahead of these when the interface is +# up) is logged and dropped. --log-limit is not a real iptables option +# (there is no such flag — rate-limiting a LOG target is done via a +# separate -m limit match ahead of it, as below); the -C check must +# match the rule as actually inserted, term for term, or it will never +# find it and silently re-append a duplicate on every run. +if ! iptables -C "${CHAIN}" -m limit --limit 5/minute -j LOG \ + --log-prefix "PIA-KILLSWITCH-DROP: " 2>/dev/null; then + iptables -A "${CHAIN}" -m limit --limit 5/minute -j LOG \ + --log-prefix "PIA-KILLSWITCH-DROP: " +fi + +if ! iptables -C "${CHAIN}" -j DROP 2>/dev/null; then + iptables -A "${CHAIN}" -j DROP +fi diff --git a/ansible/roles/pia-gateway/templates/pia-wg.conf.j2 b/ansible/roles/pia-gateway/templates/pia-wg.conf.j2 new file mode 100644 index 0000000..07bddd8 --- /dev/null +++ b/ansible/roles/pia-gateway/templates/pia-wg.conf.j2 @@ -0,0 +1,56 @@ +# Managed by Ansible (role: pia-gateway). Do not edit by hand — changes +# are overwritten on the next run of ansible/playbooks/pia-gateway.yaml. +# +# Table = off: wg-quick must NOT touch the main routing table. All routing +# for this interface lives in the dedicated "{{ pia_route_table_name }}" +# table (see tasks/routing.yaml), reached only by traffic sourced from +# {{ pia_vlan50_subnet }}. minisforum's own default route is never +# affected by this interface coming up or down. +[Interface] +PrivateKey = {{ pia_private_key }} +Address = {{ pia_peer.peer_ip }}/32 +Table = off + +# Kill-switch ordering: the boot-time pia-killswitch.service (see +# tasks/routing.yaml) has already seeded an "unreachable" default route in +# table {{ pia_route_table_name }} and created the dedicated +# {{ pia_iptables_chain }} chain (jumped into from FORWARD position 1 for +# source {{ pia_vlan50_subnet }} only — never touches Flannel/k3s traffic) +# ending in an unconditional LOG+DROP, before this unit ever starts +# (systemd After=/Requires=). PostUp only ever *opens* the path narrower +# than that pre-seeded closed state, by inserting ACCEPT rules ahead of +# the chain's own catch-all; PreDown removes exactly those same rules, +# re-closing it before removing anything else. +PostUp = ip route replace default dev %i table {{ pia_route_table_name }} +PostUp = iptables -I {{ pia_iptables_chain }} 1 -s {{ pia_vlan50_subnet }} -o %i -j ACCEPT +PostUp = iptables -I {{ pia_iptables_chain }} 1 -d {{ pia_vlan50_subnet }} -i %i -m state --state ESTABLISHED,RELATED -j ACCEPT +PostUp = iptables -t nat -A POSTROUTING -s {{ pia_vlan50_subnet }} -o %i -j MASQUERADE +{% if pia_mss_clamp_enabled %} +PostUp = iptables -t mangle -A FORWARD -s {{ pia_vlan50_subnet }} -o %i -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu +{% endif %} + +PreDown = ip route replace unreachable default table {{ pia_route_table_name }} +PreDown = iptables -t nat -D POSTROUTING -s {{ pia_vlan50_subnet }} -o %i -j MASQUERADE +PreDown = iptables -D {{ pia_iptables_chain }} -d {{ pia_vlan50_subnet }} -i %i -m state --state ESTABLISHED,RELATED -j ACCEPT +PreDown = iptables -D {{ pia_iptables_chain }} -s {{ pia_vlan50_subnet }} -o %i -j ACCEPT +{% if pia_mss_clamp_enabled %} +PreDown = iptables -t mangle -D FORWARD -s {{ pia_vlan50_subnet }} -o %i -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu +{% endif %} + +[Peer] +# PIA WireGuard server actually used — {{ pia_region_name_used }} +# ({{ pia_region_used }}), {{ pia_wg_server_used.cn }}. This is whichever +# region+server in the pia_region_candidates/pia_region fallback order +# (see defaults/main.yaml) responded first with status: OK — not +# necessarily the first-configured region. Re-run the playbook with +# pia_force_reregister (or check journalctl for the last apply's +# per-attempt diagnostics) to see what else was tried before this one. +PublicKey = {{ pia_peer.server_key }} +# The endpoint IP is the server we actually connected to +# (pia_wg_server_used, set by addkey-attempt.yaml on whichever candidate +# in servers.wg succeeded), not a field from the addKey JSON response — +# matching connect_to_wireguard_with_token.sh, which builds the Endpoint +# from its own $WG_SERVER_IP rather than trusting a response field for it. +Endpoint = {{ pia_wg_server_used.ip }}:{{ pia_peer.server_port }} +AllowedIPs = 0.0.0.0/0 +PersistentKeepalive = 25