nik/homelab
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: nik:939a7c6ed13f8da56a815463c6848402b71b65a6
Branches Tags
nik:main
nik:restore-public-ingress
..
compare: nik:7eb7072ec59ad11095f02cc3933a275d7d26c70e
Branches Tags
nik:main
nik:restore-public-ingress

2 Commits
939a7c6ed1 ... 7eb7072ec5

Author SHA1 Message Date
Nik Afiq
7eb7072ec5 feat: add vlan50-parent role for nik-debian's tagged VLAN 50 interface
Some checks failed
validate / lint (push) Failing after 1s
Details 
Creates enp1s0.50 on nik-debian (no L3 address, VLAN 40/node IP/default
route untouched) for Multus to later attach macvlan/ipvlan workloads to.

Verified live: enp1s0.50 is up with no IPv4 address (only the automatic
IPv6 link-local, which is expected and harmless), default route
unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-08-24 17:56:59 +09:00
Nik Afiq
325d3bc5c7 feat: add pia-gateway role for minisforum PIA WireGuard egress 
Registers minisforum as a PIA WireGuard peer for VPN VLAN 50, with a
boot-ordered kill switch (dedicated PIA-VLAN50 iptables chain + a
terminal unreachable route in a dedicated routing table), multi-region
addKey fallback (Hong Kong -> Taiwan -> JP Tokyo, each region's full
server list, in order), and an observability-only health check.

Verified live against minisforum: registration succeeds, wg-quick@pia-wg
is up.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
 2026-08-24 17:54:16 +09:00
26 changed files with 1726 additions and 1 deletions
Show Stats Expand all files Collapse all files

6
ansible/README.md
View File

@ -49,6 +49,8 @@ ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-ollama.yaml -
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/deploy-watch-party.yaml ansible-playbook -i ansible/inventory.yaml ansible/playbooks/deploy-watch-party.yaml
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/wireguard.yaml -K ansible-playbook -i ansible/inventory.yaml ansible/playbooks/wireguard.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/homeassistant.yaml -K ansible-playbook -i ansible/inventory.yaml ansible/playbooks/homeassistant.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/pia-gateway.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/vlan50-parent.yaml -K
``` ```
## Roles ## Roles
@ -66,7 +68,9 @@ ansible-playbook -i ansible/inventory.yaml ansible/playbooks/homeassistant.yaml
| `glances` | Host-level Glances service | | `glances` | Host-level Glances service |
| `ollama` | Ollama service on the Mac Mini and GPU node (branches on OS) | | `ollama` | Ollama service on the Mac Mini and GPU node (branches on OS) |
| `watch-party` | Watch Party Docker Compose deployment on the Mac Mini | | `watch-party` | Watch Party Docker Compose deployment on the Mac Mini |
| `wireguard` | WireGuard server configuration | | `wireguard` | WireGuard server configuration (inbound home-VPN access — phone/Mac clients) |
| `pia-gateway` | Minisforum's PIA WireGuard *egress* gateway for VPN VLAN 50 (policy routing, kill switch, health checks) — see its own README |
| `vlan50-parent` | nik-debian's tagged VLAN 50 parent interface (`enp1s0.50`) for Multus — see its own README |
| `homeassistant` | Standalone Home Assistant deployment (Docker Compose + systemd on `minisforum`) — this is the **only** thing serving `ha.home.arpa`, not legacy/dead | | `homeassistant` | Standalone Home Assistant deployment (Docker Compose + systemd on `minisforum`) — this is the **only** thing serving `ha.home.arpa`, not legacy/dead |
## Notes ## Notes

12
ansible/host_vars/minisforum.yaml
View File

@ -1,2 +1,14 @@
# Host vars for: minisforum (k3s server) # Host vars for: minisforum (k3s server)
ansible_python_interpreter: /usr/bin/python3.13 ansible_python_interpreter: /usr/bin/python3.13
# ── pia-gateway ────────────────────────────────────────────────────────────────
# Read directly from the repo-root .env (same PIA_USER/PIA_PASSWORD keys
# manifests/media/pia-secret.sh already uses) rather than ansible-vault —
# .env is already the established, gitignored, plaintext-at-rest secret
# source for this account elsewhere in the repo, and lookups run on the
# control node, so this never touches minisforum or gets committed.
# playbook_dir is always ansible/playbooks for every playbook in this
# repo, so ../../.env resolves to the repo root regardless of which
# playbook triggers evaluation of these two vars.
pia_user: "{{ lookup('ansible.builtin.file', playbook_dir + '/../../.env') | regex_search('^PIA_USER=(.*)$', '\\1', multiline=True) | first }}"
pia_password: "{{ lookup('ansible.builtin.file', playbook_dir + '/../../.env') | regex_search('^PIA_PASSWORD=(.*)$', '\\1', multiline=True) | first }}"

6
ansible/playbooks/pia-gateway.yaml Normal file
View File

@ -0,0 +1,6 @@
---
- name: PIA WireGuard egress gateway for VPN VLAN 50
hosts: minisforum
become: true
roles:
- pia-gateway

6
ansible/playbooks/vlan50-parent.yaml Normal file
View File

@ -0,0 +1,6 @@
---
- name: Tagged VLAN 50 parent interface on nik-debian
hosts: debian
become: true
roles:
- vlan50-parent

217
ansible/roles/pia-gateway/README.md Normal file
View File

@ -0,0 +1,217 @@
# pia-gateway
Makes `minisforum` a PIA WireGuard egress gateway for VPN VLAN 50
(`10.10.50.0/24`), without changing its own default route. Implements
Phase 2 of `~/repo/homelab/plan.md`.
## Before the first real run
1. **The PIA API calls in `tasks/register.yaml` have gone through three
rounds of correction against real live testing (2026-08-24):**
- `serverlist.piaservers.net/vpninfo/servers/v6`: works. Region values
everywhere in this role are the API's **id** field (e.g. `japan`,
`hk`, `taiwan`), never a display name ("JP Tokyo", "Hong Kong",
"Taiwan" — those are Gluetun's labels, not something this role or
PIA's API accepts).
- Token acquisition originally guessed a regional-meta-server
endpoint (`/authv3/generateToken`) that turned out not to be the
real flow at all — it was rebuilt from PIA's actual
`pia-foss/manual-connections` `get_token.sh` (fetched and read in
full): a single fixed `POST
https://www.privateinternetaccess.com/api/client/v2/token`,
multipart `username`/`password`, **normal system CA validation**
(no `--cacert`, no `validate_certs: false`), independent of region
entirely — only WireGuard server *selection* is regional. Not yet
exercised with real credentials end to end.
- `{wg}:1337/addKey` (WireGuard key registration) — confirmed against
`connect_to_wireguard_with_token.sh` (fetched and read in full):
this one genuinely does need PIA's own CA bundle
(`files/pia-ca.crt`, `--cacert`) and `--connect-to`. Originally
failed over only across servers *within* a single fixed region
(`pia_region: japan`) — in practice all three JP Tokyo servers
failed the same way, one after another, so failover now also
crosses regions: every server in Hong Kong, then every server in
Taiwan, then every server in JP Tokyo (`pia_region_candidates` in
`defaults/main.yaml`), stopping at the first `status: OK`. Token
passed via stdin rather than argv. See `tasks/addkey-attempt.yaml`
and "Region fallback" below.
2. Make sure the repo-root `.env` has `PIA_USER`/`PIA_PASSWORD` set (same
keys `manifests/media/pia-secret.sh` already reads) — `ansible/
host_vars/minisforum.yaml` reads them directly from there via a
`lookup('ansible.builtin.file', ...)` + regex, deliberately not
ansible-vault, so this needs no vault password at all. The lookup runs
on the control node, so `.env` never touches minisforum and is never
committed (already gitignored).
3. Confirm console/recovery access to `minisforum` (IPMI/physical/other
out-of-band path) before applying — this role changes host routing and
firewall policy. If SSH becomes unreachable, use that path.
4. `ufw status verbose` and `wg show` (existing `wg0`) were not inspected
live before writing this role (no passwordless sudo in discovery) —
spot-check that `wg0` (the separate home-VPN role) is unaffected after
applying.
## Region fallback
Registration tries every WireGuard server in every configured region, in
order, stopping at the first `status: OK`:
```text
hk (Hong Kong):
wg server 1
wg server 2
taiwan (Taiwan):
wg server 1
wg server 2
japan (JP Tokyo):
wg server 1
wg server 2
wg server 3
```
The actual number and order of servers within each region always comes
from the live serverlist response — never hardcoded here.
**Automatic fallback** (default — tries `pia_region_candidates` in
order):
```bash
ansible-playbook -i ansible/inventory.yaml \
ansible/playbooks/pia-gateway.yaml -K -J
```
**Forced single region**, for troubleshooting one region in isolation —
bypasses `pia_region_candidates` entirely, does not fall back to the
others:
```bash
ansible-playbook -i ansible/inventory.yaml \
ansible/playbooks/pia-gateway.yaml -K -J \
-e pia_region=hk
```
Both `pia_region` and every entry in `pia_region_candidates` are PIA API
**ids** — `hk`, `taiwan`, `japan` — not the display names Gluetun's
`SERVER_REGIONS` config uses ("Hong Kong", "Taiwan", "JP Tokyo"). Passing
a display name here matches nothing and fails the "region exists"
assertion before any network call is even made.
Which region/server actually ended up registered is recorded as
`pia_region_used`/`pia_region_name_used`/`pia_wg_server_used`, written
into `pia-wg.conf`'s own `[Peer]` comment, and surfaced by the health
check (`INFO configured-peer: ...`) — it is whichever one answered first,
not necessarily `pia_region_candidates[0]`.
If every region fails: `journalctl` (or the play's own output) has one
`debug` line per failed attempt, each showing region name+id, server
hostname+IP, curl exit code, a classified reason (connection timeout /
TLS-SSL failure / HTTP failure / empty response / malformed JSON /
parsed-but-not-OK status, never a generic catch-all), and sanitized
stderr — never the token, credentials, the full curl invocation, request
stdin, or a complete response body. If Hong Kong and Taiwan fail exactly
the same way Tokyo did, that's a strong signal the problem is shared
across all three (the request shape, the account credentials, or
minisforum's own network path) rather than one region being down — see
`tasks/register.yaml`'s header comment.
**Worked example (2026-08-24):** exactly that happened — all 7 servers
across all 3 regions came back `HTTP failure (non-2xx response)`, curl
exit 22, `The requested URL returned error: 401`, uniformly. The cause
was in `addkey-attempt.yaml`'s own curl task, not PIA or any region:
`ansible.builtin.command`'s `stdin` argument appends a trailing newline
by default (`stdin_add_newline` defaults to `true`), and
`--data-urlencode pt@-` does not strip it — every server was receiving
`"<real token>\n"` (URL-encoded, so a trailing `%0A`) as `pt`, a
different and invalid value, and correctly rejecting it. Fixed with
`stdin_add_newline: false` on that task. `connect_to_wireguard_with_token
.sh` never hits this because it passes the token as a literal shell
variable, not via stdin — the stdin delivery here is this repo's own
addition (to keep the token out of `ps`), so it needed the flag PIA's
reference script never had to think about. Left as a worked example
because "every candidate failed identically" pointing at one shared bug
in *this* code, not PIA, is exactly the diagnostic story this section
promises — and it happened to be true the first time it was tested for
real.
## What it does
- Registers minisforum as a PIA WireGuard peer — see "Region fallback"
above — and writes `/etc/wireguard/pia-wg.conf` (`Table = off` — this
role owns all routing for the interface, not wg-quick).
- Installs `pia-killswitch.service`, ordered `Before=
wg-quick@pia-wg.service`, that seeds a closed state at every boot (and
after any UFW reload — see handlers/main.yaml): the
`from 10.10.50.0/24 lookup pia` rule, a terminal `unreachable default`
route in the `pia` table (only when `pia-wg` isn't already up — see
the script's own comment for why), and a dedicated `PIA-VLAN50`
iptables chain jumped into by a single rule at the very top of
`FORWARD`, matching only source `10.10.50.0/24`, ending in an
unconditional rate-limited log+drop. This is deliberately **not** a
global `FORWARD` default-policy change — minisforum runs Flannel,
which needs its own broad `FORWARD` ACCEPTs for pod traffic (source
`10.42.0.0/16`, disjoint from VLAN 50) — so the kill switch is scoped
to a chain Flannel/k3s traffic can never enter, rather than risking a
chain-wide policy that was never actually proven safe against it.
Technitium (`10.10.40.53`, minisforum's own address) gets a normal UFW
**input** allow for 53/tcp+udp — it's locally-terminated traffic, not
routed — see `tasks/firewall.yaml`.
- `pia-wg.conf`'s own `PostUp`/`PreDown` open/close the narrower path on
top of that closed baseline: default route via `pia-wg` in the `pia`
table, ACCEPT + established/related return inserted into the
`PIA-VLAN50` chain (not `FORWARD` directly), and source-NAT/MASQUERADE
scoped to `10.10.50.0/24` on `pia-wg` only.
- Installs an observability-only health check (`pia-gateway-healthcheck
.timer`, every `pia_healthcheck_interval_sec`) that logs interface,
handshake age, rule/route, and firewall-policy state to the journal. It
never remediates — see the script's header comment for why, and for what
its "route-decision" check does and does not prove.
## What it deliberately does not do
- Does not touch minisforum's own default route (asserted at the start of
every run — `tasks/assert-baseline.yaml` fails loudly if that's already
wrong).
- Does not enable IPv6 forwarding or any IPv6 handling for VLAN 50.
- Does not enable MSS clamping (`pia_mss_clamp_enabled: false` by
default) — flip only after Phase 5 canary MTU testing shows it's
actually needed.
- Does not modify `ansible/roles/wireguard` (the separate `wg0` home-VPN
server role) or its interface.
## Rollback
No tag-based rollback is defined — this role doesn't have an "absent"
mode. Roll back manually on minisforum (reverses this role without
touching `wg0`, k3s, or the host default route):
```bash
sudo systemctl disable --now wg-quick@pia-wg pia-killswitch.service pia-gateway-healthcheck.timer
sudo rm -f /etc/systemd/system/pia-killswitch.service \
/etc/systemd/system/pia-gateway-healthcheck.service \
/etc/systemd/system/pia-gateway-healthcheck.timer
sudo systemctl daemon-reload
sudo ip rule del from 10.10.50.0/24 table pia priority 100
sudo ip route flush table pia
sudo sed -i '/^[0-9]\+\s\+pia$/d' /etc/iproute2/rt_tables
sudo iptables -D FORWARD -s 10.10.50.0/24 -j PIA-VLAN50
sudo iptables -F PIA-VLAN50
sudo iptables -X PIA-VLAN50
sudo ufw delete allow from 10.10.50.0/24 to 10.10.40.53 port 53 proto tcp
sudo ufw delete allow from 10.10.50.0/24 to 10.10.40.53 port 53 proto udp
sudo rm -f /etc/wireguard/pia-wg.conf /etc/wireguard/pia-wg.key /etc/wireguard/pia-wg.key.new /etc/wireguard/pia-ca.crt
```
Inspection commands to confirm rollback actually took (read-only):
```bash
sudo iptables -S FORWARD | grep PIA-VLAN50 # expect: no output
sudo iptables -L PIA-VLAN50 # expect: "iptables: No chain/target/match by that name"
sudo ufw status verbose | grep 10.10.50 # expect: no output
ip rule show | grep pia # expect: no output
ip route show table pia # expect: empty/error (table gone)
```
Does not remove `pia-credentials`/`pia-credentials-sealed.yaml` (the K8s
Secret used by the Gluetun sidecars) — that's a separate, unrelated
secret and rollback path. Does not touch `ansible/roles/common`'s own
UFW rules (Flannel pod-to-pod, pod-to-Technitium) — this role never
modified those.

116
ansible/roles/pia-gateway/defaults/main.yaml Normal file
View File

@ -0,0 +1,116 @@
---
# Part of role: pia-gateway
# Called by: ansible/playbooks/pia-gateway.yaml
# Description: Default variables for minisforum's PIA WireGuard egress
# gateway for VPN VLAN 50 — policy routing, kill switch, and health
# checks. See ~/repo/homelab/plan.md for the cross-repo contract this
# role implements (fixed values below match that contract).
# PIA regions to try, in order, for WireGuard addKey registration —
# first server (in the first region that has any) to return
# `status: OK` wins. These are API region **ids** from PIA's serverlist
# (https://serverlist.piaservers.net/vpninfo/servers/v6), e.g.
# {"id": "japan", "name": "JP Tokyo", ...} — NOT Gluetun's display names
# ("JP Tokyo", "Hong Kong", "Taiwan") and not the slug you might guess
# from one (confirmed live 2026-08-24: Tokyo's id is "japan", not
# "jp_tokyo" — that matched zero regions on a first pass). Order matches
# the region set already used by the Gluetun sidecars, Hong Kong/Taiwan
# moved ahead of Tokyo after all three Tokyo servers were observed
# failing in practice — see tasks/register.yaml's header for what that
# means and doesn't mean.
pia_region_candidates:
- hk
- taiwan
- japan
# Force a SINGLE region for troubleshooting, bypassing
# pia_region_candidates entirely — e.g. `-e pia_region=hk`. This is also
# an API id, not a display name; same rule as above. Leave null (the
# default) to use the ordered candidate list. tasks/register.yaml
# computes the actual list to try (pia_effective_regions) from whichever
# of these two is set — see that file's own comment for the exact logic,
# not duplicated here to avoid the two drifting out of sync.
pia_region: null
# WireGuard interface identity. Deliberately not "wg0" — that name is
# already used by ansible/roles/wireguard, minisforum's separate *inbound*
# home-VPN server role. Colliding with it would tear down remote access.
pia_wg_interface: pia-wg
pia_wg_config_dir: /etc/wireguard
# Re-registration behaviour — two independent, unambiguous knobs (an
# earlier single pia_force_reregister conflated "re-run the API calls"
# with "get a new keypair", which isn't the same thing and isn't safe to
# conflate — the key-generation task uses `creates:`, so a naive
# "force" only re-ran the PIA API calls with the SAME existing key,
# silently, unless you also knew to delete the key file yourself):
#
# - pia_force_reregister: re-run PIA registration (serverlist lookup,
# token, addKey) even if {{ pia_wg_interface }}.conf already exists.
# Reuses whatever keypair is already on disk (generates one only if
# none exists yet). Use this to pick up a new PIA-assigned peer/server
# without changing minisforum's own WireGuard identity.
# - pia_force_key_rotation: additionally generate a NEW keypair and
# register THAT with PIA (implies pia_force_reregister). Rotation is
# safe-by-construction: the new key is generated to a *.key.new path
# and only promoted (with the old key backed up alongside a timestamp)
# after PIA has actually accepted the new key and the new config has
# been written — see tasks/register.yaml. If registration fails
# partway, the previous key/config are left completely untouched
# rather than ending up in a broken half-state.
pia_force_reregister: false
pia_force_key_rotation: false
# Dedicated policy-routing table for VLAN 50 egress. 0/253/254/255 are
# kernel-reserved (unspec/default/main/local) — this must stay clear of
# those and of any other custom table already registered on the host.
pia_route_table_id: 52
pia_route_table_name: pia
# Rule priority for "from 10.10.50.0/24 lookup pia". Must sort after the
# kernel's built-in local table (priority 0, always evaluated first, keeps
# traffic addressed to the host's own local/Technitium address delivered
# locally per plan.md) and before main (priority 32766).
pia_rule_priority: 100
# Fixed cross-repo contract (see plan.md "Fixed cross-repository
# contract") — VLAN 50 subnet and the Flint gateway that owns the return
# route back to VLAN 50. Not derived from live discovery; change only if
# the contract itself changes.
pia_vlan50_subnet: 10.10.50.0/24
pia_flint_gateway: 10.10.50.1
# Dedicated iptables chain the kill switch governs, jumped into by a
# single rule at FORWARD position 1 matching only
# {{ pia_vlan50_subnet }} — deliberately not a global FORWARD default-
# policy change, so k3s/Flannel's own FORWARD-chain traffic (source
# 10.42.0.0/16, disjoint from VLAN 50) is never affected by this role at
# all. See templates/pia-killswitch.sh.j2 for the full reasoning.
pia_iptables_chain: PIA-VLAN50
# Real parent LAN interface on minisforum — verified live 2026-08-23 via
# `ip -brief addr`; it is enp1s0, not "eth0". Used for MASQUERADE-exclusion
# bookkeeping and documentation only, since NAT/forward rules below key off
# {{ pia_wg_interface }} rather than enumerating the normal interface.
pia_lan_interface: enp1s0
# Reverse-path filtering, applied to {{ pia_wg_interface }} only. The
# kernel default on minisforum (conf.default.rp_filter) is already loose
# (2) at time of writing, so new interfaces inherit loose mode — this task
# pins it explicitly rather than relying on that inherited default holding
# across future changes. Loose mode is required because PIA's return
# traffic can legitimately arrive with routing asymmetry relative to how
# minisforum would route toward that source itself.
pia_rp_filter_mode: 2
# MSS clamping on the pia-wg forward path — OFF by default. plan.md is
# explicit: do not guess this. Flip to true only after Phase 5 canary
# MTU/path-MTU testing (large-packet ping with DF set, across the
# VLAN 50 -> Flint -> minisforum -> pia-wg path) shows fragmentation or
# black-holing that clamping fixes.
pia_mss_clamp_enabled: false
# Health check cadence and thresholds. The check only observes and logs —
# see tasks/healthcheck.yaml — it must never itself open the kill switch.
pia_healthcheck_interval_sec: 60
pia_healthcheck_handshake_max_age_sec: 180

43
ansible/roles/pia-gateway/files/pia-ca.crt Normal file
View File

@ -0,0 +1,43 @@
-----BEGIN CERTIFICATE-----
MIIHqzCCBZOgAwIBAgIJAJ0u+vODZJntMA0GCSqGSIb3DQEBDQUAMIHoMQswCQYD
VQQGEwJVUzELMAkGA1UECBMCQ0ExEzARBgNVBAcTCkxvc0FuZ2VsZXMxIDAeBgNV
BAoTF1ByaXZhdGUgSW50ZXJuZXQgQWNjZXNzMSAwHgYDVQQLExdQcml2YXRlIElu
dGVybmV0IEFjY2VzczEgMB4GA1UEAxMXUHJpdmF0ZSBJbnRlcm5ldCBBY2Nlc3Mx
IDAeBgNVBCkTF1ByaXZhdGUgSW50ZXJuZXQgQWNjZXNzMS8wLQYJKoZIhvcNAQkB
FiBzZWN1cmVAcHJpdmF0ZWludGVybmV0YWNjZXNzLmNvbTAeFw0xNDA0MTcxNzQw
MzNaFw0zNDA0MTIxNzQwMzNaMIHoMQswCQYDVQQGEwJVUzELMAkGA1UECBMCQ0Ex
EzARBgNVBAcTCkxvc0FuZ2VsZXMxIDAeBgNVBAoTF1ByaXZhdGUgSW50ZXJuZXQg
QWNjZXNzMSAwHgYDVQQLExdQcml2YXRlIEludGVybmV0IEFjY2VzczEgMB4GA1UE
AxMXUHJpdmF0ZSBJbnRlcm5ldCBBY2Nlc3MxIDAeBgNVBCkTF1ByaXZhdGUgSW50
ZXJuZXQgQWNjZXNzMS8wLQYJKoZIhvcNAQkBFiBzZWN1cmVAcHJpdmF0ZWludGVy
bmV0YWNjZXNzLmNvbTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBALVk
hjumaqBbL8aSgj6xbX1QPTfTd1qHsAZd2B97m8Vw31c/2yQgZNf5qZY0+jOIHULN
De4R9TIvyBEbvnAg/OkPw8n/+ScgYOeH876VUXzjLDBnDb8DLr/+w9oVsuDeFJ9K
V2UFM1OYX0SnkHnrYAN2QLF98ESK4NCSU01h5zkcgmQ+qKSfA9Ny0/UpsKPBFqsQ
25NvjDWFhCpeqCHKUJ4Be27CDbSl7lAkBuHMPHJs8f8xPgAbHRXZOxVCpayZ2SND
fCwsnGWpWFoMGvdMbygngCn6jA/W1VSFOlRlfLuuGe7QFfDwA0jaLCxuWt/BgZyl
p7tAzYKR8lnWmtUCPm4+BtjyVDYtDCiGBD9Z4P13RFWvJHw5aapx/5W/CuvVyI7p
Kwvc2IT+KPxCUhH1XI8ca5RN3C9NoPJJf6qpg4g0rJH3aaWkoMRrYvQ+5PXXYUzj
tRHImghRGd/ydERYoAZXuGSbPkm9Y/p2X8unLcW+F0xpJD98+ZI+tzSsI99Zs5wi
jSUGYr9/j18KHFTMQ8n+1jauc5bCCegN27dPeKXNSZ5riXFL2XX6BkY68y58UaNz
meGMiUL9BOV1iV+PMb7B7PYs7oFLjAhh0EdyvfHkrh/ZV9BEhtFa7yXp8XR0J6vz
1YV9R6DYJmLjOEbhU8N0gc3tZm4Qz39lIIG6w3FDAgMBAAGjggFUMIIBUDAdBgNV
HQ4EFgQUrsRtyWJftjpdRM0+925Y6Cl08SUwggEfBgNVHSMEggEWMIIBEoAUrsRt
yWJftjpdRM0+925Y6Cl08SWhge6kgeswgegxCzAJBgNVBAYTAlVTMQswCQYDVQQI
EwJDQTETMBEGA1UEBxMKTG9zQW5nZWxlczEgMB4GA1UEChMXUHJpdmF0ZSBJbnRl
cm5ldCBBY2Nlc3MxIDAeBgNVBAsTF1ByaXZhdGUgSW50ZXJuZXQgQWNjZXNzMSAw
HgYDVQQDExdQcml2YXRlIEludGVybmV0IEFjY2VzczEgMB4GA1UEKRMXUHJpdmF0
ZSBJbnRlcm5ldCBBY2Nlc3MxLzAtBgkqhkiG9w0BCQEWIHNlY3VyZUBwcml2YXRl
aW50ZXJuZXRhY2Nlc3MuY29tggkAnS7684Nkme0wDAYDVR0TBAUwAwEB/zANBgkq
hkiG9w0BAQ0FAAOCAgEAJsfhsPk3r8kLXLxY+v+vHzbr4ufNtqnL9/1Uuf8NrsCt
pXAoyZ0YqfbkWx3NHTZ7OE9ZRhdMP/RqHQE1p4N4Sa1nZKhTKasV6KhHDqSCt/dv
Em89xWm2MVA7nyzQxVlHa9AkcBaemcXEiyT19XdpiXOP4Vhs+J1R5m8zQOxZlV1G
tF9vsXmJqWZpOVPmZ8f35BCsYPvv4yMewnrtAC8PFEK/bOPeYcKN50bol22QYaZu
LfpkHfNiFTnfMh8sl/ablPyNY7DUNiP5DRcMdIwmfGQxR5WEQoHL3yPJ42LkB5zs
6jIm26DGNXfwura/mi105+ENH1CaROtRYwkiHb08U6qLXXJz80mWJkT90nr8Asj3
5xN2cUppg74nG3YVav/38P48T56hG1NHbYF5uOCske19F6wi9maUoto/3vEr0rnX
JUp2KODmKdvBI7co245lHBABWikk8VfejQSlCtDBXn644ZMtAdoxKNfR2WTFVEwJ
iyd1Fzx0yujuiXDROLhISLQDRjVVAvawrAtLZWYK31bY7KlezPlQnl/D9Asxe85l
8jO5+0LdJ6VyOs/Hd4w52alDW/MFySDZSfQHMTIc30hLBJ8OnCEIvluVQQ2UQvoW
+no177N9L2Y+M9TcTA62ZyMXShHQGeh20rb4kK8f+iFX8NxtdHVSkxMEFSfDDyQ=
-----END CERTIFICATE-----

46
ansible/roles/pia-gateway/handlers/main.yaml Normal file
View File

@ -0,0 +1,46 @@
---
# Part of role: pia-gateway
# Handlers fire in the order listed here (not the order notified):
# 1. systemd daemon reload must stay first so unit-file content changes
# are picked up before any restart handler below acts on them.
# 2. "Reload ufw" must come before "Restart pia-killswitch" — a UFW
# reload rebuilds ufw's own chains and, depending on how it's
# triggered, can in principle disturb rules inserted directly into
# the base FORWARD chain (our position-1 jump to
# {{ pia_iptables_chain }}) outside ufw's own bookkeeping. Restarting
# pia-killswitch immediately after re-establishes that jump and the
# chain's contents idempotently, closing the window rather than
# leaving it open until the next reboot. This only protects against
# ufw reloads that happen *through this Ansible run* (i.e., a task
# here notifies both) — a `ufw reload` run manually and separately,
# outside of Ansible entirely, is not something a handler can react
# to; that residual gap is closed only by every real boot re-running
# pia-killswitch.service fresh (WantedBy=multi-user.target).
# 3. "Restart pia-killswitch" must come before "Restart pia-wg" — if
# both fire together (e.g. pia-wg's config changed AND the
# killswitch script changed in the same run), pia-wg's own PostUp
# needs to run last so its "open" route is the final word, not
# killswitch's route-table seeding.
- name: Reload systemd daemon
ansible.builtin.systemd:
daemon_reload: true
- name: Reload ufw
community.general.ufw:
state: reloaded
- name: Restart pia-killswitch
ansible.builtin.systemd:
name: pia-killswitch.service
state: restarted
- name: Restart pia-wg
ansible.builtin.systemd:
name: "wg-quick@{{ pia_wg_interface }}"
state: restarted
- name: Restart pia-gateway-healthcheck timer
ansible.builtin.systemd:
name: pia-gateway-healthcheck.timer
state: restarted

179
ansible/roles/pia-gateway/tasks/addkey-attempt.yaml Normal file
View File

@ -0,0 +1,179 @@
---
# Part of role: pia-gateway
# Included by: tasks/register.yaml, once per candidate in pia_candidates
# (loop_var: pia_candidate — a flattened {region_id, region_name, cn, ip}
# dict, one server, built by register.yaml via subelements over every
# requested region in order), in order, until one succeeds.
#
# Every meaningful task below is individually guarded with
# `when: pia_peer is not defined` — confirmed empirically (not assumed)
# that putting this guard only on the include statement in register.yaml
# does NOT stop later loop iterations from running; ansible re-evaluates
# a `when` on tasks inside an included file per-iteration, but does not
# re-evaluate a `when` on the include statement itself the same way. The
# guard has to live here, on every task, or a candidate that comes up
# after an earlier one already worked would still get hit.
- name: "Reset per-attempt response state for {{ pia_candidate.cn }}"
ansible.builtin.set_fact:
pia_addkey_response: {}
pia_addkey_rc: -1
pia_addkey_stderr: ""
pia_addkey_parsed: null
pia_addkey_diagnosis: ""
when: pia_peer is not defined
# Explicit reset, every iteration, before this candidate's own curl
# call runs — a prior version of this file only ever *conditionally*
# set pia_addkey_parsed (rc == 0 and non-empty stdout), so a candidate
# whose curl call failed outright (rc != 0) would skip that task and
# silently keep the PREVIOUS candidate's parsed response sitting in
# pia_addkey_parsed. That's not just cosmetic: it means a stale
# {"status": "..."} from an earlier failure could read as if it came
# from the current candidate in diagnostics, or — if the guard logic
# were ever refactored slightly differently — actually be accepted for
# the wrong server. Resetting to a known-empty/null state here, every
# time, makes each iteration self-contained regardless of what
# happened before it.
- name: "Register WireGuard key with server {{ pia_candidate.cn }}"
ansible.builtin.command:
cmd: >-
curl --silent --show-error --fail --location -G
--connect-timeout 10 --max-time 30
--cacert {{ pia_wg_config_dir }}/pia-ca.crt
--connect-to {{ pia_candidate.cn }}::{{ pia_candidate.ip }}:
--data-urlencode pt@-
--data-urlencode pubkey={{ pia_public_key.stdout }}
https://{{ pia_candidate.cn }}:1337/addKey
args:
stdin: "{{ pia_auth_token }}"
stdin_add_newline: false
register: pia_addkey_response
changed_when: false
failed_when: false
no_log: true
when: pia_peer is not defined
# Same request shape as PIA's own connect_to_wireguard_with_token.sh
# for every region — WireGuard hostname for TLS/SNI, its server IP via
# --connect-to, PIA's private CA (never -k/--insecure/validate_certs:
# false), port 1337, /addKey, generated pubkey. Token passed via stdin
# (--data-urlencode pt@-) rather than as a plain argv value — verified
# empirically (this session) that curl's argv, including a literal
# "pt@-", never the token itself, is what shows up in `ps`, so the
# token is never visible to another local user on the host for the
# process's lifetime. pubkey is not sensitive (public by design) and
# stays a normal argument. No curl --retry here — retries happen by
# moving to the next candidate in the loop instead, so one slow/dead
# server can't eat the whole attempt budget retrying itself before
# failover to the next server, or the next region, gets a chance.
#
# stdin_add_newline: false is load-bearing, not cosmetic. Ansible's
# command module appends a trailing newline to `stdin` by default
# (stdin_add_newline defaults to true) — confirmed live (this session)
# against a real run: every one of 7 servers across all 3 regions
# rejected the token with a uniform HTTP 401, and reproducing the exact
# same stdin mechanism against a plain echo endpoint showed why —
# `--data-urlencode pt@-` does not strip that trailing newline, so PIA
# received "<real token>\n" (URL-encoded, with a trailing %0A) as the
# pt value: a different, invalid string. This is what actually broke
# every region identically, not a credentials or connectivity problem
# in any of them. get_token.sh/connect_to_wireguard_with_token.sh never
# hit this because they pass the token as a literal shell variable
# (`"pt=${PIA_TOKEN}"`), not via stdin — the stdin delivery mechanism
# itself was introduced in this repo specifically to keep the token out
# of `ps` output, and needs this flag to behave the same as PIA's own
# reference flow.
- name: "Capture safe diagnostics for {{ pia_candidate.cn }}"
ansible.builtin.set_fact:
pia_addkey_rc: "{{ pia_addkey_response.rc | default(-1) }}"
pia_addkey_stderr: "{{ pia_addkey_response.stderr | default('') }}"
when: pia_peer is not defined
# rc/stderr never contain the token or pubkey — same reasoning as the
# token request's diagnostics in register.yaml. Never derived from
# pia_addkey_response.cmd/.invocation (the literal argv, harmless here
# since the token isn't in it either, but still not something to print
# wholesale) — only the two specific safe fields are ever extracted.
# --- Classify the outcome into one human-readable diagnosis. Each of
# these is mutually exclusive by its `when:`, and pia_addkey_diagnosis
# was reset to "" above, so exactly one (or zero, if this candidate was
# skipped because an earlier one already succeeded) ends up set. Curl
# exit codes confirmed against curl's own documented libcurl error list,
# not guessed — 28 is CURLE_OPERATION_TIMEDOUT, 22 is
# CURLE_HTTP_RETURNED_ERROR (only possible here because of --fail), and
# the TLS set is every CURLE_SSL_*/CURLE_PEER_FAILED_VERIFICATION code
# curl currently defines (35, 53, 54, 58, 59, 60, 66, 77, 80, 82, 83).
- name: "Diagnose a connection timeout for {{ pia_candidate.cn }}"
ansible.builtin.set_fact:
pia_addkey_diagnosis: connection timeout
when: (pia_peer is not defined) and (pia_addkey_rc | int == 28)
- name: "Diagnose a TLS validation failure for {{ pia_candidate.cn }}"
ansible.builtin.set_fact:
pia_addkey_diagnosis: TLS/SSL failure
when: (pia_peer is not defined) and (pia_addkey_rc | int in [35, 53, 54, 58, 59, 60, 66, 77, 80, 82, 83])
- name: "Diagnose an HTTP failure for {{ pia_candidate.cn }}"
ansible.builtin.set_fact:
pia_addkey_diagnosis: HTTP failure (non-2xx response)
when: (pia_peer is not defined) and (pia_addkey_rc | int == 22)
- name: "Diagnose an unclassified curl error for {{ pia_candidate.cn }}"
ansible.builtin.set_fact:
pia_addkey_diagnosis: "curl error (exit {{ pia_addkey_rc }})"
when: (pia_peer is not defined) and (pia_addkey_rc | int not in [0, 22, 28, 35, 53, 54, 58, 59, 60, 66, 77, 80, 82, 83])
- name: "Parse the addKey response from {{ pia_candidate.cn }}"
ansible.builtin.set_fact:
pia_addkey_parsed: "{{ pia_addkey_response.stdout | from_json }}"
when: (pia_peer is not defined) and (pia_addkey_rc | int == 0) and (pia_addkey_response.stdout | trim | length > 0)
ignore_errors: true
# ignore_errors here on purpose: a genuinely malformed body from one
# misbehaving server must not abort the whole play — it should just
# leave pia_addkey_parsed at the null it was reset to above, so this
# candidate is diagnosed as malformed (below) and the loop moves on.
- name: "Diagnose an empty response for {{ pia_candidate.cn }}"
ansible.builtin.set_fact:
pia_addkey_diagnosis: empty response body
when: (pia_peer is not defined) and (pia_addkey_rc | int == 0) and (pia_addkey_response.stdout | trim | length == 0)
- name: "Diagnose a malformed JSON response for {{ pia_candidate.cn }}"
ansible.builtin.set_fact:
pia_addkey_diagnosis: malformed JSON response
when: (pia_peer is not defined) and (pia_addkey_rc | int == 0) and (pia_addkey_response.stdout | trim | length > 0) and (pia_addkey_parsed is none)
- name: "Diagnose a non-OK status for {{ pia_candidate.cn }}"
ansible.builtin.set_fact:
pia_addkey_diagnosis: "parsed OK but status is '{{ pia_addkey_parsed.status | default('missing') }}', not OK"
when: (pia_peer is not defined) and (pia_addkey_parsed is not none) and ((pia_addkey_parsed.status | default('')) != 'OK')
- name: "Log the attempt outcome for {{ pia_candidate.cn }}"
ansible.builtin.debug:
msg: >-
region={{ pia_candidate.region_name }} ({{ pia_candidate.region_id }})
server={{ pia_candidate.cn }} ip={{ pia_candidate.ip }}
rc={{ pia_addkey_rc }} reason=[{{ pia_addkey_diagnosis }}]
stderr=[{{ pia_addkey_stderr }}]
status={{ pia_addkey_parsed.status | default('n/a') }}
when: (pia_peer is not defined) and (pia_addkey_diagnosis | length > 0)
# Deliberately never includes: the token, username/password, the full
# curl invocation/cmd array, request stdin, the complete response body
# (only its parsed .status field, when parsing succeeded), or any key
# material beyond what pubkey already is (public, safe by design).
- name: "Accept the response from {{ pia_candidate.cn }}"
ansible.builtin.set_fact:
pia_peer: "{{ pia_addkey_parsed }}"
pia_region_used: "{{ pia_candidate.region_id }}"
pia_region_name_used: "{{ pia_candidate.region_name }}"
pia_wg_server_used: "{{ pia_candidate }}"
when: (pia_peer is not defined) and (pia_addkey_parsed is not none) and ((pia_addkey_parsed.status | default('')) == 'OK')
# Deliberately checks status == OK here, not just "did it parse" — a
# response that parses but isn't actually OK must not stop the loop
# from trying the remaining candidates. Once this fires, every
# subsequent iteration's tasks are no-ops (all guarded on
# `pia_peer is not defined`), so pia_region_used/pia_wg_server_used are
# never overwritten by a later candidate.

34
ansible/roles/pia-gateway/tasks/assert-baseline.yaml Normal file
View File

@ -0,0 +1,34 @@
---
# Part of role: pia-gateway
# Included by: tasks/main.yaml
# Description: Enables IP forwarding persistently (required for minisforum
# to route VLAN 50 traffic at all) and asserts the host's own default
# route is still the normal LAN gateway before any PIA-specific routing
# is touched. This is the guard behind "never change minisforum's normal
# default route globally" — it fails the play loudly instead of layering
# PIA routing on top of an already-wrong baseline.
- name: Enable IP forwarding persistently
ansible.posix.sysctl:
name: net.ipv4.ip_forward
value: "1"
sysctl_set: true
state: present
reload: true
- name: Read current default route
ansible.builtin.command: ip -4 route show default
register: pia_current_default_route
changed_when: false
- name: Assert the host default route is still the normal LAN interface
ansible.builtin.assert:
that:
- pia_current_default_route.stdout is search('dev ' + pia_lan_interface)
- not (pia_current_default_route.stdout is search('dev ' + pia_wg_interface))
fail_msg: >-
minisforum's default route is not via {{ pia_lan_interface }}
({{ pia_current_default_route.stdout }}). Refusing to continue —
this role must never run against a host whose own default route
has already been changed.
success_msg: "Host default route confirmed via {{ pia_lan_interface }}"

66
ansible/roles/pia-gateway/tasks/firewall.yaml Normal file
View File

@ -0,0 +1,66 @@
---
# Part of role: pia-gateway
# Included by: tasks/main.yaml
# Description: The one thing this file does now: permit VLAN 50 clients
# to reach the local Technitium listener on minisforum. Everything
# else the kill switch needs (the dedicated FORWARD-chain scoping, the
# routing-table unreachable fallback) lives in tasks/routing.yaml /
# templates/pia-killswitch.sh.j2 — see those for the actual fail-closed
# layers.
#
# Two corrections from an earlier version of this file, both from code
# review against the live topology rather than assumption:
#
# 1. Technitium (10.10.40.53) IS minisforum's own address — traffic from
# a VLAN 50 client to it terminates LOCALLY on minisforum. That's
# INPUT traffic, not FORWARD/routed traffic, so it needs a normal UFW
# input allow, not `route: true` (which adds a FORWARD-chain rule for
# traffic passing *through* the host to some other destination — the
# wrong chain for a locally-terminated flow, and it would never
# actually have matched real Technitium queries).
#
# 2. This file used to also force UFW's DEFAULT_FORWARD_POLICY to DROP,
# intended as a second kill-switch layer. That was never actually
# proven safe: minisforum is the k3s server and runs Flannel, which
# depends on the FORWARD chain for pod-to-pod and pod-to-internet
# traffic (source 10.42.0.0/16) — changing the chain-wide default
# policy risked breaking that, for a benefit (fail-closed for VLAN 50
# specifically) that's already fully covered by the dedicated
# {{ pia_iptables_chain }} chain in pia-killswitch.sh.j2, which is
# scoped to source {{ pia_vlan50_subnet }} only and can structurally
# never see Flannel's traffic (disjoint source CIDR). Removed rather
# than shipped unproven.
#
# Inspection commands (read-only):
# sudo iptables -S FORWARD | head -3 # confirm the single
# # jump to {{ pia_iptables_chain }}
# # at position 1, and that
# # the chain-wide policy
# # is still whatever it
# # was before this role
# # (ACCEPT, typically)
# sudo iptables -S {{ pia_iptables_chain }} # the kill-switch chain
# # itself: ACCEPT pair
# # when pia-wg is up,
# # LOG+DROP catch-all
# # always present
# sudo ufw status verbose | grep 53 # the Technitium INPUT
# # allow, both protocols
#
# Rollback (this file's part only — see routing.yaml/README.md for the
# rest of the role's rollback):
# sudo ufw delete allow from 10.10.50.0/24 to 10.10.40.53 port 53 proto tcp
# sudo ufw delete allow from 10.10.50.0/24 to 10.10.40.53 port 53 proto udp
- name: Allow VLAN 50 clients to reach Technitium DNS (local INPUT traffic)
community.general.ufw:
rule: allow
direction: in
src: "{{ pia_vlan50_subnet }}"
dest: "{{ k3s_dns_resolver }}"
port: "53"
proto: "{{ item }}"
comment: VLAN 50 clients to Technitium (local, not routed)
loop:
- tcp
- udp

42
ansible/roles/pia-gateway/tasks/healthcheck.yaml Normal file
View File

@ -0,0 +1,42 @@
---
# Part of role: pia-gateway
# Included by: tasks/main.yaml
# Description: Installs the observability-only health check (script,
# oneshot service, timer). See templates/pia-gateway-healthcheck.sh.j2
# for exactly what it checks and why it never remediates.
- name: Deploy the health check script
ansible.builtin.template:
src: pia-gateway-healthcheck.sh.j2
dest: /usr/local/sbin/pia-gateway-healthcheck.sh
mode: "0700"
owner: root
group: root
- name: Deploy the health check systemd service
ansible.builtin.template:
src: pia-gateway-healthcheck.service.j2
dest: /etc/systemd/system/pia-gateway-healthcheck.service
mode: "0644"
owner: root
group: root
notify:
- Reload systemd daemon
- Restart pia-gateway-healthcheck timer
- name: Deploy the health check systemd timer
ansible.builtin.template:
src: pia-gateway-healthcheck.timer.j2
dest: /etc/systemd/system/pia-gateway-healthcheck.timer
mode: "0644"
owner: root
group: root
notify:
- Reload systemd daemon
- Restart pia-gateway-healthcheck timer
- name: Enable and start the health check timer
ansible.builtin.systemd:
name: pia-gateway-healthcheck.timer
enabled: true
state: started

32
ansible/roles/pia-gateway/tasks/interface.yaml Normal file
View File

@ -0,0 +1,32 @@
---
# Part of role: pia-gateway
# Included by: tasks/main.yaml
# Description: Enables and starts the pia-wg WireGuard interface itself.
# The config file was already written by tasks/register.yaml (only on
# first run / forced re-registration); this task just makes sure the
# systemd unit is enabled and running, and that pia-killswitch.service
# (tasks/routing.yaml) has already run first — both are WantedBy
# multi-user.target with an explicit Before= from the kill switch, so a
# normal boot orders them correctly without help here, but a first
# Ansible run on an already-booted host needs the same ordering applied
# explicitly since neither unit was part of a boot transaction yet.
- name: Confirm the kill switch is active before bringing up pia-wg
ansible.builtin.systemd:
name: pia-killswitch.service
register: pia_killswitch_status
failed_when: false
- name: Assert the kill switch is active
ansible.builtin.assert:
that:
- pia_killswitch_status.status.ActiveState == "active"
fail_msg: >-
pia-killswitch.service is not active — refusing to start
{{ pia_wg_interface }} without the kill switch already seeded.
- name: Enable and start wg-quick@{{ pia_wg_interface }}
ansible.builtin.systemd:
name: "wg-quick@{{ pia_wg_interface }}"
enabled: true
state: started

53
ansible/roles/pia-gateway/tasks/main.yaml Normal file
View File

@ -0,0 +1,53 @@
---
# Part of role: pia-gateway
# Called by: ansible/playbooks/pia-gateway.yaml
# Description: Orchestrates minisforum's PIA WireGuard egress gateway for
# VPN VLAN 50 — registration, boot-ordered kill switch + policy routing,
# the WireGuard interface itself, and health checks. Split into included
# files because each stage has a distinct idempotency/ordering concern;
# see the comment at the top of each included file.
- name: Install WireGuard tooling and iproute2
ansible.builtin.apt:
name:
- wireguard
- wireguard-tools
- iproute2
- python3-requests
state: present
update_cache: true
- name: Pin reverse-path filtering for {{ pia_wg_interface }}
ansible.posix.sysctl:
name: "net.ipv4.conf.{{ pia_wg_interface }}.rp_filter"
value: "{{ pia_rp_filter_mode | string }}"
sysctl_file: /etc/sysctl.d/60-pia-gateway.conf
reload: false # interface does not exist yet on a first run; applied for real after the WG interface comes up (handlers/main.yaml)
register: pia_rp_filter_pinned
- name: Confirm host default route and ip_forward are untouched
ansible.builtin.include_tasks: assert-baseline.yaml
- name: Register minisforum with PIA and obtain WireGuard peer config
ansible.builtin.include_tasks: register.yaml
no_log: true # PIA credentials and the derived private key pass through this include
- name: Configure boot-ordered kill switch and policy routing
ansible.builtin.include_tasks: routing.yaml
- name: Configure firewall-layer kill switch and Technitium access
ansible.builtin.include_tasks: firewall.yaml
- name: Write and enable the pia-wg WireGuard interface
ansible.builtin.include_tasks: interface.yaml
- name: Install gateway health checks
ansible.builtin.include_tasks: healthcheck.yaml
- name: Re-apply rp_filter now that {{ pia_wg_interface }} exists
ansible.posix.sysctl:
name: "net.ipv4.conf.{{ pia_wg_interface }}.rp_filter"
value: "{{ pia_rp_filter_mode | string }}"
sysctl_file: /etc/sysctl.d/60-pia-gateway.conf
reload: true
when: pia_rp_filter_pinned is succeeded

383
ansible/roles/pia-gateway/tasks/register.yaml Normal file
View File

@ -0,0 +1,383 @@
---
# Part of role: pia-gateway
# Included by: tasks/main.yaml (with no_log: true — this file handles the
# PIA account password, the derived WireGuard private key, and the
# short-lived PIA auth token)
# Description: Registers minisforum as a PIA WireGuard peer and writes
# {{ pia_wg_config_dir }}/{{ pia_wg_interface }}.conf. Skipped once that
# file exists, unless pia_force_reregister or pia_force_key_rotation is
# set — see defaults/main.yaml for exactly what each does.
#
# Status as of 2026-08-24 (third pass — multi-region fallback added after
# all three JP Tokyo servers were observed failing live, one after
# another, against the single-region version of this file):
# - Token acquisition (`Request a PIA auth token` below) was completely
# rebuilt after live testing against the *actual* PIA API showed the
# first version's endpoint (a regional meta server's
# /authv3/generateToken, with PIA's private CA) doesn't exist as
# documented — it's unreachable/wrong. The correct, current flow
# (confirmed against PIA's own pia-foss/manual-connections
# get_token.sh, fetched and read in full, not from memory) is a
# single fixed POST to www.privateinternetaccess.com — a normal
# public domain with a normal publicly-trusted cert — with
# username/password as multipart form fields. It does not depend on
# pia_region/pia_region_candidates at all; only WireGuard server
# SELECTION does.
# - addKey (WireGuard key registration) was already close to correct on
# the first pass and is confirmed against
# connect_to_wireguard_with_token.sh, fetched and read in full: a
# regional server IP with SNI pinned to its hostname via
# --connect-to, validated against PIA's own CA bundle (still needed
# here — only the token endpoint uses the system CA store), token via
# stdin not argv. Failing over across only the servers *within* one
# region turned out not to be enough in practice — all three JP Tokyo
# servers failed the same way, one after another. This version tries
# every server in every configured region, in order (pia_region_
# candidates, or a single pia_region override — see defaults/
# main.yaml), not just every server within a single fixed region. The
# underlying request shape sent to each server is unchanged from the
# previous pass (still connect_to_wireguard_with_token.sh's flow) —
# if Hong Kong and Taiwan fail identically to how Tokyo did, that
# points at something shared across every region (the request itself,
# credentials, or minisforum's own network path), not at Tokyo
# specifically, and the per-attempt diagnostics below are what
# distinguishes those cases from three-independent-regions-all-down.
# - serverlist fetch + parse: unchanged, exercised live and working.
- name: Determine the effective list of PIA regions to try
ansible.builtin.set_fact:
pia_effective_regions: "{{ [pia_region] if (pia_region is not none and (pia_region | trim | length) > 0) else pia_region_candidates }}"
# A single -e pia_region=<id> override takes over completely (for
# troubleshooting one region in isolation) rather than being prepended
# to the candidate list — mixing the two would make "did it use my
# override or fall through to the list" ambiguous.
- name: Assert the effective region list is usable
ansible.builtin.assert:
that:
- pia_effective_regions is defined
- pia_effective_regions | length > 0
- pia_effective_regions | select('none') | list | length == 0
- pia_effective_regions | map('string') | map('trim') | select('equalto', '') | list | length == 0
fail_msg: >-
pia_region_candidates (or a single -e pia_region override) resolved
to an empty or invalid region list:
{{ pia_effective_regions | default('undefined') }}. Set
pia_region_candidates in defaults/main.yaml to a nonempty list of
PIA region ids (not display names), or pass -e pia_region=<id> for
a single region.
- name: Check whether a PIA WireGuard config already exists
ansible.builtin.stat:
path: "{{ pia_wg_config_dir }}/{{ pia_wg_interface }}.conf"
register: pia_wg_conf_stat
- name: PIA registration
when: pia_force_reregister or pia_force_key_rotation or not pia_wg_conf_stat.stat.exists
block:
- name: Ensure WireGuard config directory exists
ansible.builtin.file:
path: "{{ pia_wg_config_dir }}"
state: directory
mode: "0700"
owner: root
group: root
- name: Determine which private key path this run will use
ansible.builtin.set_fact:
pia_active_key_path: "{{ (pia_wg_config_dir + '/' + pia_wg_interface + '.key.new') if pia_force_key_rotation else (pia_wg_config_dir + '/' + pia_wg_interface + '.key') }}"
# Rotation stages the new key at a separate *.key.new path and only
# promotes it (see the "Promote the rotated key" block at the end
# of this file) after PIA has accepted it AND the new config has
# been written successfully — the previous key/config are never
# touched until both of those have actually succeeded, so a failed
# rotation leaves the working gateway exactly as it was rather
# than half-migrated.
- name: Generate minisforum's PIA WireGuard private key for this run
ansible.builtin.shell: |
set -euo pipefail
umask 077
wg genkey > {{ pia_active_key_path }}
args:
creates: "{{ pia_active_key_path }}"
- name: Set permissions on the active private key
ansible.builtin.file:
path: "{{ pia_active_key_path }}"
mode: "0600"
owner: root
group: root
- name: Read the active private key
ansible.builtin.slurp:
src: "{{ pia_active_key_path }}"
register: pia_private_key_raw
- name: Derive the public key
ansible.builtin.command: wg pubkey
args:
stdin: "{{ pia_private_key_raw.content | b64decode | trim }}"
register: pia_public_key
changed_when: false
- name: Fetch PIA's region/server list
ansible.builtin.uri:
url: https://serverlist.piaservers.net/vpninfo/servers/v6
return_content: true
register: pia_serverlist_raw
# The response body is one line of minified JSON followed by a
# detached signature block. Take just the first line (matching the
# standard `head -1` extraction used elsewhere for this endpoint) —
# do NOT split on a blank-line separator: that assumption was wrong
# in practice (observed failing against the live response) and
# silently handed the whole blob, signature included, to from_json.
- name: Parse the full region list
# Deliberately one physical line, not a folded (>-) block: an
# earlier version split a similar expression across two lines with
# the second more indented than the first, which YAML's folding
# rule ("more-indented lines are not folded") turns into a literal
# embedded newline instead of a space — that corrupted from_json's
# view of the string in practice (confirmed against the live
# endpoint) even though the newline sits outside any bracket. Keep
# this on one line if it's ever touched again.
ansible.builtin.set_fact:
pia_all_regions: "{{ (pia_serverlist_raw.content.split('\n')[0] | trim | from_json)['regions'] }}"
- name: Look up each requested region's metadata, in order
ansible.builtin.set_fact:
pia_region_lookup: "{{ (pia_region_lookup | default([])) + [pia_match] }}"
loop: "{{ pia_effective_regions }}"
loop_control:
loop_var: pia_region_id
vars:
pia_match_list: "{{ pia_all_regions | selectattr('id', 'equalto', pia_region_id) | list }}"
pia_match: "{{ pia_match_list[0] if (pia_match_list | length > 0) else {'id': pia_region_id, 'name': None, 'servers': {'wg': []}} }}"
# Builds pia_region_lookup as one entry per requested region, in
# the SAME order as pia_effective_regions (not the API's own region
# order) — that order is what makes "try hk, then taiwan, then
# japan" actually mean that. A region that doesn't exist in the
# live serverlist gets a sentinel entry (name: null, no wg
# servers) rather than being silently skipped, so it can be
# reported precisely below instead of just quietly trying one
# fewer region than requested.
- name: Identify any requested regions that don't exist in the current serverlist
ansible.builtin.set_fact:
pia_missing_regions: "{{ pia_region_lookup | selectattr('name', 'none') | map(attribute='id') | list }}"
- name: Identify any requested regions with no WireGuard servers
ansible.builtin.set_fact:
pia_empty_wg_regions: "{{ pia_region_lookup | rejectattr('name', 'none') | rejectattr('servers.wg') | map(attribute='id') | list }}"
- name: Assert every requested region exists and offers WireGuard
ansible.builtin.assert:
that:
- pia_missing_regions | length == 0
- pia_empty_wg_regions | length == 0
fail_msg: >-
One or more requested PIA regions are unusable — check each id
against the 'id' field of
https://serverlist.piaservers.net/vpninfo/servers/v6 (these are
API ids, not Gluetun's display names like "JP Tokyo"). Not
found in the current serverlist at all:
{{ pia_missing_regions | default([]) }}. Found but offer no
WireGuard servers right now:
{{ pia_empty_wg_regions | default([]) }}. Requested (in order):
{{ pia_effective_regions }}. Not silently substituting a
different region — fix pia_region_candidates/pia_region and
rerun.
- name: Build the ordered, flattened list of region+server candidates
ansible.builtin.set_fact:
pia_candidates: "{{ (pia_candidates | default([])) + [{'region_id': pia_pair.0.id, 'region_name': pia_pair.0.name, 'cn': pia_pair.1.cn, 'ip': pia_pair.1.ip}] }}"
loop: "{{ pia_region_lookup | subelements('servers.wg') }}"
loop_control:
loop_var: pia_pair
# subelements preserves both list orders: outer (region, matching
# pia_effective_regions) and inner (server, matching whatever order
# the live API returned for that region — never hardcoded here).
# Each flattened candidate keeps region id, region display name,
# WireGuard hostname, and WireGuard IP together, so
# addkey-attempt.yaml and diagnostics never need to re-look-up the
# parent region for a given server.
- name: Assert at least one candidate server was found
ansible.builtin.assert:
that:
- pia_candidates | length > 0
fail_msg: >-
No WireGuard servers found across any requested region
({{ pia_effective_regions }}) — this should be unreachable given
the region-level assert above; if you see this, the serverlist
response shape has likely changed.
- name: Deploy PIA's CA bundle (WireGuard addKey endpoint only)
ansible.builtin.copy:
src: pia-ca.crt
dest: "{{ pia_wg_config_dir }}/pia-ca.crt"
mode: "0644"
owner: root
group: root
# Only the regional addKey endpoint (1337/addKey, below) needs
# this — it presents a cert chain that does not validate against a
# normal system CA store (confirmed live: curl exit 60, "unable to
# get local issuer certificate"), and PIA's own tooling pins this
# exact bundle rather than trusting the system store. The token
# endpoint (www.privateinternetaccess.com, below) is a normal
# public domain with a normal publicly-trusted cert and uses
# validate_certs: true / the system store — do not add --cacert or
# validate_certs: false there.
#
# Source: https://raw.githubusercontent.com/pia-foss/manual-connections/master/ca.rsa.4096.crt
# Re-fetched fresh and diffed byte-for-byte identical against the
# vendored copy on 2026-08-24.
# File SHA-256 (sha256sum of the .crt file's bytes — this is what
# to compare against a fresh `curl ... | sha256sum` to verify the
# vendored copy, NOT the same thing as the X.509 certificate
# fingerprint below):
# 32e9b1d1433ea97614f2a14c6e358e3f57c0570cc9f6b2ee812699ba696c66ab
# X.509 certificate fingerprint (openssl x509 -noout -fingerprint
# -sha256 — a hash of the DER-encoded certificate structure, a
# different value from the file SHA-256 above; useful for
# comparing against a cert viewed some other way, e.g. in a
# browser or `openssl s_client`):
# SHA256 Fingerprint=1F:D2:56:58:45:6E:AB:30:41:FB:A7:7C:CD:39:8A:B8:12:4E:DC:C1:B8:B2:FC:1D:55:FD:F6:B1:BB:FC:9D:70
- name: Request a PIA auth token
ansible.builtin.uri:
url: https://www.privateinternetaccess.com/api/client/v2/token
method: POST
body_format: form-multipart
body:
username: "{{ pia_user }}"
password: "{{ pia_password }}"
validate_certs: true
return_content: true
timeout: 15
register: pia_token_response
until: pia_token_response.status | default(0) == 200
retries: 3
delay: 5
failed_when: false
no_log: true
# Fixed public endpoint, system CA validation — see get_token.sh.
# Independent of pia_region entirely.
- name: Capture safe (non-credential) diagnostics from the token request
ansible.builtin.set_fact:
pia_token_status: "{{ pia_token_response.status | default(-1) }}"
pia_token_msg: "{{ pia_token_response.msg | default('') }}"
# Deliberately NOT no_log. .status (an HTTP code) and .msg (Ansible's
# own generated "Status code was N and not [200]: ..." text) never
# contain the credential, the token, or the request body — verified
# empirically before use here, not assumed. What DOES contain
# secrets and stays behind no_log: pia_token_response itself (its
# .json/.content on a 200 IS the token; its .invocation.module_args
# is the literal username/password), and pia_user/pia_password.
- name: Fail clearly if the auth token request failed
ansible.builtin.fail:
msg: >-
PIA auth token request failed: HTTP {{ pia_token_status }}
({{ pia_token_msg }}) from
www.privateinternetaccess.com/api/client/v2/token
when: pia_token_status | int != 200
- name: Extract the auth token
ansible.builtin.set_fact:
pia_auth_token: "{{ pia_token_response.json.token | default('') }}"
no_log: true
- name: Assert the token is non-empty
ansible.builtin.assert:
that:
- pia_auth_token | length > 0
fail_msg: "PIA token response (HTTP 200) did not contain a non-empty token"
- name: Attempt WireGuard key registration against each region/server candidate, in order
ansible.builtin.include_tasks: addkey-attempt.yaml
loop: "{{ pia_candidates }}"
loop_control:
loop_var: pia_candidate
# pia_candidates is already flattened and ordered: every server in
# the first requested region, then every server in the second, and
# so on (built above via subelements over pia_region_lookup — never
# re-sorted or grouped differently here). Every meaningful task in
# addkey-attempt.yaml is individually guarded with
# `when: pia_peer is not defined` (re-evaluated fresh for each loop
# iteration) — confirmed empirically that a guard on this include
# statement itself does NOT re-evaluate per iteration and would NOT
# actually stop later candidates from being attempted after an
# earlier one already succeeded; the guard has to live on the inner
# tasks, which is also where addkey-attempt.yaml resets its
# per-attempt state so a stale parsed response from one candidate
# can never be mistaken for another's.
- name: Assert PIA accepted the registration against at least one candidate
ansible.builtin.assert:
that:
- pia_peer is defined
- pia_peer.status == "OK"
- pia_region_used is defined
- pia_wg_server_used is defined
fail_msg: >-
PIA addKey failed against every candidate: {{ pia_candidates |
length }} server(s) across {{ pia_effective_regions | length }}
region(s) ({{ pia_effective_regions }}) all failed. See the
per-attempt diagnostics logged above for exactly why each one
failed (timeout, TLS, HTTP error, empty/malformed response, or
a parsed non-OK status). If every region failed the same way,
that points at something shared across all of them — the
request itself, credentials, or minisforum's own network path —
not at any one region being down.
- name: Write the pia-wg WireGuard interface config
ansible.builtin.template:
src: pia-wg.conf.j2
dest: "{{ pia_wg_config_dir }}/{{ pia_wg_interface }}.conf"
mode: "0600"
owner: root
group: root
vars:
pia_private_key: "{{ pia_private_key_raw.content | b64decode | trim }}"
no_log: true
notify: Restart pia-wg
# ansible.builtin.template writes atomically (temp file + rename),
# so a failure partway through this specific task cannot leave a
# partially-written config — the previous file, if any, is left
# untouched. Everything that can fail on PIA's side (registration,
# both API calls) happens strictly before this point, so the
# previous working config is never at risk from those failures
# either.
- name: Promote the rotated key now that registration and config write both succeeded
when: pia_force_key_rotation
block:
- name: Check whether a previous key exists to back up
ansible.builtin.stat:
path: "{{ pia_wg_config_dir }}/{{ pia_wg_interface }}.key"
register: pia_previous_key_stat
- name: Back up the previous key
ansible.builtin.command:
cmd: >-
cp -p {{ pia_wg_config_dir }}/{{ pia_wg_interface }}.key
{{ pia_wg_config_dir }}/{{ pia_wg_interface }}.key.bak-{{ ansible_date_time.iso8601_basic_short }}
when: pia_previous_key_stat.stat.exists
changed_when: true
- name: Promote the new key into place
ansible.builtin.command:
cmd: >-
mv {{ pia_wg_config_dir }}/{{ pia_wg_interface }}.key.new
{{ pia_wg_config_dir }}/{{ pia_wg_interface }}.key
changed_when: true
# Only reached after the config template task above has already
# succeeded (Ansible blocks/tasks run sequentially and stop on
# first failure) — by this point PIA has accepted the new key and
# the new config referencing it is already safely on disk, so
# promoting the key file itself (for next run's `creates:` check
# to find it under the normal, non-.new name) is the only
# remaining step, not a point where failure could strand anything.

42
ansible/roles/pia-gateway/tasks/routing.yaml Normal file
View File

@ -0,0 +1,42 @@
---
# Part of role: pia-gateway
# Included by: tasks/main.yaml
# Description: Registers the dedicated PIA policy-routing table and
# installs+enables the boot-time kill-switch service that seeds it
# (rule, terminal unreachable route, static drop rule) before
# wg-quick@{{ pia_wg_interface }} is allowed to start. See
# templates/pia-killswitch.sh.j2 for what the seeded state actually is.
- name: Register the PIA routing table name
ansible.builtin.lineinfile:
path: /etc/iproute2/rt_tables
line: "{{ pia_route_table_id }}\t{{ pia_route_table_name }}"
regexp: "^[0-9]+\\s+{{ pia_route_table_name }}$"
create: true
mode: "0644"
- name: Deploy the kill-switch seed script
ansible.builtin.template:
src: pia-killswitch.sh.j2
dest: /usr/local/sbin/pia-killswitch.sh
mode: "0700"
owner: root
group: root
notify: Restart pia-killswitch
- name: Deploy the pia-killswitch systemd unit
ansible.builtin.template:
src: pia-killswitch.service.j2
dest: /etc/systemd/system/pia-killswitch.service
mode: "0644"
owner: root
group: root
notify:
- Reload systemd daemon
- Restart pia-killswitch
- name: Enable and start the kill-switch service now
ansible.builtin.systemd:
name: pia-killswitch.service
enabled: true
state: started

7
ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.service.j2 Normal file
View File

@ -0,0 +1,7 @@
# Managed by Ansible (role: pia-gateway). Do not edit by hand.
[Unit]
Description=PIA gateway health check (observability only, no remediation)
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/pia-gateway-healthcheck.sh

110
ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.sh.j2 Normal file
View File

@ -0,0 +1,110 @@
#!/bin/bash
# Managed by Ansible (role: pia-gateway). Do not edit by hand.
#
# Observability only. This script NEVER modifies routing, firewall, or
# WireGuard state — a failed check must close the path (already true by
# construction, via the kill switch in tasks/routing.yaml/firewall.yaml),
# never "recover" it. Output goes to the journal
# (journalctl -u pia-gateway-healthcheck); wire alerting off of that
# separately (see plan.md Phase 9) rather than adding remediation here.
#
# The "route decision" check below confirms what the kernel FIB *would*
# do for a VLAN 50 source — it does not send a packet and is not proof of
# actual PIA egress or identity. That proof requires the packet-capture
# test matrix in plan.md Phase 5; do not treat a clean run of this script
# as substituting for it.
set -uo pipefail
WG_IF="{{ pia_wg_interface }}"
TABLE="{{ pia_route_table_name }}"
SUBNET="{{ pia_vlan50_subnet }}"
PRIORITY="{{ pia_rule_priority }}"
CHAIN="{{ pia_iptables_chain }}"
MAX_HANDSHAKE_AGE={{ pia_healthcheck_handshake_max_age_sec }}
PROBE_SRC="${SUBNET%.*}.2" # a representative VLAN 50 address for the route-decision probe only; never actually used as a source
fail=0
# Informational only (not pass/fail) — which region/server this host is
# actually registered against right now, per the comment
# tasks/register.yaml writes into the generated config. Since PIA
# WireGuard registration now falls back across multiple regions
# (pia_region_candidates in defaults/main.yaml), "which one is live"
# isn't implied by config alone — it's whichever one answered OK last
# time the playbook ran, not necessarily the first-configured one.
configured_peer=$(grep -m1 '^# PIA WireGuard server actually used' "{{ pia_wg_config_dir }}/{{ pia_wg_interface }}.conf" 2>/dev/null || true)
if [[ -n "${configured_peer}" ]]; then
echo "INFO configured-peer: ${configured_peer#\# }"
fi
if ip link show "${WG_IF}" up &>/dev/null; then
echo "PASS interface: ${WG_IF} is up"
else
echo "FAIL interface: ${WG_IF} is not up"
fail=1
fi
if command -v wg &>/dev/null && wg show "${WG_IF}" latest-handshakes &>/dev/null; then
handshake_epoch=$(wg show "${WG_IF}" latest-handshakes 2>/dev/null | awk '{print $2}')
now=$(date +%s)
if [[ -n "${handshake_epoch}" && "${handshake_epoch}" -gt 0 ]]; then
age=$((now - handshake_epoch))
if (( age <= MAX_HANDSHAKE_AGE )); then
echo "PASS handshake: ${age}s old (max ${MAX_HANDSHAKE_AGE}s)"
else
echo "FAIL handshake: ${age}s old, exceeds max ${MAX_HANDSHAKE_AGE}s"
fail=1
fi
else
echo "FAIL handshake: no handshake recorded yet"
fail=1
fi
else
echo "WARN handshake: unable to query wg show (interface absent or wg missing)"
fail=1
fi
if ip rule show | grep -qE "^${PRIORITY}:[[:space:]]*from ${SUBNET} lookup ${TABLE}\$"; then
echo "PASS rule: from ${SUBNET} lookup ${TABLE} present at priority ${PRIORITY}"
else
echo "FAIL rule: from ${SUBNET} lookup ${TABLE} missing"
fail=1
fi
default_route=$(ip route show table "${TABLE}" 2>/dev/null | grep '^default' || true)
if [[ "${default_route}" == *"dev ${WG_IF}"* ]]; then
echo "PASS route: table ${TABLE} default is via ${WG_IF} (open)"
elif [[ "${default_route}" == *unreachable* ]] || ip route show table "${TABLE}" 2>/dev/null | grep -q '^unreachable default'; then
echo "PASS route: table ${TABLE} default is unreachable (closed, kill switch engaged)"
else
echo "FAIL route: table ${TABLE} has no default route at all (neither open via ${WG_IF} nor a terminal unreachable) — investigate immediately"
fail=1
fi
# Not a check on FORWARD's own default policy — that's deliberately left
# alone (see tasks/firewall.yaml for why: minisforum runs Flannel, which
# needs its own broad FORWARD ACCEPTs). What actually matters is that the
# dedicated chain exists, is jumped into first for this source, and ends
# in a real DROP.
if iptables -C FORWARD -s "${SUBNET}" -j "${CHAIN}" 2>/dev/null; then
echo "PASS firewall: FORWARD jumps to ${CHAIN} for ${SUBNET}"
else
echo "FAIL firewall: FORWARD does not jump to ${CHAIN} for ${SUBNET}"
fail=1
fi
if iptables -S "${CHAIN}" 2>/dev/null | grep -qE '^-A '"${CHAIN}"' -j DROP$'; then
echo "PASS firewall: ${CHAIN} ends in an unconditional DROP"
else
echo "FAIL firewall: ${CHAIN} has no unconditional DROP catch-all"
fail=1
fi
route_decision=$(ip route get 1.1.1.1 from "${PROBE_SRC}" 2>&1 || true)
if [[ "${route_decision}" == *"dev ${WG_IF}"* ]]; then
echo "PASS route-decision (not proof of live egress — see header comment): ${SUBNET} sourced traffic resolves via ${WG_IF}"
else
echo "WARN route-decision (not proof of live egress — see header comment): ${route_decision}"
fi
exit "${fail}"

11
ansible/roles/pia-gateway/templates/pia-gateway-healthcheck.timer.j2 Normal file
View File

@ -0,0 +1,11 @@
# Managed by Ansible (role: pia-gateway). Do not edit by hand.
[Unit]
Description=Run the PIA gateway health check every {{ pia_healthcheck_interval_sec }}s
[Timer]
OnBootSec={{ pia_healthcheck_interval_sec }}
OnUnitActiveSec={{ pia_healthcheck_interval_sec }}
AccuracySec=5s
[Install]
WantedBy=timers.target

20
ansible/roles/pia-gateway/templates/pia-killswitch.service.j2 Normal file
View File

@ -0,0 +1,20 @@
# Managed by Ansible (role: pia-gateway). Do not edit by hand.
#
# Seeds the VLAN 50 kill switch (policy-routing rule, terminal unreachable
# route, static drop rule) before wg-quick@{{ pia_wg_interface }}.service
# is allowed to start — see the drop-in this role installs on that unit
# (wg-quick-order.conf.j2) for the other half of the ordering.
[Unit]
Description=PIA gateway kill switch (VLAN 50 policy routing seed)
Before=wg-quick@{{ pia_wg_interface }}.service
DefaultDependencies=no
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/sbin/pia-killswitch.sh
[Install]
WantedBy=multi-user.target

92
ansible/roles/pia-gateway/templates/pia-killswitch.sh.j2 Normal file
View File

@ -0,0 +1,92 @@
#!/bin/bash
# Managed by Ansible (role: pia-gateway). Do not edit by hand.
#
# Runs once at boot, before wg-quick@{{ pia_wg_interface }}.service
# (enforced by the systemd drop-in in pia-killswitch.service's
# Before=/wg-quick's Requires=After=), and again any time UFW is
# reloaded (see handlers/main.yaml — "Reload ufw" fires before "Restart
# pia-killswitch" so a reload can never leave a permanent gap). Seeds the
# closed state that PIA gateway's kill switch relies on:
#
# - a source rule sending {{ pia_vlan50_subnet }} to the dedicated
# "{{ pia_route_table_name }}" table, evaluated after the kernel's
# built-in "local" rule (priority 0) and before "main" (32766) — this
# is what keeps Technitium (10.10.40.53, a local address) delivered
# via the local table rather than pulled into policy routing.
# - a terminal "unreachable" default route in that table — but only
# when {{ pia_wg_interface }} isn't already up — so VLAN 50 traffic
# fails closed instead of falling through to main if the interface is
# down, absent, or its PostUp hasn't run yet, without this script
# stomping an already-open, healthy route on a restart unrelated to
# the tunnel's own state (see the check below).
# - a DEDICATED "{{ pia_iptables_chain }}" chain, jumped into by exactly
# one rule at the very top of FORWARD (position 1), matching only
# source {{ pia_vlan50_subnet }}. This is deliberately NOT a global
# FORWARD default-policy change: minisforum is the k3s server and
# runs Flannel, which depends on its own broad FORWARD-chain
# ACCEPTs for pod-to-pod and pod-to-internet traffic (source
# 10.42.0.0/16, entirely disjoint from 10.10.50.0/24) — changing the
# chain-wide default policy was never actually proven safe against
# that, so this scopes the kill switch to a chain that Flannet/k3s
# traffic can never enter in the first place, rather than risking it.
# Being the FIRST rule in FORWARD also means no other, later rule
# (ufw's own, Docker's, anything) can accidentally pre-empt this
# source's fate with a broader ACCEPT — the jump is unconditional for
# this source and the chain itself ends in an unconditional DROP, not
# a RETURN, so nothing after it in FORWARD is ever consulted for
# this source either.
#
# Idempotent: safe to run more than once (e.g. `systemctl restart
# pia-killswitch`, or after this script re-runs following a UFW reload)
# without creating duplicate rules or duplicate chains.
set -euo pipefail
TABLE="{{ pia_route_table_name }}"
SUBNET="{{ pia_vlan50_subnet }}"
WG_IF="{{ pia_wg_interface }}"
PRIORITY="{{ pia_rule_priority }}"
CHAIN="{{ pia_iptables_chain }}"
if ! ip rule show | grep -qE "^${PRIORITY}:[[:space:]]*from ${SUBNET} lookup ${TABLE}\$"; then
ip rule add from "${SUBNET}" table "${TABLE}" priority "${PRIORITY}"
fi
# Only seed the terminal-unreachable fallback if {{ pia_wg_interface }}
# isn't already up. This script re-runs any time it's restarted for
# reasons unrelated to the tunnel's own state (a UFW reload, a content
# fix to this script itself) — if the interface is already up and
# healthy, its own PostUp already set the real "default dev %i" route in
# this table, and blindly overwriting that back to "unreachable" here
# would break a working tunnel's routing for no reason until something
# re-runs PostUp. Only actual absence of the interface should close it.
if ! ip link show "${WG_IF}" up &>/dev/null; then
ip route replace unreachable default table "${TABLE}"
fi
# Dedicated chain: create if missing (iptables -N fails harmlessly if it
# already exists, hence || true rather than a -C-style existence check —
# there is no direct "does this chain exist" check short of grepping -L).
iptables -N "${CHAIN}" 2>/dev/null || true
# Exactly one jump from FORWARD into it, at absolute position 1.
if ! iptables -C FORWARD -s "${SUBNET}" -j "${CHAIN}" 2>/dev/null; then
iptables -I FORWARD 1 -s "${SUBNET}" -j "${CHAIN}"
fi
# Static catch-all inside the chain: anything that reaches this point
# (i.e., wasn't already ACCEPTed by a more specific rule that
# pia-wg.conf.j2's PostUp inserts ahead of these when the interface is
# up) is logged and dropped. --log-limit is not a real iptables option
# (there is no such flag — rate-limiting a LOG target is done via a
# separate -m limit match ahead of it, as below); the -C check must
# match the rule as actually inserted, term for term, or it will never
# find it and silently re-append a duplicate on every run.
if ! iptables -C "${CHAIN}" -m limit --limit 5/minute -j LOG \
--log-prefix "PIA-KILLSWITCH-DROP: " 2>/dev/null; then
iptables -A "${CHAIN}" -m limit --limit 5/minute -j LOG \
--log-prefix "PIA-KILLSWITCH-DROP: "
fi
if ! iptables -C "${CHAIN}" -j DROP 2>/dev/null; then
iptables -A "${CHAIN}" -j DROP
fi

56
ansible/roles/pia-gateway/templates/pia-wg.conf.j2 Normal file
View File

@ -0,0 +1,56 @@
# Managed by Ansible (role: pia-gateway). Do not edit by hand — changes
# are overwritten on the next run of ansible/playbooks/pia-gateway.yaml.
#
# Table = off: wg-quick must NOT touch the main routing table. All routing
# for this interface lives in the dedicated "{{ pia_route_table_name }}"
# table (see tasks/routing.yaml), reached only by traffic sourced from
# {{ pia_vlan50_subnet }}. minisforum's own default route is never
# affected by this interface coming up or down.
[Interface]
PrivateKey = {{ pia_private_key }}
Address = {{ pia_peer.peer_ip }}/32
Table = off
# Kill-switch ordering: the boot-time pia-killswitch.service (see
# tasks/routing.yaml) has already seeded an "unreachable" default route in
# table {{ pia_route_table_name }} and created the dedicated
# {{ pia_iptables_chain }} chain (jumped into from FORWARD position 1 for
# source {{ pia_vlan50_subnet }} only — never touches Flannel/k3s traffic)
# ending in an unconditional LOG+DROP, before this unit ever starts
# (systemd After=/Requires=). PostUp only ever *opens* the path narrower
# than that pre-seeded closed state, by inserting ACCEPT rules ahead of
# the chain's own catch-all; PreDown removes exactly those same rules,
# re-closing it before removing anything else.
PostUp = ip route replace default dev %i table {{ pia_route_table_name }}
PostUp = iptables -I {{ pia_iptables_chain }} 1 -s {{ pia_vlan50_subnet }} -o %i -j ACCEPT
PostUp = iptables -I {{ pia_iptables_chain }} 1 -d {{ pia_vlan50_subnet }} -i %i -m state --state ESTABLISHED,RELATED -j ACCEPT
PostUp = iptables -t nat -A POSTROUTING -s {{ pia_vlan50_subnet }} -o %i -j MASQUERADE
{% if pia_mss_clamp_enabled %}
PostUp = iptables -t mangle -A FORWARD -s {{ pia_vlan50_subnet }} -o %i -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
{% endif %}
PreDown = ip route replace unreachable default table {{ pia_route_table_name }}
PreDown = iptables -t nat -D POSTROUTING -s {{ pia_vlan50_subnet }} -o %i -j MASQUERADE
PreDown = iptables -D {{ pia_iptables_chain }} -d {{ pia_vlan50_subnet }} -i %i -m state --state ESTABLISHED,RELATED -j ACCEPT
PreDown = iptables -D {{ pia_iptables_chain }} -s {{ pia_vlan50_subnet }} -o %i -j ACCEPT
{% if pia_mss_clamp_enabled %}
PreDown = iptables -t mangle -D FORWARD -s {{ pia_vlan50_subnet }} -o %i -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
{% endif %}
[Peer]
# PIA WireGuard server actually used — {{ pia_region_name_used }}
# ({{ pia_region_used }}), {{ pia_wg_server_used.cn }}. This is whichever
# region+server in the pia_region_candidates/pia_region fallback order
# (see defaults/main.yaml) responded first with status: OK — not
# necessarily the first-configured region. Re-run the playbook with
# pia_force_reregister (or check journalctl for the last apply's
# per-attempt diagnostics) to see what else was tried before this one.
PublicKey = {{ pia_peer.server_key }}
# The endpoint IP is the server we actually connected to
# (pia_wg_server_used, set by addkey-attempt.yaml on whichever candidate
# in servers.wg succeeded), not a field from the addKey JSON response —
# matching connect_to_wireguard_with_token.sh, which builds the Endpoint
# from its own $WG_SERVER_IP rather than trusting a response field for it.
Endpoint = {{ pia_wg_server_used.ip }}:{{ pia_peer.server_port }}
AllowedIPs = 0.0.0.0/0
PersistentKeepalive = 25

41
ansible/roles/vlan50-parent/README.md Normal file
View File

@ -0,0 +1,41 @@
# vlan50-parent
Creates `enp1s0.50`, a tagged VLAN 50 parent interface on `nik-debian`, for
Multus macvlan/ipvlan attachments (Phase 3/4 of `~/repo/homelab/plan.md`).
Leaves the existing untagged VLAN 40 config (`enp1s0`'s own address,
default route, k3s node identity, host DNS) untouched.
## Before running
- Requires the managed-switch port for `nik-debian` already carrying VLAN
50 tagged in addition to its existing VLAN 40 untagged/PVID 40 — this is
the `home-network`-repo-owned switch/Flint handoff from plan.md Phase 1.
This role has no way to verify that from the host side; if the switch
isn't actually passing tagged VLAN 50 frames yet, the subinterface will
come up with link state but no VLAN 50 traffic will ever arrive.
- Confirm console/recovery access to `nik-debian` (physical/IPMI/other
out-of-band) before applying, same as any host networking change.
## What it does
- Installs the `vlan` package and loads/persists the `8021q` kernel
module.
- Writes `/etc/network/interfaces.d/enp1s0.50` — `iface ... inet manual`
with `vlan-raw-device enp1s0` and `vlan-id 50`. No IP address is ever
assigned to it.
- Brings the interface up now (`ifup`) if not already present, and
asserts afterward that it's UP, carries no IPv4/IPv6 address, and that
the host's own default route is still via `enp1s0` — never via
`enp1s0.50`.
## Rollback
```bash
sudo ifdown enp1s0.50 || true
sudo ip link delete enp1s0.50 2>/dev/null || true
sudo rm -f /etc/network/interfaces.d/enp1s0.50
sudo rm -f /etc/modules-load.d/8021q.conf
```
Leaves the `vlan` package and `8021q` module load itself in place (harmless
if unused); remove manually only if desired.

20
ansible/roles/vlan50-parent/defaults/main.yaml Normal file
View File

@ -0,0 +1,20 @@
---
# Part of role: vlan50-parent
# Called by: ansible/playbooks/vlan50-parent.yaml
# Description: Default variables for nik-debian's tagged VLAN 50 parent
# interface. Implements Phase 3 of ~/repo/homelab/plan.md. Only apply
# this role after the home-network handoff confirms the managed-switch
# port for nik-debian is VLAN 40 untagged/PVID 40 *and* VLAN 50 tagged —
# this role does not and cannot verify switch-side config itself.
# Real parent interface on nik-debian — verified live 2026-08-23 via
# `ip -brief addr`; it is enp1s0, not "eth0".
vlan50_parent_interface: enp1s0
vlan50_id: 50
vlan50_subinterface: "{{ vlan50_parent_interface }}.{{ vlan50_id }}"
# No L3 address is assigned to the subinterface by this role (plan.md:
# "prefer an UP L2 parent with no L3 address" — Multus's macvlan/ipvlan
# CNI attaches directly to it at L2; the host itself never needs an
# address on VLAN 50).
vlan50_mtu: 1500

74
ansible/roles/vlan50-parent/tasks/main.yaml Normal file
View File

@ -0,0 +1,74 @@
---
# Part of role: vlan50-parent
# Called by: ansible/playbooks/vlan50-parent.yaml
# Description: Creates nik-debian's persistent tagged VLAN 50 subinterface
# ({{ vlan50_subinterface }}) without touching its existing untagged
# VLAN 40 config (node IP, default route, k3s identity, host DNS). See
# this role's defaults/main.yaml for the switch-readiness precondition.
- name: Install the vlan package (ifupdown VLAN support)
ansible.builtin.apt:
name: vlan
state: present
update_cache: true
- name: Load the 8021q kernel module now
community.general.modprobe:
name: 8021q
state: present
- name: Persist the 8021q kernel module across reboots
ansible.builtin.copy:
dest: /etc/modules-load.d/8021q.conf
content: |
# Managed by Ansible (role: vlan50-parent).
8021q
mode: "0644"
owner: root
group: root
- name: Write the VLAN 50 subinterface definition
ansible.builtin.template:
src: vlan50.interface.j2
dest: "/etc/network/interfaces.d/{{ vlan50_subinterface }}"
mode: "0644"
owner: root
group: root
register: vlan50_iface_file
- name: Check whether the subinterface is already up
ansible.builtin.command: "ip link show {{ vlan50_subinterface }}"
register: vlan50_iface_check
changed_when: false
failed_when: false
- name: Bring up the VLAN 50 subinterface
ansible.builtin.command: "ifup {{ vlan50_subinterface }}"
when: vlan50_iface_check.rc != 0 or vlan50_iface_file is changed
- name: Confirm the subinterface is up with no L3 address
ansible.builtin.command: "ip -brief addr show {{ vlan50_subinterface }}"
register: vlan50_iface_state
changed_when: false
- name: Assert VLAN 50 parent is UP and carries no IPv4/IPv6 address
ansible.builtin.assert:
that:
- "'UP' in vlan50_iface_state.stdout or 'UNKNOWN' in vlan50_iface_state.stdout"
- vlan50_iface_state.stdout.split() | select('match', '^[0-9]+\\.[0-9]+\\.[0-9]+\\.[0-9]+/') | list | length == 0
fail_msg: "{{ vlan50_subinterface }} is either down or unexpectedly carries an IPv4 address: {{ vlan50_iface_state.stdout }}"
- name: Read current default route
ansible.builtin.command: ip -4 route show default
register: vlan50_current_default_route
changed_when: false
- name: Assert the host default route is still the untagged VLAN 40 interface
ansible.builtin.assert:
that:
- vlan50_current_default_route.stdout is search('dev ' + vlan50_parent_interface + '(\s|$)')
- not (vlan50_current_default_route.stdout is search('dev ' + vlan50_subinterface + '(\s|$)'))
fail_msg: >-
nik-debian's default route is not via {{ vlan50_parent_interface }}
({{ vlan50_current_default_route.stdout }}) — VLAN 50 must never
become the host default route.

13
ansible/roles/vlan50-parent/templates/vlan50.interface.j2 Normal file
View File

@ -0,0 +1,13 @@
# Managed by Ansible (role: vlan50-parent). Do not edit by hand.
#
# Tagged VLAN 50 parent for Multus macvlan/ipvlan attachments — see
# ~/repo/homelab/plan.md Phase 3/4. Deliberately "manual" (no inet
# config): this interface must never carry a host IP or become a route
# candidate. Untagged VLAN 40 on {{ vlan50_parent_interface }} itself
# (node IP, default route, k3s identity) is defined elsewhere in
# /etc/network/interfaces and is not touched by this file.
auto {{ vlan50_subinterface }}
iface {{ vlan50_subinterface }} inet manual
vlan-raw-device {{ vlan50_parent_interface }}
vlan-id {{ vlan50_id }}
mtu {{ vlan50_mtu }}