Found live, 2026-09-22, on the addKey step of the first successful kill-switch-check pass: every candidate failed with kind=tls detail="certificate verification failed". Root-caused on minisforum: PIA's own CA certificate (files/pia-ca.crt) has a CA:TRUE Basic Constraints extension that isn't marked critical - a real RFC 5280 non-conformance, confirmed not a stale vendored copy (a fresh fetch from pia-foss/manual-connections is byte-identical, same sha256 already recorded in the file's own header, same defect). Recent OpenSSL (3.2+; confirmed via a direct repro against minisforum's 3.5.7 - SSLCertVerificationError, verify_message "Basic Constraints of CA cert not marked critical") rejects this by default, a check promoted from X509_STRICT-only into the baseline chain-verification path. openssl s_client (LibreSSL, this dev machine) and curl both accept the identical cert/chain/CA file - only Python's ssl module, via this host's system OpenSSL, rejects it. This went unnoticed until now because the three prior successful registrations all went through curl --cacert (the legacy Ansible path); this was the first time the Python ssl code path ran for real. Python's stdlib ssl module has no custom verify-callback hook, so there's no clean way to tolerate just this one known, third-party-controlled defect while keeping full validation for everything else. Rather than weakening verify_mode or adding a new dependency (pyOpenSSL, for a real callback - considered, rejected as a bigger lift than necessary) to route around a gap in a library we don't control, addKey now shells out to curl (HttpClient.get_pinned_curl, curl added to this role's apt list) - the same tool PIA's own reference client already uses for this exact endpoint, with --cacert and never -k/--insecure, so this is a different, already-proven-correct TLS backend for the one endpoint that needs it, not a downgrade. serverlist/token are unaffected (public WebPKI certs) and still use the stdlib ssl path. Verified: a real subprocess curl call against the actual failing candidate (Server-11669-10a / 69.33.198.97) gets a genuine HTTP response from PIA, proving TLS validation succeeds via this path. Also found and handled: curl's own exit code for a --fail-triggered rejection isn't fully portable (macOS/SecureTransport curl reports 56 where GNU/OpenSSL curl reports 22 for the same case) - classification now also matches curl's own error text as a second signal, not just the exit code. Added 8 tests (94 total, all passing) covering the curl invocation shape, stdin token delivery (no trailing newline, never in argv), and exit-code + text-based error classification including the cross-platform exit-code case found live. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
169 lines
6.7 KiB
YAML
169 lines
6.7 KiB
YAML
---
|
|
# Part of role: pia-gateway
|
|
# Called by: ansible/playbooks/pia-gateway.yaml
|
|
# Description: Orchestrates minisforum's PIA WireGuard egress gateway for
|
|
# VPN VLAN 50 — kill switch + policy routing, the on-host recovery
|
|
# controller, registration, the WireGuard interface itself, and health
|
|
# checks. Split into included files because each stage has a distinct
|
|
# idempotency/ordering concern; see the comment at the top of each.
|
|
#
|
|
# Ordering (changed with the auto-recovery work, deliberately):
|
|
# 1. tooling, baseline assertions
|
|
# 2. controller program + config + credentials (NOT its timers)
|
|
# 3. quiesce the recovery/renewal timers for the rest of the run so a
|
|
# timer-driven restart can never race an Ansible handler restart
|
|
# 4. kill switch + routing + firewall, handlers flushed so the closed
|
|
# state is live BEFORE anything can activate a tunnel
|
|
# 5. registration (shared backend activates + verifies under the lock;
|
|
# the legacy backend only writes the config and notifies a restart)
|
|
# 6. interface unit enabled, health check installed, handlers flushed
|
|
# 7. timers enabled/disabled per pia_auto_recovery_enabled /
|
|
# pia_renewal_enabled
|
|
|
|
- name: Install WireGuard tooling and iproute2
|
|
ansible.builtin.apt:
|
|
name:
|
|
- wireguard
|
|
- wireguard-tools
|
|
- iproute2
|
|
- python3
|
|
- curl
|
|
state: present
|
|
update_cache: true
|
|
# python3-requests is no longer needed: the controller uses only the
|
|
# standard library, EXCEPT for the addKey endpoint specifically, which
|
|
# shells out to curl — see HttpClient.get_pinned_curl's own docstring
|
|
# for why (a non-critical Basic Constraints extension on PIA's own CA
|
|
# cert that recent OpenSSL rejects by default and Python's ssl module
|
|
# has no way to selectively tolerate). curl is virtually always already
|
|
# present on Debian, but declared explicitly rather than assumed.
|
|
|
|
- name: Pin reverse-path filtering for {{ pia_wg_interface }}
|
|
ansible.posix.sysctl:
|
|
name: "net.ipv4.conf.{{ pia_wg_interface }}.rp_filter"
|
|
value: "{{ pia_rp_filter_mode | string }}"
|
|
sysctl_file: /etc/sysctl.d/60-pia-gateway.conf
|
|
reload: false # interface does not exist yet on a first run; applied for real after the WG interface comes up
|
|
register: pia_rp_filter_pinned
|
|
|
|
- name: Confirm host default route and ip_forward are untouched
|
|
ansible.builtin.include_tasks: assert-baseline.yaml
|
|
|
|
- name: Install the recovery controller, its config and credentials
|
|
ansible.builtin.include_tasks: controller.yaml
|
|
|
|
- name: Check whether the recovery timer is installed from a previous run
|
|
ansible.builtin.stat:
|
|
path: /etc/systemd/system/pia-gateway-recover.timer
|
|
register: pia_recover_timer_stat
|
|
|
|
- name: Quiesce the recovery and renewal timers for the duration of this run
|
|
ansible.builtin.systemd:
|
|
name: "{{ item }}"
|
|
state: stopped
|
|
loop:
|
|
- pia-gateway-recover.timer
|
|
- pia-gateway-renew.timer
|
|
when: pia_recover_timer_stat.stat.exists
|
|
failed_when: false
|
|
# failed_when: false — on the very first install the units were just
|
|
# deployed and never started; "stopping" them is a harmless no-op that
|
|
# some systemd versions report as an error for never-loaded timers.
|
|
|
|
- name: Wait for any in-flight recovery/renewal pass to finish
|
|
ansible.builtin.command:
|
|
cmd: systemctl is-active pia-gateway-recover.service pia-gateway-renew.service
|
|
register: pia_inflight
|
|
changed_when: false
|
|
failed_when: false
|
|
until: "'active' not in (pia_inflight.stdout_lines | default([]) | map('trim') | list)"
|
|
retries: 90
|
|
delay: 10
|
|
when: pia_recover_timer_stat.stat.exists
|
|
# A pass can take up to pia_recovery_max_duration_sec (+ restart waits);
|
|
# we never kill it mid-activation. Registration below also takes the
|
|
# program's own lock, so this is belt-and-braces for the handler restarts.
|
|
|
|
- name: Configure boot-ordered kill switch and policy routing
|
|
ansible.builtin.include_tasks: routing.yaml
|
|
|
|
- name: Configure firewall-layer kill switch and Technitium access
|
|
ansible.builtin.include_tasks: firewall.yaml
|
|
|
|
- name: Apply kill-switch handlers before anything can activate a tunnel
|
|
ansible.builtin.meta: flush_handlers
|
|
|
|
- name: Register minisforum with PIA (shared on-host transaction)
|
|
ansible.builtin.include_tasks: register-shared.yaml
|
|
when: pia_registration_backend == "shared"
|
|
|
|
- name: Register minisforum with PIA (legacy in-Ansible flow)
|
|
ansible.builtin.include_tasks: register-legacy.yaml
|
|
no_log: true # PIA credentials and the derived private key pass through this include
|
|
when: pia_registration_backend == "legacy"
|
|
|
|
- name: Write and enable the pia-wg WireGuard interface
|
|
ansible.builtin.include_tasks: interface.yaml
|
|
|
|
- name: Install gateway health checks
|
|
ansible.builtin.include_tasks: healthcheck.yaml
|
|
|
|
- name: Re-apply rp_filter now that {{ pia_wg_interface }} exists
|
|
ansible.posix.sysctl:
|
|
name: "net.ipv4.conf.{{ pia_wg_interface }}.rp_filter"
|
|
value: "{{ pia_rp_filter_mode | string }}"
|
|
sysctl_file: /etc/sysctl.d/60-pia-gateway.conf
|
|
reload: true
|
|
when: pia_rp_filter_pinned is succeeded
|
|
|
|
- name: Apply remaining handlers while the timers are still quiesced
|
|
ansible.builtin.meta: flush_handlers
|
|
|
|
- name: Re-render pia-wg.conf when only the controller config changed (no PIA call)
|
|
ansible.builtin.command:
|
|
cmd: "{{ pia_gateway_program }} rewrite-config"
|
|
register: pia_rewrite_result
|
|
changed_when: pia_rewrite_result.rc == 0
|
|
failed_when: pia_rewrite_result.rc not in [0, 5]
|
|
when:
|
|
- pia_registration_backend == "shared"
|
|
- pia_gateway_config_result is changed
|
|
- not (pia_registered_this_run | default(false) | bool)
|
|
# Picks up hook-content changes (e.g. pia_mss_clamp_enabled) without a
|
|
# forced re-registration — the gap noted in the 2026-08-25 incident.
|
|
# Exit 5 = "no verified registration recorded yet" (legacy-registered
|
|
# host); that is not an error here, the next registration will render
|
|
# the current hooks.
|
|
|
|
- name: Enable the recovery timer
|
|
ansible.builtin.systemd:
|
|
name: pia-gateway-recover.timer
|
|
enabled: true
|
|
state: started
|
|
daemon_reload: true
|
|
when: pia_auto_recovery_enabled | bool
|
|
|
|
- name: Disable the recovery timer (pia_auto_recovery_enabled=false)
|
|
ansible.builtin.systemd:
|
|
name: pia-gateway-recover.timer
|
|
enabled: false
|
|
state: stopped
|
|
daemon_reload: true
|
|
when: not (pia_auto_recovery_enabled | bool)
|
|
|
|
- name: Enable the renewal timer
|
|
ansible.builtin.systemd:
|
|
name: pia-gateway-renew.timer
|
|
enabled: true
|
|
state: started
|
|
daemon_reload: true
|
|
when: (pia_auto_recovery_enabled | bool) and (pia_renewal_enabled | bool)
|
|
|
|
- name: Disable the renewal timer
|
|
ansible.builtin.systemd:
|
|
name: pia-gateway-renew.timer
|
|
enabled: false
|
|
state: stopped
|
|
daemon_reload: true
|
|
when: not ((pia_auto_recovery_enabled | bool) and (pia_renewal_enabled | bool))
|