Found live, 2026-09-22, on the addKey step of the first successful kill-switch-check pass: every candidate failed with kind=tls detail="certificate verification failed". Root-caused on minisforum: PIA's own CA certificate (files/pia-ca.crt) has a CA:TRUE Basic Constraints extension that isn't marked critical - a real RFC 5280 non-conformance, confirmed not a stale vendored copy (a fresh fetch from pia-foss/manual-connections is byte-identical, same sha256 already recorded in the file's own header, same defect). Recent OpenSSL (3.2+; confirmed via a direct repro against minisforum's 3.5.7 - SSLCertVerificationError, verify_message "Basic Constraints of CA cert not marked critical") rejects this by default, a check promoted from X509_STRICT-only into the baseline chain-verification path. openssl s_client (LibreSSL, this dev machine) and curl both accept the identical cert/chain/CA file - only Python's ssl module, via this host's system OpenSSL, rejects it. This went unnoticed until now because the three prior successful registrations all went through curl --cacert (the legacy Ansible path); this was the first time the Python ssl code path ran for real. Python's stdlib ssl module has no custom verify-callback hook, so there's no clean way to tolerate just this one known, third-party-controlled defect while keeping full validation for everything else. Rather than weakening verify_mode or adding a new dependency (pyOpenSSL, for a real callback - considered, rejected as a bigger lift than necessary) to route around a gap in a library we don't control, addKey now shells out to curl (HttpClient.get_pinned_curl, curl added to this role's apt list) - the same tool PIA's own reference client already uses for this exact endpoint, with --cacert and never -k/--insecure, so this is a different, already-proven-correct TLS backend for the one endpoint that needs it, not a downgrade. serverlist/token are unaffected (public WebPKI certs) and still use the stdlib ssl path. Verified: a real subprocess curl call against the actual failing candidate (Server-11669-10a / 69.33.198.97) gets a genuine HTTP response from PIA, proving TLS validation succeeds via this path. Also found and handled: curl's own exit code for a --fail-triggered rejection isn't fully portable (macOS/SecureTransport curl reports 56 where GNU/OpenSSL curl reports 22 for the same case) - classification now also matches curl's own error text as a second signal, not just the exit code. Added 8 tests (94 total, all passing) covering the curl invocation shape, stdin token delivery (no trailing newline, never in argv), and exit-code + text-based error classification including the cross-platform exit-code case found live. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Ansible
This directory contains host-level automation. It bootstraps machines, installs K3s, prepares storage, and manages services that intentionally run outside the cluster.
Inventory
inventory.yaml defines four groups:
| Group | Host | Purpose |
|---|---|---|
k3s_server |
minisforum |
K3s server at 10.10.40.53 |
k3s_agents |
debian |
K3s agent and NFS storage at 10.10.40.20 |
mac_mini |
mac-mini |
Docker/Ollama host at 10.10.40.30 |
gpu_workstation |
gpu-node |
K3s agent with NVIDIA GPU passthrough at 10.10.40.12 (spot-tainted) |
All hosts use the nik user and the SSH key configured in inventory.yaml.
Collections
Install the third-party collections this repo's roles depend on before running any playbook:
ansible-galaxy collection install -r ansible/requirements.yml
(community.general, ansible.posix, community.docker.)
Common Playbooks
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/bootstrap-minisforum.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-k3s.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-nfs-debian.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/join-debian-agent.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-gpu-node.yaml -K
Additional services:
export GITEA_RUNNER_TOKEN=...
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-monitoring.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-gitea-runner.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-glances-debian.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-ollama.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/deploy-watch-party.yaml
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/wireguard.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/homeassistant.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/pia-gateway.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/vlan50-parent.yaml -K
Roles
| Role | Responsibility |
|---|---|
common |
Packages, user setup, firewall, base data directories |
docker |
Docker CE install (Debian and Ubuntu); depended on by homeassistant |
nvidia |
NVIDIA driver, CUDA toolkit, and containerd/Docker GPU runtime config |
k3s-server |
K3s server install, kubeconfig fetch, Helm install, primary node label |
k3s-agent |
K3s agent join and storage/GPU node label |
nfs-server |
Export /mnt/storage from Debian to the K3s server |
monitoring |
Host directories and ownership for Prometheus/Loki |
gitea-runner |
Gitea Actions runner systemd service |
glances |
Host-level Glances service |
ollama |
Ollama service on the Mac Mini and GPU node (branches on OS) |
watch-party |
Watch Party Docker Compose deployment on the Mac Mini |
wireguard |
WireGuard server configuration (inbound home-VPN access — phone/Mac clients) |
pia-gateway |
Minisforum's PIA WireGuard egress gateway for VPN VLAN 50 (policy routing, kill switch, health checks) — see its own README |
vlan50-parent |
nik-debian's tagged VLAN 50 parent interface (enp1s0.50) for Multus — see its own README |
homeassistant |
Standalone Home Assistant deployment (Docker Compose + systemd on minisforum) — this is the only thing serving ha.home.arpa, not legacy/dead |
Notes
- K3s version is defined in three places and must be kept in sync:
roles/k3s-server/defaults/main.yaml,roles/k3s-agent/defaults/main.yaml, and the override inhost_vars/gpu-node.yaml. setup-gitea-runner.yamlreadsGITEA_RUNNER_TOKENfrom the local environment.- The K3s role disables bundled Traefik because Traefik is managed by Argo CD.
- The Debian storage role exports
/mnt/storage; several Kubernetes manifests mount that export directly. - Keep host automation idempotent where practical. These playbooks are meant to be rerunnable during rebuilds.
- To see the real K3s join token (needed once, to populate
vault_k3s_node_token), pass-e k3s_show_token=truetosetup-k3s.yaml; it's suppressed by default. Same pattern for WireGuard client configs via-e wireguard_show_client_configs=trueonwireguard.yaml.