Nik Afiq 69d880186a
Some checks failed
validate / lint (push) Failing after 1s
fix: open Flannel/K3s ports in gpu-node's UFW rules
host_vars/gpu-node.yaml's ufw_allowed_ports has overridden (not extended)
the common role's default list since the node was added, silently dropping
the Flannel VXLAN (8472/udp), K3s API (6443/tcp), and Kubelet (10250/tcp)
rules every other node gets. Went unnoticed because kubectl logs/exec/stats
tunnel through the agent's outbound connection to the k3s server rather
than needing a direct inbound path - but real pod dataplane traffic (e.g.
tts-gateway on nik-gpu resolving DNS against CoreDNS on nik-debian) needs
actual VXLAN connectivity and was blackholing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-25 00:05:54 +09:00
..

Ansible

This directory contains host-level automation. It bootstraps machines, installs K3s, prepares storage, and manages services that intentionally run outside the cluster.

Inventory

inventory.yaml defines four groups:

Group Host Purpose
k3s_server minisforum K3s server at 192.168.7.77
k3s_agents debian K3s agent and NFS storage at 192.168.7.183
mac_mini mac-mini Docker/Ollama host at 192.168.7.96
gpu_workstation gpu-node K3s agent with NVIDIA GPU passthrough at 192.168.7.98 (spot-tainted)

All hosts use the nik user and the SSH key configured in inventory.yaml.

Collections

Install the third-party collections this repo's roles depend on before running any playbook:

ansible-galaxy collection install -r ansible/requirements.yml

(community.general, ansible.posix, community.docker.)

Common Playbooks

ansible-playbook -i ansible/inventory.yaml ansible/playbooks/bootstrap-minisforum.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-k3s.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-nfs-debian.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/join-debian-agent.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-gpu-node.yaml -K

Additional services:

export GITEA_RUNNER_TOKEN=...
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-monitoring.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-gitea-runner.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-glances-debian.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-ollama.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/deploy-watch-party.yaml
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/wireguard.yaml -K
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/homeassistant.yaml -K

Roles

Role Responsibility
common Packages, user setup, firewall, base data directories
docker Docker CE install (Debian and Ubuntu); depended on by homeassistant
nvidia NVIDIA driver, CUDA toolkit, and containerd/Docker GPU runtime config
k3s-server K3s server install, kubeconfig fetch, Helm install, primary node label
k3s-agent K3s agent join and storage/GPU node label
nfs-server Export /mnt/storage from Debian to the K3s server
monitoring Host directories and ownership for Prometheus/Loki
gitea-runner Gitea Actions runner systemd service
glances Host-level Glances service
ollama Ollama service on the Mac Mini and GPU node (branches on OS)
watch-party Watch Party Docker Compose deployment on the Mac Mini
wireguard WireGuard server configuration
homeassistant Standalone Home Assistant deployment (Docker Compose + systemd on minisforum) — this is the only thing serving ha.home.arpa, not legacy/dead

Notes

  • K3s version is defined in three places and must be kept in sync: roles/k3s-server/defaults/main.yaml, roles/k3s-agent/defaults/main.yaml, and the override in host_vars/gpu-node.yaml.
  • setup-gitea-runner.yaml reads GITEA_RUNNER_TOKEN from the local environment.
  • The K3s role disables bundled Traefik because Traefik is managed by Argo CD.
  • The Debian storage role exports /mnt/storage; several Kubernetes manifests mount that export directly.
  • Keep host automation idempotent where practical. These playbooks are meant to be rerunnable during rebuilds.
  • To see the real K3s join token (needed once, to populate vault_k3s_node_token), pass -e k3s_show_token=true to setup-k3s.yaml; it's suppressed by default. Same pattern for WireGuard client configs via -e wireguard_show_client_configs=true on wireguard.yaml.