homelab/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## What this is

Infrastructure-as-code for a small K3s homelab. There is no application source
code to build or test here — this repo *is* the infrastructure: Ansible for
host bootstrap, Argo CD for GitOps reconciliation of the cluster, and raw
Kubernetes manifests / Helm values for service configuration. "Testing" a
change means applying it (via `ansible-playbook`, `kubectl`, or letting Argo
CD sync) and checking real cluster/service state — there are no unit tests.

Full operational detail (bootstrap sequence, secrets workflow, storage
layout, URL list, gotchas) lives in the root `README.md` — read it before
making non-trivial changes. Scoped `README.md` files in `ansible/`,
`argocd/`, and `manifests/` cover directory-specific workflows.

## Operating rules for Claude Code

- **There is no staging environment.** Argo CD watches `main` on the Gitea
  remote directly, and most `Application`s run with `selfHeal: true` (see
  Architecture below) — so `git push` to `main`, especially anything under
  `manifests/`, `values/`, or `argocd/`, is a production deploy, not just a
  code change, and Argo CD will keep re-asserting it even against manual
  cluster edits. Never push to `main` without the user's explicit go-ahead
  for that specific push — an earlier approval to commit is not approval to
  push.
- **Never run a command that needs the user's own credentials** (API/personal
  access tokens, passwords, anything destined for `.env` or an MCP `-e`
  flag) directly, and don't hand it back as an inline `! command` either.
  Write the exact command to a file (e.g. under the session scratch/tmp
  directory — never committed) and ask the user to open it, review it, and
  run it themselves. This keeps secrets out of the conversation transcript.

## Architecture

Three-layer flow: **Ansible** brings up hosts and anything that must run
outside Kubernetes → **Argo CD** (`manifests/argocd/app-of-apps.yaml` →
`argocd/apps/*.yaml`) reconciles everything else from Git → each
`Application` points at either a raw manifest directory under `manifests/`
or a Helm chart configured by `values/*.yaml`.

Hosts (see `ansible/inventory.yaml`):

| Host | IP | Role |
| --- | --- | --- |
| `minisforum` | `10.10.40.53` | K3s server, Traefik entrypoint, primary app node |
| `debian` | `10.10.40.20` | K3s agent, NFS storage (`/mnt/storage`), secondary Pi-hole |
| `mac-mini` | `10.10.40.30` | Standalone Docker host — Watch Party, Ollama (not in the K3s cluster) |
| `gpu-node` | `10.10.40.12` | K3s agent with NVIDIA GPU, tainted `spot=true:NoSchedule`, labeled `node-role: gpu`; runs Ollama directly on the host, not as a pod |

K3s's bundled Traefik and ingress controller are disabled — Traefik is
installed and managed through Argo CD instead. Internal services live under
`home.arpa` (internal CA via cert-manager, no wildcard DNS — every hostname
must be added explicitly to **both** `values/pihole.yaml` and
`values/pihole-debian.yaml`). Public services live under `nik4nao.com`
(Let's Encrypt).

### Repo layout

| Path | Purpose |
| --- | --- |
| `ansible/` | Host bootstrap and non-Kubernetes services (inventory, roles, playbooks, `group_vars/`, `host_vars/`) |
| `argocd/apps/` | One file per Argo CD `Application` |
| `argocd/values/` | Helm values for installing/reconciling Argo CD itself |
| `manifests/` | Raw Kubernetes resources grouped by service area (see `manifests/README.md`) |
| `values/` | Helm values consumed by Argo CD `Application`s for third-party charts |
| `config/dashy/` | Dashy dashboard config, injected via a script (not the raw manifest) |
| `router/` | OpenWRT (UCI) config backup for the network router — not part of the Ansible/K3s flow |

### Ansible

Three inventory groups (`k3s_server`, `k3s_agents`, `mac_mini`) plus
`gpu_workstation` (see `ansible/inventory.yaml`). Per-host variables live in
`ansible/host_vars/<host>.yaml`; shared/secret vars in
`ansible/group_vars/all/vault.yaml`. Roles are composed per playbook, e.g.
`ansible/playbooks/setup-gpu-node.yaml` runs `common → docker → nvidia →
k3s-agent → ollama → glances` against `gpu_workstation`. Note there are two
`ansible.cfg` files (repo root and `ansible/`) — check which one is active
for the working directory a command is run from before assuming inventory
defaults.

The `ollama` role branches on `ansible_facts['system']` to support both the
Mac Mini (Homebrew + launchd) and Linux GPU nodes (install script +
systemd), so changes to it must keep both code paths working.

### Argo CD

Most `Application`s use automated sync with `selfHeal: true` and
`prune: false` — Argo CD corrects drift but does **not** delete cluster
resources removed from Git; matching manual cleanup is often required.
`targetRevision: main` is the default for repo-managed sources.

## Secrets

Never commit plaintext secrets. Two patterns coexist, both listed in
`manifests/README.md`:

- **Runtime scripts** (`manifests/**/*-secret.sh`) read `.env` (copy from
  `.env.example`) and create Kubernetes `Secret`s directly against the live
  cluster — nothing plaintext is committed.
- **Sealed Secrets** (`*-sealed.yaml`) are safe to commit; regenerate them
  with the matching script and commit the resulting YAML. `kubeseal` must
  point at the in-cluster `sealed-secrets-controller` (`kube-system`).

`ansible/group_vars/all/vault.yaml` holds Ansible-side secrets (e.g.
`vault_k3s_node_token`) referenced from `host_vars`.

## Validation commands

No cluster access is required for any of these — run them before reporting a
change done, and show the actual output, not just a claim it passed.

```bash
yamllint -c .yamllint.yml .
ansible-lint ansible/
for pb in ansible/playbooks/*.yaml; do
  ansible-playbook --syntax-check -i ansible/inventory.yaml "$pb"
done
kubeconform -summary -ignore-missing-schemas -kubernetes-version 1.32.0 \
  -schema-location default \
  -schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' \
  $(find manifests -name "*.yaml" -not -name "*-sealed.yaml") argocd/apps/*.yaml
```

`ansible-lint` needs the collections in `ansible/requirements.yml` installed
first (`ansible-galaxy collection install -r ansible/requirements.yml`),
otherwise it reports spurious `unknown-module` errors for `community.general`/
`community.docker` modules that are actually fine.

## Key gotchas (see README.md "Gotchas" for the full list)

- Pi-hole has no wildcard DNS — add every new `home.arpa` hostname to both
  Pi-hole values files.
- Gitea uses a manual public `IngressRoute`; its Helm-chart ingress is
  disabled in `values/gitea.yaml`. Changing `ROOT_URL` may require deleting
  the generated inline config secret before Argo CD reconciles cleanly.
- Secondary Pi-hole's external IPs can be lost on chart upgrades — rerun
  `manifests/network/pihole-debian-patch.sh`.
- Dashy's manifest ships an empty ConfigMap shell; real config only lands
  after running `manifests/core/apply-dashy-config.sh`.
- A newly pushed Argo CD `Application` doesn't create its namespace
  instantly — `app-of-apps` only picks it up on its next Git poll. A
  namespace-scoped secret script run immediately after pushing can fail
  with `namespaces "<ns>" not found` until then (or until a forced
  refresh).