feat: update README files to enhance documentation for Ansible, Argo CD, and Kubernetes manifests
This commit is contained in:
parent
f9b0e53988
commit
4cc3af39d6
370
README.md
370
README.md
@ -1,221 +1,219 @@
|
|||||||
# homelab
|
# homelab
|
||||||
|
|
||||||
Infrastructure-as-Code for a 3-machine homelab running K3s.
|
Infrastructure-as-code for a small K3s homelab. Host setup is handled with
|
||||||
|
Ansible, cluster state is reconciled by Argo CD, and service configuration lives
|
||||||
|
in Kubernetes manifests plus Helm values.
|
||||||
|
|
||||||
## Architecture
|
## Current Architecture
|
||||||
|
|
||||||
| Machine | IP | Role |
|
| Host | IP | Role |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| Minisforum UM780 XTX | `192.168.7.77` | K3s control-plane |
|
| `minisforum` | `192.168.7.77` | K3s server, Traefik entrypoint, primary app node |
|
||||||
| nik-debian (HP ProDesk) | `192.168.7.183` | K3s storage agent |
|
| `debian` / `nik-debian` | `192.168.7.183` | K3s agent, NFS storage, secondary Pi-hole |
|
||||||
| Mac Mini M2 | `192.168.7.96` | Standalone Docker (outside cluster) |
|
| `mac-mini` | `192.168.7.96` | Standalone services such as Watch Party and Ollama |
|
||||||
|
|
||||||
## Roadmap
|
The cluster uses Traefik instead of the bundled K3s ingress controller. Internal
|
||||||
|
services are published under `home.arpa` with certificates from an internal CA.
|
||||||
|
Public services under `nik4nao.com` use Let's Encrypt.
|
||||||
|
|
||||||
| Phase | Description | Status |
|
## Repository Layout
|
||||||
|
|
||||||
|
| Path | Purpose |
|
||||||
|
| --- | --- |
|
||||||
|
| `ansible/` | Host bootstrap and non-Kubernetes services |
|
||||||
|
| `argocd/apps/` | Argo CD `Application` objects for Helm charts and manifest directories |
|
||||||
|
| `argocd/values/` | Helm values used to install or reconcile Argo CD itself |
|
||||||
|
| `config/` | App configuration that is injected into manifests, currently Dashy |
|
||||||
|
| `manifests/` | Raw Kubernetes resources grouped by service area |
|
||||||
|
| `values/` | Helm values consumed by Argo CD Applications |
|
||||||
|
|
||||||
|
## Managed Services
|
||||||
|
|
||||||
|
| Area | Services |
|
||||||
|
| --- | --- |
|
||||||
|
| GitOps | Argo CD and app-of-apps |
|
||||||
|
| Networking | Traefik, Pi-hole primary and secondary, DDNS, CoreDNS custom forwarding |
|
||||||
|
| TLS | cert-manager, internal CA issuer, Let's Encrypt issuers, CA installer page |
|
||||||
|
| Identity | Authentik, Traefik forward-auth middleware, OIDC integrations |
|
||||||
|
| Observability | kube-prometheus-stack, Grafana, Loki, Tempo, OpenTelemetry Collector, Glances |
|
||||||
|
| Git and CI | Gitea, Gitea Actions runner, registry pull secrets, Gitea backup CronJob |
|
||||||
|
| Media | Jellyfin, qBittorrent, JDownloader, Photoview, Immich |
|
||||||
|
| Home services | Home Assistant ingress, HA gateway, AI gateway, Discord bot |
|
||||||
|
| Public apps | Portfolio, Watch Party ingress to the Mac Mini |
|
||||||
|
| Dashboard | Dashy |
|
||||||
|
|
||||||
|
## Important URLs
|
||||||
|
|
||||||
|
| URL | Service | Certificate |
|
||||||
| --- | --- | --- |
|
| --- | --- | --- |
|
||||||
| P0 | CA web installer (`ca.home.arpa`) | ✅ Done |
|
| `https://argocd.home.arpa` | Argo CD | Internal CA |
|
||||||
| P1 | Prometheus + Grafana + Loki | ✅ Done |
|
| `https://auth.home.arpa` | Authentik internal | Internal CA |
|
||||||
| P2 | Authentik SSO | ✅ Done |
|
| `https://auth.nik4nao.com` | Authentik public | Let's Encrypt |
|
||||||
| P3 | Photoview | ✅ Done |
|
| `https://traefik.home.arpa` | Traefik dashboard | Internal CA |
|
||||||
| P4 | Home Assistant | 🔜 Planned |
|
| `https://grafana.nik4nao.com` | Grafana | Let's Encrypt |
|
||||||
| P5 | Portfolio site (`nik4nao.com`) | ✅ Done |
|
| `https://gitea.nik4nao.com` | Gitea | Let's Encrypt |
|
||||||
| P6 | WireGuard split tunnel | 🔜 Planned |
|
| `https://pihole.home.arpa` | Pi-hole | Internal CA |
|
||||||
|
| `https://dashy.home.arpa` | Dashy | Internal CA |
|
||||||
|
| `https://jellyfin.home.arpa` | Jellyfin | Internal CA |
|
||||||
|
| `https://qbittorrent.home.arpa` | qBittorrent | Internal CA |
|
||||||
|
| `https://jdownloader.home.arpa` | JDownloader | Internal CA |
|
||||||
|
| `https://photoview.home.arpa` | Photoview | Internal CA |
|
||||||
|
| `https://immich.home.arpa` | Immich | Internal CA |
|
||||||
|
| `https://ha.home.arpa` | Home Assistant | Internal CA |
|
||||||
|
| `https://glances.home.arpa` | Glances on K3s | Internal CA |
|
||||||
|
| `https://glances-debian.home.arpa` | Glances on Debian | Internal CA |
|
||||||
|
| `https://watch-party.nik4nao.com` | Watch Party on Mac Mini | Let's Encrypt |
|
||||||
|
| `https://nik4nao.com` | Portfolio | Let's Encrypt |
|
||||||
|
| `http://ca.home.arpa` | Internal CA installer | Plain HTTP |
|
||||||
|
|
||||||
## Live Services
|
`home.arpa` names are defined explicitly in `values/pihole.yaml` and
|
||||||
|
`values/pihole-debian.yaml`; Pi-hole is not configured as a wildcard DNS server.
|
||||||
|
|
||||||
| Service | URL | TLS | Notes |
|
## Bootstrap
|
||||||
|---|---|---|---|
|
|
||||||
| Traefik dashboard | `https://traefik.home.arpa` | Internal CA | Protected by Authentik |
|
|
||||||
| Authentik | `https://auth.home.arpa` / `https://auth.nik4nao.com` | Internal CA / Let's Encrypt | SSO for all services |
|
|
||||||
| Gitea | `https://gitea.nik4nao.com` | Let's Encrypt | Git + Docker registry |
|
|
||||||
| Pi-hole (primary) | `https://pihole.home.arpa` | Internal CA | DNS, runs on Minisforum |
|
|
||||||
| Pi-hole (secondary) | — | — | externalIPs on Debian (`192.168.7.183`) |
|
|
||||||
| Grafana | `https://grafana.nik4nao.com` | Let's Encrypt | Protected by Authentik OIDC |
|
|
||||||
| Prometheus | internal | — | kube-prometheus-stack |
|
|
||||||
| Loki + Promtail | internal | — | Log aggregation |
|
|
||||||
| Jellyfin | `https://jellyfin.home.arpa` | Internal CA | Media server, NFS storage |
|
|
||||||
| qBittorrent | `https://qbittorrent.home.arpa` | Internal CA | `/mnt/storage/torrents` |
|
|
||||||
| JDownloader | `https://jdownloader.home.arpa` | Internal CA | `/mnt/storage/dl` |
|
|
||||||
| Dashy | `https://dashy.home.arpa` | Internal CA | Config via ConfigMap |
|
|
||||||
| Glances | `https://glances.home.arpa` | Internal CA | DaemonSet + Debian Docker |
|
|
||||||
| Photoview | `https://photoview.home.arpa` | Internal CA | NFS photo gallery |
|
|
||||||
| Watch Party | `https://watch-party.nik4nao.com` | Let's Encrypt | Mac Mini, CI/CD deployed |
|
|
||||||
| Portfolio | `https://nik4nao.com` | Let's Encrypt | Hugo + terminal theme |
|
|
||||||
| CA installer | `http://ca.home.arpa` | — | Internal CA cert download page |
|
|
||||||
|
|
||||||
## Auth
|
Install workstation tools:
|
||||||
|
|
||||||
- **SSO:** Authentik at `auth.home.arpa` (internal) / `auth.nik4nao.com` (public)
|
|
||||||
- **Protected services:** Traefik dashboard, Grafana (OIDC), Gitea (OIDC)
|
|
||||||
- **MFA:** TOTP enforced, 8hr sessions
|
|
||||||
- **Users:** `nik` (admin), `akadmin` (break-glass)
|
|
||||||
- **Gitea:** local login disabled, OIDC only
|
|
||||||
|
|
||||||
## TLS
|
|
||||||
|
|
||||||
- **`*.home.arpa`** — internal self-signed CA (cert-manager). Install CA cert via `http://ca.home.arpa`
|
|
||||||
- **`*.nik4nao.com`** — Let's Encrypt via HTTP-01 (Traefik)
|
|
||||||
|
|
||||||
## Repo Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
ansible/
|
|
||||||
inventory.yaml
|
|
||||||
playbooks/
|
|
||||||
bootstrap-minisforum.yaml # OS hardening, packages, UFW, /data dirs
|
|
||||||
deploy-watch-party.yaml # deploy watch-party app on Mac Mini
|
|
||||||
join-debian-agent.yaml # join Debian as K3s agent
|
|
||||||
setup-gitea-runner.yaml # Gitea Actions runner (act_runner systemd)
|
|
||||||
setup-glances-debian.yaml # Glances on Debian host
|
|
||||||
setup-k3s.yaml # K3s server install, Helm, kubeconfig
|
|
||||||
setup-monitoring.yaml # Prometheus + Grafana + Loki stack
|
|
||||||
setup-nfs-debian.yaml # NFS server on Debian
|
|
||||||
roles/
|
|
||||||
common/ # user, SSH hardening, UFW, base packages
|
|
||||||
gitea-runner/ # act_runner v0.2.11 systemd service
|
|
||||||
glances/ # Glances system monitor
|
|
||||||
k3s-agent/ # K3s agent node join
|
|
||||||
k3s-server/ # K3s server install + Helm
|
|
||||||
monitoring/ # Prometheus/Grafana stack
|
|
||||||
nfs-server/ # NFS server configuration
|
|
||||||
watch-party/ # Watch Party Docker Compose on Mac Mini
|
|
||||||
config/
|
|
||||||
dashy/conf.yaml # Dashy dashboard config (applied via ConfigMap)
|
|
||||||
manifests/
|
|
||||||
authentik/ # Ingress, middleware, proxy outpost, secrets
|
|
||||||
cert-manager/ # ClusterIssuers, porkbun-secret.sh
|
|
||||||
core/ # Dashy, Glances, CA installer, CoreDNS config
|
|
||||||
gitea/ # PV, runner, backup CronJob, public ingress
|
|
||||||
media/ # Jellyfin, qBittorrent, JDownloader, Photoview
|
|
||||||
monitoring/ # Grafana/Loki datasource ConfigMap, PVs, grafana-secret.sh
|
|
||||||
network/ # DDNS CronJob, Traefik dashboard, pihole-debian-patch.sh
|
|
||||||
portfolio/ # Portfolio deployment, registry pull secret
|
|
||||||
values/
|
|
||||||
authentik.yaml
|
|
||||||
cert-manager.yaml
|
|
||||||
gitea.yaml
|
|
||||||
kube-prometheus-stack.yaml
|
|
||||||
loki-stack.yaml
|
|
||||||
pihole.yaml
|
|
||||||
pihole-debian.yaml
|
|
||||||
traefik.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
- Ansible on workstation: `pip install ansible`
|
|
||||||
- Ansible collections: `ansible-galaxy collection install community.general ansible.posix`
|
|
||||||
- SSH key: `~/.ssh/id_ed25519-nik-macbookair`
|
|
||||||
- kubectl + helm installed
|
|
||||||
|
|
||||||
## Connecting
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# SSH
|
pip install ansible
|
||||||
ssh minisforum # port 430, via ~/.ssh/config
|
ansible-galaxy collection install community.general ansible.posix
|
||||||
ssh nik-debian # port 22
|
```
|
||||||
|
|
||||||
# Kubectl
|
Also install `kubectl`, `helm`, and `kubeseal`. The inventory expects SSH
|
||||||
export KUBECONFIG=/tmp/k3s-minisforum.yaml
|
access with `~/.ssh/id_ed25519-nik-macbookair`.
|
||||||
|
|
||||||
|
Bring up hosts and base services:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/bootstrap-minisforum.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-k3s.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-nfs-debian.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/join-debian-agent.yaml -K
|
||||||
|
```
|
||||||
|
|
||||||
|
Install Argo CD once, then hand control to the app-of-apps:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
helm repo add argo https://argoproj.github.io/argo-helm
|
||||||
|
helm repo update
|
||||||
|
helm upgrade --install argocd argo/argo-cd \
|
||||||
|
--namespace argocd --create-namespace \
|
||||||
|
--version 9.4.15 \
|
||||||
|
--values argocd/values/argocd.yaml
|
||||||
|
|
||||||
|
kubectl apply -f manifests/argocd/app-of-apps.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
After that, normal changes should flow through Git and Argo CD.
|
||||||
|
|
||||||
|
## Daily Operations
|
||||||
|
|
||||||
|
Check cluster and sync state:
|
||||||
|
|
||||||
|
```bash
|
||||||
kubectl get nodes
|
kubectl get nodes
|
||||||
kubectl get pods -A
|
kubectl get pods -A
|
||||||
|
kubectl get applications -n argocd
|
||||||
```
|
```
|
||||||
|
|
||||||
## Deploying / Re-deploying
|
Apply Dashy config after editing `config/dashy/conf.yaml`:
|
||||||
|
|
||||||
### Ansible (host-level)
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Bootstrap Minisforum
|
bash manifests/core/apply-dashy-config.sh
|
||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/bootstrap-minisforum.yaml
|
```
|
||||||
|
|
||||||
# K3s server
|
Patch secondary Pi-hole DNS services after Helm upgrades if needed:
|
||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-k3s.yaml
|
|
||||||
|
|
||||||
# Join Debian as agent
|
```bash
|
||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/join-debian-agent.yaml
|
bash manifests/network/pihole-debian-patch.sh
|
||||||
|
```
|
||||||
|
|
||||||
# NFS on Debian
|
Deploy host-level services:
|
||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-nfs-debian.yaml
|
|
||||||
|
|
||||||
# Gitea Actions runner
|
```bash
|
||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-gitea-runner.yaml
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-gitea-runner.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-glances-debian.yaml -K
|
||||||
# Glances on Debian
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-ollama.yaml -K
|
||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-glances-debian.yaml
|
|
||||||
|
|
||||||
# Watch Party on Mac Mini
|
|
||||||
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/deploy-watch-party.yaml
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/deploy-watch-party.yaml
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/wireguard.yaml -K
|
||||||
```
|
```
|
||||||
|
|
||||||
### Helm (cluster services)
|
## Secrets
|
||||||
|
|
||||||
|
Do not commit plaintext secrets. Runtime-only scripts read `.env` and create
|
||||||
|
Kubernetes Secrets directly. Sealed Secret scripts regenerate committed
|
||||||
|
`*-sealed.yaml` resources.
|
||||||
|
|
||||||
|
Start from `.env.example` and keep the filled `.env` local.
|
||||||
|
|
||||||
|
Runtime secret scripts:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Traefik
|
|
||||||
helm repo add traefik https://helm.traefik.io/traefik && helm repo update
|
|
||||||
helm upgrade --install traefik traefik/traefik \
|
|
||||||
--namespace traefik --create-namespace -f values/traefik.yaml
|
|
||||||
|
|
||||||
# cert-manager
|
|
||||||
helm repo add jetstack https://charts.jetstack.io && helm repo update
|
|
||||||
helm upgrade --install cert-manager jetstack/cert-manager \
|
|
||||||
--namespace cert-manager --create-namespace -f values/cert-manager.yaml
|
|
||||||
|
|
||||||
# Gitea
|
|
||||||
helm repo add gitea-charts https://dl.gitea.com/charts/ && helm repo update
|
|
||||||
helm upgrade --install gitea gitea-charts/gitea \
|
|
||||||
--namespace gitea --create-namespace -f values/gitea.yaml
|
|
||||||
|
|
||||||
# Pi-hole (Minisforum)
|
|
||||||
helm repo add mojo2600 https://mojo2600.github.io/pihole-kubernetes/ && helm repo update
|
|
||||||
helm upgrade --install pihole mojo2600/pihole \
|
|
||||||
--namespace pihole --create-namespace -f values/pihole.yaml
|
|
||||||
# Note: re-run manifests/network/pihole-debian-patch.sh after every Pi-hole upgrade
|
|
||||||
# (externalIPs for Debian secondary are lost on upgrade)
|
|
||||||
|
|
||||||
# Authentik
|
|
||||||
helm repo add authentik https://charts.goauthentik.io && helm repo update
|
|
||||||
helm upgrade --install authentik authentik/authentik \
|
|
||||||
--namespace authentik --create-namespace -f values/authentik.yaml
|
|
||||||
|
|
||||||
# kube-prometheus-stack
|
|
||||||
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts && helm repo update
|
|
||||||
helm upgrade --install kube-prometheus-stack prometheus-community/kube-prometheus-stack \
|
|
||||||
--namespace monitoring --create-namespace -f values/kube-prometheus-stack.yaml
|
|
||||||
|
|
||||||
# Loki
|
|
||||||
helm repo add grafana https://grafana.github.io/helm-charts && helm repo update
|
|
||||||
helm upgrade --install loki grafana/loki-stack \
|
|
||||||
--namespace monitoring --create-namespace -f values/loki-stack.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
### Secrets (create before applying manifests)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Porkbun API (cert-manager DNS-01)
|
|
||||||
bash manifests/cert-manager/porkbun-secret.sh
|
bash manifests/cert-manager/porkbun-secret.sh
|
||||||
|
bash manifests/authentik/authentik-secret.sh
|
||||||
# Grafana admin password
|
bash manifests/authentik/authentik-proxy-secret.sh
|
||||||
|
bash manifests/authentik/authentik-gitea-secret.sh
|
||||||
|
bash manifests/authentik/authentik-grafana-secret.sh
|
||||||
|
bash manifests/home-services/registry-secret.sh
|
||||||
bash manifests/monitoring/grafana-secret.sh
|
bash manifests/monitoring/grafana-secret.sh
|
||||||
|
|
||||||
# DDNS credentials
|
|
||||||
bash manifests/network/ddns-secret.sh
|
bash manifests/network/ddns-secret.sh
|
||||||
|
bash manifests/portfolio/registry-secret.sh
|
||||||
# Gitea Actions runner token
|
|
||||||
bash manifests/gitea/runner-secret.sh
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Known Gotchas
|
Sealed Secret regeneration:
|
||||||
|
|
||||||
- **Gitea ROOT_URL:** changing `ROOT_URL` in `values/gitea.yaml` is not enough — must also delete the `gitea-inline-config` secret and re-run `helm upgrade`. Disable the built-in ingress (`ingress.enabled=false`) and use the manual IngressRoute in `manifests/gitea/`.
|
```bash
|
||||||
- **Pihole secondary externalIPs:** lost on every Helm upgrade — re-run `manifests/network/pihole-debian-patch.sh` after each upgrade.
|
bash manifests/home-services/discord-bot-secret.sh
|
||||||
- **Prometheus hostPath:** `/data/prometheus` requires `chmod -R 777` (owned by UID 65534).
|
bash manifests/home-services/ha-gateway-secret.sh
|
||||||
- **Grafana PVC:** use `local-path` dynamic provisioning — do not use a static hostPath PV, K3s overrides `storageClassName: ""`.
|
bash manifests/media/immich-postgres-secret.sh
|
||||||
- **Loki datasource:** provisioned via labeled ConfigMap (`grafana_datasource`), not the Grafana UI — the bundled plugin in loki-stack v2.9.3 is incompatible with Grafana 12.
|
```
|
||||||
- **Authentik forwardAuth:** `Cookie` header must be in `authRequestHeaders` in the Traefik middleware or you get an infinite redirect loop after login.
|
|
||||||
- **Traefik v3 `api@internal`:** requires both `traefik` and `websecure` entrypoints in the IngressRoute, otherwise 404.
|
Some sealed secrets are maintained directly in the repo, including Argo CD OIDC,
|
||||||
- **CoreDNS custom config:** use `.server` suffix for zone blocks. `.override` suffix cannot contain `zone {}` syntax — crashes CoreDNS.
|
Gitea admin/OIDC, Grafana admin/OIDC, Pi-hole admin, and home-service secrets.
|
||||||
- **Photoview video:** `PHOTOVIEW_DISABLE_VIDEO=true` only takes effect on a fresh scan — delete the SQLite DB and restart before rescanning.
|
The host-level Gitea runner reads `GITEA_RUNNER_TOKEN` from the environment when
|
||||||
- **CI/CD Buildkit CA:** internal CA must be injected into the `buildx_buildkit_multiarch0` container on every CI run (does not persist across restarts).
|
running `ansible/playbooks/setup-gitea-runner.yaml`.
|
||||||
- **Pihole DNS:** no wildcard support — every new `home.arpa` subdomain needs an explicit entry in `values/pihole.yaml`.
|
|
||||||
|
## Storage
|
||||||
|
|
||||||
|
K3s `local-path` is used for many app PVCs. Static hostPath PVs are used for
|
||||||
|
state that must live on known disks:
|
||||||
|
|
||||||
|
| Location | Use |
|
||||||
|
| --- | --- |
|
||||||
|
| `/data/gitea` on `minisforum` | Gitea shared storage |
|
||||||
|
| `/data/prometheus` on `minisforum` | Prometheus |
|
||||||
|
| `/data/grafana` on `minisforum` | Grafana |
|
||||||
|
| `/data/loki` on `minisforum` | Loki |
|
||||||
|
| `/mnt/storage` on `debian` | NFS media library and backups |
|
||||||
|
|
||||||
|
The Debian NFS server exports `/mnt/storage` to `192.168.7.77`.
|
||||||
|
|
||||||
|
## TLS and Trust
|
||||||
|
|
||||||
|
`*.home.arpa` certificates use `internal-ca-issuer` from `manifests/cert-manager`.
|
||||||
|
The CA installer at `http://ca.home.arpa` serves `ca.crt` and an iOS/macOS
|
||||||
|
mobileconfig profile. The `ca-sync` CronJob updates those files from the
|
||||||
|
`cert-manager/internal-ca-cert` secret when the CA changes.
|
||||||
|
|
||||||
|
`*.nik4nao.com` certificates use the Let's Encrypt issuers in
|
||||||
|
`manifests/cert-manager/cluster-issuer-letsencrypt.yaml`.
|
||||||
|
|
||||||
|
## Gotchas
|
||||||
|
|
||||||
|
- Argo CD Applications mostly set `prune: false`; removing resources from Git may
|
||||||
|
require manual cleanup.
|
||||||
|
- Gitea uses a manual public `IngressRoute`; the chart ingress is disabled in
|
||||||
|
`values/gitea.yaml`.
|
||||||
|
- Gitea `ROOT_URL` changes can require deleting the generated inline config
|
||||||
|
secret before reconciling.
|
||||||
|
- Pi-hole does not provide wildcard DNS here; add each new internal hostname to
|
||||||
|
both Pi-hole values files.
|
||||||
|
- Secondary Pi-hole external IPs can be lost during chart upgrades; rerun
|
||||||
|
`manifests/network/pihole-debian-patch.sh`.
|
||||||
|
- Authentik forward-auth depends on the `Cookie` header in
|
||||||
|
`authRequestHeaders`; removing it causes redirect loops.
|
||||||
|
- CoreDNS custom zone snippets must use the `.server` key suffix.
|
||||||
|
- The Dashy manifest contains an empty ConfigMap shell; use
|
||||||
|
`manifests/core/apply-dashy-config.sh` to load the real config.
|
||||||
|
|
||||||
|
See the scoped READMEs in `ansible/`, `argocd/`, and `manifests/` for workflows
|
||||||
|
specific to those directories.
|
||||||
|
|||||||
66
ansible/README.md
Normal file
66
ansible/README.md
Normal file
@ -0,0 +1,66 @@
|
|||||||
|
# Ansible
|
||||||
|
|
||||||
|
This directory contains host-level automation. It bootstraps machines, installs
|
||||||
|
K3s, prepares storage, and manages services that intentionally run outside the
|
||||||
|
cluster.
|
||||||
|
|
||||||
|
## Inventory
|
||||||
|
|
||||||
|
`inventory.yaml` defines three groups:
|
||||||
|
|
||||||
|
| Group | Host | Purpose |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `k3s_server` | `minisforum` | K3s server at `192.168.7.77` |
|
||||||
|
| `k3s_agents` | `debian` | K3s agent and NFS storage at `192.168.7.183` |
|
||||||
|
| `mac_mini` | `mac-mini` | Docker/Ollama host at `192.168.7.96` |
|
||||||
|
|
||||||
|
All hosts use the `nik` user and the SSH key configured in `inventory.yaml`.
|
||||||
|
|
||||||
|
## Common Playbooks
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/bootstrap-minisforum.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-k3s.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-nfs-debian.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/join-debian-agent.yaml -K
|
||||||
|
```
|
||||||
|
|
||||||
|
Additional services:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export GITEA_RUNNER_TOKEN=...
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-monitoring.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-gitea-runner.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-glances-debian.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/setup-ollama.yaml -K
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/deploy-watch-party.yaml
|
||||||
|
ansible-playbook -i ansible/inventory.yaml ansible/playbooks/wireguard.yaml -K
|
||||||
|
```
|
||||||
|
|
||||||
|
## Roles
|
||||||
|
|
||||||
|
| Role | Responsibility |
|
||||||
|
| --- | --- |
|
||||||
|
| `common` | Packages, user setup, firewall, base data directories |
|
||||||
|
| `k3s-server` | K3s server install, kubeconfig fetch, Helm install, primary node label |
|
||||||
|
| `k3s-agent` | K3s agent join and storage node label |
|
||||||
|
| `nfs-server` | Export `/mnt/storage` from Debian to the K3s server |
|
||||||
|
| `monitoring` | Host directories and ownership for Prometheus/Loki |
|
||||||
|
| `gitea-runner` | Gitea Actions runner systemd service |
|
||||||
|
| `glances` | Host-level Glances service |
|
||||||
|
| `ollama` | Ollama service on the Mac Mini |
|
||||||
|
| `watch-party` | Watch Party Docker Compose deployment on the Mac Mini |
|
||||||
|
| `wireguard` | WireGuard server configuration |
|
||||||
|
| `homeassistant` | Legacy standalone Home Assistant deployment |
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- K3s version is set in `roles/k3s-server/defaults/main.yaml` and
|
||||||
|
`roles/k3s-agent/defaults/main.yaml`.
|
||||||
|
- `setup-gitea-runner.yaml` reads `GITEA_RUNNER_TOKEN` from the local
|
||||||
|
environment.
|
||||||
|
- The K3s role disables bundled Traefik because Traefik is managed by Argo CD.
|
||||||
|
- The Debian storage role exports `/mnt/storage`; several Kubernetes manifests
|
||||||
|
mount that export directly.
|
||||||
|
- Keep host automation idempotent where practical. These playbooks are meant to
|
||||||
|
be rerunnable during rebuilds.
|
||||||
51
argocd/README.md
Normal file
51
argocd/README.md
Normal file
@ -0,0 +1,51 @@
|
|||||||
|
# Argo CD
|
||||||
|
|
||||||
|
Argo CD is the cluster reconciler for this repo. `manifests/argocd/app-of-apps.yaml`
|
||||||
|
points Argo CD at `argocd/apps`, where each file defines one child
|
||||||
|
`Application`.
|
||||||
|
|
||||||
|
## Bootstrap
|
||||||
|
|
||||||
|
Install or upgrade Argo CD with the pinned chart values:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
helm repo add argo https://argoproj.github.io/argo-helm
|
||||||
|
helm repo update
|
||||||
|
helm upgrade --install argocd argo/argo-cd \
|
||||||
|
--namespace argocd --create-namespace \
|
||||||
|
--version 9.4.15 \
|
||||||
|
--values argocd/values/argocd.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
Then apply the app-of-apps:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl apply -f manifests/argocd/app-of-apps.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
Argo CD is exposed at `https://argocd.home.arpa` by
|
||||||
|
`manifests/argocd/argocd.yaml`.
|
||||||
|
|
||||||
|
## Application Types
|
||||||
|
|
||||||
|
| Type | Examples | Source |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Helm chart plus values | Traefik, cert-manager, Gitea, Pi-hole, monitoring, Loki, Tempo | `argocd/apps/*.yaml` and `values/*.yaml` |
|
||||||
|
| Raw manifest directory | Core, media, network secrets, home services, portfolio | `manifests/*` |
|
||||||
|
| Argo CD self-management | `argocd-self`, `argocd-config` | `argocd/values` and `manifests/argocd` |
|
||||||
|
|
||||||
|
Most Applications use automated sync with `selfHeal: true` and `prune: false`.
|
||||||
|
Expect Argo CD to correct drift, but do not expect deleted Git resources to be
|
||||||
|
pruned automatically.
|
||||||
|
|
||||||
|
## Adding a Service
|
||||||
|
|
||||||
|
1. Add raw manifests under `manifests/<area>` or Helm values under `values/`.
|
||||||
|
2. Add an `Application` in `argocd/apps/`.
|
||||||
|
3. Add any required DNS entries to both Pi-hole values files.
|
||||||
|
4. Add certificates, secrets, and registry pull secrets if the service needs
|
||||||
|
them.
|
||||||
|
5. Commit and let the app-of-apps reconcile.
|
||||||
|
|
||||||
|
Use `targetRevision: main` for repo-managed services unless there is a specific
|
||||||
|
reason to track `HEAD`.
|
||||||
57
manifests/README.md
Normal file
57
manifests/README.md
Normal file
@ -0,0 +1,57 @@
|
|||||||
|
# Kubernetes Manifests
|
||||||
|
|
||||||
|
This directory contains raw Kubernetes resources grouped by service area. Most
|
||||||
|
subdirectories are consumed by Argo CD Applications in `argocd/apps`.
|
||||||
|
|
||||||
|
## Directories
|
||||||
|
|
||||||
|
| Directory | Contents |
|
||||||
|
| --- | --- |
|
||||||
|
| `argocd/` | App-of-apps, Argo CD ingress, Argo CD OIDC sealed secret |
|
||||||
|
| `authentik/` | Authentik ingress, public ingress, proxy outpost, middleware, secret scripts |
|
||||||
|
| `cert-manager/` | Internal and Let's Encrypt ClusterIssuers, Porkbun secret script |
|
||||||
|
| `core/` | Dashy, Glances, CoreDNS custom config, CA installer |
|
||||||
|
| `gitea/` | Gitea storage, backup, public ingress, runner and OIDC/admin secrets |
|
||||||
|
| `home-services/` | HA gateway, AI gateway, Discord bot, service TLS, registry secret |
|
||||||
|
| `homeassistant/` | Home Assistant external service and ingress |
|
||||||
|
| `media/` | Jellyfin, qBittorrent, JDownloader, Photoview, Immich |
|
||||||
|
| `monitoring/` | Monitoring PVs, Grafana datasource, Grafana/Auth OIDC secrets |
|
||||||
|
| `network/` | Pi-hole secrets, DDNS, Traefik dashboard, external host ingresses |
|
||||||
|
| `portfolio/` | Portfolio deployment, ingress, registry pull secret |
|
||||||
|
|
||||||
|
## Secrets
|
||||||
|
|
||||||
|
There are two patterns:
|
||||||
|
|
||||||
|
- `*-sealed.yaml` files are safe to commit and are reconciled by Sealed Secrets.
|
||||||
|
- `*.sh` scripts create runtime Secrets from `.env` directly in the cluster.
|
||||||
|
|
||||||
|
Use `.env.example` as the template for local secret names. `kubeseal` must point
|
||||||
|
at the in-cluster controller named `sealed-secrets-controller` in `kube-system`.
|
||||||
|
|
||||||
|
Regenerate committed sealed secrets with the matching script, then commit the
|
||||||
|
resulting YAML. Runtime secret scripts should be run against the target cluster
|
||||||
|
and should not produce committed plaintext.
|
||||||
|
|
||||||
|
## Certificates
|
||||||
|
|
||||||
|
Internal services generally use `internal-ca-issuer` and `home.arpa` hostnames.
|
||||||
|
Public services use Let's Encrypt issuers and `nik4nao.com` hostnames.
|
||||||
|
|
||||||
|
The CA installer lives in `core/ca-installer`. Its `ca-sync` CronJob keeps the
|
||||||
|
served `ca.crt` and Apple mobileconfig in sync with the cert-manager CA secret.
|
||||||
|
|
||||||
|
## DNS
|
||||||
|
|
||||||
|
Internal DNS records are configured in `values/pihole.yaml` and
|
||||||
|
`values/pihole-debian.yaml`. Add a new hostname to both files when adding a
|
||||||
|
`home.arpa` service.
|
||||||
|
|
||||||
|
## Dashy
|
||||||
|
|
||||||
|
`core/dashy.yaml` defines the deployment and a placeholder ConfigMap. The real
|
||||||
|
dashboard config comes from `config/dashy/conf.yaml`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash manifests/core/apply-dashy-config.sh
|
||||||
|
```
|
||||||
Loading…
x
Reference in New Issue
Block a user