Nik Afiq
7fbab2561a
Root-causes today's Gitea outage. The official Gitea Helm chart
(dl.gitea.com/charts, gitea/helm-gitea) always builds Gitea's own [database]
config directly from the plaintext postgresql.global.postgresql.auth.password
value, confirmed from templates/gitea/config.yaml:
{{- $_ := set .Values.gitea.config.database "PASSWD" .Values.postgresql.global.postgresql.auth.password -}}
`existingSecret` only affects the bundled postgresql subchart's own credential
provisioning -- it has no effect on what Gitea itself connects with. Removing
the plaintext field in the earlier Stage 1 "secret hygiene" commit made this
silently fall back to the chart's built-in default password ("gitea", 5
chars), which got re-baked into the persisted app.ini on every pod restart
regardless of what the live Postgres role's real password was -- hence the
repeating CrashLoopBackOff no amount of Postgres-side fixing could resolve.
Restores a plaintext password (the value already live on Postgres from this
incident's recovery), with a comment explaining why, so this doesn't get
"cleaned up" back into the same breakage later. Removes the now-nonfunctional
manifests/gitea/gitea-postgres-secret.sh and its .env.example entry. Getting
this fully off plaintext would need a Config Management Plugin or similar --
not attempted here.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Kubernetes Manifests
This directory contains raw Kubernetes resources grouped by service area. Most
subdirectories are consumed by Argo CD Applications in argocd/apps.
Directories
| Directory | Contents |
|---|---|
argocd/ |
App-of-apps, Argo CD ingress, Argo CD OIDC sealed secret |
authentik/ |
Authentik ingress, public ingress, proxy outpost, middleware, secret scripts |
cert-manager/ |
Internal and Let's Encrypt ClusterIssuers, Porkbun secret script |
core/ |
Dashy, Glances, CoreDNS custom config, CA installer |
gitea/ |
Gitea storage, backup, public ingress, runner and OIDC/admin secrets |
home-services/ |
HA gateway, AI gateway, Discord bot, service TLS, registry secret |
homeassistant/ |
Home Assistant external service and ingress |
media/ |
Jellyfin, qBittorrent, JDownloader, Immich |
monitoring/ |
Monitoring PVs, Grafana datasource, Grafana/Auth OIDC secrets |
network/ |
Pi-hole secrets, DDNS, Traefik dashboard, external host ingresses |
portfolio/ |
Portfolio deployment, ingress, registry pull secret |
Secrets
There are two patterns:
*-sealed.yamlfiles are safe to commit and are reconciled by Sealed Secrets.*.shscripts create runtime Secrets from.envdirectly in the cluster.
Use .env.example as the template for local secret names. kubeseal must point
at the in-cluster controller named sealed-secrets-controller in kube-system.
Regenerate committed sealed secrets with the matching script, then commit the resulting YAML. Runtime secret scripts should be run against the target cluster and should not produce committed plaintext.
Certificates
Internal services generally use internal-ca-issuer and home.arpa hostnames.
Public services use Let's Encrypt issuers and nik4nao.com hostnames.
The CA installer lives in core/ca-installer. Its ca-sync CronJob keeps the
served ca.crt and Apple mobileconfig in sync with the cert-manager CA secret.
DNS
Internal DNS records are configured in values/pihole.yaml and
values/pihole-debian.yaml. Add a new hostname to both files when adding a
home.arpa service.
Dashy
core/dashy.yaml defines the deployment and a placeholder ConfigMap. The real
dashboard config comes from config/dashy/conf.yaml:
bash manifests/core/apply-dashy-config.sh