From 7fbab2561a8ba02e66d2f1e9ba03f41f3d4a5a35 Mon Sep 17 00:00:00 2001 From: Nik Afiq Date: Thu, 23 Jul 2026 19:56:21 +0900 Subject: [PATCH] fix: revert Gitea DB password to plaintext -- existingSecret doesn't work with this chart Root-causes today's Gitea outage. The official Gitea Helm chart (dl.gitea.com/charts, gitea/helm-gitea) always builds Gitea's own [database] config directly from the plaintext postgresql.global.postgresql.auth.password value, confirmed from templates/gitea/config.yaml: {{- $_ := set .Values.gitea.config.database "PASSWD" .Values.postgresql.global.postgresql.auth.password -}} `existingSecret` only affects the bundled postgresql subchart's own credential provisioning -- it has no effect on what Gitea itself connects with. Removing the plaintext field in the earlier Stage 1 "secret hygiene" commit made this silently fall back to the chart's built-in default password ("gitea", 5 chars), which got re-baked into the persisted app.ini on every pod restart regardless of what the live Postgres role's real password was -- hence the repeating CrashLoopBackOff no amount of Postgres-side fixing could resolve. Restores a plaintext password (the value already live on Postgres from this incident's recovery), with a comment explaining why, so this doesn't get "cleaned up" back into the same breakage later. Removes the now-nonfunctional manifests/gitea/gitea-postgres-secret.sh and its .env.example entry. Getting this fully off plaintext would need a Config Management Plugin or similar -- not attempted here. Co-Authored-By: Claude Sonnet 5 --- .env.example | 3 --- REFACTOR_PLAN.md | 4 +++- manifests/gitea/gitea-postgres-secret.sh | 16 ---------------- values/gitea.yaml | 15 +++++++++++---- 4 files changed, 14 insertions(+), 24 deletions(-) delete mode 100755 manifests/gitea/gitea-postgres-secret.sh diff --git a/.env.example b/.env.example index ee621ee..7f06310 100644 --- a/.env.example +++ b/.env.example @@ -38,9 +38,6 @@ SWITCHBOT_SECRET=your_switchbot_secret_here # Immich database credentials IMMICH_POSTGRES_PASSWORD=your_password_here -# Gitea database credentials (rotated off the plaintext value formerly in values/gitea.yaml) -GITEA_POSTGRES_PASSWORD=your_password_here - # Dashy weather widget API key (rotated off the plaintext value formerly in config/dashy/conf.yaml) DASHY_WEATHER_API_KEY=your_api_key_here diff --git a/REFACTOR_PLAN.md b/REFACTOR_PLAN.md index 7fa75bb..1a11e81 100644 --- a/REFACTOR_PLAN.md +++ b/REFACTOR_PLAN.md @@ -26,7 +26,9 @@ citations, and the highest-severity ones were independently re-verified | 8 | Documentation & DR runbook | Done, committed locally. | | 9 | CI & Claude Code guidance | Done, committed locally, **not pushed** (new CI automation surface). | -**Secret rotation still needs you** (Stage 1): I wired up the `existingSecret`/env-injection plumbing for the Gitea DB password and Dashy API key, but I don't generate or handle the actual new credential values — that's your call per this repo's credential-handling rule. See the session summary for exact steps. +**Correction, 2026-07-23**: the original `existingSecret`-based fix for Finding #3 was wrong and caused a real Gitea outage. Confirmed directly from the chart source (`gitea/helm-gitea` `templates/gitea/config.yaml`): `{{ set .Values.gitea.config.database "PASSWD" .Values.postgresql.global.postgresql.auth.password }}` — Gitea's own `[database]` config is *always* built from this plaintext values field, regardless of `existingSecret` (which only affects the bundled postgresql subchart's own credential provisioning, a separate concern). Removing the plaintext field made it silently fall back to the chart's built-in default password, breaking every fresh Gitea pod's DB connection. `values/gitea.yaml` is back to a plaintext (rotated) password with a comment explaining why, and `manifests/gitea/gitea-postgres-secret.sh` was removed. Getting this fully off plaintext would need a Config Management Plugin or similar — not attempted here; flagging as a real open item if it matters enough to invest in. + +Dashy's API key fix (env-injection via `manifests/core/apply-dashy-config.sh`) is unaffected by this and still needs you to rotate the actual key value. ### New findings from this session's live-cluster checks (not in the original audit) diff --git a/manifests/gitea/gitea-postgres-secret.sh b/manifests/gitea/gitea-postgres-secret.sh deleted file mode 100755 index 15b007e..0000000 --- a/manifests/gitea/gitea-postgres-secret.sh +++ /dev/null @@ -1,16 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -source "$SCRIPT_DIR/../../.env" - -kubectl create secret generic gitea-postgres-secret \ - --namespace=gitea \ - --from-literal=postgresql-password="${GITEA_POSTGRES_PASSWORD}" \ - --dry-run=client -o yaml \ - | kubeseal \ - --controller-namespace=kube-system \ - --controller-name=sealed-secrets-controller \ - --format yaml \ - > "$SCRIPT_DIR/gitea-postgres-sealed.yaml" - -echo "Wrote $SCRIPT_DIR/gitea-postgres-sealed.yaml" diff --git a/values/gitea.yaml b/values/gitea.yaml index 4afb487..8ec439e 100644 --- a/values/gitea.yaml +++ b/values/gitea.yaml @@ -45,12 +45,19 @@ postgresql: global: postgresql: auth: + # NOTE: this chart's own template (templates/gitea/config.yaml) always + # reads this plaintext field directly to build gitea's [database] + # config section -- `existingSecret` only affects the bundled + # postgresql subchart's own credential provisioning, not what Gitea + # itself connects with. There is no existingSecret-based way to keep + # this out of git with this specific chart short of a Config + # Management Plugin. Do not "clean this up" back to existingSecret + # without also patching the chart's config template -- doing so + # silently reverts to the chart's built-in default password and + # breaks Gitea's DB connection (confirmed the hard way on 2026-07-23). username: gitea database: gitea - existingSecret: gitea-postgres-secret - secretKeys: - adminPasswordKey: postgresql-password - userPasswordKey: postgresql-password + password: ySelvqn2LjZ1fdJ4OFQs5ZlK6VLxHdny service: ssh: