# tts-gateway Handoff Status as of 2026-07-25: Phases 0–6 of `TTS_GATEWAY_PLAN.md` are done, and the deployed service is confirmed working in production (real `grpcurl` call through `kubectl port-forward`, real GPU inference, real playable audio). This file is a punch list for picking the remaining work up in a fresh session — it doesn't re-explain things that are already documented elsewhere; it points to where. **Update (later same day):** items 1 and 3 below are now done. Model artifact distribution ended up fully automated, not just "worked around" — after an initial attempt at a manually-built image hit a nik-gpu registry-auth wall, the checkpoint/hparams were committed directly into `tts-gateway/model/` (a deliberate one-time exception to not committing large binaries) and CI now builds+pushes the `tts-model` image itself (`build-tts-model` in `.gitea/workflows/ci.yaml`), same as every other image here — no manual nik-gpu step at all anymore. `discord-bot` has a working `/speak` command (see `TTS_GATEWAY_PLAN.md` Phase 6). Only item 2 (mTLS) remains, left untouched per an explicit decision to verify these two first. The two "what's left" entries below are kept as-written for their historical reasoning/decision trail rather than rewritten in place. ## Where things stand - `tts-gateway` (Go) + `tts-sidecar` (Python/libtorch) are both built, containerized, and running in Kubernetes (`home-services` namespace). - CI (`.gitea/workflows/ci.yaml`) builds and pushes both images on every push to `main`, gated by path-filtered per-service rebuilds and using registry-based Docker layer caching. - Full history — the Phase 0 ONNX-export/g2p investigation, why there's a Python sidecar instead of pure Go, the exact `japanese_cleaners` pipeline verification — is in `TTS_GATEWAY_PLAN.md` and `tmp/reference/uma-tts-api/spike/FINDINGS.md`. Read those before re-deriving anything; the ONNX export failure in particular took several iterations to characterize correctly and isn't worth re-investigating from scratch. ## What's left 1. **Model artifact distribution isn't automated.** The k8s Deployment (`~/repo/homelab/manifests/home-services/tts-gateway.yaml` — separate repo, see below) `hostPath`-mounts `/data/tts-gateway` on the `nik-gpu` node into the sidecar container. Someone has to manually place `G_790000.pth` + `uma.json` there before the pod goes Ready — no init-container download, no PVC, nothing automated. `TTS_GATEWAY_PLAN.md`'s Phase 5 flagged this as an open decision; it's still open, just worked around. If asked to fix it: options are a PVC populated by an init-container download step, baking the checkpoint into a private image layer, or similar — weigh against the checkpoint being ~455MB and not something to casually put in a git-tracked Dockerfile context. 2. **mTLS is off.** `tts-gateway.yaml` comments out the `TLS_DIR` env var and its volume mount, for plaintext `grpcurl` testing from outside the cluster during initial rollout — a live TODO, not a permanent decision. Every other service in this repo assumes internal-network-or-mTLS as its only access boundary (`CLAUDE.md`'s "Configuration Notes"). Re-enabling: uncomment the env var + mount in that manifest, and set up a `tts-gateway-tls` secret the same way `ha-gateway-sealed.yaml`/`ha-gateway-secret.sh` do it for ha-gateway (tts-gateway doesn't currently have its own `-secret.sh`/`-sealed.yaml` pair — it'll need one, since unlike ha-gateway/discord-bot it has no other secrets today, so this would be its first). 3. **Phase 6 (discord-bot integration) not started.** A `/speak`-style Discord slash command calling `tts-gateway`, following the existing pattern — `discord-bot` already has gRPC clients for `ha-gateway` and `ai-gateway` at `discord-bot/internal/adapters/secondary/{gateway,aigateway}`; a new `internal/adapters/secondary/ttsgateway` client would follow the same shape. Needs a product decision first: keep AAC (parity with what's already implemented) or switch to Opus (Discord's native voice codec) — deliberately deferred in `TTS_GATEWAY_PLAN.md`'s open questions until this exact moment, not yet decided. ## Things worth knowing before touching this - **`tts-gateway` and `tts-sidecar` share one Kubernetes Pod**, not two separate Deployments — the Go gateway reaches the sidecar over `localhost:50054`. This mirrors the `--network host` setup documented in `tts-gateway/README.md` for local Docker testing. Don't split them into separate Deployments/Services without also rethinking `INFERENCE_SIDECAR_ADDR` and the networking model. - **The k8s manifests live in a different repo**: `~/repo/homelab/manifests/home-services/` — not in this repo at all. `CLAUDE.md` now has an "Infrastructure / Deployment" section documenting this; check there (and that repo directly) before assuming something isn't deployed just because this repo has no manifests for it. - **`open_jtalk`'s tokenization has an intentional bug that must be preserved.** The Go text normalizer (`tts-gateway/internal/adapters/secondary/jtalk`) maps the cleaned phoneme string to symbol IDs character-by-character, not phoneme-by-phoneme, because that's what the checkpoint was actually trained on (`text/symbols.py`'s "wrong tokens" comment in the original reference implementation). Don't "fix" this without retraining the model — see the comments in `jtalk.go` and `tts-gateway/README.md`. - **ONNX export doesn't work for this checkpoint, and it's not a quick fix.** Read `tmp/reference/uma-tts-api/spike/FINDINGS.md` before re-attempting it — the model's data-dependent output length (predicted phoneme durations determine audio length at runtime) defeats `torch.export`'s guard system in a way that would need an unknown number of per-layer `torch._check` hints to resolve, not a single targeted change. - **CI's build cache needs one "priming" run per image.** If a build job still looks slow after the caching change in `ci.yaml`, check whether a `:buildcache` tag actually exists yet in the registry for that specific image — the first run after the cache was added has nothing to pull from and builds fully fresh. ## Where to look for more detail - `TTS_GATEWAY_PLAN.md` (repo root) — original plan, feasibility analysis, phase breakdown, open product questions (codec, concurrency, auth). - `tmp/reference/uma-tts-api/spike/FINDINGS.md` — Phase 0 spike results with full reasoning. - `tts-gateway/README.md` — service-level docs: gRPC API, config reference, how to run/test locally and on nik-gpu. - `~/repo/homelab/manifests/home-services/tts-gateway.yaml` — actual production config (separate repo, maintained independently — check it directly rather than trusting a stale summary).