home-services/handoff.md

tts-gateway Handoff

Status as of 2026-07-25: Phases 0–6 of TTS_GATEWAY_PLAN.md are done, and the deployed service is confirmed working in production (real grpcurl call through kubectl port-forward, real GPU inference, real playable audio). This file is a punch list for picking the remaining work up in a fresh session — it doesn't re-explain things that are already documented elsewhere; it points to where.

Update (later same day): items 1 and 3 below are now done. Model artifact distribution ended up fully automated, not just "worked around" — after an initial attempt at a manually-built image hit a nik-gpu registry-auth wall, the checkpoint/hparams were committed directly into tts-gateway/model/ (a deliberate one-time exception to not committing large binaries) and CI now builds+pushes the tts-model image itself (build-tts-model in .gitea/workflows/ci.yaml), same as every other image here — no manual nik-gpu step at all anymore. discord-bot has a working /speak command (see TTS_GATEWAY_PLAN.md Phase 6). Only item 2 (mTLS) remains, left untouched per an explicit decision to verify these two first. The two "what's left" entries below are kept as-written for their historical reasoning/decision trail rather than rewritten in place.

Where things stand

• tts-gateway (Go) + tts-sidecar (Python/libtorch) are both built, containerized, and running in Kubernetes (home-services namespace).
• CI (.gitea/workflows/ci.yaml) builds and pushes both images on every push to main, gated by path-filtered per-service rebuilds and using registry-based Docker layer caching.
• Full history — the Phase 0 ONNX-export/g2p investigation, why there's a Python sidecar instead of pure Go, the exact japanese_cleaners pipeline verification — is in TTS_GATEWAY_PLAN.md and tmp/reference/uma-tts-api/spike/FINDINGS.md. Read those before re-deriving anything; the ONNX export failure in particular took several iterations to characterize correctly and isn't worth re-investigating from scratch.

What's left

1. Model artifact distribution isn't automated. The k8s Deployment (~/repo/homelab/manifests/home-services/tts-gateway.yaml — separate repo, see below) hostPath-mounts /data/tts-gateway on the nik-gpu node into the sidecar container. Someone has to manually place G_790000.pth + uma.json there before the pod goes Ready — no init-container download, no PVC, nothing automated. TTS_GATEWAY_PLAN.md's Phase 5 flagged this as an open decision; it's still open, just worked around. If asked to fix it: options are a PVC populated by an init-container download step, baking the checkpoint into a private image layer, or similar — weigh against the checkpoint being ~455MB and not something to casually put in a git-tracked Dockerfile context.

2. mTLS is off. tts-gateway.yaml comments out the TLS_DIR env var and its volume mount, for plaintext grpcurl testing from outside the cluster during initial rollout — a live TODO, not a permanent decision. Every other service in this repo assumes internal-network-or-mTLS as its only access boundary (CLAUDE.md's "Configuration Notes"). Re-enabling: uncomment the env var + mount in that manifest, and set up a tts-gateway-tls secret the same way ha-gateway-sealed.yaml/ha-gateway-secret.sh do it for ha-gateway (tts-gateway doesn't currently have its own -secret.sh/-sealed.yaml pair — it'll need one, since unlike ha-gateway/discord-bot it has no other secrets today, so this would be its first).

3. Phase 6 (discord-bot integration) not started. A /speak-style Discord slash command calling tts-gateway, following the existing pattern — discord-bot already has gRPC clients for ha-gateway and ai-gateway at discord-bot/internal/adapters/secondary/{gateway,aigateway}; a new internal/adapters/secondary/ttsgateway client would follow the same shape. Needs a product decision first: keep AAC (parity with what's already implemented) or switch to Opus (Discord's native voice codec) — deliberately deferred in TTS_GATEWAY_PLAN.md's open questions until this exact moment, not yet decided.

Things worth knowing before touching this

• tts-gateway and tts-sidecar share one Kubernetes Pod, not two separate Deployments — the Go gateway reaches the sidecar over localhost:50054. This mirrors the --network host setup documented in tts-gateway/README.md for local Docker testing. Don't split them into separate Deployments/Services without also rethinking INFERENCE_SIDECAR_ADDR and the networking model.
• The k8s manifests live in a different repo: ~/repo/homelab/manifests/home-services/ — not in this repo at all. CLAUDE.md now has an "Infrastructure / Deployment" section documenting this; check there (and that repo directly) before assuming something isn't deployed just because this repo has no manifests for it.
• open_jtalk's tokenization has an intentional bug that must be preserved. The Go text normalizer (tts-gateway/internal/adapters/secondary/jtalk) maps the cleaned phoneme string to symbol IDs character-by-character, not phoneme-by-phoneme, because that's what the checkpoint was actually trained on (text/symbols.py's "wrong tokens" comment in the original reference implementation). Don't "fix" this without retraining the model — see the comments in jtalk.go and tts-gateway/README.md.
• ONNX export doesn't work for this checkpoint, and it's not a quick fix. Read tmp/reference/uma-tts-api/spike/FINDINGS.md before re-attempting it — the model's data-dependent output length (predicted phoneme durations determine audio length at runtime) defeats torch.export's guard system in a way that would need an unknown number of per-layer torch._check hints to resolve, not a single targeted change.
• CI's build cache needs one "priming" run per image. If a build job still looks slow after the caching change in ci.yaml, check whether a :buildcache tag actually exists yet in the registry for that specific image — the first run after the cache was added has nothing to pull from and builds fully fresh.

Where to look for more detail

• TTS_GATEWAY_PLAN.md (repo root) — original plan, feasibility analysis, phase breakdown, open product questions (codec, concurrency, auth).
• tmp/reference/uma-tts-api/spike/FINDINGS.md — Phase 0 spike results with full reasoning.
• tts-gateway/README.md — service-level docs: gRPC API, config reference, how to run/test locally and on nik-gpu.
• ~/repo/homelab/manifests/home-services/tts-gateway.yaml — actual production config (separate repo, maintained independently — check it directly rather than trusting a stale summary).