Nik Afiq
7d9a75a3e7
All checks were successful
CI / changes (push) Successful in 19s
CI / test (push) Successful in 23s
CI / build-ai-gateway (push) Has been skipped
CI / build-ha-gateway (push) Has been skipped
CI / build-discord-bot (push) Has been skipped
CI / build-tts-gateway (push) Successful in 54s
CI / build-tts-sidecar (push) Has been skipped
CI / build-tts-model (push) Successful in 39s
95 lines
6.7 KiB
Markdown
95 lines
6.7 KiB
Markdown
# tts-gateway Handoff
|
||
|
||
Status as of 2026-07-25: Phases 0–6 of `TTS_GATEWAY_PLAN.md` are done, and the deployed service is
|
||
confirmed working in production (real `grpcurl` call through `kubectl port-forward`, real GPU
|
||
inference, real playable audio). This file is a punch list for picking the remaining work up in a
|
||
fresh session — it doesn't re-explain things that are already documented elsewhere; it points to
|
||
where.
|
||
|
||
**Update (later same day):** items 1 and 3 below are now done. Model artifact distribution ended
|
||
up fully automated, not just "worked around" — after an initial attempt at a manually-built image
|
||
hit a nik-gpu registry-auth wall, the checkpoint/hparams were committed directly into
|
||
`tts-gateway/model/` (a deliberate one-time exception to not committing large binaries) and CI now
|
||
builds+pushes the `tts-model` image itself (`build-tts-model` in `.gitea/workflows/ci.yaml`), same
|
||
as every other image here — no manual nik-gpu step at all anymore. `discord-bot` has a working
|
||
`/speak` command (see `TTS_GATEWAY_PLAN.md` Phase 6). Only item 2 (mTLS) remains, left untouched
|
||
per an explicit decision to verify these two first. The two "what's left" entries below are kept
|
||
as-written for their historical reasoning/decision trail rather than rewritten in place.
|
||
|
||
## Where things stand
|
||
|
||
- `tts-gateway` (Go) + `tts-sidecar` (Python/libtorch) are both built, containerized, and running
|
||
in Kubernetes (`home-services` namespace).
|
||
- CI (`.gitea/workflows/ci.yaml`) builds and pushes both images on every push to `main`, gated by
|
||
path-filtered per-service rebuilds and using registry-based Docker layer caching.
|
||
- Full history — the Phase 0 ONNX-export/g2p investigation, why there's a Python sidecar instead
|
||
of pure Go, the exact `japanese_cleaners` pipeline verification — is in `TTS_GATEWAY_PLAN.md`
|
||
and `tmp/reference/uma-tts-api/spike/FINDINGS.md`. Read those before re-deriving anything; the
|
||
ONNX export failure in particular took several iterations to characterize correctly and isn't
|
||
worth re-investigating from scratch.
|
||
|
||
## What's left
|
||
|
||
1. **Model artifact distribution isn't automated.** The k8s Deployment
|
||
(`~/repo/homelab/manifests/home-services/tts-gateway.yaml` — separate repo, see below)
|
||
`hostPath`-mounts `/data/tts-gateway` on the `nik-gpu` node into the sidecar container.
|
||
Someone has to manually place `G_790000.pth` + `uma.json` there before the pod goes Ready — no
|
||
init-container download, no PVC, nothing automated. `TTS_GATEWAY_PLAN.md`'s Phase 5 flagged
|
||
this as an open decision; it's still open, just worked around. If asked to fix it: options are
|
||
a PVC populated by an init-container download step, baking the checkpoint into a private image
|
||
layer, or similar — weigh against the checkpoint being ~455MB and not something to casually put
|
||
in a git-tracked Dockerfile context.
|
||
|
||
2. **mTLS is off.** `tts-gateway.yaml` comments out the `TLS_DIR` env var and its volume mount,
|
||
for plaintext `grpcurl` testing from outside the cluster during initial rollout — a live TODO,
|
||
not a permanent decision. Every other service in this repo assumes internal-network-or-mTLS as
|
||
its only access boundary (`CLAUDE.md`'s "Configuration Notes"). Re-enabling: uncomment the env
|
||
var + mount in that manifest, and set up a `tts-gateway-tls` secret the same way
|
||
`ha-gateway-sealed.yaml`/`ha-gateway-secret.sh` do it for ha-gateway (tts-gateway doesn't
|
||
currently have its own `-secret.sh`/`-sealed.yaml` pair — it'll need one, since unlike
|
||
ha-gateway/discord-bot it has no other secrets today, so this would be its first).
|
||
|
||
3. **Phase 6 (discord-bot integration) not started.** A `/speak`-style Discord slash command
|
||
calling `tts-gateway`, following the existing pattern — `discord-bot` already has gRPC clients
|
||
for `ha-gateway` and `ai-gateway` at `discord-bot/internal/adapters/secondary/{gateway,aigateway}`;
|
||
a new `internal/adapters/secondary/ttsgateway` client would follow the same shape. Needs a
|
||
product decision first: keep AAC (parity with what's already implemented) or switch to Opus
|
||
(Discord's native voice codec) — deliberately deferred in `TTS_GATEWAY_PLAN.md`'s open
|
||
questions until this exact moment, not yet decided.
|
||
|
||
## Things worth knowing before touching this
|
||
|
||
- **`tts-gateway` and `tts-sidecar` share one Kubernetes Pod**, not two separate Deployments — the
|
||
Go gateway reaches the sidecar over `localhost:50054`. This mirrors the `--network host` setup
|
||
documented in `tts-gateway/README.md` for local Docker testing. Don't split them into separate
|
||
Deployments/Services without also rethinking `INFERENCE_SIDECAR_ADDR` and the networking model.
|
||
- **The k8s manifests live in a different repo**: `~/repo/homelab/manifests/home-services/` — not
|
||
in this repo at all. `CLAUDE.md` now has an "Infrastructure / Deployment" section documenting
|
||
this; check there (and that repo directly) before assuming something isn't deployed just because
|
||
this repo has no manifests for it.
|
||
- **`open_jtalk`'s tokenization has an intentional bug that must be preserved.** The Go text
|
||
normalizer (`tts-gateway/internal/adapters/secondary/jtalk`) maps the cleaned phoneme string to
|
||
symbol IDs character-by-character, not phoneme-by-phoneme, because that's what the checkpoint
|
||
was actually trained on (`text/symbols.py`'s "wrong tokens" comment in the original reference
|
||
implementation). Don't "fix" this without retraining the model — see the comments in `jtalk.go`
|
||
and `tts-gateway/README.md`.
|
||
- **ONNX export doesn't work for this checkpoint, and it's not a quick fix.** Read
|
||
`tmp/reference/uma-tts-api/spike/FINDINGS.md` before re-attempting it — the model's
|
||
data-dependent output length (predicted phoneme durations determine audio length at runtime)
|
||
defeats `torch.export`'s guard system in a way that would need an unknown number of per-layer
|
||
`torch._check` hints to resolve, not a single targeted change.
|
||
- **CI's build cache needs one "priming" run per image.** If a build job still looks slow after
|
||
the caching change in `ci.yaml`, check whether a `:buildcache` tag actually exists yet in the
|
||
registry for that specific image — the first run after the cache was added has nothing to pull
|
||
from and builds fully fresh.
|
||
|
||
## Where to look for more detail
|
||
|
||
- `TTS_GATEWAY_PLAN.md` (repo root) — original plan, feasibility analysis, phase breakdown, open
|
||
product questions (codec, concurrency, auth).
|
||
- `tmp/reference/uma-tts-api/spike/FINDINGS.md` — Phase 0 spike results with full reasoning.
|
||
- `tts-gateway/README.md` — service-level docs: gRPC API, config reference, how to run/test
|
||
locally and on nik-gpu.
|
||
- `~/repo/homelab/manifests/home-services/tts-gateway.yaml` — actual production config (separate
|
||
repo, maintained independently — check it directly rather than trusting a stale summary).
|