Nik Afiq
7d9a75a3e7
All checks were successful
CI / changes (push) Successful in 19s
CI / test (push) Successful in 23s
CI / build-ai-gateway (push) Has been skipped
CI / build-ha-gateway (push) Has been skipped
CI / build-discord-bot (push) Has been skipped
CI / build-tts-gateway (push) Successful in 54s
CI / build-tts-sidecar (push) Has been skipped
CI / build-tts-model (push) Successful in 39s
204 lines
8.8 KiB
Markdown
204 lines
8.8 KiB
Markdown
# tts-gateway
|
|
|
|
`tts-gateway` is the internal gRPC boundary for text-to-speech synthesis. It
|
|
wraps a VITS voice model (92 Umamusume: Pretty Derby character voices, ported
|
|
from `tmp/reference/uma-tts-api`) behind a small protobuf API.
|
|
|
|
Unlike the other three services, `tts-gateway` is two deployables:
|
|
|
|
- **`tts-gateway`** (this directory, Go) - the gRPC service itself: Japanese
|
|
text normalization (`open_jtalk` CLI) and audio encoding (WAV + `ffmpeg`
|
|
AAC transcode). CPU-only, no GPU needed.
|
|
- **`tts-gateway/sidecar`** (Python/libtorch) - a minimal HTTP service that
|
|
does only `net_g.infer()` (the actual VITS forward pass). This exists
|
|
because the checkpoint's output length is genuinely data-dependent at
|
|
runtime (predicted phoneme durations), which `torch.export`/ONNX couldn't
|
|
trace cleanly - see `tmp/reference/uma-tts-api/spike/FINDINGS.md` for the
|
|
full investigation. Needs an Nvidia GPU to be useful; only meaningfully
|
|
runs on `nik-gpu`.
|
|
|
|
## Runtime Flow
|
|
|
|
1. `tts-gateway` loads `.env`, configures logging/telemetry, and starts gRPC
|
|
on `GRPC_PORT`.
|
|
2. `TTSService.Synthesize` normalizes the request text via the `open_jtalk`
|
|
CLI adapter (reproducing `uma-tts-api`'s `japanese_cleaners` pipeline
|
|
character-for-character, including its "wrong tokens" quirk - the
|
|
checkpoint was trained on that exact tokenization), sends the resulting
|
|
symbol IDs to the inference sidecar over HTTP, and transcodes the
|
|
returned PCM to AAC via `ffmpeg`.
|
|
3. `TTSService.ListSpeakers` returns the 92-speaker roster (optionally
|
|
filtered by a case-insensitive substring), no sidecar call needed.
|
|
|
|
## gRPC API
|
|
|
|
Contract: `proto/tts/v1/tts.proto`.
|
|
|
|
- `TTSService.Synthesize` - `speaker_name`, `text`, optional `noise_scale`
|
|
(default `0.37`) / `noise_scale_w` (default `0.46`) / `length_scale`
|
|
(default `1.3`) → `audio` bytes + `mime_type` (`audio/aac`). Unknown
|
|
speaker → `INVALID_ARGUMENT`.
|
|
- `TTSService.ListSpeakers` - optional `search` substring → speaker names.
|
|
|
|
The server also registers gRPC health checks and reflection.
|
|
|
|
## Configuration
|
|
|
|
`tts-gateway` environment variables:
|
|
|
|
| Variable | Default | Description |
|
|
| --- | --- | --- |
|
|
| `GRPC_PORT` | `50053` | gRPC listen port |
|
|
| `TLS_DIR` | empty | Enables mTLS for the gRPC server when set |
|
|
| `OTEL_ENDPOINT` | empty | OTLP gRPC collector endpoint; empty disables telemetry |
|
|
| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, or `error` |
|
|
| `LOG_FORMAT` | `json` | `json` or `text` |
|
|
| `OPEN_JTALK_BIN` | `open_jtalk` | Path/name of the open_jtalk CLI binary |
|
|
| `OPEN_JTALK_DICT_DIR` | empty | Dictionary dir; empty auto-discovers under `/usr`/`/var` |
|
|
| `OPEN_JTALK_VOICE` | empty | `.htsvoice` path; empty auto-discovers under `/usr/share/hts-voice` |
|
|
| `INFERENCE_SIDECAR_ADDR` | `localhost:50054` | `host:port` of the inference sidecar |
|
|
|
|
`sidecar/server.py` environment variables:
|
|
|
|
| Variable | Default | Description |
|
|
| --- | --- | --- |
|
|
| `CONFIG_PATH` | `/models/uma.json` | Path to the VITS hparams JSON |
|
|
| `CHECKPOINT_PATH` | `/models/G_790000.pth` | Path to the model checkpoint |
|
|
| `PORT` | `50054` | HTTP listen port |
|
|
|
|
The checkpoint and hparams file live directly in `tts-gateway/model/` (a
|
|
deliberate, one-time exception to not committing large binaries - this asset
|
|
never changes and there's no other artifact store in play). CI builds and
|
|
pushes `gitea.nik4nao.com/nik/tts-model` from there exactly like the other
|
|
four images (`build-tts-model` in `.gitea/workflows/ci.yaml`), path-filtered
|
|
so it only rebuilds when `tts-gateway/model/**` actually changes. In
|
|
production, a k8s `initContainer` copies that image's files into a shared
|
|
volume before `tts-sidecar` starts - see
|
|
`~/repo/homelab/manifests/home-services/tts-gateway.yaml`'s `model-init`
|
|
container. To update the checkpoint: replace `tts-gateway/model/G_790000.pth`
|
|
and/or `uma.json`, commit, push to `main` - CI handles the rest.
|
|
|
|
## Running And Testing On nik-gpu
|
|
|
|
The Go gateway only needs `ffmpeg`/`open_jtalk`, which its image already
|
|
bundles - it can run anywhere. The sidecar needs an actual Nvidia GPU, so the
|
|
full system is really only testable on `nik-gpu`.
|
|
|
|
### Option A: pull the images CI already built and pushed
|
|
|
|
```bash
|
|
docker --context nik-gpu pull gitea.nik4nao.com/nik/tts-gateway:latest
|
|
docker --context nik-gpu pull gitea.nik4nao.com/nik/tts-sidecar:latest
|
|
```
|
|
|
|
### Option B: build from source (use the `nik-gpu-sync` and `nik-gpu-docker-build` skills)
|
|
|
|
```bash
|
|
# from the repo root, on your local machine
|
|
# 1. sync the repo to nik-gpu (nik-gpu-sync skill)
|
|
# 2. build both images there (nik-gpu-docker-build skill), e.g.:
|
|
docker --context nik-gpu build --platform linux/amd64 \
|
|
-f tts-gateway/Dockerfile -t gitea.nik4nao.com/nik/tts-gateway:latest ~/repo/home-service
|
|
docker --context nik-gpu build --platform linux/amd64 \
|
|
-f tts-gateway/sidecar/Dockerfile -t gitea.nik4nao.com/nik/tts-sidecar:latest ~/repo/home-service/tts-gateway/sidecar
|
|
```
|
|
|
|
### Run both containers on nik-gpu
|
|
|
|
The checkpoint/config aren't baked into the sidecar image - bind-mount them
|
|
from wherever they live on nik-gpu (e.g. a synced copy of
|
|
`tmp/reference/uma-tts-api/`). Both containers use `--network host` so the
|
|
gateway can reach the sidecar over `localhost`:
|
|
|
|
```bash
|
|
docker --context nik-gpu run -d --name tts-sidecar --network host --gpus all \
|
|
-v /home/nik/repo/home-service/tmp/reference/uma-tts-api/G_790000.pth:/models/G_790000.pth:ro \
|
|
-v /home/nik/repo/home-service/tmp/reference/uma-tts-api/configs/uma.json:/models/uma.json:ro \
|
|
gitea.nik4nao.com/nik/tts-sidecar:latest
|
|
|
|
docker --context nik-gpu run -d --name tts-gateway --network host \
|
|
-e GRPC_PORT=50053 -e INFERENCE_SIDECAR_ADDR=localhost:50054 -e LOG_FORMAT=text \
|
|
gitea.nik4nao.com/nik/tts-gateway:latest
|
|
```
|
|
|
|
Check the sidecar loaded the checkpoint before testing:
|
|
|
|
```bash
|
|
docker --context nik-gpu logs tts-sidecar
|
|
# expect: "model loaded from /models/G_790000.pth" then "inference sidecar listening on :50054"
|
|
```
|
|
|
|
**Use absolute remote paths in `-v`, not `~`** - with a `docker --context`
|
|
pointed at a remote host, `~` still gets expanded by your *local* shell
|
|
before the command is sent, not by nik-gpu, so it silently resolves to a
|
|
path that doesn't exist there and Docker bind-mounts an empty directory
|
|
instead of the real file.
|
|
|
|
### Smoke test with grpcurl
|
|
|
|
No local `grpcurl` needed - run it in a container against the host network:
|
|
|
|
```bash
|
|
docker --context nik-gpu run --rm --network host fullstorydev/grpcurl \
|
|
-plaintext -d '{"search":"rice"}' localhost:50053 tts.v1.TTSService/ListSpeakers
|
|
|
|
docker --context nik-gpu run --rm --network host fullstorydev/grpcurl \
|
|
-plaintext -d '{"speaker_name":"Rice Shower","text":"おはようございます"}' \
|
|
localhost:50053 tts.v1.TTSService/Synthesize > /tmp/synth.json
|
|
|
|
python3 -c "
|
|
import json, base64
|
|
data = json.load(open('/tmp/synth.json'))
|
|
open('/tmp/synth.m4a', 'wb').write(base64.b64decode(data['audio']))
|
|
"
|
|
ffprobe /tmp/synth.m4a # sanity-check duration/codec
|
|
```
|
|
|
|
### Clean up
|
|
|
|
```bash
|
|
docker --context nik-gpu rm -f tts-gateway tts-sidecar
|
|
```
|
|
|
|
## Test And Build
|
|
|
|
```bash
|
|
go test ./...
|
|
go build ./...
|
|
```
|
|
|
|
Note: `go vet`/`go test`/`go build` don't need `ffmpeg`/`open_jtalk` present -
|
|
the tests that touch those adapters only exercise pure parsing/encoding
|
|
logic (`internal/adapters/secondary/jtalk`, `internal/adapters/secondary/ffmpeg`),
|
|
never the actual binaries.
|
|
|
|
## Package Map
|
|
|
|
```text
|
|
cmd/gateway/ # process entrypoint and wiring
|
|
internal/adapters/primary/grpc/ # gRPC service implementation
|
|
internal/adapters/secondary/jtalk/ # open_jtalk CLI text normalizer
|
|
internal/adapters/secondary/inferencesidecar/ # HTTP client for the Python sidecar
|
|
internal/adapters/secondary/ffmpeg/ # WAV + ffmpeg AAC encoder
|
|
internal/app/ # synthesis orchestration
|
|
internal/config/ # environment loading
|
|
internal/core/domain/ # domain types, symbol table, speaker roster
|
|
internal/core/ports/ # driving and driven interfaces
|
|
internal/logger/ # slog setup
|
|
internal/telemetry/ # OpenTelemetry setup
|
|
sidecar/ # Python/libtorch inference service (see above)
|
|
```
|
|
|
|
## Limitations
|
|
|
|
- No app-layer authorization, same as the other three services - keep this
|
|
internal or protect it with mTLS.
|
|
- Concurrency is unbounded on the Go side, but the sidecar's model forward
|
|
pass is inherently single-threaded per GPU; heavy concurrent load will
|
|
just queue at the sidecar.
|
|
- The model checkpoint is committed directly in `tts-gateway/model/` rather
|
|
than fetched from an external artifact store - a deliberate exception
|
|
given how rarely it changes, not a pattern to repeat for other assets.
|
|
- `discord-bot`'s `/speak` command calls this service and plays the result
|
|
in a Discord voice channel (`discord-bot/internal/adapters/primary/discord/voice.go`).
|