# tts-gateway

`tts-gateway` is the internal gRPC boundary for text-to-speech synthesis. It
wraps a VITS voice model (92 Umamusume: Pretty Derby character voices, ported
from `tmp/reference/uma-tts-api`) behind a small protobuf API.

Unlike the other three services, `tts-gateway` is two deployables:

- **`tts-gateway`** (this directory, Go) - the gRPC service itself: Japanese
  text normalization (`open_jtalk` CLI) and audio encoding (WAV + `ffmpeg`
  AAC transcode). CPU-only, no GPU needed.
- **`tts-gateway/sidecar`** (Python/libtorch) - a minimal HTTP service that
  does only `net_g.infer()` (the actual VITS forward pass). This exists
  because the checkpoint's output length is genuinely data-dependent at
  runtime (predicted phoneme durations), which `torch.export`/ONNX couldn't
  trace cleanly - see `tmp/reference/uma-tts-api/spike/FINDINGS.md` for the
  full investigation. Needs an Nvidia GPU to be useful; only meaningfully
  runs on `nik-gpu`.

## Runtime Flow

1. `tts-gateway` loads `.env`, configures logging/telemetry, and starts gRPC
   on `GRPC_PORT`.
2. `TTSService.Synthesize` normalizes the request text via the `open_jtalk`
   CLI adapter (reproducing `uma-tts-api`'s `japanese_cleaners` pipeline
   character-for-character, including its "wrong tokens" quirk - the
   checkpoint was trained on that exact tokenization), sends the resulting
   symbol IDs to the inference sidecar over HTTP, and transcodes the
   returned PCM to AAC via `ffmpeg`.
3. `TTSService.ListSpeakers` returns the 92-speaker roster (optionally
   filtered by a case-insensitive substring), no sidecar call needed.

## gRPC API

Contract: `proto/tts/v1/tts.proto`.

- `TTSService.Synthesize` - `speaker_name`, `text`, optional `noise_scale`
  (default `0.37`) / `noise_scale_w` (default `0.46`) / `length_scale`
  (default `1.3`) → `audio` bytes + `mime_type` (`audio/aac`). Unknown
  speaker → `INVALID_ARGUMENT`.
- `TTSService.ListSpeakers` - optional `search` substring → speaker names.

The server also registers gRPC health checks and reflection.

## Configuration

`tts-gateway` environment variables:

| Variable | Default | Description |
| --- | --- | --- |
| `GRPC_PORT` | `50053` | gRPC listen port |
| `TLS_DIR` | empty | Enables mTLS for the gRPC server when set |
| `OTEL_ENDPOINT` | empty | OTLP gRPC collector endpoint; empty disables telemetry |
| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, or `error` |
| `LOG_FORMAT` | `json` | `json` or `text` |
| `OPEN_JTALK_BIN` | `open_jtalk` | Path/name of the open_jtalk CLI binary |
| `OPEN_JTALK_DICT_DIR` | empty | Dictionary dir; empty auto-discovers under `/usr`/`/var` |
| `OPEN_JTALK_VOICE` | empty | `.htsvoice` path; empty auto-discovers under `/usr/share/hts-voice` |
| `INFERENCE_SIDECAR_ADDR` | `localhost:50054` | `host:port` of the inference sidecar |

`sidecar/server.py` environment variables:

| Variable | Default | Description |
| --- | --- | --- |
| `CONFIG_PATH` | `/models/uma.json` | Path to the VITS hparams JSON |
| `CHECKPOINT_PATH` | `/models/G_790000.pth` | Path to the model checkpoint |
| `PORT` | `50054` | HTTP listen port |

Neither the checkpoint nor the hparams file is committed to git - both must
be supplied at runtime (see below). In production, they're baked into a
small versioned image (`tts-gateway/model/Dockerfile`) that a k8s
`initContainer` copies into a shared volume before `tts-sidecar` starts -
see "Model artifact distribution (production)" below. For local/manual
testing on nik-gpu, a plain bind mount (as shown below) is simpler.

### Model artifact distribution (production)

CI can never build the model image - the checkpoint (~455MB) and hparams
aren't committed to git, and nik-gpu's local disk at `/data/tts-gateway` is
the only place they exist. Build and push it manually, directly on nik-gpu
(not via `docker --context nik-gpu` from elsewhere - the build context is
gathered client-side, so it would look for `/data/tts-gateway` on the wrong
machine), whenever the checkpoint or config change:

```bash
ssh nik-gpu
docker build -f ~/repo/home-service/tts-gateway/model/Dockerfile \
  -t gitea.nik4nao.com/nik/tts-model:g790000-v1 /data/tts-gateway
docker push gitea.nik4nao.com/nik/tts-model:g790000-v1
```

Bump the tag (`v2`, `v3`, ...) rather than overwriting one - non-`:latest`
tags default to `imagePullPolicy: IfNotPresent`, so a node that already
pulled a tag won't re-pull an overwritten one. The k8s Deployment
(`~/repo/homelab/manifests/home-services/tts-gateway.yaml`) references the
tag explicitly in its `model-init` init container.

## Running And Testing On nik-gpu

The Go gateway only needs `ffmpeg`/`open_jtalk`, which its image already
bundles - it can run anywhere. The sidecar needs an actual Nvidia GPU, so the
full system is really only testable on `nik-gpu`.

### Option A: pull the images CI already built and pushed

```bash
docker --context nik-gpu pull gitea.nik4nao.com/nik/tts-gateway:latest
docker --context nik-gpu pull gitea.nik4nao.com/nik/tts-sidecar:latest
```

### Option B: build from source (use the `nik-gpu-sync` and `nik-gpu-docker-build` skills)

```bash
# from the repo root, on your local machine
# 1. sync the repo to nik-gpu (nik-gpu-sync skill)
# 2. build both images there (nik-gpu-docker-build skill), e.g.:
docker --context nik-gpu build --platform linux/amd64 \
  -f tts-gateway/Dockerfile -t gitea.nik4nao.com/nik/tts-gateway:latest ~/repo/home-service
docker --context nik-gpu build --platform linux/amd64 \
  -f tts-gateway/sidecar/Dockerfile -t gitea.nik4nao.com/nik/tts-sidecar:latest ~/repo/home-service/tts-gateway/sidecar
```

### Run both containers on nik-gpu

The checkpoint/config aren't baked into the sidecar image - bind-mount them
from wherever they live on nik-gpu (e.g. a synced copy of
`tmp/reference/uma-tts-api/`). Both containers use `--network host` so the
gateway can reach the sidecar over `localhost`:

```bash
docker --context nik-gpu run -d --name tts-sidecar --network host --gpus all \
  -v /home/nik/repo/home-service/tmp/reference/uma-tts-api/G_790000.pth:/models/G_790000.pth:ro \
  -v /home/nik/repo/home-service/tmp/reference/uma-tts-api/configs/uma.json:/models/uma.json:ro \
  gitea.nik4nao.com/nik/tts-sidecar:latest

docker --context nik-gpu run -d --name tts-gateway --network host \
  -e GRPC_PORT=50053 -e INFERENCE_SIDECAR_ADDR=localhost:50054 -e LOG_FORMAT=text \
  gitea.nik4nao.com/nik/tts-gateway:latest
```

Check the sidecar loaded the checkpoint before testing:

```bash
docker --context nik-gpu logs tts-sidecar
# expect: "model loaded from /models/G_790000.pth" then "inference sidecar listening on :50054"
```

**Use absolute remote paths in `-v`, not `~`** - with a `docker --context`
pointed at a remote host, `~` still gets expanded by your *local* shell
before the command is sent, not by nik-gpu, so it silently resolves to a
path that doesn't exist there and Docker bind-mounts an empty directory
instead of the real file.

### Smoke test with grpcurl

No local `grpcurl` needed - run it in a container against the host network:

```bash
docker --context nik-gpu run --rm --network host fullstorydev/grpcurl \
  -plaintext -d '{"search":"rice"}' localhost:50053 tts.v1.TTSService/ListSpeakers

docker --context nik-gpu run --rm --network host fullstorydev/grpcurl \
  -plaintext -d '{"speaker_name":"Rice Shower","text":"おはようございます"}' \
  localhost:50053 tts.v1.TTSService/Synthesize > /tmp/synth.json

python3 -c "
import json, base64
data = json.load(open('/tmp/synth.json'))
open('/tmp/synth.m4a', 'wb').write(base64.b64decode(data['audio']))
"
ffprobe /tmp/synth.m4a   # sanity-check duration/codec
```

### Clean up

```bash
docker --context nik-gpu rm -f tts-gateway tts-sidecar
```

## Test And Build

```bash
go test ./...
go build ./...
```

Note: `go vet`/`go test`/`go build` don't need `ffmpeg`/`open_jtalk` present -
the tests that touch those adapters only exercise pure parsing/encoding
logic (`internal/adapters/secondary/jtalk`, `internal/adapters/secondary/ffmpeg`),
never the actual binaries.

## Package Map

```text
cmd/gateway/                                  # process entrypoint and wiring
internal/adapters/primary/grpc/               # gRPC service implementation
internal/adapters/secondary/jtalk/            # open_jtalk CLI text normalizer
internal/adapters/secondary/inferencesidecar/ # HTTP client for the Python sidecar
internal/adapters/secondary/ffmpeg/           # WAV + ffmpeg AAC encoder
internal/app/                                 # synthesis orchestration
internal/config/                              # environment loading
internal/core/domain/                         # domain types, symbol table, speaker roster
internal/core/ports/                          # driving and driven interfaces
internal/logger/                              # slog setup
internal/telemetry/                           # OpenTelemetry setup
sidecar/                                      # Python/libtorch inference service (see above)
```

## Limitations

- No app-layer authorization, same as the other three services - keep this
  internal or protect it with mTLS.
- Concurrency is unbounded on the Go side, but the sidecar's model forward
  pass is inherently single-threaded per GPU; heavy concurrent load will
  just queue at the sidecar.
- Model artifact distribution to production is a manually-built-and-pushed
  versioned image (see above), not something CI can automate - the
  checkpoint only ever existed on nik-gpu's local disk.
- `discord-bot`'s `/speak` command calls this service and plays the result
  in a Discord voice channel (`discord-bot/internal/adapters/primary/discord/voice.go`).