feat: add tts-gateway and sidecar Dockerfiles, update CI workflow for TTS builds

2026-07-24 22:55:11 +09:00 · 2026-07-24 22:55:11 +09:00 · 99b624ef20
commit 99b624ef20
parent 1b025301d0
5 changed files with 262 additions and 11 deletions
--- a/.gitea/workflows/ci.yaml
+++ b/.gitea/workflows/ci.yaml
@ -124,3 +124,56 @@ jobs:
          tags: |
            ${{ env.IMAGE_PREFIX }}/discord-bot:${{ github.sha }}
            ${{ env.IMAGE_PREFIX }}/discord-bot:latest
+
+  build-tts-gateway:
+    needs: test
+    if: github.ref == 'refs/heads/main'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - uses: docker/setup-buildx-action@v4
+
+      - uses: docker/login-action@v4
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ secrets.REGISTRY_USER }}
+          password: ${{ secrets.REGISTRY_PASSWORD }}
+
+      - uses: docker/build-push-action@v7
+        with:
+          context: .
+          file: tts-gateway/Dockerfile
+          push: true
+          platforms: linux/amd64
+          tags: |
+            ${{ env.IMAGE_PREFIX }}/tts-gateway:${{ github.sha }}
+            ${{ env.IMAGE_PREFIX }}/tts-gateway:latest
+
+  build-tts-sidecar:
+    needs: test
+    if: github.ref == 'refs/heads/main'
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+
+      - uses: docker/setup-buildx-action@v4
+
+      - uses: docker/login-action@v4
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ secrets.REGISTRY_USER }}
+          password: ${{ secrets.REGISTRY_PASSWORD }}
+
+      # CUDA base image makes this a large (~13GB) build/push - it doesn't
+      # need a GPU to build (only to usefully run), but expect this job to be
+      # the slowest one in the pipeline by a wide margin.
+      - uses: docker/build-push-action@v7
+        with:
+          context: tts-gateway/sidecar
+          file: tts-gateway/sidecar/Dockerfile
+          push: true
+          platforms: linux/amd64
+          tags: |
+            ${{ env.IMAGE_PREFIX }}/tts-sidecar:${{ github.sha }}
+            ${{ env.IMAGE_PREFIX }}/tts-sidecar:latest
--- a/CLAUDE.md
+++ b/CLAUDE.md
@ -139,4 +139,4 @@ mutation, and is fine to run.

 ## CI

-`.gitea/workflows/ci.yaml` runs `go vet` and `go test` for all five modules (`gen`, `ai-gateway`, `ha-gateway`, `discord-bot`, `tts-gateway`), then builds and pushes Docker images for `ai-gateway`, `ha-gateway`, and `discord-bot` on pushes to `main`. `tts-gateway` is deliberately excluded from the build/push step for now — its production image (CUDA base, model artifact distribution, etc.) is a separate decision, not yet made. Note that `ai-gateway/Dockerfile`, `ha-gateway/Dockerfile`, and `discord-bot/Dockerfile` each `COPY` every other service directory (not just their own) because `go.work` lists all five modules as workspace members — Go's workspace-mode module resolution needs every listed directory present in the build context, even ones a given service doesn't otherwise depend on. Adding a new module to `go.work` means adding a matching `COPY` line to the other Dockerfiles too, or their builds break.
+`.gitea/workflows/ci.yaml` runs `go vet` and `go test` for all five modules (`gen`, `ai-gateway`, `ha-gateway`, `discord-bot`, `tts-gateway`), then builds and pushes Docker images on pushes to `main`: `ai-gateway`, `ha-gateway`, `discord-bot`, `tts-gateway`, and `tts-sidecar` (the Python/CUDA inference sidecar under `tts-gateway/sidecar/` — a large ~13GB image; it builds fine on a generic runner since only *running* it needs a GPU, not building it). Note that `ai-gateway/Dockerfile`, `ha-gateway/Dockerfile`, `discord-bot/Dockerfile`, and `tts-gateway/Dockerfile` each `COPY` every other service directory (not just their own) because `go.work` lists all five Go modules as workspace members — Go's workspace-mode module resolution needs every listed directory present in the build context, even ones a given service doesn't otherwise depend on. Adding a new module to `go.work` means adding a matching `COPY` line to the other Dockerfiles too, or their builds break.
--- a/tts-gateway/Dockerfile.dev
+++ b/tts-gateway/Dockerfile.dev
@ -1,9 +1,9 @@
-# Dev/smoke-test image for tts-gateway (Phase 4 of TTS_GATEWAY_PLAN.md).
-# NOT the polished production image - Phase 5 (deferred) decides the final
-# base image/multi-stage layout. This one just needs to prove the pipeline
-# end-to-end: unlike ha-gateway/ai-gateway/discord-bot, tts-gateway needs
-# `ffmpeg` and the `open_jtalk` CLI (+ dictionary + voice) present at
-# runtime, which rules out a distroless base for now.
+# Unlike ha-gateway/ai-gateway/discord-bot, tts-gateway needs `ffmpeg` and the
+# `open_jtalk` CLI (+ dictionary + voice) present at runtime, which rules out
+# a distroless base - hence the Ubuntu runtime stage below instead. Model
+# artifact distribution for the inference sidecar (tts-gateway/sidecar/) is
+# still an open decision (TTS_GATEWAY_PLAN.md Phase 5); this image is only the
+# Go gateway, which has no model weights to worry about.
 FROM golang:1.26-bookworm AS builder
 WORKDIR /workspace

--- a/tts-gateway/README.md
+++ b/tts-gateway/README.md
@ -0,0 +1,195 @@
+# tts-gateway
+
+`tts-gateway` is the internal gRPC boundary for text-to-speech synthesis. It
+wraps a VITS voice model (92 Umamusume: Pretty Derby character voices, ported
+from `tmp/reference/uma-tts-api`) behind a small protobuf API.
+
+Unlike the other three services, `tts-gateway` is two deployables:
+
+- **`tts-gateway`** (this directory, Go) - the gRPC service itself: Japanese
+  text normalization (`open_jtalk` CLI) and audio encoding (WAV + `ffmpeg`
+  AAC transcode). CPU-only, no GPU needed.
+- **`tts-gateway/sidecar`** (Python/libtorch) - a minimal HTTP service that
+  does only `net_g.infer()` (the actual VITS forward pass). This exists
+  because the checkpoint's output length is genuinely data-dependent at
+  runtime (predicted phoneme durations), which `torch.export`/ONNX couldn't
+  trace cleanly - see `tmp/reference/uma-tts-api/spike/FINDINGS.md` for the
+  full investigation. Needs an Nvidia GPU to be useful; only meaningfully
+  runs on `nik-gpu`.
+
+## Runtime Flow
+
+1. `tts-gateway` loads `.env`, configures logging/telemetry, and starts gRPC
+   on `GRPC_PORT`.
+2. `TTSService.Synthesize` normalizes the request text via the `open_jtalk`
+   CLI adapter (reproducing `uma-tts-api`'s `japanese_cleaners` pipeline
+   character-for-character, including its "wrong tokens" quirk - the
+   checkpoint was trained on that exact tokenization), sends the resulting
+   symbol IDs to the inference sidecar over HTTP, and transcodes the
+   returned PCM to AAC via `ffmpeg`.
+3. `TTSService.ListSpeakers` returns the 92-speaker roster (optionally
+   filtered by a case-insensitive substring), no sidecar call needed.
+
+## gRPC API
+
+Contract: `proto/tts/v1/tts.proto`.
+
+- `TTSService.Synthesize` - `speaker_name`, `text`, optional `noise_scale`
+  (default `0.37`) / `noise_scale_w` (default `0.46`) / `length_scale`
+  (default `1.3`) → `audio` bytes + `mime_type` (`audio/aac`). Unknown
+  speaker → `INVALID_ARGUMENT`.
+- `TTSService.ListSpeakers` - optional `search` substring → speaker names.
+
+The server also registers gRPC health checks and reflection.
+
+## Configuration
+
+`tts-gateway` environment variables:
+
+| Variable | Default | Description |
+| --- | --- | --- |
+| `GRPC_PORT` | `50053` | gRPC listen port |
+| `TLS_DIR` | empty | Enables mTLS for the gRPC server when set |
+| `OTEL_ENDPOINT` | empty | OTLP gRPC collector endpoint; empty disables telemetry |
+| `LOG_LEVEL` | `info` | `debug`, `info`, `warn`, or `error` |
+| `LOG_FORMAT` | `json` | `json` or `text` |
+| `OPEN_JTALK_BIN` | `open_jtalk` | Path/name of the open_jtalk CLI binary |
+| `OPEN_JTALK_DICT_DIR` | empty | Dictionary dir; empty auto-discovers under `/usr`/`/var` |
+| `OPEN_JTALK_VOICE` | empty | `.htsvoice` path; empty auto-discovers under `/usr/share/hts-voice` |
+| `INFERENCE_SIDECAR_ADDR` | `localhost:50054` | `host:port` of the inference sidecar |
+
+`sidecar/server.py` environment variables:
+
+| Variable | Default | Description |
+| --- | --- | --- |
+| `CONFIG_PATH` | `/models/uma.json` | Path to the VITS hparams JSON |
+| `CHECKPOINT_PATH` | `/models/G_790000.pth` | Path to the model checkpoint |
+| `PORT` | `50054` | HTTP listen port |
+
+Neither the checkpoint nor the hparams file is committed to git - both must
+be supplied at runtime (see below). Model artifact distribution (bind mount
+vs. build-time download vs. registry) is still an open decision.
+
+## Running And Testing On nik-gpu
+
+The Go gateway only needs `ffmpeg`/`open_jtalk`, which its image already
+bundles - it can run anywhere. The sidecar needs an actual Nvidia GPU, so the
+full system is really only testable on `nik-gpu`.
+
+### Option A: pull the images CI already built and pushed
+
+```bash
+docker --context nik-gpu pull gitea.nik4nao.com/nik/tts-gateway:latest
+docker --context nik-gpu pull gitea.nik4nao.com/nik/tts-sidecar:latest
+```
+
+### Option B: build from source (use the `nik-gpu-sync` and `nik-gpu-docker-build` skills)
+
+```bash
+# from the repo root, on your local machine
+# 1. sync the repo to nik-gpu (nik-gpu-sync skill)
+# 2. build both images there (nik-gpu-docker-build skill), e.g.:
+docker --context nik-gpu build --platform linux/amd64 \
+  -f tts-gateway/Dockerfile -t gitea.nik4nao.com/nik/tts-gateway:latest ~/repo/home-service
+docker --context nik-gpu build --platform linux/amd64 \
+  -f tts-gateway/sidecar/Dockerfile -t gitea.nik4nao.com/nik/tts-sidecar:latest ~/repo/home-service/tts-gateway/sidecar
+```
+
+### Run both containers on nik-gpu
+
+The checkpoint/config aren't baked into the sidecar image - bind-mount them
+from wherever they live on nik-gpu (e.g. a synced copy of
+`tmp/reference/uma-tts-api/`). Both containers use `--network host` so the
+gateway can reach the sidecar over `localhost`:
+
+```bash
+docker --context nik-gpu run -d --name tts-sidecar --network host --gpus all \
+  -v /home/nik/repo/home-service/tmp/reference/uma-tts-api/G_790000.pth:/models/G_790000.pth:ro \
+  -v /home/nik/repo/home-service/tmp/reference/uma-tts-api/configs/uma.json:/models/uma.json:ro \
+  gitea.nik4nao.com/nik/tts-sidecar:latest
+
+docker --context nik-gpu run -d --name tts-gateway --network host \
+  -e GRPC_PORT=50053 -e INFERENCE_SIDECAR_ADDR=localhost:50054 -e LOG_FORMAT=text \
+  gitea.nik4nao.com/nik/tts-gateway:latest
+```
+
+Check the sidecar loaded the checkpoint before testing:
+
+```bash
+docker --context nik-gpu logs tts-sidecar
+# expect: "model loaded from /models/G_790000.pth" then "inference sidecar listening on :50054"
+```
+
+**Use absolute remote paths in `-v`, not `~`** - with a `docker --context`
+pointed at a remote host, `~` still gets expanded by your *local* shell
+before the command is sent, not by nik-gpu, so it silently resolves to a
+path that doesn't exist there and Docker bind-mounts an empty directory
+instead of the real file.
+
+### Smoke test with grpcurl
+
+No local `grpcurl` needed - run it in a container against the host network:
+
+```bash
+docker --context nik-gpu run --rm --network host fullstorydev/grpcurl \
+  -plaintext -d '{"search":"rice"}' localhost:50053 tts.v1.TTSService/ListSpeakers
+
+docker --context nik-gpu run --rm --network host fullstorydev/grpcurl \
+  -plaintext -d '{"speaker_name":"Rice Shower","text":"おはようございます"}' \
+  localhost:50053 tts.v1.TTSService/Synthesize > /tmp/synth.json
+
+python3 -c "
+import json, base64
+data = json.load(open('/tmp/synth.json'))
+open('/tmp/synth.m4a', 'wb').write(base64.b64decode(data['audio']))
+"
+ffprobe /tmp/synth.m4a   # sanity-check duration/codec
+```
+
+### Clean up
+
+```bash
+docker --context nik-gpu rm -f tts-gateway tts-sidecar
+```
+
+## Test And Build
+
+```bash
+go test ./...
+go build ./...
+```
+
+Note: `go vet`/`go test`/`go build` don't need `ffmpeg`/`open_jtalk` present -
+the tests that touch those adapters only exercise pure parsing/encoding
+logic (`internal/adapters/secondary/jtalk`, `internal/adapters/secondary/ffmpeg`),
+never the actual binaries.
+
+## Package Map
+
+```text
+cmd/gateway/                                  # process entrypoint and wiring
+internal/adapters/primary/grpc/               # gRPC service implementation
+internal/adapters/secondary/jtalk/            # open_jtalk CLI text normalizer
+internal/adapters/secondary/inferencesidecar/ # HTTP client for the Python sidecar
+internal/adapters/secondary/ffmpeg/           # WAV + ffmpeg AAC encoder
+internal/app/                                 # synthesis orchestration
+internal/config/                              # environment loading
+internal/core/domain/                         # domain types, symbol table, speaker roster
+internal/core/ports/                          # driving and driven interfaces
+internal/logger/                              # slog setup
+internal/telemetry/                           # OpenTelemetry setup
+sidecar/                                      # Python/libtorch inference service (see above)
+```
+
+## Limitations
+
+- No app-layer authorization, same as the other three services - keep this
+  internal or protect it with mTLS.
+- Concurrency is unbounded on the Go side, but the sidecar's model forward
+  pass is inherently single-threaded per GPU; heavy concurrent load will
+  just queue at the sidecar.
+- Model artifact distribution isn't finalized - the sidecar currently
+  expects a bind-mounted checkpoint/config, not something the image ships
+  with.
+- No Discord-bot integration yet (a `/speak`-style command calling this
+  service is a deliberate follow-up, not yet built).
--- a/tts-gateway/sidecar/Dockerfile
+++ b/tts-gateway/sidecar/Dockerfile
@ -1,7 +1,10 @@
-# Dev/smoke-test image for the tts-gateway inference sidecar (Phase 3/4 of
-# TTS_GATEWAY_PLAN.md). NOT the polished production image - Phase 5 (deferred)
-# covers the final containerization/CI/model-artifact-distribution decisions.
-# Base image matches nik-gpu's driver 570.211.01 / CUDA 12.8.
+# Inference sidecar for tts-gateway (TTS_GATEWAY_PLAN.md Phase 3). Base image
+# matches nik-gpu's driver 570.211.01 / CUDA 12.8 - only meaningfully runnable
+# on a CUDA host, even though it builds fine anywhere. Does NOT bake in model
+# weights: CHECKPOINT_PATH/CONFIG_PATH must point at a mounted G_790000.pth +
+# uma.json at runtime (model artifact distribution is still an open decision,
+# TTS_GATEWAY_PLAN.md Phase 5 - for now this means a bind mount, see the repo
+# README/CLAUDE.md for the exact nik-gpu run command).
 FROM pytorch/pytorch:2.9.1-cuda12.8-cudnn9-runtime

 RUN apt-get update && apt-get install -y --no-install-recommends \