feat: add /speak command for TTS integration
All checks were successful
CI / changes (push) Successful in 1s
CI / test (push) Successful in 6s
CI / build-ai-gateway (push) Has been skipped
CI / build-ha-gateway (push) Has been skipped
CI / build-discord-bot (push) Successful in 1m33s
CI / build-tts-gateway (push) Successful in 37s
CI / build-tts-sidecar (push) Has been skipped

- Implemented the /speak command in Discord bot to synthesize speech using the TTS gateway.
- Added voice handling logic to join voice channels and play synthesized audio.
- Created tests for the new command and voice functionalities.
- Introduced TTSGateway interface for TTS service communication.
- Updated configuration to include TTS gateway address.
- Documented the TTS gateway integration and model artifact distribution process.
This commit is contained in:
Nik Afiq 2026-07-25 02:09:02 +09:00
parent 7851c49dba
commit 0d58e46740
20 changed files with 937 additions and 50 deletions

View File

@ -17,12 +17,14 @@ ai-gateway ------> Ollama
|
v
ha-gateway
discord-bot -----> tts-gateway -----> tts-sidecar (GPU inference)
```
- **ha-gateway** (port `50051`) — gRPC boundary for Home Assistant. Talks to HA's REST API; implements entity state, light control/discovery, switch control/discovery, and climate (HVAC) control/discovery; also relays SwitchBot Cloud remote commands (`RemoteApp`) when `SWITCHBOT_TOKEN`/`SWITCHBOT_SECRET` are set. Event streaming is stubbed.
- **ai-gateway** (port `50052`) — gRPC service that turns free-form text into home actions. Calls Ollama for intent extraction, resolves intents against a cached light list from `ha-gateway`, and calls `ha-gateway` to execute approved actions.
- **discord-bot** — registers `/light`, `/switch`, `/ac`, `/ai` slash commands and calls `ha-gateway`/`ai-gateway` via gRPC clients.
- **tts-gateway** (port `50053`) — gRPC text-to-speech service (VITS voice model, 92 Umamusume voices), paired with a Python/libtorch inference sidecar (`tts-gateway/sidecar/`, deployed as a second container in the same pod) that needs an Nvidia GPU — see `TTS_GATEWAY_PLAN.md` for why. Not yet wired into the diagram above; nothing calls it yet (a discord-bot `/speak` command is the planned follow-up). See `tts-gateway/README.md` for its API/config/local-run details.
- **discord-bot** — registers `/light`, `/switch`, `/ac`, `/ai`, `/speak` slash commands and calls `ha-gateway`/`ai-gateway`/`tts-gateway` via gRPC clients. `/speak` synthesizes speech via `tts-gateway` (which returns AAC) and transcodes it to Opus locally (via `ffmpeg`/`github.com/jonas747/dca`) to stream into the invoking user's current voice channel — see `internal/adapters/primary/discord/voice.go`.
- **tts-gateway** (port `50053`) — gRPC text-to-speech service (VITS voice model, 92 Umamusume voices), paired with a Python/libtorch inference sidecar (`tts-gateway/sidecar/`, deployed as a second container in the same pod) that needs an Nvidia GPU — see `TTS_GATEWAY_PLAN.md` for why. Called by `discord-bot`'s `/speak` command. See `tts-gateway/README.md` for its API/config/local-run details.
Each service is a separate Go module (own `go.mod`) joined by `go.work` at the root, plus a `gen` module for shared generated code. Module paths are `gitea.nik4nao.com/nik/home-services/{ha-gateway,ai-gateway,discord-bot,tts-gateway,gen}`.
@ -172,10 +174,13 @@ rather than trusting this description to stay accurate):
`nvidia.com/gpu: 1` request/limit on the sidecar container. `strategy: Recreate` instead of the
default `RollingUpdate` — nik-gpu only has one allocatable GPU, so a rolling update would
deadlock waiting for a GPU still held by the pod it's replacing.
- Model artifact distribution (checkpoint + hparams) is a `hostPath` volume at
`/data/tts-gateway` on the nik-gpu node, not baked into the image or automated — someone has to
manually place `G_790000.pth`/`uma.json` there. This is a known-open gap (see `handoff.md`),
not a deliberate final design.
- Model artifact distribution (checkpoint + hparams) is an `emptyDir` volume populated at pod
start by an `initContainers` entry (`model-init`) that copies from a small, versioned image
(`gitea.nik4nao.com/nik/tts-model:<tag>`, built from `tts-gateway/model/Dockerfile`) — not a
raw `hostPath` into the node's disk anymore. That model image still can't be built by CI (the
checkpoint isn't committed to git, ~455MB, and only ever existed on nik-gpu's local disk) — it's
built and pushed manually, on nik-gpu, whenever the checkpoint/config change. See
`tts-gateway/README.md` for the exact commands.
- mTLS (`TLS_DIR`) is currently commented out on `tts-gateway`, for plaintext `grpcurl` testing
from outside the cluster during initial rollout — a live TODO, not a permanent decision, unlike
every other service here which assumes mTLS-or-trusted-network as its only access boundary (see

View File

@ -170,16 +170,18 @@ repo's hand-written-mock testing convention rather than golden-byte comparison.
WAV writer, `ffmpeg` transcode adapter, full `Synthesize` RPC wiring, speaker-not-found →
`INVALID_ARGUMENT` (mirroring `app.py`'s current 400 behavior), `ListSpeakers` RPC.
**Phase 5 — Containerize & deploy to nik-gpu**
See [GPU/deployment notes](#gpu-and-deployment-notes) below — this phase needs a Dockerfile that
diverges from the other three services' pattern, plus a decision on how the model artifact
reaches the container.
**Phase 5 — Containerize & deploy to nik-gpu** — **done.** Deployed and confirmed working in
production; see `handoff.md` for status. Model artifact distribution is resolved as a small
versioned image (`tts-gateway/model/Dockerfile`) copied into a shared volume by a k8s
`initContainer`, built manually on nik-gpu (the checkpoint never existed anywhere CI can reach) —
see `tts-gateway/README.md`'s "Model artifact distribution (production)" section.
**Phase 6 — Client integration (optional, separate follow-up)**
A `/speak` (or similar) Discord command in `discord-bot` calling this gateway, following the
same secondary-adapter + gRPC-client pattern already used for its `ha-gateway`/`ai-gateway`
clients. Not assumed in scope here — a deliberate follow-on decision once the service itself
works.
**Phase 6 — Client integration** — **done.** `discord-bot` registers a `/speak` command calling
this gateway via the same secondary-adapter + gRPC-client pattern used for its
`ha-gateway`/`ai-gateway` clients (`discord-bot/internal/adapters/secondary/ttsgateway/`), then
transcodes the AAC response to Opus locally (`ffmpeg` + `github.com/jonas747/dca`) and streams it
into the invoking user's current voice channel
(`discord-bot/internal/adapters/primary/discord/voice.go`).
## GPU and deployment notes
@ -226,9 +228,10 @@ nik-gpu's host state untouched and the setup reproducible from the Dockerfile al
## Open questions (product/design, not technical blockers)
- **Output codec:** keep AAC (parity with today) or switch to Opus? If the eventual consumer is
`discord-bot`, Opus is Discord's native voice codec — worth deciding once Phase 6 is actually
on the table, not before.
- **Output codec:** ~~keep AAC (parity with today) or switch to Opus?~~ **Resolved: kept AAC.**
`tts-gateway` itself is unchanged; `discord-bot` transcodes AAC → Opus locally for voice
playback (see Phase 6 above), rather than pushing an Opus-specific output format onto every
other current/future consumer of this gateway.
- **Concurrency:** the reference implementation serializes all inference behind one lock. Same
starting point is reasonable here — revisit only if real latency/throughput needs show up.
- **Auth:** consistent with every other service in this repo, no app-layer authorization —

View File

@ -2,6 +2,7 @@ DISCORD_TOKEN=your-bot-token-here
GUILD_ID=your-guild-id-here
HA_GATEWAY_ADDR=localhost:50051
AI_GATEWAY_ADDR=localhost:50052
TTS_GATEWAY_ADDR=localhost:50053
OTEL_ENDPOINT=
LOG_LEVEL=info
LOG_FORMAT=text

View File

@ -27,6 +27,14 @@ RUN CGO_ENABLED=0 GOOS=linux go build \
-ldflags="-s -w -X main.version=${VERSION}" \
-o /discord-bot ./cmd/bot
FROM gcr.io/distroless/static:nonroot
# /speak needs `ffmpeg` at runtime (github.com/jonas747/dca shells out to it to transcode
# tts-gateway's AAC output to Opus for Discord voice), which rules out a distroless base -
# same reasoning as tts-gateway/Dockerfile.
FROM ubuntu:22.04
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /discord-bot /discord-bot
ENTRYPOINT ["/discord-bot"]

View File

@ -14,6 +14,7 @@ import (
discordadapter "gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/primary/discord"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/aigateway"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/gateway"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/ttsgateway"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/app"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/config"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
@ -42,6 +43,7 @@ func main() {
"version", version,
"ha_gateway_addr", cfg.HAGatewayAddr,
"ai_gateway_addr", cfg.AIGatewayAddr,
"tts_gateway_addr", cfg.TTSGatewayAddr,
"discord_token", redactToken(cfg.DiscordToken),
"tls_dir", cfg.TLSDir,
"log_level", cfg.LogLevel,
@ -86,9 +88,20 @@ func main() {
}
}()
ttsClient, err := ttsgateway.New(ctx, cfg.TTSGatewayAddr, cfg.TLSDir, log)
if err != nil {
log.Error("tts-gateway client setup failed", "err", err)
os.Exit(1)
}
defer func() {
if err := ttsClient.Close(); err != nil {
log.Error("tts-gateway client close failed", "err", err)
}
}()
modelStore := modelstore.New()
validator := modelvalidator.New(aiClient, 30*time.Second)
commandApp := app.NewCommandApp(haClient, aiClient, modelStore, validator)
commandApp := app.NewCommandApp(haClient, aiClient, modelStore, validator, ttsClient)
tracker := discordadapter.NewTracker(context.Background())
// Discord-specific wiring stays at the edge so the app layer remains transport-agnostic.
@ -97,7 +110,9 @@ func main() {
log.Error("create discord session failed", "err", err)
os.Exit(1)
}
session.Identify.Intents = discordgo.IntentsGuilds
// GuildVoiceStates is needed to resolve which voice channel the invoking
// user is currently in for /speak (see internal/adapters/primary/discord/voice.go).
session.Identify.Intents = discordgo.IntentsGuilds | discordgo.IntentsGuildVoiceStates
handler := discordadapter.NewHandler(commandApp, tracker)
handler.Register(session)
@ -121,6 +136,7 @@ func main() {
"command_scope", scope,
"ha_gateway_addr", cfg.HAGatewayAddr,
"ai_gateway_addr", cfg.AIGatewayAddr,
"tts_gateway_addr", cfg.TTSGatewayAddr,
"version", version,
)

View File

@ -6,6 +6,7 @@ require (
gitea.nik4nao.com/nik/home-services/gen v0.0.0
github.com/bwmarrin/discordgo v0.29.0
github.com/joho/godotenv v1.5.1
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0
go.opentelemetry.io/otel v1.39.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.35.0
@ -25,6 +26,7 @@ require (
github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.4.2 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 // indirect
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.35.0 // indirect
go.opentelemetry.io/proto/otlp v1.5.0 // indirect

View File

@ -23,6 +23,10 @@ github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 h1:e9Rjr40Z98/clHv5Yg79Is0Ntos
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1/go.mod h1:tIxuGz/9mpox++sgp9fJjHO0+q1X9/UOWd798aAm22M=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7 h1:Iw1pZVDiq4tY8hO7Wt1D1EqDC0BmMw/bFO/Rdt4gs9Q=
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7/go.mod h1:rxjYX9OJU81unMxQDHChU/lAiOhlY9MV+faPX/NmwLk=
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757 h1:Kyv+zTfWIGRNaz/4+lS+CxvuKVZSKFz/6G8E3BKKBRs=
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757/go.mod h1:cZnNmdLiLpihzgIVqiaQppi9Ts3D4qF/M45//yW35nI=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=

View File

@ -38,10 +38,12 @@ type commandHandler interface {
HandleAIModelSet(ctx context.Context, name string) (string, error)
HandleAIModelGet(ctx context.Context) (string, error)
HandleAIModelList(ctx context.Context) (string, error)
HandleSpeak(ctx context.Context, speakerName, text string) (audio []byte, mimeType string, err error)
AutocompleteLights(ctx context.Context) ([]apppkg.Choice, error)
AutocompleteSwitches(ctx context.Context) ([]apppkg.Choice, error)
AutocompleteClimates(ctx context.Context) ([]apppkg.Choice, error)
AutocompleteAIModels(ctx context.Context) ([]apppkg.Choice, error)
AutocompleteSpeakers(ctx context.Context) ([]apppkg.Choice, error)
}
// Handler adapts Discord interactions to the command application layer.
@ -84,7 +86,7 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
data := i.ApplicationCommandData()
command := data.Name
if len(data.Options) > 0 {
if len(data.Options) > 0 && isSubcommandOption(data.Options[0]) {
command += "." + data.Options[0].Name
if len(data.Options[0].Options) > 0 && data.Options[0].Type == discordgo.ApplicationCommandOptionSubCommandGroup {
command += "." + data.Options[0].Options[0].Name
@ -104,6 +106,15 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
),
)
defer span.End()
// speak is a flat command (speaker/text values directly under data.Options, no
// subcommand wrapper) unlike light/switch/ac/ai, so it's dispatched before the
// subcommand-assuming logic below rather than through commandPath's switch.
if data.Name == "speak" {
h.handleSpeak(ctx, s, i, data, start)
return
}
if len(data.Options) == 0 {
h.respondError(ctx, s, i.Interaction, true, start, fmt.Errorf("missing subcommand"))
return
@ -305,6 +316,57 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
}
}
// handleSpeak synthesizes speech via tts-gateway and plays it in the invoking
// user's current voice channel, following the same deferred-ack + background +
// followup shape as the ai.query case (synthesis + voice join + playback will
// exceed Discord's ~3s initial-ack window).
func (h *Handler) handleSpeak(ctx context.Context, s *discordgo.Session, i *discordgo.InteractionCreate, data discordgo.ApplicationCommandInteractionData, start time.Time) {
log := logger.FromContext(ctx)
if err := h.deferResponse(s, i.Interaction, true); err != nil {
log.Error("discord response failed",
"duration_ms", time.Since(start).Milliseconds(),
"error", err.Error(),
)
return
}
speakerName := stringOption(data.Options, "speaker")
text := stringOption(data.Options, "text")
guildID := i.GuildID
userID := ""
if i.Member != nil && i.Member.User != nil {
userID = i.Member.User.ID
} else if i.User != nil {
userID = i.User.ID
}
interaction := i.Interaction
reqLog := log
h.tracker.Go(func(trackerCtx context.Context) {
asyncCtx := logger.WithLogger(trackerCtx, reqLog)
asyncCtx, cancel := context.WithTimeout(asyncCtx, 2*time.Minute)
defer cancel()
audio, mimeType, err := h.app.HandleSpeak(asyncCtx, speakerName, text)
if err == nil {
err = speak(asyncCtx, s, guildID, userID, audio, mimeType)
}
msg := fmt.Sprintf("Spoke as `%s`.", speakerName)
if err != nil {
reqLog.Error("speak command failed", "error", err.Error())
msg = mapSpeakError(err)
}
if _, followErr := s.FollowupMessageCreate(interaction, true, &discordgo.WebhookParams{
Content: msg,
Flags: discordgo.MessageFlagsEphemeral,
}); followErr != nil {
reqLog.Error("discord response failed", "error", followErr.Error())
}
})
log.Info("command handled", "duration_ms", time.Since(start).Milliseconds())
}
// handleAutocomplete keeps autocomplete fast by filtering already-fetched choices
// instead of requiring Discord-specific logic in the app layer.
func (h *Handler) handleAutocomplete(ctx context.Context, s *discordgo.Session, i *discordgo.InteractionCreate) {
@ -326,6 +388,10 @@ func (h *Handler) handleAutocomplete(ctx context.Context, s *discordgo.Session,
if focusedOptionName(data) == "name" {
choices, err = h.app.AutocompleteAIModels(ctx)
}
case "speak":
if focusedOptionName(data) == "speaker" {
choices, err = h.app.AutocompleteSpeakers(ctx)
}
default:
choices = nil
}
@ -419,6 +485,24 @@ func (h *Handler) followup(ctx context.Context, s *discordgo.Session, interactio
}
}
func mapSpeakError(err error) string {
if errors.Is(err, ErrNotInVoiceChannel) {
return "Join a voice channel first."
}
if errors.Is(err, context.DeadlineExceeded) {
return "Speaking took too long. Try again later."
}
if status, ok := grpcstatus.FromError(err); ok {
switch status.Code() {
case codes.InvalidArgument:
return "Unknown speaker. Try /speak again with a different name."
case codes.Unavailable:
return "The TTS service is unreachable right now. Try again in a moment."
}
}
return "Sorry, something went wrong speaking that."
}
func mapAIError(err error) string {
if errors.Is(err, context.DeadlineExceeded) {
return "The AI took too long to respond. Try again later."
@ -448,7 +532,14 @@ func interactionLogger(ctx context.Context, i *discordgo.InteractionCreate) *slo
}
func requiredStringOption(sub *discordgo.ApplicationCommandInteractionDataOption, name string) string {
for _, opt := range sub.Options {
return stringOption(sub.Options, name)
}
// stringOption looks up a named string-valued option in a flat options slice -
// used directly for flat commands like /speak, and via requiredStringOption for
// commands nested one level under a subcommand.
func stringOption(opts []*discordgo.ApplicationCommandInteractionDataOption, name string) string {
for _, opt := range opts {
if opt.Name == name {
return opt.StringValue()
}
@ -456,6 +547,12 @@ func requiredStringOption(sub *discordgo.ApplicationCommandInteractionDataOption
return ""
}
// isSubcommandOption reports whether opt represents a subcommand/subcommand-group
// node, as opposed to a plain value option (e.g. a flat command's own arguments).
func isSubcommandOption(opt *discordgo.ApplicationCommandInteractionDataOption) bool {
return opt.Type == discordgo.ApplicationCommandOptionSubCommand || opt.Type == discordgo.ApplicationCommandOptionSubCommandGroup
}
func optionalUint32Option(sub *discordgo.ApplicationCommandInteractionDataOption, name string) *uint32 {
for _, opt := range sub.Options {
if opt.Name == name {
@ -468,6 +565,9 @@ func optionalUint32Option(sub *discordgo.ApplicationCommandInteractionDataOption
func focusedOptionValue(data discordgo.ApplicationCommandInteractionData) string {
for _, sub := range data.Options {
if sub.Focused {
return sub.StringValue()
}
if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup {
for _, nested := range sub.Options {
for _, opt := range nested.Options {
@ -488,6 +588,9 @@ func focusedOptionValue(data discordgo.ApplicationCommandInteractionData) string
func focusedOptionName(data discordgo.ApplicationCommandInteractionData) string {
for _, sub := range data.Options {
if sub.Focused {
return sub.Name
}
if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup {
for _, nested := range sub.Options {
for _, opt := range nested.Options {

View File

@ -0,0 +1,108 @@
package discord
import (
"testing"
"github.com/bwmarrin/discordgo"
)
func TestStringOption(t *testing.T) {
opts := []*discordgo.ApplicationCommandInteractionDataOption{
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice Shower"},
{Name: "text", Type: discordgo.ApplicationCommandOptionString, Value: "hello"},
}
if got := stringOption(opts, "speaker"); got != "Rice Shower" {
t.Fatalf("stringOption(speaker) = %q, want %q", got, "Rice Shower")
}
if got := stringOption(opts, "missing"); got != "" {
t.Fatalf("stringOption(missing) = %q, want empty", got)
}
}
func TestIsSubcommandOption(t *testing.T) {
tests := []struct {
name string
typ discordgo.ApplicationCommandOptionType
want bool
}{
{name: "subcommand", typ: discordgo.ApplicationCommandOptionSubCommand, want: true},
{name: "subcommand group", typ: discordgo.ApplicationCommandOptionSubCommandGroup, want: true},
{name: "plain string option", typ: discordgo.ApplicationCommandOptionString, want: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := isSubcommandOption(&discordgo.ApplicationCommandInteractionDataOption{Type: tt.typ})
if got != tt.want {
t.Fatalf("isSubcommandOption(%v) = %v, want %v", tt.typ, got, tt.want)
}
})
}
}
func TestFocusedOptionNameAndValue(t *testing.T) {
tests := []struct {
name string
data discordgo.ApplicationCommandInteractionData
wantName string
wantValue string
}{
{
name: "flat command - top-level option focused",
data: discordgo.ApplicationCommandInteractionData{
Name: "speak",
Options: []*discordgo.ApplicationCommandInteractionDataOption{
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice", Focused: true},
{Name: "text", Type: discordgo.ApplicationCommandOptionString, Value: "hi"},
},
},
wantName: "speaker",
wantValue: "Rice",
},
{
name: "nested subcommand option focused (regression)",
data: discordgo.ApplicationCommandInteractionData{
Name: "ai",
Options: []*discordgo.ApplicationCommandInteractionDataOption{
{
Name: "model",
Type: discordgo.ApplicationCommandOptionSubCommandGroup,
Options: []*discordgo.ApplicationCommandInteractionDataOption{
{
Name: "set",
Type: discordgo.ApplicationCommandOptionSubCommand,
Options: []*discordgo.ApplicationCommandInteractionDataOption{
{Name: "name", Type: discordgo.ApplicationCommandOptionString, Value: "llama", Focused: true},
},
},
},
},
},
},
wantName: "name",
wantValue: "llama",
},
{
name: "nothing focused",
data: discordgo.ApplicationCommandInteractionData{
Name: "speak",
Options: []*discordgo.ApplicationCommandInteractionDataOption{
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice"},
},
},
wantName: "",
wantValue: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := focusedOptionName(tt.data); got != tt.wantName {
t.Fatalf("focusedOptionName() = %q, want %q", got, tt.wantName)
}
if got := focusedOptionValue(tt.data); got != tt.wantValue {
t.Fatalf("focusedOptionValue() = %q, want %q", got, tt.wantValue)
}
})
}
}

View File

@ -207,6 +207,25 @@ func RegisterCommands(s *discordgo.Session, guildID string) error {
},
},
},
{
Name: "speak",
Description: "Speak text in your current voice channel",
Options: []*discordgo.ApplicationCommandOption{
{
Type: discordgo.ApplicationCommandOptionString,
Name: "speaker",
Description: "Speaker voice",
Required: true,
Autocomplete: true,
},
{
Type: discordgo.ApplicationCommandOptionString,
Name: "text",
Description: "Text to speak",
Required: true,
},
},
},
}
if _, err := s.ApplicationCommandBulkOverwrite(appID, guildID, commands); err != nil {

View File

@ -0,0 +1,131 @@
package discord
import (
"context"
"errors"
"fmt"
"io"
"os"
"time"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
"github.com/bwmarrin/discordgo"
"github.com/jonas747/dca"
)
// ErrNotInVoiceChannel is returned when the invoking user isn't in a voice channel.
var ErrNotInVoiceChannel = errors.New("join a voice channel first")
// resolveVoiceChannel returns the voice channel the given guild member currently
// occupies, or ErrNotInVoiceChannel if they aren't in one.
func resolveVoiceChannel(s *discordgo.Session, guildID, userID string) (string, error) {
vs, err := s.State.VoiceState(guildID, userID)
if err != nil {
return "", ErrNotInVoiceChannel
}
return vs.ChannelID, nil
}
// speak joins the invoking user's current voice channel, transcodes audio (as
// returned by tts-gateway, e.g. AAC) to Opus via ffmpeg/dca, streams it, then
// leaves. This is Discord-specific playback plumbing that the transport-agnostic
// app layer has no business knowing about - see internal/app/command.go's
// HandleSpeak for the RPC half of this feature.
func speak(ctx context.Context, s *discordgo.Session, guildID, userID string, audio []byte, mimeType string) error {
log := logger.FromContext(ctx).With("mime_type", mimeType)
channelID, err := resolveVoiceChannel(s, guildID, userID)
if err != nil {
return err
}
vc, err := s.ChannelVoiceJoin(guildID, channelID, false, true)
if err != nil {
return fmt.Errorf("join voice channel: %w", err)
}
defer func() {
if err := vc.Disconnect(); err != nil {
log.Error("voice disconnect failed", "error", err.Error())
}
}()
if err := waitForVoiceReady(ctx, vc); err != nil {
return err
}
// dca shells out to ffmpeg, which needs a seekable input for containers like the
// AAC/M4A tts-gateway returns - its own moov atom sits at the end of the file (see
// tts-gateway/internal/adapters/secondary/ffmpeg/ffmpeg.go), so a non-seekable pipe
// input would fail. Write to a temp file first, mirroring that same adapter's idiom
// on the output side.
tmpPath, err := writeTempAudio(audio)
if err != nil {
return err
}
defer os.Remove(tmpPath)
encodeSession, err := dca.EncodeFile(tmpPath, dca.StdEncodeOptions)
if err != nil {
return fmt.Errorf("encode audio to opus: %w", err)
}
defer encodeSession.Cleanup()
if err := vc.Speaking(true); err != nil {
return fmt.Errorf("set speaking state: %w", err)
}
defer func() {
if err := vc.Speaking(false); err != nil {
log.Error("clear speaking state failed", "error", err.Error())
}
}()
done := make(chan error, 1)
dca.NewStream(encodeSession, vc, done)
select {
case err := <-done:
if err != nil && err != io.EOF {
return fmt.Errorf("stream audio: %w", err)
}
return nil
case <-ctx.Done():
return ctx.Err()
}
}
// writeTempAudio persists synthesized audio to a temp file for ffmpeg to read from.
func writeTempAudio(audio []byte) (string, error) {
tmpFile, err := os.CreateTemp("", "discord-bot-speak-*")
if err != nil {
return "", fmt.Errorf("create temp audio file: %w", err)
}
tmpPath := tmpFile.Name()
if _, err := tmpFile.Write(audio); err != nil {
_ = tmpFile.Close()
os.Remove(tmpPath)
return "", fmt.Errorf("write temp audio file: %w", err)
}
if err := tmpFile.Close(); err != nil {
os.Remove(tmpPath)
return "", fmt.Errorf("close temp audio file: %w", err)
}
return tmpPath, nil
}
// waitForVoiceReady polls until the voice connection's UDP socket is ready to
// send, or ctx is done - discordgo has no blocking/channel-based signal for this.
func waitForVoiceReady(ctx context.Context, vc *discordgo.VoiceConnection) error {
ticker := time.NewTicker(50 * time.Millisecond)
defer ticker.Stop()
for {
if vc.Ready {
return nil
}
select {
case <-ticker.C:
case <-ctx.Done():
return fmt.Errorf("wait for voice connection ready: %w", ctx.Err())
}
}
}

View File

@ -0,0 +1,73 @@
package discord
import (
"errors"
"testing"
"github.com/bwmarrin/discordgo"
)
func newTestSession(t *testing.T, guildID string, voiceStates []*discordgo.VoiceState) *discordgo.Session {
t.Helper()
state := discordgo.NewState()
if err := state.GuildAdd(&discordgo.Guild{ID: guildID, VoiceStates: voiceStates}); err != nil {
t.Fatalf("GuildAdd() error = %v", err)
}
return &discordgo.Session{State: state}
}
func TestResolveVoiceChannel(t *testing.T) {
const guildID = "guild-1"
const userID = "user-1"
tests := []struct {
name string
voiceStates []*discordgo.VoiceState
userID string
wantChannel string
wantErr error
}{
{
name: "user in a voice channel",
voiceStates: []*discordgo.VoiceState{
{GuildID: guildID, UserID: userID, ChannelID: "channel-1"},
},
userID: userID,
wantChannel: "channel-1",
},
{
name: "user not in any voice channel",
voiceStates: nil,
userID: userID,
wantErr: ErrNotInVoiceChannel,
},
{
name: "user in a different guild's voice channel is not found",
voiceStates: []*discordgo.VoiceState{
{GuildID: guildID, UserID: "someone-else", ChannelID: "channel-1"},
},
userID: userID,
wantErr: ErrNotInVoiceChannel,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
session := newTestSession(t, guildID, tt.voiceStates)
got, err := resolveVoiceChannel(session, guildID, tt.userID)
if tt.wantErr != nil {
if !errors.Is(err, tt.wantErr) {
t.Fatalf("resolveVoiceChannel() error = %v, want %v", err, tt.wantErr)
}
return
}
if err != nil {
t.Fatalf("resolveVoiceChannel() error = %v", err)
}
if got != tt.wantChannel {
t.Fatalf("resolveVoiceChannel() = %q, want %q", got, tt.wantChannel)
}
})
}
}

View File

@ -0,0 +1,124 @@
package ttsgateway
import (
"context"
"crypto/tls"
"crypto/x509"
"fmt"
"log/slog"
"os"
"path/filepath"
"time"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
ttsv1 "gitea.nik4nao.com/nik/home-services/gen/tts/v1"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/credentials/insecure"
)
// Client implements the app's TTS driven port over gRPC.
type Client struct {
conn *grpc.ClientConn
client ttsv1.TTSServiceClient
log *slog.Logger
}
// New constructs a gRPC client for the internal tts-gateway service.
func New(ctx context.Context, addr, tlsDir string, log *slog.Logger) (*Client, error) {
transportCreds := insecure.NewCredentials()
if tlsDir != "" {
creds, err := loadTransportCredentials(tlsDir)
if err != nil {
return nil, fmt.Errorf("load mTLS credentials: %w", err)
}
transportCreds = creds
}
conn, err := grpc.NewClient(
addr,
grpc.WithTransportCredentials(transportCreds),
grpc.WithStatsHandler(otelgrpc.NewClientHandler()),
)
if err != nil {
return nil, fmt.Errorf("dial tts-gateway: %w", err)
}
return &Client{
conn: conn,
client: ttsv1.NewTTSServiceClient(conn),
log: log,
}, nil
}
// Close closes the underlying gRPC connection.
func (c *Client) Close() error {
if err := c.conn.Close(); err != nil {
return fmt.Errorf("close tts-gateway client: %w", err)
}
return nil
}
// Synthesize forwards a speech synthesis request to tts-gateway. noise_scale/
// noise_scale_w/length_scale are left unset so tts-gateway applies its own defaults.
func (c *Client) Synthesize(ctx context.Context, speakerName, text string) ([]byte, string, error) {
start := time.Now()
log := logger.FromContext(ctx).With("grpc.method", "TTSService/Synthesize")
resp, err := c.client.Synthesize(ctx, &ttsv1.SynthesizeRequest{
SpeakerName: speakerName,
Text: text,
})
if err != nil {
log.Error("grpc call failed",
"duration_ms", time.Since(start).Milliseconds(),
"error", err.Error(),
)
return nil, "", fmt.Errorf("synthesize speech: %w", err)
}
log.Debug("grpc call completed", "duration_ms", time.Since(start).Milliseconds())
return resp.GetAudio(), resp.GetMimeType(), nil
}
// ListSpeakers returns the speaker roster from tts-gateway, optionally filtered by search.
func (c *Client) ListSpeakers(ctx context.Context, search string) ([]string, error) {
start := time.Now()
log := logger.FromContext(ctx).With("grpc.method", "TTSService/ListSpeakers")
resp, err := c.client.ListSpeakers(ctx, &ttsv1.ListSpeakersRequest{Search: search})
if err != nil {
log.Error("grpc call failed",
"duration_ms", time.Since(start).Milliseconds(),
"error", err.Error(),
)
return nil, fmt.Errorf("list tts-gateway speakers: %w", err)
}
log.Debug("grpc call completed", "duration_ms", time.Since(start).Milliseconds())
return append([]string(nil), resp.GetSpeakerNames()...), nil
}
func loadTransportCredentials(tlsDir string) (credentials.TransportCredentials, error) {
cert, err := tls.LoadX509KeyPair(
filepath.Join(tlsDir, "tls.crt"),
filepath.Join(tlsDir, "tls.key"),
)
if err != nil {
return nil, fmt.Errorf("load client key pair: %w", err)
}
caPEM, err := os.ReadFile(filepath.Join(tlsDir, "ca.crt"))
if err != nil {
return nil, fmt.Errorf("read server CA: %w", err)
}
rootCAs := x509.NewCertPool()
if !rootCAs.AppendCertsFromPEM(caPEM) {
return nil, fmt.Errorf("append server CA: invalid PEM")
}
return credentials.NewTLS(&tls.Config{
Certificates: []tls.Certificate{cert},
RootCAs: rootCAs,
ServerName: "tts-gateway.home-services.svc.cluster.local",
MinVersion: tls.VersionTLS13,
}), nil
}

View File

@ -23,11 +23,12 @@ type CommandApp struct {
ai driven.AIGateway
models *modelstore.Store
validator *modelvalidator.Validator
tts driven.TTSGateway
}
// NewCommandApp constructs the Discord command application service.
func NewCommandApp(ha driven.HAGateway, ai driven.AIGateway, models *modelstore.Store, validator *modelvalidator.Validator) *CommandApp {
return &CommandApp{ha: ha, ai: ai, models: models, validator: validator}
func NewCommandApp(ha driven.HAGateway, ai driven.AIGateway, models *modelstore.Store, validator *modelvalidator.Validator, tts driven.TTSGateway) *CommandApp {
return &CommandApp{ha: ha, ai: ai, models: models, validator: validator, tts: tts}
}
// HandleLightList formats discovered lights into a monospace-friendly response.
@ -279,6 +280,33 @@ func (a *CommandApp) HandleAIModelList(ctx context.Context) (string, error) {
return strings.Join(lines, "\n"), nil
}
// HandleSpeak synthesizes speech via tts-gateway for the invoking Discord adapter
// to play back in a voice channel; the app layer stays transport-agnostic and
// leaves voice-channel playback itself to the Discord adapter.
func (a *CommandApp) HandleSpeak(ctx context.Context, speakerName, text string) ([]byte, string, error) {
audio, mimeType, err := a.tts.Synthesize(ctx, speakerName, text)
if err != nil {
return nil, "", fmt.Errorf("handle speak: %w", err)
}
return audio, mimeType, nil
}
// AutocompleteSpeakers returns the full tts-gateway speaker roster for the /speak
// command; handleAutocomplete's existing substring filter narrows it client-side,
// same as the light/switch/ac autocompletes above.
func (a *CommandApp) AutocompleteSpeakers(ctx context.Context) ([]Choice, error) {
speakers, err := a.tts.ListSpeakers(ctx, "")
if err != nil {
return nil, fmt.Errorf("autocomplete speakers: %w", err)
}
choices := make([]Choice, 0, len(speakers))
for _, speaker := range speakers {
choices = append(choices, Choice{Label: speaker, Value: speaker})
}
return choices, nil
}
// AutocompleteAIModels returns model names for the /ai model set command.
func (a *CommandApp) AutocompleteAIModels(ctx context.Context) ([]Choice, error) {
models, err := a.validator.Known(ctx)

View File

@ -35,6 +35,11 @@ type mockAIGateway struct {
listModelsFunc func(ctx context.Context) ([]string, error)
}
type mockTTSGateway struct {
synthesizeFunc func(ctx context.Context, speakerName, text string) ([]byte, string, error)
listSpeakersFunc func(ctx context.Context, search string) ([]string, error)
}
func (m *mockHAGateway) ListLights(ctx context.Context) ([]driven.Light, error) {
if m.listLightsFunc == nil {
return nil, nil
@ -147,8 +152,22 @@ func (m *mockAIGateway) ListModels(ctx context.Context) ([]string, error) {
return m.listModelsFunc(ctx)
}
func (m *mockTTSGateway) Synthesize(ctx context.Context, speakerName, text string) ([]byte, string, error) {
if m.synthesizeFunc == nil {
return nil, "", nil
}
return m.synthesizeFunc(ctx, speakerName, text)
}
func (m *mockTTSGateway) ListSpeakers(ctx context.Context, search string) ([]string, error) {
if m.listSpeakersFunc == nil {
return nil, nil
}
return m.listSpeakersFunc(ctx, search)
}
func newTestCommandApp(ha *mockHAGateway, ai *mockAIGateway) *CommandApp {
return NewCommandApp(ha, ai, modelstore.New(), modelvalidator.New(ai, time.Minute))
return NewCommandApp(ha, ai, modelstore.New(), modelvalidator.New(ai, time.Minute), &mockTTSGateway{})
}
func TestCommandAppHandleLightList(t *testing.T) {
@ -1081,7 +1100,7 @@ func TestCommandAppHandleAIQuery(t *testing.T) {
}
return "Turning on Kitchen.", "llama3:latest", nil
},
}, store, modelvalidator.New(&mockAIGateway{}, time.Minute))
}, store, modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{})
got, err := app.HandleAIQuery(context.Background(), "turn on kitchen")
if err != nil {
@ -1092,6 +1111,116 @@ func TestCommandAppHandleAIQuery(t *testing.T) {
}
}
func TestCommandAppHandleSpeak(t *testing.T) {
tests := []struct {
name string
speakerName string
text string
audio []byte
mimeType string
synthErr error
wantErr string
}{
{
name: "synthesize success",
speakerName: "Rice Shower",
text: "おはようございます",
audio: []byte{0x00, 0x01, 0x02},
mimeType: "audio/aac",
},
{
name: "synthesize error propagated",
speakerName: "Unknown Speaker",
text: "hello",
synthErr: errors.New("boom"),
wantErr: "handle speak: boom",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var gotSpeaker, gotText string
app := NewCommandApp(&mockHAGateway{}, &mockAIGateway{}, modelstore.New(), modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{
synthesizeFunc: func(ctx context.Context, speakerName, text string) ([]byte, string, error) {
gotSpeaker, gotText = speakerName, text
if tt.synthErr != nil {
return nil, "", tt.synthErr
}
return tt.audio, tt.mimeType, nil
},
})
audio, mimeType, err := app.HandleSpeak(context.Background(), tt.speakerName, tt.text)
if tt.wantErr != "" {
if err == nil || err.Error() != tt.wantErr {
t.Fatalf("HandleSpeak() error = %v, want %q", err, tt.wantErr)
}
return
}
if err != nil {
t.Fatalf("HandleSpeak() error = %v", err)
}
if gotSpeaker != tt.speakerName || gotText != tt.text {
t.Fatalf("Synthesize() called with (%q, %q), want (%q, %q)", gotSpeaker, gotText, tt.speakerName, tt.text)
}
if !reflect.DeepEqual(audio, tt.audio) || mimeType != tt.mimeType {
t.Fatalf("HandleSpeak() = (%#v, %q), want (%#v, %q)", audio, mimeType, tt.audio, tt.mimeType)
}
})
}
}
func TestCommandAppAutocompleteSpeakers(t *testing.T) {
tests := []struct {
name string
speakers []string
listErr error
want []Choice
wantErr string
}{
{
name: "maps speaker names to choices",
speakers: []string{"Rice Shower", "Special Week"},
want: []Choice{
{Label: "Rice Shower", Value: "Rice Shower"},
{Label: "Special Week", Value: "Special Week"},
},
},
{
name: "ListSpeakers error",
listErr: errors.New("boom"),
wantErr: "autocomplete speakers: boom",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
app := NewCommandApp(&mockHAGateway{}, &mockAIGateway{}, modelstore.New(), modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{
listSpeakersFunc: func(ctx context.Context, search string) ([]string, error) {
if tt.listErr != nil {
return nil, tt.listErr
}
return tt.speakers, nil
},
})
got, err := app.AutocompleteSpeakers(context.Background())
if tt.wantErr != "" {
if err == nil || err.Error() != tt.wantErr {
t.Fatalf("AutocompleteSpeakers() error = %v, want %q", err, tt.wantErr)
}
return
}
if err != nil {
t.Fatalf("AutocompleteSpeakers() error = %v", err)
}
if !reflect.DeepEqual(got, tt.want) {
t.Fatalf("AutocompleteSpeakers() = %#v, want %#v", got, tt.want)
}
})
}
}
func TestCommandAppHandleAIModelSet(t *testing.T) {
app := newTestCommandApp(&mockHAGateway{}, &mockAIGateway{
listModelsFunc: func(ctx context.Context) ([]string, error) {

View File

@ -11,6 +11,7 @@ type Config struct {
GuildID string
HAGatewayAddr string
AIGatewayAddr string
TTSGatewayAddr string
TLSDir string
OTELEndpoint string
LogLevel string
@ -34,6 +35,7 @@ func Load() (*Config, error) {
GuildID: os.Getenv("GUILD_ID"),
HAGatewayAddr: addr,
AIGatewayAddr: getenvDefault("AI_GATEWAY_ADDR", "ai-gateway.home-services.svc.cluster.local:50052"),
TTSGatewayAddr: getenvDefault("TTS_GATEWAY_ADDR", "tts-gateway.home-services.svc.cluster.local:50053"),
TLSDir: os.Getenv("TLS_DIR"),
OTELEndpoint: os.Getenv("OTEL_ENDPOINT"),
LogLevel: getenvDefault("LOG_LEVEL", "info"),

View File

@ -0,0 +1,9 @@
package driven
import "context"
// TTSGateway exposes the text-to-speech synthesis API used by the Discord bot.
type TTSGateway interface {
Synthesize(ctx context.Context, speakerName, text string) (audio []byte, mimeType string, err error)
ListSpeakers(ctx context.Context, search string) ([]string, error)
}

90
handoff.md Normal file
View File

@ -0,0 +1,90 @@
# tts-gateway Handoff
Status as of 2026-07-25: Phases 06 of `TTS_GATEWAY_PLAN.md` are done, and the deployed service is
confirmed working in production (real `grpcurl` call through `kubectl port-forward`, real GPU
inference, real playable audio). This file is a punch list for picking the remaining work up in a
fresh session — it doesn't re-explain things that are already documented elsewhere; it points to
where.
**Update (later same day):** items 1 and 3 below are now done — model artifact distribution is a
versioned image + k8s `initContainer` (see `tts-gateway/README.md`), and `discord-bot` has a
working `/speak` command (see `TTS_GATEWAY_PLAN.md` Phase 6). Only item 2 (mTLS) remains, left
untouched per an explicit decision to verify these two first. The two "what's left" entries below
are kept as-written for their historical reasoning/decision trail rather than rewritten in place.
## Where things stand
- `tts-gateway` (Go) + `tts-sidecar` (Python/libtorch) are both built, containerized, and running
in Kubernetes (`home-services` namespace).
- CI (`.gitea/workflows/ci.yaml`) builds and pushes both images on every push to `main`, gated by
path-filtered per-service rebuilds and using registry-based Docker layer caching.
- Full history — the Phase 0 ONNX-export/g2p investigation, why there's a Python sidecar instead
of pure Go, the exact `japanese_cleaners` pipeline verification — is in `TTS_GATEWAY_PLAN.md`
and `tmp/reference/uma-tts-api/spike/FINDINGS.md`. Read those before re-deriving anything; the
ONNX export failure in particular took several iterations to characterize correctly and isn't
worth re-investigating from scratch.
## What's left
1. **Model artifact distribution isn't automated.** The k8s Deployment
(`~/repo/homelab/manifests/home-services/tts-gateway.yaml` — separate repo, see below)
`hostPath`-mounts `/data/tts-gateway` on the `nik-gpu` node into the sidecar container.
Someone has to manually place `G_790000.pth` + `uma.json` there before the pod goes Ready — no
init-container download, no PVC, nothing automated. `TTS_GATEWAY_PLAN.md`'s Phase 5 flagged
this as an open decision; it's still open, just worked around. If asked to fix it: options are
a PVC populated by an init-container download step, baking the checkpoint into a private image
layer, or similar — weigh against the checkpoint being ~455MB and not something to casually put
in a git-tracked Dockerfile context.
2. **mTLS is off.** `tts-gateway.yaml` comments out the `TLS_DIR` env var and its volume mount,
for plaintext `grpcurl` testing from outside the cluster during initial rollout — a live TODO,
not a permanent decision. Every other service in this repo assumes internal-network-or-mTLS as
its only access boundary (`CLAUDE.md`'s "Configuration Notes"). Re-enabling: uncomment the env
var + mount in that manifest, and set up a `tts-gateway-tls` secret the same way
`ha-gateway-sealed.yaml`/`ha-gateway-secret.sh` do it for ha-gateway (tts-gateway doesn't
currently have its own `-secret.sh`/`-sealed.yaml` pair — it'll need one, since unlike
ha-gateway/discord-bot it has no other secrets today, so this would be its first).
3. **Phase 6 (discord-bot integration) not started.** A `/speak`-style Discord slash command
calling `tts-gateway`, following the existing pattern — `discord-bot` already has gRPC clients
for `ha-gateway` and `ai-gateway` at `discord-bot/internal/adapters/secondary/{gateway,aigateway}`;
a new `internal/adapters/secondary/ttsgateway` client would follow the same shape. Needs a
product decision first: keep AAC (parity with what's already implemented) or switch to Opus
(Discord's native voice codec) — deliberately deferred in `TTS_GATEWAY_PLAN.md`'s open
questions until this exact moment, not yet decided.
## Things worth knowing before touching this
- **`tts-gateway` and `tts-sidecar` share one Kubernetes Pod**, not two separate Deployments — the
Go gateway reaches the sidecar over `localhost:50054`. This mirrors the `--network host` setup
documented in `tts-gateway/README.md` for local Docker testing. Don't split them into separate
Deployments/Services without also rethinking `INFERENCE_SIDECAR_ADDR` and the networking model.
- **The k8s manifests live in a different repo**: `~/repo/homelab/manifests/home-services/` — not
in this repo at all. `CLAUDE.md` now has an "Infrastructure / Deployment" section documenting
this; check there (and that repo directly) before assuming something isn't deployed just because
this repo has no manifests for it.
- **`open_jtalk`'s tokenization has an intentional bug that must be preserved.** The Go text
normalizer (`tts-gateway/internal/adapters/secondary/jtalk`) maps the cleaned phoneme string to
symbol IDs character-by-character, not phoneme-by-phoneme, because that's what the checkpoint
was actually trained on (`text/symbols.py`'s "wrong tokens" comment in the original reference
implementation). Don't "fix" this without retraining the model — see the comments in `jtalk.go`
and `tts-gateway/README.md`.
- **ONNX export doesn't work for this checkpoint, and it's not a quick fix.** Read
`tmp/reference/uma-tts-api/spike/FINDINGS.md` before re-attempting it — the model's
data-dependent output length (predicted phoneme durations determine audio length at runtime)
defeats `torch.export`'s guard system in a way that would need an unknown number of per-layer
`torch._check` hints to resolve, not a single targeted change.
- **CI's build cache needs one "priming" run per image.** If a build job still looks slow after
the caching change in `ci.yaml`, check whether a `:buildcache` tag actually exists yet in the
registry for that specific image — the first run after the cache was added has nothing to pull
from and builds fully fresh.
## Where to look for more detail
- `TTS_GATEWAY_PLAN.md` (repo root) — original plan, feasibility analysis, phase breakdown, open
product questions (codec, concurrency, auth).
- `tmp/reference/uma-tts-api/spike/FINDINGS.md` — Phase 0 spike results with full reasoning.
- `tts-gateway/README.md` — service-level docs: gRPC API, config reference, how to run/test
locally and on nik-gpu.
- `~/repo/homelab/manifests/home-services/tts-gateway.yaml` — actual production config (separate
repo, maintained independently — check it directly rather than trusting a stale summary).

View File

@ -67,8 +67,33 @@ The server also registers gRPC health checks and reflection.
| `PORT` | `50054` | HTTP listen port |
Neither the checkpoint nor the hparams file is committed to git - both must
be supplied at runtime (see below). Model artifact distribution (bind mount
vs. build-time download vs. registry) is still an open decision.
be supplied at runtime (see below). In production, they're baked into a
small versioned image (`tts-gateway/model/Dockerfile`) that a k8s
`initContainer` copies into a shared volume before `tts-sidecar` starts -
see "Model artifact distribution (production)" below. For local/manual
testing on nik-gpu, a plain bind mount (as shown below) is simpler.
### Model artifact distribution (production)
CI can never build the model image - the checkpoint (~455MB) and hparams
aren't committed to git, and nik-gpu's local disk at `/data/tts-gateway` is
the only place they exist. Build and push it manually, directly on nik-gpu
(not via `docker --context nik-gpu` from elsewhere - the build context is
gathered client-side, so it would look for `/data/tts-gateway` on the wrong
machine), whenever the checkpoint or config change:
```bash
ssh nik-gpu
docker build -f ~/repo/home-service/tts-gateway/model/Dockerfile \
-t gitea.nik4nao.com/nik/tts-model:g790000-v1 /data/tts-gateway
docker push gitea.nik4nao.com/nik/tts-model:g790000-v1
```
Bump the tag (`v2`, `v3`, ...) rather than overwriting one - non-`:latest`
tags default to `imagePullPolicy: IfNotPresent`, so a node that already
pulled a tag won't re-pull an overwritten one. The k8s Deployment
(`~/repo/homelab/manifests/home-services/tts-gateway.yaml`) references the
tag explicitly in its `model-init` init container.
## Running And Testing On nik-gpu
@ -188,8 +213,8 @@ sidecar/ # Python/libtorch inference servic
- Concurrency is unbounded on the Go side, but the sidecar's model forward
pass is inherently single-threaded per GPU; heavy concurrent load will
just queue at the sidecar.
- Model artifact distribution isn't finalized - the sidecar currently
expects a bind-mounted checkpoint/config, not something the image ships
with.
- No Discord-bot integration yet (a `/speak`-style command calling this
service is a deliberate follow-up, not yet built).
- Model artifact distribution to production is a manually-built-and-pushed
versioned image (see above), not something CI can automate - the
checkpoint only ever existed on nik-gpu's local disk.
- `discord-bot`'s `/speak` command calls this service and plays the result
in a Discord voice channel (`discord-bot/internal/adapters/primary/discord/voice.go`).

View File

@ -0,0 +1,7 @@
# One-time (or only-when-the-checkpoint-changes) manually built and pushed by a human on
# nik-gpu - CI can never build this: the checkpoint and hparams aren't committed to git
# (~455MB, and nik-gpu's local disk at /data/tts-gateway is the only place they exist).
# Build CONTEXT must be that directory, not a checkout of this repo - see
# tts-gateway/README.md for the exact commands.
FROM busybox:1.36
COPY G_790000.pth uma.json /models/