feat: add /speak command for TTS integration
All checks were successful
CI / changes (push) Successful in 1s
CI / test (push) Successful in 6s
CI / build-ai-gateway (push) Has been skipped
CI / build-ha-gateway (push) Has been skipped
CI / build-discord-bot (push) Successful in 1m33s
CI / build-tts-gateway (push) Successful in 37s
CI / build-tts-sidecar (push) Has been skipped

- Implemented the /speak command in Discord bot to synthesize speech using the TTS gateway.
- Added voice handling logic to join voice channels and play synthesized audio.
- Created tests for the new command and voice functionalities.
- Introduced TTSGateway interface for TTS service communication.
- Updated configuration to include TTS gateway address.
- Documented the TTS gateway integration and model artifact distribution process.
This commit is contained in:
Nik Afiq 2026-07-25 02:09:02 +09:00
parent 7851c49dba
commit 0d58e46740
20 changed files with 937 additions and 50 deletions

View File

@ -17,12 +17,14 @@ ai-gateway ------> Ollama
| |
v v
ha-gateway ha-gateway
discord-bot -----> tts-gateway -----> tts-sidecar (GPU inference)
``` ```
- **ha-gateway** (port `50051`) — gRPC boundary for Home Assistant. Talks to HA's REST API; implements entity state, light control/discovery, switch control/discovery, and climate (HVAC) control/discovery; also relays SwitchBot Cloud remote commands (`RemoteApp`) when `SWITCHBOT_TOKEN`/`SWITCHBOT_SECRET` are set. Event streaming is stubbed. - **ha-gateway** (port `50051`) — gRPC boundary for Home Assistant. Talks to HA's REST API; implements entity state, light control/discovery, switch control/discovery, and climate (HVAC) control/discovery; also relays SwitchBot Cloud remote commands (`RemoteApp`) when `SWITCHBOT_TOKEN`/`SWITCHBOT_SECRET` are set. Event streaming is stubbed.
- **ai-gateway** (port `50052`) — gRPC service that turns free-form text into home actions. Calls Ollama for intent extraction, resolves intents against a cached light list from `ha-gateway`, and calls `ha-gateway` to execute approved actions. - **ai-gateway** (port `50052`) — gRPC service that turns free-form text into home actions. Calls Ollama for intent extraction, resolves intents against a cached light list from `ha-gateway`, and calls `ha-gateway` to execute approved actions.
- **discord-bot** — registers `/light`, `/switch`, `/ac`, `/ai` slash commands and calls `ha-gateway`/`ai-gateway` via gRPC clients. - **discord-bot** — registers `/light`, `/switch`, `/ac`, `/ai`, `/speak` slash commands and calls `ha-gateway`/`ai-gateway`/`tts-gateway` via gRPC clients. `/speak` synthesizes speech via `tts-gateway` (which returns AAC) and transcodes it to Opus locally (via `ffmpeg`/`github.com/jonas747/dca`) to stream into the invoking user's current voice channel — see `internal/adapters/primary/discord/voice.go`.
- **tts-gateway** (port `50053`) — gRPC text-to-speech service (VITS voice model, 92 Umamusume voices), paired with a Python/libtorch inference sidecar (`tts-gateway/sidecar/`, deployed as a second container in the same pod) that needs an Nvidia GPU — see `TTS_GATEWAY_PLAN.md` for why. Not yet wired into the diagram above; nothing calls it yet (a discord-bot `/speak` command is the planned follow-up). See `tts-gateway/README.md` for its API/config/local-run details. - **tts-gateway** (port `50053`) — gRPC text-to-speech service (VITS voice model, 92 Umamusume voices), paired with a Python/libtorch inference sidecar (`tts-gateway/sidecar/`, deployed as a second container in the same pod) that needs an Nvidia GPU — see `TTS_GATEWAY_PLAN.md` for why. Called by `discord-bot`'s `/speak` command. See `tts-gateway/README.md` for its API/config/local-run details.
Each service is a separate Go module (own `go.mod`) joined by `go.work` at the root, plus a `gen` module for shared generated code. Module paths are `gitea.nik4nao.com/nik/home-services/{ha-gateway,ai-gateway,discord-bot,tts-gateway,gen}`. Each service is a separate Go module (own `go.mod`) joined by `go.work` at the root, plus a `gen` module for shared generated code. Module paths are `gitea.nik4nao.com/nik/home-services/{ha-gateway,ai-gateway,discord-bot,tts-gateway,gen}`.
@ -172,10 +174,13 @@ rather than trusting this description to stay accurate):
`nvidia.com/gpu: 1` request/limit on the sidecar container. `strategy: Recreate` instead of the `nvidia.com/gpu: 1` request/limit on the sidecar container. `strategy: Recreate` instead of the
default `RollingUpdate` — nik-gpu only has one allocatable GPU, so a rolling update would default `RollingUpdate` — nik-gpu only has one allocatable GPU, so a rolling update would
deadlock waiting for a GPU still held by the pod it's replacing. deadlock waiting for a GPU still held by the pod it's replacing.
- Model artifact distribution (checkpoint + hparams) is a `hostPath` volume at - Model artifact distribution (checkpoint + hparams) is an `emptyDir` volume populated at pod
`/data/tts-gateway` on the nik-gpu node, not baked into the image or automated — someone has to start by an `initContainers` entry (`model-init`) that copies from a small, versioned image
manually place `G_790000.pth`/`uma.json` there. This is a known-open gap (see `handoff.md`), (`gitea.nik4nao.com/nik/tts-model:<tag>`, built from `tts-gateway/model/Dockerfile`) — not a
not a deliberate final design. raw `hostPath` into the node's disk anymore. That model image still can't be built by CI (the
checkpoint isn't committed to git, ~455MB, and only ever existed on nik-gpu's local disk) — it's
built and pushed manually, on nik-gpu, whenever the checkpoint/config change. See
`tts-gateway/README.md` for the exact commands.
- mTLS (`TLS_DIR`) is currently commented out on `tts-gateway`, for plaintext `grpcurl` testing - mTLS (`TLS_DIR`) is currently commented out on `tts-gateway`, for plaintext `grpcurl` testing
from outside the cluster during initial rollout — a live TODO, not a permanent decision, unlike from outside the cluster during initial rollout — a live TODO, not a permanent decision, unlike
every other service here which assumes mTLS-or-trusted-network as its only access boundary (see every other service here which assumes mTLS-or-trusted-network as its only access boundary (see

View File

@ -170,16 +170,18 @@ repo's hand-written-mock testing convention rather than golden-byte comparison.
WAV writer, `ffmpeg` transcode adapter, full `Synthesize` RPC wiring, speaker-not-found → WAV writer, `ffmpeg` transcode adapter, full `Synthesize` RPC wiring, speaker-not-found →
`INVALID_ARGUMENT` (mirroring `app.py`'s current 400 behavior), `ListSpeakers` RPC. `INVALID_ARGUMENT` (mirroring `app.py`'s current 400 behavior), `ListSpeakers` RPC.
**Phase 5 — Containerize & deploy to nik-gpu** **Phase 5 — Containerize & deploy to nik-gpu** — **done.** Deployed and confirmed working in
See [GPU/deployment notes](#gpu-and-deployment-notes) below — this phase needs a Dockerfile that production; see `handoff.md` for status. Model artifact distribution is resolved as a small
diverges from the other three services' pattern, plus a decision on how the model artifact versioned image (`tts-gateway/model/Dockerfile`) copied into a shared volume by a k8s
reaches the container. `initContainer`, built manually on nik-gpu (the checkpoint never existed anywhere CI can reach) —
see `tts-gateway/README.md`'s "Model artifact distribution (production)" section.
**Phase 6 — Client integration (optional, separate follow-up)** **Phase 6 — Client integration** — **done.** `discord-bot` registers a `/speak` command calling
A `/speak` (or similar) Discord command in `discord-bot` calling this gateway, following the this gateway via the same secondary-adapter + gRPC-client pattern used for its
same secondary-adapter + gRPC-client pattern already used for its `ha-gateway`/`ai-gateway` `ha-gateway`/`ai-gateway` clients (`discord-bot/internal/adapters/secondary/ttsgateway/`), then
clients. Not assumed in scope here — a deliberate follow-on decision once the service itself transcodes the AAC response to Opus locally (`ffmpeg` + `github.com/jonas747/dca`) and streams it
works. into the invoking user's current voice channel
(`discord-bot/internal/adapters/primary/discord/voice.go`).
## GPU and deployment notes ## GPU and deployment notes
@ -226,9 +228,10 @@ nik-gpu's host state untouched and the setup reproducible from the Dockerfile al
## Open questions (product/design, not technical blockers) ## Open questions (product/design, not technical blockers)
- **Output codec:** keep AAC (parity with today) or switch to Opus? If the eventual consumer is - **Output codec:** ~~keep AAC (parity with today) or switch to Opus?~~ **Resolved: kept AAC.**
`discord-bot`, Opus is Discord's native voice codec — worth deciding once Phase 6 is actually `tts-gateway` itself is unchanged; `discord-bot` transcodes AAC → Opus locally for voice
on the table, not before. playback (see Phase 6 above), rather than pushing an Opus-specific output format onto every
other current/future consumer of this gateway.
- **Concurrency:** the reference implementation serializes all inference behind one lock. Same - **Concurrency:** the reference implementation serializes all inference behind one lock. Same
starting point is reasonable here — revisit only if real latency/throughput needs show up. starting point is reasonable here — revisit only if real latency/throughput needs show up.
- **Auth:** consistent with every other service in this repo, no app-layer authorization — - **Auth:** consistent with every other service in this repo, no app-layer authorization —

View File

@ -2,6 +2,7 @@ DISCORD_TOKEN=your-bot-token-here
GUILD_ID=your-guild-id-here GUILD_ID=your-guild-id-here
HA_GATEWAY_ADDR=localhost:50051 HA_GATEWAY_ADDR=localhost:50051
AI_GATEWAY_ADDR=localhost:50052 AI_GATEWAY_ADDR=localhost:50052
TTS_GATEWAY_ADDR=localhost:50053
OTEL_ENDPOINT= OTEL_ENDPOINT=
LOG_LEVEL=info LOG_LEVEL=info
LOG_FORMAT=text LOG_FORMAT=text

View File

@ -27,6 +27,14 @@ RUN CGO_ENABLED=0 GOOS=linux go build \
-ldflags="-s -w -X main.version=${VERSION}" \ -ldflags="-s -w -X main.version=${VERSION}" \
-o /discord-bot ./cmd/bot -o /discord-bot ./cmd/bot
FROM gcr.io/distroless/static:nonroot # /speak needs `ffmpeg` at runtime (github.com/jonas747/dca shells out to it to transcode
# tts-gateway's AAC output to Opus for Discord voice), which rules out a distroless base -
# same reasoning as tts-gateway/Dockerfile.
FROM ubuntu:22.04
RUN apt-get update && apt-get install -y --no-install-recommends \
ffmpeg \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
COPY --from=builder /discord-bot /discord-bot COPY --from=builder /discord-bot /discord-bot
ENTRYPOINT ["/discord-bot"] ENTRYPOINT ["/discord-bot"]

View File

@ -14,6 +14,7 @@ import (
discordadapter "gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/primary/discord" discordadapter "gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/primary/discord"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/aigateway" "gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/aigateway"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/gateway" "gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/gateway"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/ttsgateway"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/app" "gitea.nik4nao.com/nik/home-services/discord-bot/internal/app"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/config" "gitea.nik4nao.com/nik/home-services/discord-bot/internal/config"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger" "gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
@ -42,6 +43,7 @@ func main() {
"version", version, "version", version,
"ha_gateway_addr", cfg.HAGatewayAddr, "ha_gateway_addr", cfg.HAGatewayAddr,
"ai_gateway_addr", cfg.AIGatewayAddr, "ai_gateway_addr", cfg.AIGatewayAddr,
"tts_gateway_addr", cfg.TTSGatewayAddr,
"discord_token", redactToken(cfg.DiscordToken), "discord_token", redactToken(cfg.DiscordToken),
"tls_dir", cfg.TLSDir, "tls_dir", cfg.TLSDir,
"log_level", cfg.LogLevel, "log_level", cfg.LogLevel,
@ -86,9 +88,20 @@ func main() {
} }
}() }()
ttsClient, err := ttsgateway.New(ctx, cfg.TTSGatewayAddr, cfg.TLSDir, log)
if err != nil {
log.Error("tts-gateway client setup failed", "err", err)
os.Exit(1)
}
defer func() {
if err := ttsClient.Close(); err != nil {
log.Error("tts-gateway client close failed", "err", err)
}
}()
modelStore := modelstore.New() modelStore := modelstore.New()
validator := modelvalidator.New(aiClient, 30*time.Second) validator := modelvalidator.New(aiClient, 30*time.Second)
commandApp := app.NewCommandApp(haClient, aiClient, modelStore, validator) commandApp := app.NewCommandApp(haClient, aiClient, modelStore, validator, ttsClient)
tracker := discordadapter.NewTracker(context.Background()) tracker := discordadapter.NewTracker(context.Background())
// Discord-specific wiring stays at the edge so the app layer remains transport-agnostic. // Discord-specific wiring stays at the edge so the app layer remains transport-agnostic.
@ -97,7 +110,9 @@ func main() {
log.Error("create discord session failed", "err", err) log.Error("create discord session failed", "err", err)
os.Exit(1) os.Exit(1)
} }
session.Identify.Intents = discordgo.IntentsGuilds // GuildVoiceStates is needed to resolve which voice channel the invoking
// user is currently in for /speak (see internal/adapters/primary/discord/voice.go).
session.Identify.Intents = discordgo.IntentsGuilds | discordgo.IntentsGuildVoiceStates
handler := discordadapter.NewHandler(commandApp, tracker) handler := discordadapter.NewHandler(commandApp, tracker)
handler.Register(session) handler.Register(session)
@ -121,6 +136,7 @@ func main() {
"command_scope", scope, "command_scope", scope,
"ha_gateway_addr", cfg.HAGatewayAddr, "ha_gateway_addr", cfg.HAGatewayAddr,
"ai_gateway_addr", cfg.AIGatewayAddr, "ai_gateway_addr", cfg.AIGatewayAddr,
"tts_gateway_addr", cfg.TTSGatewayAddr,
"version", version, "version", version,
) )

View File

@ -6,6 +6,7 @@ require (
gitea.nik4nao.com/nik/home-services/gen v0.0.0 gitea.nik4nao.com/nik/home-services/gen v0.0.0
github.com/bwmarrin/discordgo v0.29.0 github.com/bwmarrin/discordgo v0.29.0
github.com/joho/godotenv v1.5.1 github.com/joho/godotenv v1.5.1
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0 go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0
go.opentelemetry.io/otel v1.39.0 go.opentelemetry.io/otel v1.39.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.35.0 go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.35.0
@ -25,6 +26,7 @@ require (
github.com/google/uuid v1.6.0 // indirect github.com/google/uuid v1.6.0 // indirect
github.com/gorilla/websocket v1.4.2 // indirect github.com/gorilla/websocket v1.4.2 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 // indirect github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 // indirect
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.35.0 // indirect go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.35.0 // indirect
go.opentelemetry.io/proto/otlp v1.5.0 // indirect go.opentelemetry.io/proto/otlp v1.5.0 // indirect

View File

@ -23,6 +23,10 @@ github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 h1:e9Rjr40Z98/clHv5Yg79Is0Ntos
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1/go.mod h1:tIxuGz/9mpox++sgp9fJjHO0+q1X9/UOWd798aAm22M= github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1/go.mod h1:tIxuGz/9mpox++sgp9fJjHO0+q1X9/UOWd798aAm22M=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0= github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4= github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7 h1:Iw1pZVDiq4tY8hO7Wt1D1EqDC0BmMw/bFO/Rdt4gs9Q=
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7/go.mod h1:rxjYX9OJU81unMxQDHChU/lAiOhlY9MV+faPX/NmwLk=
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757 h1:Kyv+zTfWIGRNaz/4+lS+CxvuKVZSKFz/6G8E3BKKBRs=
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757/go.mod h1:cZnNmdLiLpihzgIVqiaQppi9Ts3D4qF/M45//yW35nI=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=

View File

@ -38,10 +38,12 @@ type commandHandler interface {
HandleAIModelSet(ctx context.Context, name string) (string, error) HandleAIModelSet(ctx context.Context, name string) (string, error)
HandleAIModelGet(ctx context.Context) (string, error) HandleAIModelGet(ctx context.Context) (string, error)
HandleAIModelList(ctx context.Context) (string, error) HandleAIModelList(ctx context.Context) (string, error)
HandleSpeak(ctx context.Context, speakerName, text string) (audio []byte, mimeType string, err error)
AutocompleteLights(ctx context.Context) ([]apppkg.Choice, error) AutocompleteLights(ctx context.Context) ([]apppkg.Choice, error)
AutocompleteSwitches(ctx context.Context) ([]apppkg.Choice, error) AutocompleteSwitches(ctx context.Context) ([]apppkg.Choice, error)
AutocompleteClimates(ctx context.Context) ([]apppkg.Choice, error) AutocompleteClimates(ctx context.Context) ([]apppkg.Choice, error)
AutocompleteAIModels(ctx context.Context) ([]apppkg.Choice, error) AutocompleteAIModels(ctx context.Context) ([]apppkg.Choice, error)
AutocompleteSpeakers(ctx context.Context) ([]apppkg.Choice, error)
} }
// Handler adapts Discord interactions to the command application layer. // Handler adapts Discord interactions to the command application layer.
@ -84,7 +86,7 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
data := i.ApplicationCommandData() data := i.ApplicationCommandData()
command := data.Name command := data.Name
if len(data.Options) > 0 { if len(data.Options) > 0 && isSubcommandOption(data.Options[0]) {
command += "." + data.Options[0].Name command += "." + data.Options[0].Name
if len(data.Options[0].Options) > 0 && data.Options[0].Type == discordgo.ApplicationCommandOptionSubCommandGroup { if len(data.Options[0].Options) > 0 && data.Options[0].Type == discordgo.ApplicationCommandOptionSubCommandGroup {
command += "." + data.Options[0].Options[0].Name command += "." + data.Options[0].Options[0].Name
@ -104,6 +106,15 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
), ),
) )
defer span.End() defer span.End()
// speak is a flat command (speaker/text values directly under data.Options, no
// subcommand wrapper) unlike light/switch/ac/ai, so it's dispatched before the
// subcommand-assuming logic below rather than through commandPath's switch.
if data.Name == "speak" {
h.handleSpeak(ctx, s, i, data, start)
return
}
if len(data.Options) == 0 { if len(data.Options) == 0 {
h.respondError(ctx, s, i.Interaction, true, start, fmt.Errorf("missing subcommand")) h.respondError(ctx, s, i.Interaction, true, start, fmt.Errorf("missing subcommand"))
return return
@ -305,6 +316,57 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
} }
} }
// handleSpeak synthesizes speech via tts-gateway and plays it in the invoking
// user's current voice channel, following the same deferred-ack + background +
// followup shape as the ai.query case (synthesis + voice join + playback will
// exceed Discord's ~3s initial-ack window).
func (h *Handler) handleSpeak(ctx context.Context, s *discordgo.Session, i *discordgo.InteractionCreate, data discordgo.ApplicationCommandInteractionData, start time.Time) {
log := logger.FromContext(ctx)
if err := h.deferResponse(s, i.Interaction, true); err != nil {
log.Error("discord response failed",
"duration_ms", time.Since(start).Milliseconds(),
"error", err.Error(),
)
return
}
speakerName := stringOption(data.Options, "speaker")
text := stringOption(data.Options, "text")
guildID := i.GuildID
userID := ""
if i.Member != nil && i.Member.User != nil {
userID = i.Member.User.ID
} else if i.User != nil {
userID = i.User.ID
}
interaction := i.Interaction
reqLog := log
h.tracker.Go(func(trackerCtx context.Context) {
asyncCtx := logger.WithLogger(trackerCtx, reqLog)
asyncCtx, cancel := context.WithTimeout(asyncCtx, 2*time.Minute)
defer cancel()
audio, mimeType, err := h.app.HandleSpeak(asyncCtx, speakerName, text)
if err == nil {
err = speak(asyncCtx, s, guildID, userID, audio, mimeType)
}
msg := fmt.Sprintf("Spoke as `%s`.", speakerName)
if err != nil {
reqLog.Error("speak command failed", "error", err.Error())
msg = mapSpeakError(err)
}
if _, followErr := s.FollowupMessageCreate(interaction, true, &discordgo.WebhookParams{
Content: msg,
Flags: discordgo.MessageFlagsEphemeral,
}); followErr != nil {
reqLog.Error("discord response failed", "error", followErr.Error())
}
})
log.Info("command handled", "duration_ms", time.Since(start).Milliseconds())
}
// handleAutocomplete keeps autocomplete fast by filtering already-fetched choices // handleAutocomplete keeps autocomplete fast by filtering already-fetched choices
// instead of requiring Discord-specific logic in the app layer. // instead of requiring Discord-specific logic in the app layer.
func (h *Handler) handleAutocomplete(ctx context.Context, s *discordgo.Session, i *discordgo.InteractionCreate) { func (h *Handler) handleAutocomplete(ctx context.Context, s *discordgo.Session, i *discordgo.InteractionCreate) {
@ -326,6 +388,10 @@ func (h *Handler) handleAutocomplete(ctx context.Context, s *discordgo.Session,
if focusedOptionName(data) == "name" { if focusedOptionName(data) == "name" {
choices, err = h.app.AutocompleteAIModels(ctx) choices, err = h.app.AutocompleteAIModels(ctx)
} }
case "speak":
if focusedOptionName(data) == "speaker" {
choices, err = h.app.AutocompleteSpeakers(ctx)
}
default: default:
choices = nil choices = nil
} }
@ -419,6 +485,24 @@ func (h *Handler) followup(ctx context.Context, s *discordgo.Session, interactio
} }
} }
func mapSpeakError(err error) string {
if errors.Is(err, ErrNotInVoiceChannel) {
return "Join a voice channel first."
}
if errors.Is(err, context.DeadlineExceeded) {
return "Speaking took too long. Try again later."
}
if status, ok := grpcstatus.FromError(err); ok {
switch status.Code() {
case codes.InvalidArgument:
return "Unknown speaker. Try /speak again with a different name."
case codes.Unavailable:
return "The TTS service is unreachable right now. Try again in a moment."
}
}
return "Sorry, something went wrong speaking that."
}
func mapAIError(err error) string { func mapAIError(err error) string {
if errors.Is(err, context.DeadlineExceeded) { if errors.Is(err, context.DeadlineExceeded) {
return "The AI took too long to respond. Try again later." return "The AI took too long to respond. Try again later."
@ -448,7 +532,14 @@ func interactionLogger(ctx context.Context, i *discordgo.InteractionCreate) *slo
} }
func requiredStringOption(sub *discordgo.ApplicationCommandInteractionDataOption, name string) string { func requiredStringOption(sub *discordgo.ApplicationCommandInteractionDataOption, name string) string {
for _, opt := range sub.Options { return stringOption(sub.Options, name)
}
// stringOption looks up a named string-valued option in a flat options slice -
// used directly for flat commands like /speak, and via requiredStringOption for
// commands nested one level under a subcommand.
func stringOption(opts []*discordgo.ApplicationCommandInteractionDataOption, name string) string {
for _, opt := range opts {
if opt.Name == name { if opt.Name == name {
return opt.StringValue() return opt.StringValue()
} }
@ -456,6 +547,12 @@ func requiredStringOption(sub *discordgo.ApplicationCommandInteractionDataOption
return "" return ""
} }
// isSubcommandOption reports whether opt represents a subcommand/subcommand-group
// node, as opposed to a plain value option (e.g. a flat command's own arguments).
func isSubcommandOption(opt *discordgo.ApplicationCommandInteractionDataOption) bool {
return opt.Type == discordgo.ApplicationCommandOptionSubCommand || opt.Type == discordgo.ApplicationCommandOptionSubCommandGroup
}
func optionalUint32Option(sub *discordgo.ApplicationCommandInteractionDataOption, name string) *uint32 { func optionalUint32Option(sub *discordgo.ApplicationCommandInteractionDataOption, name string) *uint32 {
for _, opt := range sub.Options { for _, opt := range sub.Options {
if opt.Name == name { if opt.Name == name {
@ -468,6 +565,9 @@ func optionalUint32Option(sub *discordgo.ApplicationCommandInteractionDataOption
func focusedOptionValue(data discordgo.ApplicationCommandInteractionData) string { func focusedOptionValue(data discordgo.ApplicationCommandInteractionData) string {
for _, sub := range data.Options { for _, sub := range data.Options {
if sub.Focused {
return sub.StringValue()
}
if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup { if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup {
for _, nested := range sub.Options { for _, nested := range sub.Options {
for _, opt := range nested.Options { for _, opt := range nested.Options {
@ -488,6 +588,9 @@ func focusedOptionValue(data discordgo.ApplicationCommandInteractionData) string
func focusedOptionName(data discordgo.ApplicationCommandInteractionData) string { func focusedOptionName(data discordgo.ApplicationCommandInteractionData) string {
for _, sub := range data.Options { for _, sub := range data.Options {
if sub.Focused {
return sub.Name
}
if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup { if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup {
for _, nested := range sub.Options { for _, nested := range sub.Options {
for _, opt := range nested.Options { for _, opt := range nested.Options {

View File

@ -0,0 +1,108 @@
package discord
import (
"testing"
"github.com/bwmarrin/discordgo"
)
func TestStringOption(t *testing.T) {
opts := []*discordgo.ApplicationCommandInteractionDataOption{
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice Shower"},
{Name: "text", Type: discordgo.ApplicationCommandOptionString, Value: "hello"},
}
if got := stringOption(opts, "speaker"); got != "Rice Shower" {
t.Fatalf("stringOption(speaker) = %q, want %q", got, "Rice Shower")
}
if got := stringOption(opts, "missing"); got != "" {
t.Fatalf("stringOption(missing) = %q, want empty", got)
}
}
func TestIsSubcommandOption(t *testing.T) {
tests := []struct {
name string
typ discordgo.ApplicationCommandOptionType
want bool
}{
{name: "subcommand", typ: discordgo.ApplicationCommandOptionSubCommand, want: true},
{name: "subcommand group", typ: discordgo.ApplicationCommandOptionSubCommandGroup, want: true},
{name: "plain string option", typ: discordgo.ApplicationCommandOptionString, want: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := isSubcommandOption(&discordgo.ApplicationCommandInteractionDataOption{Type: tt.typ})
if got != tt.want {
t.Fatalf("isSubcommandOption(%v) = %v, want %v", tt.typ, got, tt.want)
}
})
}
}
func TestFocusedOptionNameAndValue(t *testing.T) {
tests := []struct {
name string
data discordgo.ApplicationCommandInteractionData
wantName string
wantValue string
}{
{
name: "flat command - top-level option focused",
data: discordgo.ApplicationCommandInteractionData{
Name: "speak",
Options: []*discordgo.ApplicationCommandInteractionDataOption{
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice", Focused: true},
{Name: "text", Type: discordgo.ApplicationCommandOptionString, Value: "hi"},
},
},
wantName: "speaker",
wantValue: "Rice",
},
{
name: "nested subcommand option focused (regression)",
data: discordgo.ApplicationCommandInteractionData{
Name: "ai",
Options: []*discordgo.ApplicationCommandInteractionDataOption{
{
Name: "model",
Type: discordgo.ApplicationCommandOptionSubCommandGroup,
Options: []*discordgo.ApplicationCommandInteractionDataOption{
{
Name: "set",
Type: discordgo.ApplicationCommandOptionSubCommand,
Options: []*discordgo.ApplicationCommandInteractionDataOption{
{Name: "name", Type: discordgo.ApplicationCommandOptionString, Value: "llama", Focused: true},
},
},
},
},
},
},
wantName: "name",
wantValue: "llama",
},
{
name: "nothing focused",
data: discordgo.ApplicationCommandInteractionData{
Name: "speak",
Options: []*discordgo.ApplicationCommandInteractionDataOption{
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice"},
},
},
wantName: "",
wantValue: "",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := focusedOptionName(tt.data); got != tt.wantName {
t.Fatalf("focusedOptionName() = %q, want %q", got, tt.wantName)
}
if got := focusedOptionValue(tt.data); got != tt.wantValue {
t.Fatalf("focusedOptionValue() = %q, want %q", got, tt.wantValue)
}
})
}
}

View File

@ -207,6 +207,25 @@ func RegisterCommands(s *discordgo.Session, guildID string) error {
}, },
}, },
}, },
{
Name: "speak",
Description: "Speak text in your current voice channel",
Options: []*discordgo.ApplicationCommandOption{
{
Type: discordgo.ApplicationCommandOptionString,
Name: "speaker",
Description: "Speaker voice",
Required: true,
Autocomplete: true,
},
{
Type: discordgo.ApplicationCommandOptionString,
Name: "text",
Description: "Text to speak",
Required: true,
},
},
},
} }
if _, err := s.ApplicationCommandBulkOverwrite(appID, guildID, commands); err != nil { if _, err := s.ApplicationCommandBulkOverwrite(appID, guildID, commands); err != nil {

View File

@ -0,0 +1,131 @@
package discord
import (
"context"
"errors"
"fmt"
"io"
"os"
"time"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
"github.com/bwmarrin/discordgo"
"github.com/jonas747/dca"
)
// ErrNotInVoiceChannel is returned when the invoking user isn't in a voice channel.
var ErrNotInVoiceChannel = errors.New("join a voice channel first")
// resolveVoiceChannel returns the voice channel the given guild member currently
// occupies, or ErrNotInVoiceChannel if they aren't in one.
func resolveVoiceChannel(s *discordgo.Session, guildID, userID string) (string, error) {
vs, err := s.State.VoiceState(guildID, userID)
if err != nil {
return "", ErrNotInVoiceChannel
}
return vs.ChannelID, nil
}
// speak joins the invoking user's current voice channel, transcodes audio (as
// returned by tts-gateway, e.g. AAC) to Opus via ffmpeg/dca, streams it, then
// leaves. This is Discord-specific playback plumbing that the transport-agnostic
// app layer has no business knowing about - see internal/app/command.go's
// HandleSpeak for the RPC half of this feature.
func speak(ctx context.Context, s *discordgo.Session, guildID, userID string, audio []byte, mimeType string) error {
log := logger.FromContext(ctx).With("mime_type", mimeType)
channelID, err := resolveVoiceChannel(s, guildID, userID)
if err != nil {
return err
}
vc, err := s.ChannelVoiceJoin(guildID, channelID, false, true)
if err != nil {
return fmt.Errorf("join voice channel: %w", err)
}
defer func() {
if err := vc.Disconnect(); err != nil {
log.Error("voice disconnect failed", "error", err.Error())
}
}()
if err := waitForVoiceReady(ctx, vc); err != nil {
return err
}
// dca shells out to ffmpeg, which needs a seekable input for containers like the
// AAC/M4A tts-gateway returns - its own moov atom sits at the end of the file (see
// tts-gateway/internal/adapters/secondary/ffmpeg/ffmpeg.go), so a non-seekable pipe
// input would fail. Write to a temp file first, mirroring that same adapter's idiom
// on the output side.
tmpPath, err := writeTempAudio(audio)
if err != nil {
return err
}
defer os.Remove(tmpPath)
encodeSession, err := dca.EncodeFile(tmpPath, dca.StdEncodeOptions)
if err != nil {
return fmt.Errorf("encode audio to opus: %w", err)
}
defer encodeSession.Cleanup()
if err := vc.Speaking(true); err != nil {
return fmt.Errorf("set speaking state: %w", err)
}
defer func() {
if err := vc.Speaking(false); err != nil {
log.Error("clear speaking state failed", "error", err.Error())
}
}()
done := make(chan error, 1)
dca.NewStream(encodeSession, vc, done)
select {
case err := <-done:
if err != nil && err != io.EOF {
return fmt.Errorf("stream audio: %w", err)
}
return nil
case <-ctx.Done():
return ctx.Err()
}
}
// writeTempAudio persists synthesized audio to a temp file for ffmpeg to read from.
func writeTempAudio(audio []byte) (string, error) {
tmpFile, err := os.CreateTemp("", "discord-bot-speak-*")
if err != nil {
return "", fmt.Errorf("create temp audio file: %w", err)
}
tmpPath := tmpFile.Name()
if _, err := tmpFile.Write(audio); err != nil {
_ = tmpFile.Close()
os.Remove(tmpPath)
return "", fmt.Errorf("write temp audio file: %w", err)
}
if err := tmpFile.Close(); err != nil {
os.Remove(tmpPath)
return "", fmt.Errorf("close temp audio file: %w", err)
}
return tmpPath, nil
}
// waitForVoiceReady polls until the voice connection's UDP socket is ready to
// send, or ctx is done - discordgo has no blocking/channel-based signal for this.
func waitForVoiceReady(ctx context.Context, vc *discordgo.VoiceConnection) error {
ticker := time.NewTicker(50 * time.Millisecond)
defer ticker.Stop()
for {
if vc.Ready {
return nil
}
select {
case <-ticker.C:
case <-ctx.Done():
return fmt.Errorf("wait for voice connection ready: %w", ctx.Err())
}
}
}

View File

@ -0,0 +1,73 @@
package discord
import (
"errors"
"testing"
"github.com/bwmarrin/discordgo"
)
func newTestSession(t *testing.T, guildID string, voiceStates []*discordgo.VoiceState) *discordgo.Session {
t.Helper()
state := discordgo.NewState()
if err := state.GuildAdd(&discordgo.Guild{ID: guildID, VoiceStates: voiceStates}); err != nil {
t.Fatalf("GuildAdd() error = %v", err)
}
return &discordgo.Session{State: state}
}
func TestResolveVoiceChannel(t *testing.T) {
const guildID = "guild-1"
const userID = "user-1"
tests := []struct {
name string
voiceStates []*discordgo.VoiceState
userID string
wantChannel string
wantErr error
}{
{
name: "user in a voice channel",
voiceStates: []*discordgo.VoiceState{
{GuildID: guildID, UserID: userID, ChannelID: "channel-1"},
},
userID: userID,
wantChannel: "channel-1",
},
{
name: "user not in any voice channel",
voiceStates: nil,
userID: userID,
wantErr: ErrNotInVoiceChannel,
},
{
name: "user in a different guild's voice channel is not found",
voiceStates: []*discordgo.VoiceState{
{GuildID: guildID, UserID: "someone-else", ChannelID: "channel-1"},
},
userID: userID,
wantErr: ErrNotInVoiceChannel,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
session := newTestSession(t, guildID, tt.voiceStates)
got, err := resolveVoiceChannel(session, guildID, tt.userID)
if tt.wantErr != nil {
if !errors.Is(err, tt.wantErr) {
t.Fatalf("resolveVoiceChannel() error = %v, want %v", err, tt.wantErr)
}
return
}
if err != nil {
t.Fatalf("resolveVoiceChannel() error = %v", err)
}
if got != tt.wantChannel {
t.Fatalf("resolveVoiceChannel() = %q, want %q", got, tt.wantChannel)
}
})
}
}

View File

@ -0,0 +1,124 @@
package ttsgateway
import (
"context"
"crypto/tls"
"crypto/x509"
"fmt"
"log/slog"
"os"
"path/filepath"
"time"
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
ttsv1 "gitea.nik4nao.com/nik/home-services/gen/tts/v1"
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials"
"google.golang.org/grpc/credentials/insecure"
)
// Client implements the app's TTS driven port over gRPC.
type Client struct {
conn *grpc.ClientConn
client ttsv1.TTSServiceClient
log *slog.Logger
}
// New constructs a gRPC client for the internal tts-gateway service.
func New(ctx context.Context, addr, tlsDir string, log *slog.Logger) (*Client, error) {
transportCreds := insecure.NewCredentials()
if tlsDir != "" {
creds, err := loadTransportCredentials(tlsDir)
if err != nil {
return nil, fmt.Errorf("load mTLS credentials: %w", err)
}
transportCreds = creds
}
conn, err := grpc.NewClient(
addr,
grpc.WithTransportCredentials(transportCreds),
grpc.WithStatsHandler(otelgrpc.NewClientHandler()),
)
if err != nil {
return nil, fmt.Errorf("dial tts-gateway: %w", err)
}
return &Client{
conn: conn,
client: ttsv1.NewTTSServiceClient(conn),
log: log,
}, nil
}
// Close closes the underlying gRPC connection.
func (c *Client) Close() error {
if err := c.conn.Close(); err != nil {
return fmt.Errorf("close tts-gateway client: %w", err)
}
return nil
}
// Synthesize forwards a speech synthesis request to tts-gateway. noise_scale/
// noise_scale_w/length_scale are left unset so tts-gateway applies its own defaults.
func (c *Client) Synthesize(ctx context.Context, speakerName, text string) ([]byte, string, error) {
start := time.Now()
log := logger.FromContext(ctx).With("grpc.method", "TTSService/Synthesize")
resp, err := c.client.Synthesize(ctx, &ttsv1.SynthesizeRequest{
SpeakerName: speakerName,
Text: text,
})
if err != nil {
log.Error("grpc call failed",
"duration_ms", time.Since(start).Milliseconds(),
"error", err.Error(),
)
return nil, "", fmt.Errorf("synthesize speech: %w", err)
}
log.Debug("grpc call completed", "duration_ms", time.Since(start).Milliseconds())
return resp.GetAudio(), resp.GetMimeType(), nil
}
// ListSpeakers returns the speaker roster from tts-gateway, optionally filtered by search.
func (c *Client) ListSpeakers(ctx context.Context, search string) ([]string, error) {
start := time.Now()
log := logger.FromContext(ctx).With("grpc.method", "TTSService/ListSpeakers")
resp, err := c.client.ListSpeakers(ctx, &ttsv1.ListSpeakersRequest{Search: search})
if err != nil {
log.Error("grpc call failed",
"duration_ms", time.Since(start).Milliseconds(),
"error", err.Error(),
)
return nil, fmt.Errorf("list tts-gateway speakers: %w", err)
}
log.Debug("grpc call completed", "duration_ms", time.Since(start).Milliseconds())
return append([]string(nil), resp.GetSpeakerNames()...), nil
}
func loadTransportCredentials(tlsDir string) (credentials.TransportCredentials, error) {
cert, err := tls.LoadX509KeyPair(
filepath.Join(tlsDir, "tls.crt"),
filepath.Join(tlsDir, "tls.key"),
)
if err != nil {
return nil, fmt.Errorf("load client key pair: %w", err)
}
caPEM, err := os.ReadFile(filepath.Join(tlsDir, "ca.crt"))
if err != nil {
return nil, fmt.Errorf("read server CA: %w", err)
}
rootCAs := x509.NewCertPool()
if !rootCAs.AppendCertsFromPEM(caPEM) {
return nil, fmt.Errorf("append server CA: invalid PEM")
}
return credentials.NewTLS(&tls.Config{
Certificates: []tls.Certificate{cert},
RootCAs: rootCAs,
ServerName: "tts-gateway.home-services.svc.cluster.local",
MinVersion: tls.VersionTLS13,
}), nil
}

View File

@ -23,11 +23,12 @@ type CommandApp struct {
ai driven.AIGateway ai driven.AIGateway
models *modelstore.Store models *modelstore.Store
validator *modelvalidator.Validator validator *modelvalidator.Validator
tts driven.TTSGateway
} }
// NewCommandApp constructs the Discord command application service. // NewCommandApp constructs the Discord command application service.
func NewCommandApp(ha driven.HAGateway, ai driven.AIGateway, models *modelstore.Store, validator *modelvalidator.Validator) *CommandApp { func NewCommandApp(ha driven.HAGateway, ai driven.AIGateway, models *modelstore.Store, validator *modelvalidator.Validator, tts driven.TTSGateway) *CommandApp {
return &CommandApp{ha: ha, ai: ai, models: models, validator: validator} return &CommandApp{ha: ha, ai: ai, models: models, validator: validator, tts: tts}
} }
// HandleLightList formats discovered lights into a monospace-friendly response. // HandleLightList formats discovered lights into a monospace-friendly response.
@ -279,6 +280,33 @@ func (a *CommandApp) HandleAIModelList(ctx context.Context) (string, error) {
return strings.Join(lines, "\n"), nil return strings.Join(lines, "\n"), nil
} }
// HandleSpeak synthesizes speech via tts-gateway for the invoking Discord adapter
// to play back in a voice channel; the app layer stays transport-agnostic and
// leaves voice-channel playback itself to the Discord adapter.
func (a *CommandApp) HandleSpeak(ctx context.Context, speakerName, text string) ([]byte, string, error) {
audio, mimeType, err := a.tts.Synthesize(ctx, speakerName, text)
if err != nil {
return nil, "", fmt.Errorf("handle speak: %w", err)
}
return audio, mimeType, nil
}
// AutocompleteSpeakers returns the full tts-gateway speaker roster for the /speak
// command; handleAutocomplete's existing substring filter narrows it client-side,
// same as the light/switch/ac autocompletes above.
func (a *CommandApp) AutocompleteSpeakers(ctx context.Context) ([]Choice, error) {
speakers, err := a.tts.ListSpeakers(ctx, "")
if err != nil {
return nil, fmt.Errorf("autocomplete speakers: %w", err)
}
choices := make([]Choice, 0, len(speakers))
for _, speaker := range speakers {
choices = append(choices, Choice{Label: speaker, Value: speaker})
}
return choices, nil
}
// AutocompleteAIModels returns model names for the /ai model set command. // AutocompleteAIModels returns model names for the /ai model set command.
func (a *CommandApp) AutocompleteAIModels(ctx context.Context) ([]Choice, error) { func (a *CommandApp) AutocompleteAIModels(ctx context.Context) ([]Choice, error) {
models, err := a.validator.Known(ctx) models, err := a.validator.Known(ctx)

View File

@ -35,6 +35,11 @@ type mockAIGateway struct {
listModelsFunc func(ctx context.Context) ([]string, error) listModelsFunc func(ctx context.Context) ([]string, error)
} }
type mockTTSGateway struct {
synthesizeFunc func(ctx context.Context, speakerName, text string) ([]byte, string, error)
listSpeakersFunc func(ctx context.Context, search string) ([]string, error)
}
func (m *mockHAGateway) ListLights(ctx context.Context) ([]driven.Light, error) { func (m *mockHAGateway) ListLights(ctx context.Context) ([]driven.Light, error) {
if m.listLightsFunc == nil { if m.listLightsFunc == nil {
return nil, nil return nil, nil
@ -147,8 +152,22 @@ func (m *mockAIGateway) ListModels(ctx context.Context) ([]string, error) {
return m.listModelsFunc(ctx) return m.listModelsFunc(ctx)
} }
func (m *mockTTSGateway) Synthesize(ctx context.Context, speakerName, text string) ([]byte, string, error) {
if m.synthesizeFunc == nil {
return nil, "", nil
}
return m.synthesizeFunc(ctx, speakerName, text)
}
func (m *mockTTSGateway) ListSpeakers(ctx context.Context, search string) ([]string, error) {
if m.listSpeakersFunc == nil {
return nil, nil
}
return m.listSpeakersFunc(ctx, search)
}
func newTestCommandApp(ha *mockHAGateway, ai *mockAIGateway) *CommandApp { func newTestCommandApp(ha *mockHAGateway, ai *mockAIGateway) *CommandApp {
return NewCommandApp(ha, ai, modelstore.New(), modelvalidator.New(ai, time.Minute)) return NewCommandApp(ha, ai, modelstore.New(), modelvalidator.New(ai, time.Minute), &mockTTSGateway{})
} }
func TestCommandAppHandleLightList(t *testing.T) { func TestCommandAppHandleLightList(t *testing.T) {
@ -1081,7 +1100,7 @@ func TestCommandAppHandleAIQuery(t *testing.T) {
} }
return "Turning on Kitchen.", "llama3:latest", nil return "Turning on Kitchen.", "llama3:latest", nil
}, },
}, store, modelvalidator.New(&mockAIGateway{}, time.Minute)) }, store, modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{})
got, err := app.HandleAIQuery(context.Background(), "turn on kitchen") got, err := app.HandleAIQuery(context.Background(), "turn on kitchen")
if err != nil { if err != nil {
@ -1092,6 +1111,116 @@ func TestCommandAppHandleAIQuery(t *testing.T) {
} }
} }
func TestCommandAppHandleSpeak(t *testing.T) {
tests := []struct {
name string
speakerName string
text string
audio []byte
mimeType string
synthErr error
wantErr string
}{
{
name: "synthesize success",
speakerName: "Rice Shower",
text: "おはようございます",
audio: []byte{0x00, 0x01, 0x02},
mimeType: "audio/aac",
},
{
name: "synthesize error propagated",
speakerName: "Unknown Speaker",
text: "hello",
synthErr: errors.New("boom"),
wantErr: "handle speak: boom",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
var gotSpeaker, gotText string
app := NewCommandApp(&mockHAGateway{}, &mockAIGateway{}, modelstore.New(), modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{
synthesizeFunc: func(ctx context.Context, speakerName, text string) ([]byte, string, error) {
gotSpeaker, gotText = speakerName, text
if tt.synthErr != nil {
return nil, "", tt.synthErr
}
return tt.audio, tt.mimeType, nil
},
})
audio, mimeType, err := app.HandleSpeak(context.Background(), tt.speakerName, tt.text)
if tt.wantErr != "" {
if err == nil || err.Error() != tt.wantErr {
t.Fatalf("HandleSpeak() error = %v, want %q", err, tt.wantErr)
}
return
}
if err != nil {
t.Fatalf("HandleSpeak() error = %v", err)
}
if gotSpeaker != tt.speakerName || gotText != tt.text {
t.Fatalf("Synthesize() called with (%q, %q), want (%q, %q)", gotSpeaker, gotText, tt.speakerName, tt.text)
}
if !reflect.DeepEqual(audio, tt.audio) || mimeType != tt.mimeType {
t.Fatalf("HandleSpeak() = (%#v, %q), want (%#v, %q)", audio, mimeType, tt.audio, tt.mimeType)
}
})
}
}
func TestCommandAppAutocompleteSpeakers(t *testing.T) {
tests := []struct {
name string
speakers []string
listErr error
want []Choice
wantErr string
}{
{
name: "maps speaker names to choices",
speakers: []string{"Rice Shower", "Special Week"},
want: []Choice{
{Label: "Rice Shower", Value: "Rice Shower"},
{Label: "Special Week", Value: "Special Week"},
},
},
{
name: "ListSpeakers error",
listErr: errors.New("boom"),
wantErr: "autocomplete speakers: boom",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
app := NewCommandApp(&mockHAGateway{}, &mockAIGateway{}, modelstore.New(), modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{
listSpeakersFunc: func(ctx context.Context, search string) ([]string, error) {
if tt.listErr != nil {
return nil, tt.listErr
}
return tt.speakers, nil
},
})
got, err := app.AutocompleteSpeakers(context.Background())
if tt.wantErr != "" {
if err == nil || err.Error() != tt.wantErr {
t.Fatalf("AutocompleteSpeakers() error = %v, want %q", err, tt.wantErr)
}
return
}
if err != nil {
t.Fatalf("AutocompleteSpeakers() error = %v", err)
}
if !reflect.DeepEqual(got, tt.want) {
t.Fatalf("AutocompleteSpeakers() = %#v, want %#v", got, tt.want)
}
})
}
}
func TestCommandAppHandleAIModelSet(t *testing.T) { func TestCommandAppHandleAIModelSet(t *testing.T) {
app := newTestCommandApp(&mockHAGateway{}, &mockAIGateway{ app := newTestCommandApp(&mockHAGateway{}, &mockAIGateway{
listModelsFunc: func(ctx context.Context) ([]string, error) { listModelsFunc: func(ctx context.Context) ([]string, error) {

View File

@ -11,6 +11,7 @@ type Config struct {
GuildID string GuildID string
HAGatewayAddr string HAGatewayAddr string
AIGatewayAddr string AIGatewayAddr string
TTSGatewayAddr string
TLSDir string TLSDir string
OTELEndpoint string OTELEndpoint string
LogLevel string LogLevel string
@ -34,6 +35,7 @@ func Load() (*Config, error) {
GuildID: os.Getenv("GUILD_ID"), GuildID: os.Getenv("GUILD_ID"),
HAGatewayAddr: addr, HAGatewayAddr: addr,
AIGatewayAddr: getenvDefault("AI_GATEWAY_ADDR", "ai-gateway.home-services.svc.cluster.local:50052"), AIGatewayAddr: getenvDefault("AI_GATEWAY_ADDR", "ai-gateway.home-services.svc.cluster.local:50052"),
TTSGatewayAddr: getenvDefault("TTS_GATEWAY_ADDR", "tts-gateway.home-services.svc.cluster.local:50053"),
TLSDir: os.Getenv("TLS_DIR"), TLSDir: os.Getenv("TLS_DIR"),
OTELEndpoint: os.Getenv("OTEL_ENDPOINT"), OTELEndpoint: os.Getenv("OTEL_ENDPOINT"),
LogLevel: getenvDefault("LOG_LEVEL", "info"), LogLevel: getenvDefault("LOG_LEVEL", "info"),

View File

@ -0,0 +1,9 @@
package driven
import "context"
// TTSGateway exposes the text-to-speech synthesis API used by the Discord bot.
type TTSGateway interface {
Synthesize(ctx context.Context, speakerName, text string) (audio []byte, mimeType string, err error)
ListSpeakers(ctx context.Context, search string) ([]string, error)
}

90
handoff.md Normal file
View File

@ -0,0 +1,90 @@
# tts-gateway Handoff
Status as of 2026-07-25: Phases 06 of `TTS_GATEWAY_PLAN.md` are done, and the deployed service is
confirmed working in production (real `grpcurl` call through `kubectl port-forward`, real GPU
inference, real playable audio). This file is a punch list for picking the remaining work up in a
fresh session — it doesn't re-explain things that are already documented elsewhere; it points to
where.
**Update (later same day):** items 1 and 3 below are now done — model artifact distribution is a
versioned image + k8s `initContainer` (see `tts-gateway/README.md`), and `discord-bot` has a
working `/speak` command (see `TTS_GATEWAY_PLAN.md` Phase 6). Only item 2 (mTLS) remains, left
untouched per an explicit decision to verify these two first. The two "what's left" entries below
are kept as-written for their historical reasoning/decision trail rather than rewritten in place.
## Where things stand
- `tts-gateway` (Go) + `tts-sidecar` (Python/libtorch) are both built, containerized, and running
in Kubernetes (`home-services` namespace).
- CI (`.gitea/workflows/ci.yaml`) builds and pushes both images on every push to `main`, gated by
path-filtered per-service rebuilds and using registry-based Docker layer caching.
- Full history — the Phase 0 ONNX-export/g2p investigation, why there's a Python sidecar instead
of pure Go, the exact `japanese_cleaners` pipeline verification — is in `TTS_GATEWAY_PLAN.md`
and `tmp/reference/uma-tts-api/spike/FINDINGS.md`. Read those before re-deriving anything; the
ONNX export failure in particular took several iterations to characterize correctly and isn't
worth re-investigating from scratch.
## What's left
1. **Model artifact distribution isn't automated.** The k8s Deployment
(`~/repo/homelab/manifests/home-services/tts-gateway.yaml` — separate repo, see below)
`hostPath`-mounts `/data/tts-gateway` on the `nik-gpu` node into the sidecar container.
Someone has to manually place `G_790000.pth` + `uma.json` there before the pod goes Ready — no
init-container download, no PVC, nothing automated. `TTS_GATEWAY_PLAN.md`'s Phase 5 flagged
this as an open decision; it's still open, just worked around. If asked to fix it: options are
a PVC populated by an init-container download step, baking the checkpoint into a private image
layer, or similar — weigh against the checkpoint being ~455MB and not something to casually put
in a git-tracked Dockerfile context.
2. **mTLS is off.** `tts-gateway.yaml` comments out the `TLS_DIR` env var and its volume mount,
for plaintext `grpcurl` testing from outside the cluster during initial rollout — a live TODO,
not a permanent decision. Every other service in this repo assumes internal-network-or-mTLS as
its only access boundary (`CLAUDE.md`'s "Configuration Notes"). Re-enabling: uncomment the env
var + mount in that manifest, and set up a `tts-gateway-tls` secret the same way
`ha-gateway-sealed.yaml`/`ha-gateway-secret.sh` do it for ha-gateway (tts-gateway doesn't
currently have its own `-secret.sh`/`-sealed.yaml` pair — it'll need one, since unlike
ha-gateway/discord-bot it has no other secrets today, so this would be its first).
3. **Phase 6 (discord-bot integration) not started.** A `/speak`-style Discord slash command
calling `tts-gateway`, following the existing pattern — `discord-bot` already has gRPC clients
for `ha-gateway` and `ai-gateway` at `discord-bot/internal/adapters/secondary/{gateway,aigateway}`;
a new `internal/adapters/secondary/ttsgateway` client would follow the same shape. Needs a
product decision first: keep AAC (parity with what's already implemented) or switch to Opus
(Discord's native voice codec) — deliberately deferred in `TTS_GATEWAY_PLAN.md`'s open
questions until this exact moment, not yet decided.
## Things worth knowing before touching this
- **`tts-gateway` and `tts-sidecar` share one Kubernetes Pod**, not two separate Deployments — the
Go gateway reaches the sidecar over `localhost:50054`. This mirrors the `--network host` setup
documented in `tts-gateway/README.md` for local Docker testing. Don't split them into separate
Deployments/Services without also rethinking `INFERENCE_SIDECAR_ADDR` and the networking model.
- **The k8s manifests live in a different repo**: `~/repo/homelab/manifests/home-services/` — not
in this repo at all. `CLAUDE.md` now has an "Infrastructure / Deployment" section documenting
this; check there (and that repo directly) before assuming something isn't deployed just because
this repo has no manifests for it.
- **`open_jtalk`'s tokenization has an intentional bug that must be preserved.** The Go text
normalizer (`tts-gateway/internal/adapters/secondary/jtalk`) maps the cleaned phoneme string to
symbol IDs character-by-character, not phoneme-by-phoneme, because that's what the checkpoint
was actually trained on (`text/symbols.py`'s "wrong tokens" comment in the original reference
implementation). Don't "fix" this without retraining the model — see the comments in `jtalk.go`
and `tts-gateway/README.md`.
- **ONNX export doesn't work for this checkpoint, and it's not a quick fix.** Read
`tmp/reference/uma-tts-api/spike/FINDINGS.md` before re-attempting it — the model's
data-dependent output length (predicted phoneme durations determine audio length at runtime)
defeats `torch.export`'s guard system in a way that would need an unknown number of per-layer
`torch._check` hints to resolve, not a single targeted change.
- **CI's build cache needs one "priming" run per image.** If a build job still looks slow after
the caching change in `ci.yaml`, check whether a `:buildcache` tag actually exists yet in the
registry for that specific image — the first run after the cache was added has nothing to pull
from and builds fully fresh.
## Where to look for more detail
- `TTS_GATEWAY_PLAN.md` (repo root) — original plan, feasibility analysis, phase breakdown, open
product questions (codec, concurrency, auth).
- `tmp/reference/uma-tts-api/spike/FINDINGS.md` — Phase 0 spike results with full reasoning.
- `tts-gateway/README.md` — service-level docs: gRPC API, config reference, how to run/test
locally and on nik-gpu.
- `~/repo/homelab/manifests/home-services/tts-gateway.yaml` — actual production config (separate
repo, maintained independently — check it directly rather than trusting a stale summary).

View File

@ -67,8 +67,33 @@ The server also registers gRPC health checks and reflection.
| `PORT` | `50054` | HTTP listen port | | `PORT` | `50054` | HTTP listen port |
Neither the checkpoint nor the hparams file is committed to git - both must Neither the checkpoint nor the hparams file is committed to git - both must
be supplied at runtime (see below). Model artifact distribution (bind mount be supplied at runtime (see below). In production, they're baked into a
vs. build-time download vs. registry) is still an open decision. small versioned image (`tts-gateway/model/Dockerfile`) that a k8s
`initContainer` copies into a shared volume before `tts-sidecar` starts -
see "Model artifact distribution (production)" below. For local/manual
testing on nik-gpu, a plain bind mount (as shown below) is simpler.
### Model artifact distribution (production)
CI can never build the model image - the checkpoint (~455MB) and hparams
aren't committed to git, and nik-gpu's local disk at `/data/tts-gateway` is
the only place they exist. Build and push it manually, directly on nik-gpu
(not via `docker --context nik-gpu` from elsewhere - the build context is
gathered client-side, so it would look for `/data/tts-gateway` on the wrong
machine), whenever the checkpoint or config change:
```bash
ssh nik-gpu
docker build -f ~/repo/home-service/tts-gateway/model/Dockerfile \
-t gitea.nik4nao.com/nik/tts-model:g790000-v1 /data/tts-gateway
docker push gitea.nik4nao.com/nik/tts-model:g790000-v1
```
Bump the tag (`v2`, `v3`, ...) rather than overwriting one - non-`:latest`
tags default to `imagePullPolicy: IfNotPresent`, so a node that already
pulled a tag won't re-pull an overwritten one. The k8s Deployment
(`~/repo/homelab/manifests/home-services/tts-gateway.yaml`) references the
tag explicitly in its `model-init` init container.
## Running And Testing On nik-gpu ## Running And Testing On nik-gpu
@ -188,8 +213,8 @@ sidecar/ # Python/libtorch inference servic
- Concurrency is unbounded on the Go side, but the sidecar's model forward - Concurrency is unbounded on the Go side, but the sidecar's model forward
pass is inherently single-threaded per GPU; heavy concurrent load will pass is inherently single-threaded per GPU; heavy concurrent load will
just queue at the sidecar. just queue at the sidecar.
- Model artifact distribution isn't finalized - the sidecar currently - Model artifact distribution to production is a manually-built-and-pushed
expects a bind-mounted checkpoint/config, not something the image ships versioned image (see above), not something CI can automate - the
with. checkpoint only ever existed on nik-gpu's local disk.
- No Discord-bot integration yet (a `/speak`-style command calling this - `discord-bot`'s `/speak` command calls this service and plays the result
service is a deliberate follow-up, not yet built). in a Discord voice channel (`discord-bot/internal/adapters/primary/discord/voice.go`).

View File

@ -0,0 +1,7 @@
# One-time (or only-when-the-checkpoint-changes) manually built and pushed by a human on
# nik-gpu - CI can never build this: the checkpoint and hparams aren't committed to git
# (~455MB, and nik-gpu's local disk at /data/tts-gateway is the only place they exist).
# Build CONTEXT must be that directory, not a checkout of this repo - see
# tts-gateway/README.md for the exact commands.
FROM busybox:1.36
COPY G_790000.pth uma.json /models/