feat: add /speak command for TTS integration
All checks were successful
CI / changes (push) Successful in 1s
CI / test (push) Successful in 6s
CI / build-ai-gateway (push) Has been skipped
CI / build-ha-gateway (push) Has been skipped
CI / build-discord-bot (push) Successful in 1m33s
CI / build-tts-gateway (push) Successful in 37s
CI / build-tts-sidecar (push) Has been skipped
All checks were successful
CI / changes (push) Successful in 1s
CI / test (push) Successful in 6s
CI / build-ai-gateway (push) Has been skipped
CI / build-ha-gateway (push) Has been skipped
CI / build-discord-bot (push) Successful in 1m33s
CI / build-tts-gateway (push) Successful in 37s
CI / build-tts-sidecar (push) Has been skipped
- Implemented the /speak command in Discord bot to synthesize speech using the TTS gateway. - Added voice handling logic to join voice channels and play synthesized audio. - Created tests for the new command and voice functionalities. - Introduced TTSGateway interface for TTS service communication. - Updated configuration to include TTS gateway address. - Documented the TTS gateway integration and model artifact distribution process.
This commit is contained in:
parent
7851c49dba
commit
0d58e46740
17
CLAUDE.md
17
CLAUDE.md
@ -17,12 +17,14 @@ ai-gateway ------> Ollama
|
||||
|
|
||||
v
|
||||
ha-gateway
|
||||
|
||||
discord-bot -----> tts-gateway -----> tts-sidecar (GPU inference)
|
||||
```
|
||||
|
||||
- **ha-gateway** (port `50051`) — gRPC boundary for Home Assistant. Talks to HA's REST API; implements entity state, light control/discovery, switch control/discovery, and climate (HVAC) control/discovery; also relays SwitchBot Cloud remote commands (`RemoteApp`) when `SWITCHBOT_TOKEN`/`SWITCHBOT_SECRET` are set. Event streaming is stubbed.
|
||||
- **ai-gateway** (port `50052`) — gRPC service that turns free-form text into home actions. Calls Ollama for intent extraction, resolves intents against a cached light list from `ha-gateway`, and calls `ha-gateway` to execute approved actions.
|
||||
- **discord-bot** — registers `/light`, `/switch`, `/ac`, `/ai` slash commands and calls `ha-gateway`/`ai-gateway` via gRPC clients.
|
||||
- **tts-gateway** (port `50053`) — gRPC text-to-speech service (VITS voice model, 92 Umamusume voices), paired with a Python/libtorch inference sidecar (`tts-gateway/sidecar/`, deployed as a second container in the same pod) that needs an Nvidia GPU — see `TTS_GATEWAY_PLAN.md` for why. Not yet wired into the diagram above; nothing calls it yet (a discord-bot `/speak` command is the planned follow-up). See `tts-gateway/README.md` for its API/config/local-run details.
|
||||
- **discord-bot** — registers `/light`, `/switch`, `/ac`, `/ai`, `/speak` slash commands and calls `ha-gateway`/`ai-gateway`/`tts-gateway` via gRPC clients. `/speak` synthesizes speech via `tts-gateway` (which returns AAC) and transcodes it to Opus locally (via `ffmpeg`/`github.com/jonas747/dca`) to stream into the invoking user's current voice channel — see `internal/adapters/primary/discord/voice.go`.
|
||||
- **tts-gateway** (port `50053`) — gRPC text-to-speech service (VITS voice model, 92 Umamusume voices), paired with a Python/libtorch inference sidecar (`tts-gateway/sidecar/`, deployed as a second container in the same pod) that needs an Nvidia GPU — see `TTS_GATEWAY_PLAN.md` for why. Called by `discord-bot`'s `/speak` command. See `tts-gateway/README.md` for its API/config/local-run details.
|
||||
|
||||
Each service is a separate Go module (own `go.mod`) joined by `go.work` at the root, plus a `gen` module for shared generated code. Module paths are `gitea.nik4nao.com/nik/home-services/{ha-gateway,ai-gateway,discord-bot,tts-gateway,gen}`.
|
||||
|
||||
@ -172,10 +174,13 @@ rather than trusting this description to stay accurate):
|
||||
`nvidia.com/gpu: 1` request/limit on the sidecar container. `strategy: Recreate` instead of the
|
||||
default `RollingUpdate` — nik-gpu only has one allocatable GPU, so a rolling update would
|
||||
deadlock waiting for a GPU still held by the pod it's replacing.
|
||||
- Model artifact distribution (checkpoint + hparams) is a `hostPath` volume at
|
||||
`/data/tts-gateway` on the nik-gpu node, not baked into the image or automated — someone has to
|
||||
manually place `G_790000.pth`/`uma.json` there. This is a known-open gap (see `handoff.md`),
|
||||
not a deliberate final design.
|
||||
- Model artifact distribution (checkpoint + hparams) is an `emptyDir` volume populated at pod
|
||||
start by an `initContainers` entry (`model-init`) that copies from a small, versioned image
|
||||
(`gitea.nik4nao.com/nik/tts-model:<tag>`, built from `tts-gateway/model/Dockerfile`) — not a
|
||||
raw `hostPath` into the node's disk anymore. That model image still can't be built by CI (the
|
||||
checkpoint isn't committed to git, ~455MB, and only ever existed on nik-gpu's local disk) — it's
|
||||
built and pushed manually, on nik-gpu, whenever the checkpoint/config change. See
|
||||
`tts-gateway/README.md` for the exact commands.
|
||||
- mTLS (`TLS_DIR`) is currently commented out on `tts-gateway`, for plaintext `grpcurl` testing
|
||||
from outside the cluster during initial rollout — a live TODO, not a permanent decision, unlike
|
||||
every other service here which assumes mTLS-or-trusted-network as its only access boundary (see
|
||||
|
||||
@ -170,16 +170,18 @@ repo's hand-written-mock testing convention rather than golden-byte comparison.
|
||||
WAV writer, `ffmpeg` transcode adapter, full `Synthesize` RPC wiring, speaker-not-found →
|
||||
`INVALID_ARGUMENT` (mirroring `app.py`'s current 400 behavior), `ListSpeakers` RPC.
|
||||
|
||||
**Phase 5 — Containerize & deploy to nik-gpu**
|
||||
See [GPU/deployment notes](#gpu-and-deployment-notes) below — this phase needs a Dockerfile that
|
||||
diverges from the other three services' pattern, plus a decision on how the model artifact
|
||||
reaches the container.
|
||||
**Phase 5 — Containerize & deploy to nik-gpu** — **done.** Deployed and confirmed working in
|
||||
production; see `handoff.md` for status. Model artifact distribution is resolved as a small
|
||||
versioned image (`tts-gateway/model/Dockerfile`) copied into a shared volume by a k8s
|
||||
`initContainer`, built manually on nik-gpu (the checkpoint never existed anywhere CI can reach) —
|
||||
see `tts-gateway/README.md`'s "Model artifact distribution (production)" section.
|
||||
|
||||
**Phase 6 — Client integration (optional, separate follow-up)**
|
||||
A `/speak` (or similar) Discord command in `discord-bot` calling this gateway, following the
|
||||
same secondary-adapter + gRPC-client pattern already used for its `ha-gateway`/`ai-gateway`
|
||||
clients. Not assumed in scope here — a deliberate follow-on decision once the service itself
|
||||
works.
|
||||
**Phase 6 — Client integration** — **done.** `discord-bot` registers a `/speak` command calling
|
||||
this gateway via the same secondary-adapter + gRPC-client pattern used for its
|
||||
`ha-gateway`/`ai-gateway` clients (`discord-bot/internal/adapters/secondary/ttsgateway/`), then
|
||||
transcodes the AAC response to Opus locally (`ffmpeg` + `github.com/jonas747/dca`) and streams it
|
||||
into the invoking user's current voice channel
|
||||
(`discord-bot/internal/adapters/primary/discord/voice.go`).
|
||||
|
||||
## GPU and deployment notes
|
||||
|
||||
@ -226,9 +228,10 @@ nik-gpu's host state untouched and the setup reproducible from the Dockerfile al
|
||||
|
||||
## Open questions (product/design, not technical blockers)
|
||||
|
||||
- **Output codec:** keep AAC (parity with today) or switch to Opus? If the eventual consumer is
|
||||
`discord-bot`, Opus is Discord's native voice codec — worth deciding once Phase 6 is actually
|
||||
on the table, not before.
|
||||
- **Output codec:** ~~keep AAC (parity with today) or switch to Opus?~~ **Resolved: kept AAC.**
|
||||
`tts-gateway` itself is unchanged; `discord-bot` transcodes AAC → Opus locally for voice
|
||||
playback (see Phase 6 above), rather than pushing an Opus-specific output format onto every
|
||||
other current/future consumer of this gateway.
|
||||
- **Concurrency:** the reference implementation serializes all inference behind one lock. Same
|
||||
starting point is reasonable here — revisit only if real latency/throughput needs show up.
|
||||
- **Auth:** consistent with every other service in this repo, no app-layer authorization —
|
||||
|
||||
@ -2,6 +2,7 @@ DISCORD_TOKEN=your-bot-token-here
|
||||
GUILD_ID=your-guild-id-here
|
||||
HA_GATEWAY_ADDR=localhost:50051
|
||||
AI_GATEWAY_ADDR=localhost:50052
|
||||
TTS_GATEWAY_ADDR=localhost:50053
|
||||
OTEL_ENDPOINT=
|
||||
LOG_LEVEL=info
|
||||
LOG_FORMAT=text
|
||||
|
||||
@ -27,6 +27,14 @@ RUN CGO_ENABLED=0 GOOS=linux go build \
|
||||
-ldflags="-s -w -X main.version=${VERSION}" \
|
||||
-o /discord-bot ./cmd/bot
|
||||
|
||||
FROM gcr.io/distroless/static:nonroot
|
||||
# /speak needs `ffmpeg` at runtime (github.com/jonas747/dca shells out to it to transcode
|
||||
# tts-gateway's AAC output to Opus for Discord voice), which rules out a distroless base -
|
||||
# same reasoning as tts-gateway/Dockerfile.
|
||||
FROM ubuntu:22.04
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ffmpeg \
|
||||
ca-certificates \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY --from=builder /discord-bot /discord-bot
|
||||
ENTRYPOINT ["/discord-bot"]
|
||||
|
||||
@ -14,6 +14,7 @@ import (
|
||||
discordadapter "gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/primary/discord"
|
||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/aigateway"
|
||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/gateway"
|
||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/ttsgateway"
|
||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/app"
|
||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/config"
|
||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
|
||||
@ -42,6 +43,7 @@ func main() {
|
||||
"version", version,
|
||||
"ha_gateway_addr", cfg.HAGatewayAddr,
|
||||
"ai_gateway_addr", cfg.AIGatewayAddr,
|
||||
"tts_gateway_addr", cfg.TTSGatewayAddr,
|
||||
"discord_token", redactToken(cfg.DiscordToken),
|
||||
"tls_dir", cfg.TLSDir,
|
||||
"log_level", cfg.LogLevel,
|
||||
@ -86,9 +88,20 @@ func main() {
|
||||
}
|
||||
}()
|
||||
|
||||
ttsClient, err := ttsgateway.New(ctx, cfg.TTSGatewayAddr, cfg.TLSDir, log)
|
||||
if err != nil {
|
||||
log.Error("tts-gateway client setup failed", "err", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
defer func() {
|
||||
if err := ttsClient.Close(); err != nil {
|
||||
log.Error("tts-gateway client close failed", "err", err)
|
||||
}
|
||||
}()
|
||||
|
||||
modelStore := modelstore.New()
|
||||
validator := modelvalidator.New(aiClient, 30*time.Second)
|
||||
commandApp := app.NewCommandApp(haClient, aiClient, modelStore, validator)
|
||||
commandApp := app.NewCommandApp(haClient, aiClient, modelStore, validator, ttsClient)
|
||||
tracker := discordadapter.NewTracker(context.Background())
|
||||
|
||||
// Discord-specific wiring stays at the edge so the app layer remains transport-agnostic.
|
||||
@ -97,7 +110,9 @@ func main() {
|
||||
log.Error("create discord session failed", "err", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
session.Identify.Intents = discordgo.IntentsGuilds
|
||||
// GuildVoiceStates is needed to resolve which voice channel the invoking
|
||||
// user is currently in for /speak (see internal/adapters/primary/discord/voice.go).
|
||||
session.Identify.Intents = discordgo.IntentsGuilds | discordgo.IntentsGuildVoiceStates
|
||||
|
||||
handler := discordadapter.NewHandler(commandApp, tracker)
|
||||
handler.Register(session)
|
||||
@ -121,6 +136,7 @@ func main() {
|
||||
"command_scope", scope,
|
||||
"ha_gateway_addr", cfg.HAGatewayAddr,
|
||||
"ai_gateway_addr", cfg.AIGatewayAddr,
|
||||
"tts_gateway_addr", cfg.TTSGatewayAddr,
|
||||
"version", version,
|
||||
)
|
||||
|
||||
|
||||
@ -6,6 +6,7 @@ require (
|
||||
gitea.nik4nao.com/nik/home-services/gen v0.0.0
|
||||
github.com/bwmarrin/discordgo v0.29.0
|
||||
github.com/joho/godotenv v1.5.1
|
||||
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7
|
||||
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0
|
||||
go.opentelemetry.io/otel v1.39.0
|
||||
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.35.0
|
||||
@ -25,6 +26,7 @@ require (
|
||||
github.com/google/uuid v1.6.0 // indirect
|
||||
github.com/gorilla/websocket v1.4.2 // indirect
|
||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 // indirect
|
||||
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757 // indirect
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.35.0 // indirect
|
||||
go.opentelemetry.io/proto/otlp v1.5.0 // indirect
|
||||
|
||||
@ -23,6 +23,10 @@ github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 h1:e9Rjr40Z98/clHv5Yg79Is0Ntos
|
||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1/go.mod h1:tIxuGz/9mpox++sgp9fJjHO0+q1X9/UOWd798aAm22M=
|
||||
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
|
||||
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
|
||||
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7 h1:Iw1pZVDiq4tY8hO7Wt1D1EqDC0BmMw/bFO/Rdt4gs9Q=
|
||||
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7/go.mod h1:rxjYX9OJU81unMxQDHChU/lAiOhlY9MV+faPX/NmwLk=
|
||||
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757 h1:Kyv+zTfWIGRNaz/4+lS+CxvuKVZSKFz/6G8E3BKKBRs=
|
||||
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757/go.mod h1:cZnNmdLiLpihzgIVqiaQppi9Ts3D4qF/M45//yW35nI=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||
|
||||
@ -38,10 +38,12 @@ type commandHandler interface {
|
||||
HandleAIModelSet(ctx context.Context, name string) (string, error)
|
||||
HandleAIModelGet(ctx context.Context) (string, error)
|
||||
HandleAIModelList(ctx context.Context) (string, error)
|
||||
HandleSpeak(ctx context.Context, speakerName, text string) (audio []byte, mimeType string, err error)
|
||||
AutocompleteLights(ctx context.Context) ([]apppkg.Choice, error)
|
||||
AutocompleteSwitches(ctx context.Context) ([]apppkg.Choice, error)
|
||||
AutocompleteClimates(ctx context.Context) ([]apppkg.Choice, error)
|
||||
AutocompleteAIModels(ctx context.Context) ([]apppkg.Choice, error)
|
||||
AutocompleteSpeakers(ctx context.Context) ([]apppkg.Choice, error)
|
||||
}
|
||||
|
||||
// Handler adapts Discord interactions to the command application layer.
|
||||
@ -84,7 +86,7 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
|
||||
|
||||
data := i.ApplicationCommandData()
|
||||
command := data.Name
|
||||
if len(data.Options) > 0 {
|
||||
if len(data.Options) > 0 && isSubcommandOption(data.Options[0]) {
|
||||
command += "." + data.Options[0].Name
|
||||
if len(data.Options[0].Options) > 0 && data.Options[0].Type == discordgo.ApplicationCommandOptionSubCommandGroup {
|
||||
command += "." + data.Options[0].Options[0].Name
|
||||
@ -104,6 +106,15 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
|
||||
),
|
||||
)
|
||||
defer span.End()
|
||||
|
||||
// speak is a flat command (speaker/text values directly under data.Options, no
|
||||
// subcommand wrapper) unlike light/switch/ac/ai, so it's dispatched before the
|
||||
// subcommand-assuming logic below rather than through commandPath's switch.
|
||||
if data.Name == "speak" {
|
||||
h.handleSpeak(ctx, s, i, data, start)
|
||||
return
|
||||
}
|
||||
|
||||
if len(data.Options) == 0 {
|
||||
h.respondError(ctx, s, i.Interaction, true, start, fmt.Errorf("missing subcommand"))
|
||||
return
|
||||
@ -305,6 +316,57 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
|
||||
}
|
||||
}
|
||||
|
||||
// handleSpeak synthesizes speech via tts-gateway and plays it in the invoking
|
||||
// user's current voice channel, following the same deferred-ack + background +
|
||||
// followup shape as the ai.query case (synthesis + voice join + playback will
|
||||
// exceed Discord's ~3s initial-ack window).
|
||||
func (h *Handler) handleSpeak(ctx context.Context, s *discordgo.Session, i *discordgo.InteractionCreate, data discordgo.ApplicationCommandInteractionData, start time.Time) {
|
||||
log := logger.FromContext(ctx)
|
||||
if err := h.deferResponse(s, i.Interaction, true); err != nil {
|
||||
log.Error("discord response failed",
|
||||
"duration_ms", time.Since(start).Milliseconds(),
|
||||
"error", err.Error(),
|
||||
)
|
||||
return
|
||||
}
|
||||
|
||||
speakerName := stringOption(data.Options, "speaker")
|
||||
text := stringOption(data.Options, "text")
|
||||
guildID := i.GuildID
|
||||
userID := ""
|
||||
if i.Member != nil && i.Member.User != nil {
|
||||
userID = i.Member.User.ID
|
||||
} else if i.User != nil {
|
||||
userID = i.User.ID
|
||||
}
|
||||
|
||||
interaction := i.Interaction
|
||||
reqLog := log
|
||||
h.tracker.Go(func(trackerCtx context.Context) {
|
||||
asyncCtx := logger.WithLogger(trackerCtx, reqLog)
|
||||
asyncCtx, cancel := context.WithTimeout(asyncCtx, 2*time.Minute)
|
||||
defer cancel()
|
||||
|
||||
audio, mimeType, err := h.app.HandleSpeak(asyncCtx, speakerName, text)
|
||||
if err == nil {
|
||||
err = speak(asyncCtx, s, guildID, userID, audio, mimeType)
|
||||
}
|
||||
|
||||
msg := fmt.Sprintf("Spoke as `%s`.", speakerName)
|
||||
if err != nil {
|
||||
reqLog.Error("speak command failed", "error", err.Error())
|
||||
msg = mapSpeakError(err)
|
||||
}
|
||||
if _, followErr := s.FollowupMessageCreate(interaction, true, &discordgo.WebhookParams{
|
||||
Content: msg,
|
||||
Flags: discordgo.MessageFlagsEphemeral,
|
||||
}); followErr != nil {
|
||||
reqLog.Error("discord response failed", "error", followErr.Error())
|
||||
}
|
||||
})
|
||||
log.Info("command handled", "duration_ms", time.Since(start).Milliseconds())
|
||||
}
|
||||
|
||||
// handleAutocomplete keeps autocomplete fast by filtering already-fetched choices
|
||||
// instead of requiring Discord-specific logic in the app layer.
|
||||
func (h *Handler) handleAutocomplete(ctx context.Context, s *discordgo.Session, i *discordgo.InteractionCreate) {
|
||||
@ -326,6 +388,10 @@ func (h *Handler) handleAutocomplete(ctx context.Context, s *discordgo.Session,
|
||||
if focusedOptionName(data) == "name" {
|
||||
choices, err = h.app.AutocompleteAIModels(ctx)
|
||||
}
|
||||
case "speak":
|
||||
if focusedOptionName(data) == "speaker" {
|
||||
choices, err = h.app.AutocompleteSpeakers(ctx)
|
||||
}
|
||||
default:
|
||||
choices = nil
|
||||
}
|
||||
@ -419,6 +485,24 @@ func (h *Handler) followup(ctx context.Context, s *discordgo.Session, interactio
|
||||
}
|
||||
}
|
||||
|
||||
func mapSpeakError(err error) string {
|
||||
if errors.Is(err, ErrNotInVoiceChannel) {
|
||||
return "Join a voice channel first."
|
||||
}
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
return "Speaking took too long. Try again later."
|
||||
}
|
||||
if status, ok := grpcstatus.FromError(err); ok {
|
||||
switch status.Code() {
|
||||
case codes.InvalidArgument:
|
||||
return "Unknown speaker. Try /speak again with a different name."
|
||||
case codes.Unavailable:
|
||||
return "The TTS service is unreachable right now. Try again in a moment."
|
||||
}
|
||||
}
|
||||
return "Sorry, something went wrong speaking that."
|
||||
}
|
||||
|
||||
func mapAIError(err error) string {
|
||||
if errors.Is(err, context.DeadlineExceeded) {
|
||||
return "The AI took too long to respond. Try again later."
|
||||
@ -448,7 +532,14 @@ func interactionLogger(ctx context.Context, i *discordgo.InteractionCreate) *slo
|
||||
}
|
||||
|
||||
func requiredStringOption(sub *discordgo.ApplicationCommandInteractionDataOption, name string) string {
|
||||
for _, opt := range sub.Options {
|
||||
return stringOption(sub.Options, name)
|
||||
}
|
||||
|
||||
// stringOption looks up a named string-valued option in a flat options slice -
|
||||
// used directly for flat commands like /speak, and via requiredStringOption for
|
||||
// commands nested one level under a subcommand.
|
||||
func stringOption(opts []*discordgo.ApplicationCommandInteractionDataOption, name string) string {
|
||||
for _, opt := range opts {
|
||||
if opt.Name == name {
|
||||
return opt.StringValue()
|
||||
}
|
||||
@ -456,6 +547,12 @@ func requiredStringOption(sub *discordgo.ApplicationCommandInteractionDataOption
|
||||
return ""
|
||||
}
|
||||
|
||||
// isSubcommandOption reports whether opt represents a subcommand/subcommand-group
|
||||
// node, as opposed to a plain value option (e.g. a flat command's own arguments).
|
||||
func isSubcommandOption(opt *discordgo.ApplicationCommandInteractionDataOption) bool {
|
||||
return opt.Type == discordgo.ApplicationCommandOptionSubCommand || opt.Type == discordgo.ApplicationCommandOptionSubCommandGroup
|
||||
}
|
||||
|
||||
func optionalUint32Option(sub *discordgo.ApplicationCommandInteractionDataOption, name string) *uint32 {
|
||||
for _, opt := range sub.Options {
|
||||
if opt.Name == name {
|
||||
@ -468,6 +565,9 @@ func optionalUint32Option(sub *discordgo.ApplicationCommandInteractionDataOption
|
||||
|
||||
func focusedOptionValue(data discordgo.ApplicationCommandInteractionData) string {
|
||||
for _, sub := range data.Options {
|
||||
if sub.Focused {
|
||||
return sub.StringValue()
|
||||
}
|
||||
if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup {
|
||||
for _, nested := range sub.Options {
|
||||
for _, opt := range nested.Options {
|
||||
@ -488,6 +588,9 @@ func focusedOptionValue(data discordgo.ApplicationCommandInteractionData) string
|
||||
|
||||
func focusedOptionName(data discordgo.ApplicationCommandInteractionData) string {
|
||||
for _, sub := range data.Options {
|
||||
if sub.Focused {
|
||||
return sub.Name
|
||||
}
|
||||
if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup {
|
||||
for _, nested := range sub.Options {
|
||||
for _, opt := range nested.Options {
|
||||
|
||||
108
discord-bot/internal/adapters/primary/discord/handler_test.go
Normal file
108
discord-bot/internal/adapters/primary/discord/handler_test.go
Normal file
@ -0,0 +1,108 @@
|
||||
package discord
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/bwmarrin/discordgo"
|
||||
)
|
||||
|
||||
func TestStringOption(t *testing.T) {
|
||||
opts := []*discordgo.ApplicationCommandInteractionDataOption{
|
||||
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice Shower"},
|
||||
{Name: "text", Type: discordgo.ApplicationCommandOptionString, Value: "hello"},
|
||||
}
|
||||
|
||||
if got := stringOption(opts, "speaker"); got != "Rice Shower" {
|
||||
t.Fatalf("stringOption(speaker) = %q, want %q", got, "Rice Shower")
|
||||
}
|
||||
if got := stringOption(opts, "missing"); got != "" {
|
||||
t.Fatalf("stringOption(missing) = %q, want empty", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsSubcommandOption(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
typ discordgo.ApplicationCommandOptionType
|
||||
want bool
|
||||
}{
|
||||
{name: "subcommand", typ: discordgo.ApplicationCommandOptionSubCommand, want: true},
|
||||
{name: "subcommand group", typ: discordgo.ApplicationCommandOptionSubCommandGroup, want: true},
|
||||
{name: "plain string option", typ: discordgo.ApplicationCommandOptionString, want: false},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
got := isSubcommandOption(&discordgo.ApplicationCommandInteractionDataOption{Type: tt.typ})
|
||||
if got != tt.want {
|
||||
t.Fatalf("isSubcommandOption(%v) = %v, want %v", tt.typ, got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestFocusedOptionNameAndValue(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
data discordgo.ApplicationCommandInteractionData
|
||||
wantName string
|
||||
wantValue string
|
||||
}{
|
||||
{
|
||||
name: "flat command - top-level option focused",
|
||||
data: discordgo.ApplicationCommandInteractionData{
|
||||
Name: "speak",
|
||||
Options: []*discordgo.ApplicationCommandInteractionDataOption{
|
||||
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice", Focused: true},
|
||||
{Name: "text", Type: discordgo.ApplicationCommandOptionString, Value: "hi"},
|
||||
},
|
||||
},
|
||||
wantName: "speaker",
|
||||
wantValue: "Rice",
|
||||
},
|
||||
{
|
||||
name: "nested subcommand option focused (regression)",
|
||||
data: discordgo.ApplicationCommandInteractionData{
|
||||
Name: "ai",
|
||||
Options: []*discordgo.ApplicationCommandInteractionDataOption{
|
||||
{
|
||||
Name: "model",
|
||||
Type: discordgo.ApplicationCommandOptionSubCommandGroup,
|
||||
Options: []*discordgo.ApplicationCommandInteractionDataOption{
|
||||
{
|
||||
Name: "set",
|
||||
Type: discordgo.ApplicationCommandOptionSubCommand,
|
||||
Options: []*discordgo.ApplicationCommandInteractionDataOption{
|
||||
{Name: "name", Type: discordgo.ApplicationCommandOptionString, Value: "llama", Focused: true},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
wantName: "name",
|
||||
wantValue: "llama",
|
||||
},
|
||||
{
|
||||
name: "nothing focused",
|
||||
data: discordgo.ApplicationCommandInteractionData{
|
||||
Name: "speak",
|
||||
Options: []*discordgo.ApplicationCommandInteractionDataOption{
|
||||
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice"},
|
||||
},
|
||||
},
|
||||
wantName: "",
|
||||
wantValue: "",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if got := focusedOptionName(tt.data); got != tt.wantName {
|
||||
t.Fatalf("focusedOptionName() = %q, want %q", got, tt.wantName)
|
||||
}
|
||||
if got := focusedOptionValue(tt.data); got != tt.wantValue {
|
||||
t.Fatalf("focusedOptionValue() = %q, want %q", got, tt.wantValue)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@ -207,6 +207,25 @@ func RegisterCommands(s *discordgo.Session, guildID string) error {
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
Name: "speak",
|
||||
Description: "Speak text in your current voice channel",
|
||||
Options: []*discordgo.ApplicationCommandOption{
|
||||
{
|
||||
Type: discordgo.ApplicationCommandOptionString,
|
||||
Name: "speaker",
|
||||
Description: "Speaker voice",
|
||||
Required: true,
|
||||
Autocomplete: true,
|
||||
},
|
||||
{
|
||||
Type: discordgo.ApplicationCommandOptionString,
|
||||
Name: "text",
|
||||
Description: "Text to speak",
|
||||
Required: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
if _, err := s.ApplicationCommandBulkOverwrite(appID, guildID, commands); err != nil {
|
||||
|
||||
131
discord-bot/internal/adapters/primary/discord/voice.go
Normal file
131
discord-bot/internal/adapters/primary/discord/voice.go
Normal file
@ -0,0 +1,131 @@
|
||||
package discord
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"time"
|
||||
|
||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
|
||||
"github.com/bwmarrin/discordgo"
|
||||
"github.com/jonas747/dca"
|
||||
)
|
||||
|
||||
// ErrNotInVoiceChannel is returned when the invoking user isn't in a voice channel.
|
||||
var ErrNotInVoiceChannel = errors.New("join a voice channel first")
|
||||
|
||||
// resolveVoiceChannel returns the voice channel the given guild member currently
|
||||
// occupies, or ErrNotInVoiceChannel if they aren't in one.
|
||||
func resolveVoiceChannel(s *discordgo.Session, guildID, userID string) (string, error) {
|
||||
vs, err := s.State.VoiceState(guildID, userID)
|
||||
if err != nil {
|
||||
return "", ErrNotInVoiceChannel
|
||||
}
|
||||
return vs.ChannelID, nil
|
||||
}
|
||||
|
||||
// speak joins the invoking user's current voice channel, transcodes audio (as
|
||||
// returned by tts-gateway, e.g. AAC) to Opus via ffmpeg/dca, streams it, then
|
||||
// leaves. This is Discord-specific playback plumbing that the transport-agnostic
|
||||
// app layer has no business knowing about - see internal/app/command.go's
|
||||
// HandleSpeak for the RPC half of this feature.
|
||||
func speak(ctx context.Context, s *discordgo.Session, guildID, userID string, audio []byte, mimeType string) error {
|
||||
log := logger.FromContext(ctx).With("mime_type", mimeType)
|
||||
|
||||
channelID, err := resolveVoiceChannel(s, guildID, userID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
vc, err := s.ChannelVoiceJoin(guildID, channelID, false, true)
|
||||
if err != nil {
|
||||
return fmt.Errorf("join voice channel: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if err := vc.Disconnect(); err != nil {
|
||||
log.Error("voice disconnect failed", "error", err.Error())
|
||||
}
|
||||
}()
|
||||
|
||||
if err := waitForVoiceReady(ctx, vc); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// dca shells out to ffmpeg, which needs a seekable input for containers like the
|
||||
// AAC/M4A tts-gateway returns - its own moov atom sits at the end of the file (see
|
||||
// tts-gateway/internal/adapters/secondary/ffmpeg/ffmpeg.go), so a non-seekable pipe
|
||||
// input would fail. Write to a temp file first, mirroring that same adapter's idiom
|
||||
// on the output side.
|
||||
tmpPath, err := writeTempAudio(audio)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.Remove(tmpPath)
|
||||
|
||||
encodeSession, err := dca.EncodeFile(tmpPath, dca.StdEncodeOptions)
|
||||
if err != nil {
|
||||
return fmt.Errorf("encode audio to opus: %w", err)
|
||||
}
|
||||
defer encodeSession.Cleanup()
|
||||
|
||||
if err := vc.Speaking(true); err != nil {
|
||||
return fmt.Errorf("set speaking state: %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if err := vc.Speaking(false); err != nil {
|
||||
log.Error("clear speaking state failed", "error", err.Error())
|
||||
}
|
||||
}()
|
||||
|
||||
done := make(chan error, 1)
|
||||
dca.NewStream(encodeSession, vc, done)
|
||||
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil && err != io.EOF {
|
||||
return fmt.Errorf("stream audio: %w", err)
|
||||
}
|
||||
return nil
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
}
|
||||
}
|
||||
|
||||
// writeTempAudio persists synthesized audio to a temp file for ffmpeg to read from.
|
||||
func writeTempAudio(audio []byte) (string, error) {
|
||||
tmpFile, err := os.CreateTemp("", "discord-bot-speak-*")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("create temp audio file: %w", err)
|
||||
}
|
||||
tmpPath := tmpFile.Name()
|
||||
|
||||
if _, err := tmpFile.Write(audio); err != nil {
|
||||
_ = tmpFile.Close()
|
||||
os.Remove(tmpPath)
|
||||
return "", fmt.Errorf("write temp audio file: %w", err)
|
||||
}
|
||||
if err := tmpFile.Close(); err != nil {
|
||||
os.Remove(tmpPath)
|
||||
return "", fmt.Errorf("close temp audio file: %w", err)
|
||||
}
|
||||
return tmpPath, nil
|
||||
}
|
||||
|
||||
// waitForVoiceReady polls until the voice connection's UDP socket is ready to
|
||||
// send, or ctx is done - discordgo has no blocking/channel-based signal for this.
|
||||
func waitForVoiceReady(ctx context.Context, vc *discordgo.VoiceConnection) error {
|
||||
ticker := time.NewTicker(50 * time.Millisecond)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
if vc.Ready {
|
||||
return nil
|
||||
}
|
||||
select {
|
||||
case <-ticker.C:
|
||||
case <-ctx.Done():
|
||||
return fmt.Errorf("wait for voice connection ready: %w", ctx.Err())
|
||||
}
|
||||
}
|
||||
}
|
||||
73
discord-bot/internal/adapters/primary/discord/voice_test.go
Normal file
73
discord-bot/internal/adapters/primary/discord/voice_test.go
Normal file
@ -0,0 +1,73 @@
|
||||
package discord
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"github.com/bwmarrin/discordgo"
|
||||
)
|
||||
|
||||
func newTestSession(t *testing.T, guildID string, voiceStates []*discordgo.VoiceState) *discordgo.Session {
|
||||
t.Helper()
|
||||
state := discordgo.NewState()
|
||||
if err := state.GuildAdd(&discordgo.Guild{ID: guildID, VoiceStates: voiceStates}); err != nil {
|
||||
t.Fatalf("GuildAdd() error = %v", err)
|
||||
}
|
||||
return &discordgo.Session{State: state}
|
||||
}
|
||||
|
||||
func TestResolveVoiceChannel(t *testing.T) {
|
||||
const guildID = "guild-1"
|
||||
const userID = "user-1"
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
voiceStates []*discordgo.VoiceState
|
||||
userID string
|
||||
wantChannel string
|
||||
wantErr error
|
||||
}{
|
||||
{
|
||||
name: "user in a voice channel",
|
||||
voiceStates: []*discordgo.VoiceState{
|
||||
{GuildID: guildID, UserID: userID, ChannelID: "channel-1"},
|
||||
},
|
||||
userID: userID,
|
||||
wantChannel: "channel-1",
|
||||
},
|
||||
{
|
||||
name: "user not in any voice channel",
|
||||
voiceStates: nil,
|
||||
userID: userID,
|
||||
wantErr: ErrNotInVoiceChannel,
|
||||
},
|
||||
{
|
||||
name: "user in a different guild's voice channel is not found",
|
||||
voiceStates: []*discordgo.VoiceState{
|
||||
{GuildID: guildID, UserID: "someone-else", ChannelID: "channel-1"},
|
||||
},
|
||||
userID: userID,
|
||||
wantErr: ErrNotInVoiceChannel,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
session := newTestSession(t, guildID, tt.voiceStates)
|
||||
|
||||
got, err := resolveVoiceChannel(session, guildID, tt.userID)
|
||||
if tt.wantErr != nil {
|
||||
if !errors.Is(err, tt.wantErr) {
|
||||
t.Fatalf("resolveVoiceChannel() error = %v, want %v", err, tt.wantErr)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("resolveVoiceChannel() error = %v", err)
|
||||
}
|
||||
if got != tt.wantChannel {
|
||||
t.Fatalf("resolveVoiceChannel() = %q, want %q", got, tt.wantChannel)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
124
discord-bot/internal/adapters/secondary/ttsgateway/client.go
Normal file
124
discord-bot/internal/adapters/secondary/ttsgateway/client.go
Normal file
@ -0,0 +1,124 @@
|
||||
package ttsgateway
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
|
||||
ttsv1 "gitea.nik4nao.com/nik/home-services/gen/tts/v1"
|
||||
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
|
||||
"google.golang.org/grpc"
|
||||
"google.golang.org/grpc/credentials"
|
||||
"google.golang.org/grpc/credentials/insecure"
|
||||
)
|
||||
|
||||
// Client implements the app's TTS driven port over gRPC.
|
||||
type Client struct {
|
||||
conn *grpc.ClientConn
|
||||
client ttsv1.TTSServiceClient
|
||||
log *slog.Logger
|
||||
}
|
||||
|
||||
// New constructs a gRPC client for the internal tts-gateway service.
|
||||
func New(ctx context.Context, addr, tlsDir string, log *slog.Logger) (*Client, error) {
|
||||
transportCreds := insecure.NewCredentials()
|
||||
if tlsDir != "" {
|
||||
creds, err := loadTransportCredentials(tlsDir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("load mTLS credentials: %w", err)
|
||||
}
|
||||
transportCreds = creds
|
||||
}
|
||||
|
||||
conn, err := grpc.NewClient(
|
||||
addr,
|
||||
grpc.WithTransportCredentials(transportCreds),
|
||||
grpc.WithStatsHandler(otelgrpc.NewClientHandler()),
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("dial tts-gateway: %w", err)
|
||||
}
|
||||
|
||||
return &Client{
|
||||
conn: conn,
|
||||
client: ttsv1.NewTTSServiceClient(conn),
|
||||
log: log,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Close closes the underlying gRPC connection.
|
||||
func (c *Client) Close() error {
|
||||
if err := c.conn.Close(); err != nil {
|
||||
return fmt.Errorf("close tts-gateway client: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Synthesize forwards a speech synthesis request to tts-gateway. noise_scale/
|
||||
// noise_scale_w/length_scale are left unset so tts-gateway applies its own defaults.
|
||||
func (c *Client) Synthesize(ctx context.Context, speakerName, text string) ([]byte, string, error) {
|
||||
start := time.Now()
|
||||
log := logger.FromContext(ctx).With("grpc.method", "TTSService/Synthesize")
|
||||
resp, err := c.client.Synthesize(ctx, &ttsv1.SynthesizeRequest{
|
||||
SpeakerName: speakerName,
|
||||
Text: text,
|
||||
})
|
||||
if err != nil {
|
||||
log.Error("grpc call failed",
|
||||
"duration_ms", time.Since(start).Milliseconds(),
|
||||
"error", err.Error(),
|
||||
)
|
||||
return nil, "", fmt.Errorf("synthesize speech: %w", err)
|
||||
}
|
||||
log.Debug("grpc call completed", "duration_ms", time.Since(start).Milliseconds())
|
||||
return resp.GetAudio(), resp.GetMimeType(), nil
|
||||
}
|
||||
|
||||
// ListSpeakers returns the speaker roster from tts-gateway, optionally filtered by search.
|
||||
func (c *Client) ListSpeakers(ctx context.Context, search string) ([]string, error) {
|
||||
start := time.Now()
|
||||
log := logger.FromContext(ctx).With("grpc.method", "TTSService/ListSpeakers")
|
||||
resp, err := c.client.ListSpeakers(ctx, &ttsv1.ListSpeakersRequest{Search: search})
|
||||
if err != nil {
|
||||
log.Error("grpc call failed",
|
||||
"duration_ms", time.Since(start).Milliseconds(),
|
||||
"error", err.Error(),
|
||||
)
|
||||
return nil, fmt.Errorf("list tts-gateway speakers: %w", err)
|
||||
}
|
||||
log.Debug("grpc call completed", "duration_ms", time.Since(start).Milliseconds())
|
||||
return append([]string(nil), resp.GetSpeakerNames()...), nil
|
||||
}
|
||||
|
||||
func loadTransportCredentials(tlsDir string) (credentials.TransportCredentials, error) {
|
||||
cert, err := tls.LoadX509KeyPair(
|
||||
filepath.Join(tlsDir, "tls.crt"),
|
||||
filepath.Join(tlsDir, "tls.key"),
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("load client key pair: %w", err)
|
||||
}
|
||||
|
||||
caPEM, err := os.ReadFile(filepath.Join(tlsDir, "ca.crt"))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read server CA: %w", err)
|
||||
}
|
||||
|
||||
rootCAs := x509.NewCertPool()
|
||||
if !rootCAs.AppendCertsFromPEM(caPEM) {
|
||||
return nil, fmt.Errorf("append server CA: invalid PEM")
|
||||
}
|
||||
|
||||
return credentials.NewTLS(&tls.Config{
|
||||
Certificates: []tls.Certificate{cert},
|
||||
RootCAs: rootCAs,
|
||||
ServerName: "tts-gateway.home-services.svc.cluster.local",
|
||||
MinVersion: tls.VersionTLS13,
|
||||
}), nil
|
||||
}
|
||||
@ -23,11 +23,12 @@ type CommandApp struct {
|
||||
ai driven.AIGateway
|
||||
models *modelstore.Store
|
||||
validator *modelvalidator.Validator
|
||||
tts driven.TTSGateway
|
||||
}
|
||||
|
||||
// NewCommandApp constructs the Discord command application service.
|
||||
func NewCommandApp(ha driven.HAGateway, ai driven.AIGateway, models *modelstore.Store, validator *modelvalidator.Validator) *CommandApp {
|
||||
return &CommandApp{ha: ha, ai: ai, models: models, validator: validator}
|
||||
func NewCommandApp(ha driven.HAGateway, ai driven.AIGateway, models *modelstore.Store, validator *modelvalidator.Validator, tts driven.TTSGateway) *CommandApp {
|
||||
return &CommandApp{ha: ha, ai: ai, models: models, validator: validator, tts: tts}
|
||||
}
|
||||
|
||||
// HandleLightList formats discovered lights into a monospace-friendly response.
|
||||
@ -279,6 +280,33 @@ func (a *CommandApp) HandleAIModelList(ctx context.Context) (string, error) {
|
||||
return strings.Join(lines, "\n"), nil
|
||||
}
|
||||
|
||||
// HandleSpeak synthesizes speech via tts-gateway for the invoking Discord adapter
|
||||
// to play back in a voice channel; the app layer stays transport-agnostic and
|
||||
// leaves voice-channel playback itself to the Discord adapter.
|
||||
func (a *CommandApp) HandleSpeak(ctx context.Context, speakerName, text string) ([]byte, string, error) {
|
||||
audio, mimeType, err := a.tts.Synthesize(ctx, speakerName, text)
|
||||
if err != nil {
|
||||
return nil, "", fmt.Errorf("handle speak: %w", err)
|
||||
}
|
||||
return audio, mimeType, nil
|
||||
}
|
||||
|
||||
// AutocompleteSpeakers returns the full tts-gateway speaker roster for the /speak
|
||||
// command; handleAutocomplete's existing substring filter narrows it client-side,
|
||||
// same as the light/switch/ac autocompletes above.
|
||||
func (a *CommandApp) AutocompleteSpeakers(ctx context.Context) ([]Choice, error) {
|
||||
speakers, err := a.tts.ListSpeakers(ctx, "")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("autocomplete speakers: %w", err)
|
||||
}
|
||||
|
||||
choices := make([]Choice, 0, len(speakers))
|
||||
for _, speaker := range speakers {
|
||||
choices = append(choices, Choice{Label: speaker, Value: speaker})
|
||||
}
|
||||
return choices, nil
|
||||
}
|
||||
|
||||
// AutocompleteAIModels returns model names for the /ai model set command.
|
||||
func (a *CommandApp) AutocompleteAIModels(ctx context.Context) ([]Choice, error) {
|
||||
models, err := a.validator.Known(ctx)
|
||||
|
||||
@ -35,6 +35,11 @@ type mockAIGateway struct {
|
||||
listModelsFunc func(ctx context.Context) ([]string, error)
|
||||
}
|
||||
|
||||
type mockTTSGateway struct {
|
||||
synthesizeFunc func(ctx context.Context, speakerName, text string) ([]byte, string, error)
|
||||
listSpeakersFunc func(ctx context.Context, search string) ([]string, error)
|
||||
}
|
||||
|
||||
func (m *mockHAGateway) ListLights(ctx context.Context) ([]driven.Light, error) {
|
||||
if m.listLightsFunc == nil {
|
||||
return nil, nil
|
||||
@ -147,8 +152,22 @@ func (m *mockAIGateway) ListModels(ctx context.Context) ([]string, error) {
|
||||
return m.listModelsFunc(ctx)
|
||||
}
|
||||
|
||||
func (m *mockTTSGateway) Synthesize(ctx context.Context, speakerName, text string) ([]byte, string, error) {
|
||||
if m.synthesizeFunc == nil {
|
||||
return nil, "", nil
|
||||
}
|
||||
return m.synthesizeFunc(ctx, speakerName, text)
|
||||
}
|
||||
|
||||
func (m *mockTTSGateway) ListSpeakers(ctx context.Context, search string) ([]string, error) {
|
||||
if m.listSpeakersFunc == nil {
|
||||
return nil, nil
|
||||
}
|
||||
return m.listSpeakersFunc(ctx, search)
|
||||
}
|
||||
|
||||
func newTestCommandApp(ha *mockHAGateway, ai *mockAIGateway) *CommandApp {
|
||||
return NewCommandApp(ha, ai, modelstore.New(), modelvalidator.New(ai, time.Minute))
|
||||
return NewCommandApp(ha, ai, modelstore.New(), modelvalidator.New(ai, time.Minute), &mockTTSGateway{})
|
||||
}
|
||||
|
||||
func TestCommandAppHandleLightList(t *testing.T) {
|
||||
@ -1081,7 +1100,7 @@ func TestCommandAppHandleAIQuery(t *testing.T) {
|
||||
}
|
||||
return "Turning on Kitchen.", "llama3:latest", nil
|
||||
},
|
||||
}, store, modelvalidator.New(&mockAIGateway{}, time.Minute))
|
||||
}, store, modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{})
|
||||
|
||||
got, err := app.HandleAIQuery(context.Background(), "turn on kitchen")
|
||||
if err != nil {
|
||||
@ -1092,6 +1111,116 @@ func TestCommandAppHandleAIQuery(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCommandAppHandleSpeak(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
speakerName string
|
||||
text string
|
||||
audio []byte
|
||||
mimeType string
|
||||
synthErr error
|
||||
wantErr string
|
||||
}{
|
||||
{
|
||||
name: "synthesize success",
|
||||
speakerName: "Rice Shower",
|
||||
text: "おはようございます",
|
||||
audio: []byte{0x00, 0x01, 0x02},
|
||||
mimeType: "audio/aac",
|
||||
},
|
||||
{
|
||||
name: "synthesize error propagated",
|
||||
speakerName: "Unknown Speaker",
|
||||
text: "hello",
|
||||
synthErr: errors.New("boom"),
|
||||
wantErr: "handle speak: boom",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
var gotSpeaker, gotText string
|
||||
app := NewCommandApp(&mockHAGateway{}, &mockAIGateway{}, modelstore.New(), modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{
|
||||
synthesizeFunc: func(ctx context.Context, speakerName, text string) ([]byte, string, error) {
|
||||
gotSpeaker, gotText = speakerName, text
|
||||
if tt.synthErr != nil {
|
||||
return nil, "", tt.synthErr
|
||||
}
|
||||
return tt.audio, tt.mimeType, nil
|
||||
},
|
||||
})
|
||||
|
||||
audio, mimeType, err := app.HandleSpeak(context.Background(), tt.speakerName, tt.text)
|
||||
if tt.wantErr != "" {
|
||||
if err == nil || err.Error() != tt.wantErr {
|
||||
t.Fatalf("HandleSpeak() error = %v, want %q", err, tt.wantErr)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("HandleSpeak() error = %v", err)
|
||||
}
|
||||
if gotSpeaker != tt.speakerName || gotText != tt.text {
|
||||
t.Fatalf("Synthesize() called with (%q, %q), want (%q, %q)", gotSpeaker, gotText, tt.speakerName, tt.text)
|
||||
}
|
||||
if !reflect.DeepEqual(audio, tt.audio) || mimeType != tt.mimeType {
|
||||
t.Fatalf("HandleSpeak() = (%#v, %q), want (%#v, %q)", audio, mimeType, tt.audio, tt.mimeType)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCommandAppAutocompleteSpeakers(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
speakers []string
|
||||
listErr error
|
||||
want []Choice
|
||||
wantErr string
|
||||
}{
|
||||
{
|
||||
name: "maps speaker names to choices",
|
||||
speakers: []string{"Rice Shower", "Special Week"},
|
||||
want: []Choice{
|
||||
{Label: "Rice Shower", Value: "Rice Shower"},
|
||||
{Label: "Special Week", Value: "Special Week"},
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "ListSpeakers error",
|
||||
listErr: errors.New("boom"),
|
||||
wantErr: "autocomplete speakers: boom",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
app := NewCommandApp(&mockHAGateway{}, &mockAIGateway{}, modelstore.New(), modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{
|
||||
listSpeakersFunc: func(ctx context.Context, search string) ([]string, error) {
|
||||
if tt.listErr != nil {
|
||||
return nil, tt.listErr
|
||||
}
|
||||
return tt.speakers, nil
|
||||
},
|
||||
})
|
||||
|
||||
got, err := app.AutocompleteSpeakers(context.Background())
|
||||
if tt.wantErr != "" {
|
||||
if err == nil || err.Error() != tt.wantErr {
|
||||
t.Fatalf("AutocompleteSpeakers() error = %v, want %q", err, tt.wantErr)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("AutocompleteSpeakers() error = %v", err)
|
||||
}
|
||||
if !reflect.DeepEqual(got, tt.want) {
|
||||
t.Fatalf("AutocompleteSpeakers() = %#v, want %#v", got, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCommandAppHandleAIModelSet(t *testing.T) {
|
||||
app := newTestCommandApp(&mockHAGateway{}, &mockAIGateway{
|
||||
listModelsFunc: func(ctx context.Context) ([]string, error) {
|
||||
|
||||
@ -7,14 +7,15 @@ import (
|
||||
|
||||
// Config holds runtime configuration for the Discord bot process.
|
||||
type Config struct {
|
||||
DiscordToken string
|
||||
GuildID string
|
||||
HAGatewayAddr string
|
||||
AIGatewayAddr string
|
||||
TLSDir string
|
||||
OTELEndpoint string
|
||||
LogLevel string
|
||||
LogFormat string
|
||||
DiscordToken string
|
||||
GuildID string
|
||||
HAGatewayAddr string
|
||||
AIGatewayAddr string
|
||||
TTSGatewayAddr string
|
||||
TLSDir string
|
||||
OTELEndpoint string
|
||||
LogLevel string
|
||||
LogFormat string
|
||||
}
|
||||
|
||||
// Load reads Discord bot configuration from environment variables.
|
||||
@ -30,14 +31,15 @@ func Load() (*Config, error) {
|
||||
}
|
||||
|
||||
return &Config{
|
||||
DiscordToken: token,
|
||||
GuildID: os.Getenv("GUILD_ID"),
|
||||
HAGatewayAddr: addr,
|
||||
AIGatewayAddr: getenvDefault("AI_GATEWAY_ADDR", "ai-gateway.home-services.svc.cluster.local:50052"),
|
||||
TLSDir: os.Getenv("TLS_DIR"),
|
||||
OTELEndpoint: os.Getenv("OTEL_ENDPOINT"),
|
||||
LogLevel: getenvDefault("LOG_LEVEL", "info"),
|
||||
LogFormat: getenvDefault("LOG_FORMAT", "json"),
|
||||
DiscordToken: token,
|
||||
GuildID: os.Getenv("GUILD_ID"),
|
||||
HAGatewayAddr: addr,
|
||||
AIGatewayAddr: getenvDefault("AI_GATEWAY_ADDR", "ai-gateway.home-services.svc.cluster.local:50052"),
|
||||
TTSGatewayAddr: getenvDefault("TTS_GATEWAY_ADDR", "tts-gateway.home-services.svc.cluster.local:50053"),
|
||||
TLSDir: os.Getenv("TLS_DIR"),
|
||||
OTELEndpoint: os.Getenv("OTEL_ENDPOINT"),
|
||||
LogLevel: getenvDefault("LOG_LEVEL", "info"),
|
||||
LogFormat: getenvDefault("LOG_FORMAT", "json"),
|
||||
}, nil
|
||||
}
|
||||
|
||||
|
||||
9
discord-bot/internal/core/ports/driven/tts.go
Normal file
9
discord-bot/internal/core/ports/driven/tts.go
Normal file
@ -0,0 +1,9 @@
|
||||
package driven
|
||||
|
||||
import "context"
|
||||
|
||||
// TTSGateway exposes the text-to-speech synthesis API used by the Discord bot.
|
||||
type TTSGateway interface {
|
||||
Synthesize(ctx context.Context, speakerName, text string) (audio []byte, mimeType string, err error)
|
||||
ListSpeakers(ctx context.Context, search string) ([]string, error)
|
||||
}
|
||||
90
handoff.md
Normal file
90
handoff.md
Normal file
@ -0,0 +1,90 @@
|
||||
# tts-gateway Handoff
|
||||
|
||||
Status as of 2026-07-25: Phases 0–6 of `TTS_GATEWAY_PLAN.md` are done, and the deployed service is
|
||||
confirmed working in production (real `grpcurl` call through `kubectl port-forward`, real GPU
|
||||
inference, real playable audio). This file is a punch list for picking the remaining work up in a
|
||||
fresh session — it doesn't re-explain things that are already documented elsewhere; it points to
|
||||
where.
|
||||
|
||||
**Update (later same day):** items 1 and 3 below are now done — model artifact distribution is a
|
||||
versioned image + k8s `initContainer` (see `tts-gateway/README.md`), and `discord-bot` has a
|
||||
working `/speak` command (see `TTS_GATEWAY_PLAN.md` Phase 6). Only item 2 (mTLS) remains, left
|
||||
untouched per an explicit decision to verify these two first. The two "what's left" entries below
|
||||
are kept as-written for their historical reasoning/decision trail rather than rewritten in place.
|
||||
|
||||
## Where things stand
|
||||
|
||||
- `tts-gateway` (Go) + `tts-sidecar` (Python/libtorch) are both built, containerized, and running
|
||||
in Kubernetes (`home-services` namespace).
|
||||
- CI (`.gitea/workflows/ci.yaml`) builds and pushes both images on every push to `main`, gated by
|
||||
path-filtered per-service rebuilds and using registry-based Docker layer caching.
|
||||
- Full history — the Phase 0 ONNX-export/g2p investigation, why there's a Python sidecar instead
|
||||
of pure Go, the exact `japanese_cleaners` pipeline verification — is in `TTS_GATEWAY_PLAN.md`
|
||||
and `tmp/reference/uma-tts-api/spike/FINDINGS.md`. Read those before re-deriving anything; the
|
||||
ONNX export failure in particular took several iterations to characterize correctly and isn't
|
||||
worth re-investigating from scratch.
|
||||
|
||||
## What's left
|
||||
|
||||
1. **Model artifact distribution isn't automated.** The k8s Deployment
|
||||
(`~/repo/homelab/manifests/home-services/tts-gateway.yaml` — separate repo, see below)
|
||||
`hostPath`-mounts `/data/tts-gateway` on the `nik-gpu` node into the sidecar container.
|
||||
Someone has to manually place `G_790000.pth` + `uma.json` there before the pod goes Ready — no
|
||||
init-container download, no PVC, nothing automated. `TTS_GATEWAY_PLAN.md`'s Phase 5 flagged
|
||||
this as an open decision; it's still open, just worked around. If asked to fix it: options are
|
||||
a PVC populated by an init-container download step, baking the checkpoint into a private image
|
||||
layer, or similar — weigh against the checkpoint being ~455MB and not something to casually put
|
||||
in a git-tracked Dockerfile context.
|
||||
|
||||
2. **mTLS is off.** `tts-gateway.yaml` comments out the `TLS_DIR` env var and its volume mount,
|
||||
for plaintext `grpcurl` testing from outside the cluster during initial rollout — a live TODO,
|
||||
not a permanent decision. Every other service in this repo assumes internal-network-or-mTLS as
|
||||
its only access boundary (`CLAUDE.md`'s "Configuration Notes"). Re-enabling: uncomment the env
|
||||
var + mount in that manifest, and set up a `tts-gateway-tls` secret the same way
|
||||
`ha-gateway-sealed.yaml`/`ha-gateway-secret.sh` do it for ha-gateway (tts-gateway doesn't
|
||||
currently have its own `-secret.sh`/`-sealed.yaml` pair — it'll need one, since unlike
|
||||
ha-gateway/discord-bot it has no other secrets today, so this would be its first).
|
||||
|
||||
3. **Phase 6 (discord-bot integration) not started.** A `/speak`-style Discord slash command
|
||||
calling `tts-gateway`, following the existing pattern — `discord-bot` already has gRPC clients
|
||||
for `ha-gateway` and `ai-gateway` at `discord-bot/internal/adapters/secondary/{gateway,aigateway}`;
|
||||
a new `internal/adapters/secondary/ttsgateway` client would follow the same shape. Needs a
|
||||
product decision first: keep AAC (parity with what's already implemented) or switch to Opus
|
||||
(Discord's native voice codec) — deliberately deferred in `TTS_GATEWAY_PLAN.md`'s open
|
||||
questions until this exact moment, not yet decided.
|
||||
|
||||
## Things worth knowing before touching this
|
||||
|
||||
- **`tts-gateway` and `tts-sidecar` share one Kubernetes Pod**, not two separate Deployments — the
|
||||
Go gateway reaches the sidecar over `localhost:50054`. This mirrors the `--network host` setup
|
||||
documented in `tts-gateway/README.md` for local Docker testing. Don't split them into separate
|
||||
Deployments/Services without also rethinking `INFERENCE_SIDECAR_ADDR` and the networking model.
|
||||
- **The k8s manifests live in a different repo**: `~/repo/homelab/manifests/home-services/` — not
|
||||
in this repo at all. `CLAUDE.md` now has an "Infrastructure / Deployment" section documenting
|
||||
this; check there (and that repo directly) before assuming something isn't deployed just because
|
||||
this repo has no manifests for it.
|
||||
- **`open_jtalk`'s tokenization has an intentional bug that must be preserved.** The Go text
|
||||
normalizer (`tts-gateway/internal/adapters/secondary/jtalk`) maps the cleaned phoneme string to
|
||||
symbol IDs character-by-character, not phoneme-by-phoneme, because that's what the checkpoint
|
||||
was actually trained on (`text/symbols.py`'s "wrong tokens" comment in the original reference
|
||||
implementation). Don't "fix" this without retraining the model — see the comments in `jtalk.go`
|
||||
and `tts-gateway/README.md`.
|
||||
- **ONNX export doesn't work for this checkpoint, and it's not a quick fix.** Read
|
||||
`tmp/reference/uma-tts-api/spike/FINDINGS.md` before re-attempting it — the model's
|
||||
data-dependent output length (predicted phoneme durations determine audio length at runtime)
|
||||
defeats `torch.export`'s guard system in a way that would need an unknown number of per-layer
|
||||
`torch._check` hints to resolve, not a single targeted change.
|
||||
- **CI's build cache needs one "priming" run per image.** If a build job still looks slow after
|
||||
the caching change in `ci.yaml`, check whether a `:buildcache` tag actually exists yet in the
|
||||
registry for that specific image — the first run after the cache was added has nothing to pull
|
||||
from and builds fully fresh.
|
||||
|
||||
## Where to look for more detail
|
||||
|
||||
- `TTS_GATEWAY_PLAN.md` (repo root) — original plan, feasibility analysis, phase breakdown, open
|
||||
product questions (codec, concurrency, auth).
|
||||
- `tmp/reference/uma-tts-api/spike/FINDINGS.md` — Phase 0 spike results with full reasoning.
|
||||
- `tts-gateway/README.md` — service-level docs: gRPC API, config reference, how to run/test
|
||||
locally and on nik-gpu.
|
||||
- `~/repo/homelab/manifests/home-services/tts-gateway.yaml` — actual production config (separate
|
||||
repo, maintained independently — check it directly rather than trusting a stale summary).
|
||||
@ -67,8 +67,33 @@ The server also registers gRPC health checks and reflection.
|
||||
| `PORT` | `50054` | HTTP listen port |
|
||||
|
||||
Neither the checkpoint nor the hparams file is committed to git - both must
|
||||
be supplied at runtime (see below). Model artifact distribution (bind mount
|
||||
vs. build-time download vs. registry) is still an open decision.
|
||||
be supplied at runtime (see below). In production, they're baked into a
|
||||
small versioned image (`tts-gateway/model/Dockerfile`) that a k8s
|
||||
`initContainer` copies into a shared volume before `tts-sidecar` starts -
|
||||
see "Model artifact distribution (production)" below. For local/manual
|
||||
testing on nik-gpu, a plain bind mount (as shown below) is simpler.
|
||||
|
||||
### Model artifact distribution (production)
|
||||
|
||||
CI can never build the model image - the checkpoint (~455MB) and hparams
|
||||
aren't committed to git, and nik-gpu's local disk at `/data/tts-gateway` is
|
||||
the only place they exist. Build and push it manually, directly on nik-gpu
|
||||
(not via `docker --context nik-gpu` from elsewhere - the build context is
|
||||
gathered client-side, so it would look for `/data/tts-gateway` on the wrong
|
||||
machine), whenever the checkpoint or config change:
|
||||
|
||||
```bash
|
||||
ssh nik-gpu
|
||||
docker build -f ~/repo/home-service/tts-gateway/model/Dockerfile \
|
||||
-t gitea.nik4nao.com/nik/tts-model:g790000-v1 /data/tts-gateway
|
||||
docker push gitea.nik4nao.com/nik/tts-model:g790000-v1
|
||||
```
|
||||
|
||||
Bump the tag (`v2`, `v3`, ...) rather than overwriting one - non-`:latest`
|
||||
tags default to `imagePullPolicy: IfNotPresent`, so a node that already
|
||||
pulled a tag won't re-pull an overwritten one. The k8s Deployment
|
||||
(`~/repo/homelab/manifests/home-services/tts-gateway.yaml`) references the
|
||||
tag explicitly in its `model-init` init container.
|
||||
|
||||
## Running And Testing On nik-gpu
|
||||
|
||||
@ -188,8 +213,8 @@ sidecar/ # Python/libtorch inference servic
|
||||
- Concurrency is unbounded on the Go side, but the sidecar's model forward
|
||||
pass is inherently single-threaded per GPU; heavy concurrent load will
|
||||
just queue at the sidecar.
|
||||
- Model artifact distribution isn't finalized - the sidecar currently
|
||||
expects a bind-mounted checkpoint/config, not something the image ships
|
||||
with.
|
||||
- No Discord-bot integration yet (a `/speak`-style command calling this
|
||||
service is a deliberate follow-up, not yet built).
|
||||
- Model artifact distribution to production is a manually-built-and-pushed
|
||||
versioned image (see above), not something CI can automate - the
|
||||
checkpoint only ever existed on nik-gpu's local disk.
|
||||
- `discord-bot`'s `/speak` command calls this service and plays the result
|
||||
in a Discord voice channel (`discord-bot/internal/adapters/primary/discord/voice.go`).
|
||||
|
||||
7
tts-gateway/model/Dockerfile
Normal file
7
tts-gateway/model/Dockerfile
Normal file
@ -0,0 +1,7 @@
|
||||
# One-time (or only-when-the-checkpoint-changes) manually built and pushed by a human on
|
||||
# nik-gpu - CI can never build this: the checkpoint and hparams aren't committed to git
|
||||
# (~455MB, and nik-gpu's local disk at /data/tts-gateway is the only place they exist).
|
||||
# Build CONTEXT must be that directory, not a checkout of this repo - see
|
||||
# tts-gateway/README.md for the exact commands.
|
||||
FROM busybox:1.36
|
||||
COPY G_790000.pth uma.json /models/
|
||||
Loading…
x
Reference in New Issue
Block a user