feat: add /speak command for TTS integration
All checks were successful
CI / changes (push) Successful in 1s
CI / test (push) Successful in 6s
CI / build-ai-gateway (push) Has been skipped
CI / build-ha-gateway (push) Has been skipped
CI / build-discord-bot (push) Successful in 1m33s
CI / build-tts-gateway (push) Successful in 37s
CI / build-tts-sidecar (push) Has been skipped
All checks were successful
CI / changes (push) Successful in 1s
CI / test (push) Successful in 6s
CI / build-ai-gateway (push) Has been skipped
CI / build-ha-gateway (push) Has been skipped
CI / build-discord-bot (push) Successful in 1m33s
CI / build-tts-gateway (push) Successful in 37s
CI / build-tts-sidecar (push) Has been skipped
- Implemented the /speak command in Discord bot to synthesize speech using the TTS gateway. - Added voice handling logic to join voice channels and play synthesized audio. - Created tests for the new command and voice functionalities. - Introduced TTSGateway interface for TTS service communication. - Updated configuration to include TTS gateway address. - Documented the TTS gateway integration and model artifact distribution process.
This commit is contained in:
parent
7851c49dba
commit
0d58e46740
17
CLAUDE.md
17
CLAUDE.md
@ -17,12 +17,14 @@ ai-gateway ------> Ollama
|
|||||||
|
|
|
|
||||||
v
|
v
|
||||||
ha-gateway
|
ha-gateway
|
||||||
|
|
||||||
|
discord-bot -----> tts-gateway -----> tts-sidecar (GPU inference)
|
||||||
```
|
```
|
||||||
|
|
||||||
- **ha-gateway** (port `50051`) — gRPC boundary for Home Assistant. Talks to HA's REST API; implements entity state, light control/discovery, switch control/discovery, and climate (HVAC) control/discovery; also relays SwitchBot Cloud remote commands (`RemoteApp`) when `SWITCHBOT_TOKEN`/`SWITCHBOT_SECRET` are set. Event streaming is stubbed.
|
- **ha-gateway** (port `50051`) — gRPC boundary for Home Assistant. Talks to HA's REST API; implements entity state, light control/discovery, switch control/discovery, and climate (HVAC) control/discovery; also relays SwitchBot Cloud remote commands (`RemoteApp`) when `SWITCHBOT_TOKEN`/`SWITCHBOT_SECRET` are set. Event streaming is stubbed.
|
||||||
- **ai-gateway** (port `50052`) — gRPC service that turns free-form text into home actions. Calls Ollama for intent extraction, resolves intents against a cached light list from `ha-gateway`, and calls `ha-gateway` to execute approved actions.
|
- **ai-gateway** (port `50052`) — gRPC service that turns free-form text into home actions. Calls Ollama for intent extraction, resolves intents against a cached light list from `ha-gateway`, and calls `ha-gateway` to execute approved actions.
|
||||||
- **discord-bot** — registers `/light`, `/switch`, `/ac`, `/ai` slash commands and calls `ha-gateway`/`ai-gateway` via gRPC clients.
|
- **discord-bot** — registers `/light`, `/switch`, `/ac`, `/ai`, `/speak` slash commands and calls `ha-gateway`/`ai-gateway`/`tts-gateway` via gRPC clients. `/speak` synthesizes speech via `tts-gateway` (which returns AAC) and transcodes it to Opus locally (via `ffmpeg`/`github.com/jonas747/dca`) to stream into the invoking user's current voice channel — see `internal/adapters/primary/discord/voice.go`.
|
||||||
- **tts-gateway** (port `50053`) — gRPC text-to-speech service (VITS voice model, 92 Umamusume voices), paired with a Python/libtorch inference sidecar (`tts-gateway/sidecar/`, deployed as a second container in the same pod) that needs an Nvidia GPU — see `TTS_GATEWAY_PLAN.md` for why. Not yet wired into the diagram above; nothing calls it yet (a discord-bot `/speak` command is the planned follow-up). See `tts-gateway/README.md` for its API/config/local-run details.
|
- **tts-gateway** (port `50053`) — gRPC text-to-speech service (VITS voice model, 92 Umamusume voices), paired with a Python/libtorch inference sidecar (`tts-gateway/sidecar/`, deployed as a second container in the same pod) that needs an Nvidia GPU — see `TTS_GATEWAY_PLAN.md` for why. Called by `discord-bot`'s `/speak` command. See `tts-gateway/README.md` for its API/config/local-run details.
|
||||||
|
|
||||||
Each service is a separate Go module (own `go.mod`) joined by `go.work` at the root, plus a `gen` module for shared generated code. Module paths are `gitea.nik4nao.com/nik/home-services/{ha-gateway,ai-gateway,discord-bot,tts-gateway,gen}`.
|
Each service is a separate Go module (own `go.mod`) joined by `go.work` at the root, plus a `gen` module for shared generated code. Module paths are `gitea.nik4nao.com/nik/home-services/{ha-gateway,ai-gateway,discord-bot,tts-gateway,gen}`.
|
||||||
|
|
||||||
@ -172,10 +174,13 @@ rather than trusting this description to stay accurate):
|
|||||||
`nvidia.com/gpu: 1` request/limit on the sidecar container. `strategy: Recreate` instead of the
|
`nvidia.com/gpu: 1` request/limit on the sidecar container. `strategy: Recreate` instead of the
|
||||||
default `RollingUpdate` — nik-gpu only has one allocatable GPU, so a rolling update would
|
default `RollingUpdate` — nik-gpu only has one allocatable GPU, so a rolling update would
|
||||||
deadlock waiting for a GPU still held by the pod it's replacing.
|
deadlock waiting for a GPU still held by the pod it's replacing.
|
||||||
- Model artifact distribution (checkpoint + hparams) is a `hostPath` volume at
|
- Model artifact distribution (checkpoint + hparams) is an `emptyDir` volume populated at pod
|
||||||
`/data/tts-gateway` on the nik-gpu node, not baked into the image or automated — someone has to
|
start by an `initContainers` entry (`model-init`) that copies from a small, versioned image
|
||||||
manually place `G_790000.pth`/`uma.json` there. This is a known-open gap (see `handoff.md`),
|
(`gitea.nik4nao.com/nik/tts-model:<tag>`, built from `tts-gateway/model/Dockerfile`) — not a
|
||||||
not a deliberate final design.
|
raw `hostPath` into the node's disk anymore. That model image still can't be built by CI (the
|
||||||
|
checkpoint isn't committed to git, ~455MB, and only ever existed on nik-gpu's local disk) — it's
|
||||||
|
built and pushed manually, on nik-gpu, whenever the checkpoint/config change. See
|
||||||
|
`tts-gateway/README.md` for the exact commands.
|
||||||
- mTLS (`TLS_DIR`) is currently commented out on `tts-gateway`, for plaintext `grpcurl` testing
|
- mTLS (`TLS_DIR`) is currently commented out on `tts-gateway`, for plaintext `grpcurl` testing
|
||||||
from outside the cluster during initial rollout — a live TODO, not a permanent decision, unlike
|
from outside the cluster during initial rollout — a live TODO, not a permanent decision, unlike
|
||||||
every other service here which assumes mTLS-or-trusted-network as its only access boundary (see
|
every other service here which assumes mTLS-or-trusted-network as its only access boundary (see
|
||||||
|
|||||||
@ -170,16 +170,18 @@ repo's hand-written-mock testing convention rather than golden-byte comparison.
|
|||||||
WAV writer, `ffmpeg` transcode adapter, full `Synthesize` RPC wiring, speaker-not-found →
|
WAV writer, `ffmpeg` transcode adapter, full `Synthesize` RPC wiring, speaker-not-found →
|
||||||
`INVALID_ARGUMENT` (mirroring `app.py`'s current 400 behavior), `ListSpeakers` RPC.
|
`INVALID_ARGUMENT` (mirroring `app.py`'s current 400 behavior), `ListSpeakers` RPC.
|
||||||
|
|
||||||
**Phase 5 — Containerize & deploy to nik-gpu**
|
**Phase 5 — Containerize & deploy to nik-gpu** — **done.** Deployed and confirmed working in
|
||||||
See [GPU/deployment notes](#gpu-and-deployment-notes) below — this phase needs a Dockerfile that
|
production; see `handoff.md` for status. Model artifact distribution is resolved as a small
|
||||||
diverges from the other three services' pattern, plus a decision on how the model artifact
|
versioned image (`tts-gateway/model/Dockerfile`) copied into a shared volume by a k8s
|
||||||
reaches the container.
|
`initContainer`, built manually on nik-gpu (the checkpoint never existed anywhere CI can reach) —
|
||||||
|
see `tts-gateway/README.md`'s "Model artifact distribution (production)" section.
|
||||||
|
|
||||||
**Phase 6 — Client integration (optional, separate follow-up)**
|
**Phase 6 — Client integration** — **done.** `discord-bot` registers a `/speak` command calling
|
||||||
A `/speak` (or similar) Discord command in `discord-bot` calling this gateway, following the
|
this gateway via the same secondary-adapter + gRPC-client pattern used for its
|
||||||
same secondary-adapter + gRPC-client pattern already used for its `ha-gateway`/`ai-gateway`
|
`ha-gateway`/`ai-gateway` clients (`discord-bot/internal/adapters/secondary/ttsgateway/`), then
|
||||||
clients. Not assumed in scope here — a deliberate follow-on decision once the service itself
|
transcodes the AAC response to Opus locally (`ffmpeg` + `github.com/jonas747/dca`) and streams it
|
||||||
works.
|
into the invoking user's current voice channel
|
||||||
|
(`discord-bot/internal/adapters/primary/discord/voice.go`).
|
||||||
|
|
||||||
## GPU and deployment notes
|
## GPU and deployment notes
|
||||||
|
|
||||||
@ -226,9 +228,10 @@ nik-gpu's host state untouched and the setup reproducible from the Dockerfile al
|
|||||||
|
|
||||||
## Open questions (product/design, not technical blockers)
|
## Open questions (product/design, not technical blockers)
|
||||||
|
|
||||||
- **Output codec:** keep AAC (parity with today) or switch to Opus? If the eventual consumer is
|
- **Output codec:** ~~keep AAC (parity with today) or switch to Opus?~~ **Resolved: kept AAC.**
|
||||||
`discord-bot`, Opus is Discord's native voice codec — worth deciding once Phase 6 is actually
|
`tts-gateway` itself is unchanged; `discord-bot` transcodes AAC → Opus locally for voice
|
||||||
on the table, not before.
|
playback (see Phase 6 above), rather than pushing an Opus-specific output format onto every
|
||||||
|
other current/future consumer of this gateway.
|
||||||
- **Concurrency:** the reference implementation serializes all inference behind one lock. Same
|
- **Concurrency:** the reference implementation serializes all inference behind one lock. Same
|
||||||
starting point is reasonable here — revisit only if real latency/throughput needs show up.
|
starting point is reasonable here — revisit only if real latency/throughput needs show up.
|
||||||
- **Auth:** consistent with every other service in this repo, no app-layer authorization —
|
- **Auth:** consistent with every other service in this repo, no app-layer authorization —
|
||||||
|
|||||||
@ -2,6 +2,7 @@ DISCORD_TOKEN=your-bot-token-here
|
|||||||
GUILD_ID=your-guild-id-here
|
GUILD_ID=your-guild-id-here
|
||||||
HA_GATEWAY_ADDR=localhost:50051
|
HA_GATEWAY_ADDR=localhost:50051
|
||||||
AI_GATEWAY_ADDR=localhost:50052
|
AI_GATEWAY_ADDR=localhost:50052
|
||||||
|
TTS_GATEWAY_ADDR=localhost:50053
|
||||||
OTEL_ENDPOINT=
|
OTEL_ENDPOINT=
|
||||||
LOG_LEVEL=info
|
LOG_LEVEL=info
|
||||||
LOG_FORMAT=text
|
LOG_FORMAT=text
|
||||||
|
|||||||
@ -27,6 +27,14 @@ RUN CGO_ENABLED=0 GOOS=linux go build \
|
|||||||
-ldflags="-s -w -X main.version=${VERSION}" \
|
-ldflags="-s -w -X main.version=${VERSION}" \
|
||||||
-o /discord-bot ./cmd/bot
|
-o /discord-bot ./cmd/bot
|
||||||
|
|
||||||
FROM gcr.io/distroless/static:nonroot
|
# /speak needs `ffmpeg` at runtime (github.com/jonas747/dca shells out to it to transcode
|
||||||
|
# tts-gateway's AAC output to Opus for Discord voice), which rules out a distroless base -
|
||||||
|
# same reasoning as tts-gateway/Dockerfile.
|
||||||
|
FROM ubuntu:22.04
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
ffmpeg \
|
||||||
|
ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
COPY --from=builder /discord-bot /discord-bot
|
COPY --from=builder /discord-bot /discord-bot
|
||||||
ENTRYPOINT ["/discord-bot"]
|
ENTRYPOINT ["/discord-bot"]
|
||||||
|
|||||||
@ -14,6 +14,7 @@ import (
|
|||||||
discordadapter "gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/primary/discord"
|
discordadapter "gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/primary/discord"
|
||||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/aigateway"
|
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/aigateway"
|
||||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/gateway"
|
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/gateway"
|
||||||
|
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/adapters/secondary/ttsgateway"
|
||||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/app"
|
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/app"
|
||||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/config"
|
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/config"
|
||||||
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
|
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
|
||||||
@ -42,6 +43,7 @@ func main() {
|
|||||||
"version", version,
|
"version", version,
|
||||||
"ha_gateway_addr", cfg.HAGatewayAddr,
|
"ha_gateway_addr", cfg.HAGatewayAddr,
|
||||||
"ai_gateway_addr", cfg.AIGatewayAddr,
|
"ai_gateway_addr", cfg.AIGatewayAddr,
|
||||||
|
"tts_gateway_addr", cfg.TTSGatewayAddr,
|
||||||
"discord_token", redactToken(cfg.DiscordToken),
|
"discord_token", redactToken(cfg.DiscordToken),
|
||||||
"tls_dir", cfg.TLSDir,
|
"tls_dir", cfg.TLSDir,
|
||||||
"log_level", cfg.LogLevel,
|
"log_level", cfg.LogLevel,
|
||||||
@ -86,9 +88,20 @@ func main() {
|
|||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
|
|
||||||
|
ttsClient, err := ttsgateway.New(ctx, cfg.TTSGatewayAddr, cfg.TLSDir, log)
|
||||||
|
if err != nil {
|
||||||
|
log.Error("tts-gateway client setup failed", "err", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
if err := ttsClient.Close(); err != nil {
|
||||||
|
log.Error("tts-gateway client close failed", "err", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
modelStore := modelstore.New()
|
modelStore := modelstore.New()
|
||||||
validator := modelvalidator.New(aiClient, 30*time.Second)
|
validator := modelvalidator.New(aiClient, 30*time.Second)
|
||||||
commandApp := app.NewCommandApp(haClient, aiClient, modelStore, validator)
|
commandApp := app.NewCommandApp(haClient, aiClient, modelStore, validator, ttsClient)
|
||||||
tracker := discordadapter.NewTracker(context.Background())
|
tracker := discordadapter.NewTracker(context.Background())
|
||||||
|
|
||||||
// Discord-specific wiring stays at the edge so the app layer remains transport-agnostic.
|
// Discord-specific wiring stays at the edge so the app layer remains transport-agnostic.
|
||||||
@ -97,7 +110,9 @@ func main() {
|
|||||||
log.Error("create discord session failed", "err", err)
|
log.Error("create discord session failed", "err", err)
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
session.Identify.Intents = discordgo.IntentsGuilds
|
// GuildVoiceStates is needed to resolve which voice channel the invoking
|
||||||
|
// user is currently in for /speak (see internal/adapters/primary/discord/voice.go).
|
||||||
|
session.Identify.Intents = discordgo.IntentsGuilds | discordgo.IntentsGuildVoiceStates
|
||||||
|
|
||||||
handler := discordadapter.NewHandler(commandApp, tracker)
|
handler := discordadapter.NewHandler(commandApp, tracker)
|
||||||
handler.Register(session)
|
handler.Register(session)
|
||||||
@ -121,6 +136,7 @@ func main() {
|
|||||||
"command_scope", scope,
|
"command_scope", scope,
|
||||||
"ha_gateway_addr", cfg.HAGatewayAddr,
|
"ha_gateway_addr", cfg.HAGatewayAddr,
|
||||||
"ai_gateway_addr", cfg.AIGatewayAddr,
|
"ai_gateway_addr", cfg.AIGatewayAddr,
|
||||||
|
"tts_gateway_addr", cfg.TTSGatewayAddr,
|
||||||
"version", version,
|
"version", version,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@ -6,6 +6,7 @@ require (
|
|||||||
gitea.nik4nao.com/nik/home-services/gen v0.0.0
|
gitea.nik4nao.com/nik/home-services/gen v0.0.0
|
||||||
github.com/bwmarrin/discordgo v0.29.0
|
github.com/bwmarrin/discordgo v0.29.0
|
||||||
github.com/joho/godotenv v1.5.1
|
github.com/joho/godotenv v1.5.1
|
||||||
|
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7
|
||||||
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0
|
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.60.0
|
||||||
go.opentelemetry.io/otel v1.39.0
|
go.opentelemetry.io/otel v1.39.0
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.35.0
|
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.35.0
|
||||||
@ -25,6 +26,7 @@ require (
|
|||||||
github.com/google/uuid v1.6.0 // indirect
|
github.com/google/uuid v1.6.0 // indirect
|
||||||
github.com/gorilla/websocket v1.4.2 // indirect
|
github.com/gorilla/websocket v1.4.2 // indirect
|
||||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 // indirect
|
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 // indirect
|
||||||
|
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757 // indirect
|
||||||
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
|
||||||
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.35.0 // indirect
|
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.35.0 // indirect
|
||||||
go.opentelemetry.io/proto/otlp v1.5.0 // indirect
|
go.opentelemetry.io/proto/otlp v1.5.0 // indirect
|
||||||
|
|||||||
@ -23,6 +23,10 @@ github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1 h1:e9Rjr40Z98/clHv5Yg79Is0Ntos
|
|||||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1/go.mod h1:tIxuGz/9mpox++sgp9fJjHO0+q1X9/UOWd798aAm22M=
|
github.com/grpc-ecosystem/grpc-gateway/v2 v2.26.1/go.mod h1:tIxuGz/9mpox++sgp9fJjHO0+q1X9/UOWd798aAm22M=
|
||||||
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
|
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
|
||||||
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
|
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
|
||||||
|
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7 h1:Iw1pZVDiq4tY8hO7Wt1D1EqDC0BmMw/bFO/Rdt4gs9Q=
|
||||||
|
github.com/jonas747/dca v0.0.0-20210930103944-155f5e5f0cc7/go.mod h1:rxjYX9OJU81unMxQDHChU/lAiOhlY9MV+faPX/NmwLk=
|
||||||
|
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757 h1:Kyv+zTfWIGRNaz/4+lS+CxvuKVZSKFz/6G8E3BKKBRs=
|
||||||
|
github.com/jonas747/ogg v0.0.0-20161220051205-b4f6f4cf3757/go.mod h1:cZnNmdLiLpihzgIVqiaQppi9Ts3D4qF/M45//yW35nI=
|
||||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
|
||||||
|
|||||||
@ -38,10 +38,12 @@ type commandHandler interface {
|
|||||||
HandleAIModelSet(ctx context.Context, name string) (string, error)
|
HandleAIModelSet(ctx context.Context, name string) (string, error)
|
||||||
HandleAIModelGet(ctx context.Context) (string, error)
|
HandleAIModelGet(ctx context.Context) (string, error)
|
||||||
HandleAIModelList(ctx context.Context) (string, error)
|
HandleAIModelList(ctx context.Context) (string, error)
|
||||||
|
HandleSpeak(ctx context.Context, speakerName, text string) (audio []byte, mimeType string, err error)
|
||||||
AutocompleteLights(ctx context.Context) ([]apppkg.Choice, error)
|
AutocompleteLights(ctx context.Context) ([]apppkg.Choice, error)
|
||||||
AutocompleteSwitches(ctx context.Context) ([]apppkg.Choice, error)
|
AutocompleteSwitches(ctx context.Context) ([]apppkg.Choice, error)
|
||||||
AutocompleteClimates(ctx context.Context) ([]apppkg.Choice, error)
|
AutocompleteClimates(ctx context.Context) ([]apppkg.Choice, error)
|
||||||
AutocompleteAIModels(ctx context.Context) ([]apppkg.Choice, error)
|
AutocompleteAIModels(ctx context.Context) ([]apppkg.Choice, error)
|
||||||
|
AutocompleteSpeakers(ctx context.Context) ([]apppkg.Choice, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Handler adapts Discord interactions to the command application layer.
|
// Handler adapts Discord interactions to the command application layer.
|
||||||
@ -84,7 +86,7 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
|
|||||||
|
|
||||||
data := i.ApplicationCommandData()
|
data := i.ApplicationCommandData()
|
||||||
command := data.Name
|
command := data.Name
|
||||||
if len(data.Options) > 0 {
|
if len(data.Options) > 0 && isSubcommandOption(data.Options[0]) {
|
||||||
command += "." + data.Options[0].Name
|
command += "." + data.Options[0].Name
|
||||||
if len(data.Options[0].Options) > 0 && data.Options[0].Type == discordgo.ApplicationCommandOptionSubCommandGroup {
|
if len(data.Options[0].Options) > 0 && data.Options[0].Type == discordgo.ApplicationCommandOptionSubCommandGroup {
|
||||||
command += "." + data.Options[0].Options[0].Name
|
command += "." + data.Options[0].Options[0].Name
|
||||||
@ -104,6 +106,15 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
defer span.End()
|
defer span.End()
|
||||||
|
|
||||||
|
// speak is a flat command (speaker/text values directly under data.Options, no
|
||||||
|
// subcommand wrapper) unlike light/switch/ac/ai, so it's dispatched before the
|
||||||
|
// subcommand-assuming logic below rather than through commandPath's switch.
|
||||||
|
if data.Name == "speak" {
|
||||||
|
h.handleSpeak(ctx, s, i, data, start)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
if len(data.Options) == 0 {
|
if len(data.Options) == 0 {
|
||||||
h.respondError(ctx, s, i.Interaction, true, start, fmt.Errorf("missing subcommand"))
|
h.respondError(ctx, s, i.Interaction, true, start, fmt.Errorf("missing subcommand"))
|
||||||
return
|
return
|
||||||
@ -305,6 +316,57 @@ func (h *Handler) handleApplicationCommand(ctx context.Context, s *discordgo.Ses
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// handleSpeak synthesizes speech via tts-gateway and plays it in the invoking
|
||||||
|
// user's current voice channel, following the same deferred-ack + background +
|
||||||
|
// followup shape as the ai.query case (synthesis + voice join + playback will
|
||||||
|
// exceed Discord's ~3s initial-ack window).
|
||||||
|
func (h *Handler) handleSpeak(ctx context.Context, s *discordgo.Session, i *discordgo.InteractionCreate, data discordgo.ApplicationCommandInteractionData, start time.Time) {
|
||||||
|
log := logger.FromContext(ctx)
|
||||||
|
if err := h.deferResponse(s, i.Interaction, true); err != nil {
|
||||||
|
log.Error("discord response failed",
|
||||||
|
"duration_ms", time.Since(start).Milliseconds(),
|
||||||
|
"error", err.Error(),
|
||||||
|
)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
speakerName := stringOption(data.Options, "speaker")
|
||||||
|
text := stringOption(data.Options, "text")
|
||||||
|
guildID := i.GuildID
|
||||||
|
userID := ""
|
||||||
|
if i.Member != nil && i.Member.User != nil {
|
||||||
|
userID = i.Member.User.ID
|
||||||
|
} else if i.User != nil {
|
||||||
|
userID = i.User.ID
|
||||||
|
}
|
||||||
|
|
||||||
|
interaction := i.Interaction
|
||||||
|
reqLog := log
|
||||||
|
h.tracker.Go(func(trackerCtx context.Context) {
|
||||||
|
asyncCtx := logger.WithLogger(trackerCtx, reqLog)
|
||||||
|
asyncCtx, cancel := context.WithTimeout(asyncCtx, 2*time.Minute)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
audio, mimeType, err := h.app.HandleSpeak(asyncCtx, speakerName, text)
|
||||||
|
if err == nil {
|
||||||
|
err = speak(asyncCtx, s, guildID, userID, audio, mimeType)
|
||||||
|
}
|
||||||
|
|
||||||
|
msg := fmt.Sprintf("Spoke as `%s`.", speakerName)
|
||||||
|
if err != nil {
|
||||||
|
reqLog.Error("speak command failed", "error", err.Error())
|
||||||
|
msg = mapSpeakError(err)
|
||||||
|
}
|
||||||
|
if _, followErr := s.FollowupMessageCreate(interaction, true, &discordgo.WebhookParams{
|
||||||
|
Content: msg,
|
||||||
|
Flags: discordgo.MessageFlagsEphemeral,
|
||||||
|
}); followErr != nil {
|
||||||
|
reqLog.Error("discord response failed", "error", followErr.Error())
|
||||||
|
}
|
||||||
|
})
|
||||||
|
log.Info("command handled", "duration_ms", time.Since(start).Milliseconds())
|
||||||
|
}
|
||||||
|
|
||||||
// handleAutocomplete keeps autocomplete fast by filtering already-fetched choices
|
// handleAutocomplete keeps autocomplete fast by filtering already-fetched choices
|
||||||
// instead of requiring Discord-specific logic in the app layer.
|
// instead of requiring Discord-specific logic in the app layer.
|
||||||
func (h *Handler) handleAutocomplete(ctx context.Context, s *discordgo.Session, i *discordgo.InteractionCreate) {
|
func (h *Handler) handleAutocomplete(ctx context.Context, s *discordgo.Session, i *discordgo.InteractionCreate) {
|
||||||
@ -326,6 +388,10 @@ func (h *Handler) handleAutocomplete(ctx context.Context, s *discordgo.Session,
|
|||||||
if focusedOptionName(data) == "name" {
|
if focusedOptionName(data) == "name" {
|
||||||
choices, err = h.app.AutocompleteAIModels(ctx)
|
choices, err = h.app.AutocompleteAIModels(ctx)
|
||||||
}
|
}
|
||||||
|
case "speak":
|
||||||
|
if focusedOptionName(data) == "speaker" {
|
||||||
|
choices, err = h.app.AutocompleteSpeakers(ctx)
|
||||||
|
}
|
||||||
default:
|
default:
|
||||||
choices = nil
|
choices = nil
|
||||||
}
|
}
|
||||||
@ -419,6 +485,24 @@ func (h *Handler) followup(ctx context.Context, s *discordgo.Session, interactio
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func mapSpeakError(err error) string {
|
||||||
|
if errors.Is(err, ErrNotInVoiceChannel) {
|
||||||
|
return "Join a voice channel first."
|
||||||
|
}
|
||||||
|
if errors.Is(err, context.DeadlineExceeded) {
|
||||||
|
return "Speaking took too long. Try again later."
|
||||||
|
}
|
||||||
|
if status, ok := grpcstatus.FromError(err); ok {
|
||||||
|
switch status.Code() {
|
||||||
|
case codes.InvalidArgument:
|
||||||
|
return "Unknown speaker. Try /speak again with a different name."
|
||||||
|
case codes.Unavailable:
|
||||||
|
return "The TTS service is unreachable right now. Try again in a moment."
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "Sorry, something went wrong speaking that."
|
||||||
|
}
|
||||||
|
|
||||||
func mapAIError(err error) string {
|
func mapAIError(err error) string {
|
||||||
if errors.Is(err, context.DeadlineExceeded) {
|
if errors.Is(err, context.DeadlineExceeded) {
|
||||||
return "The AI took too long to respond. Try again later."
|
return "The AI took too long to respond. Try again later."
|
||||||
@ -448,7 +532,14 @@ func interactionLogger(ctx context.Context, i *discordgo.InteractionCreate) *slo
|
|||||||
}
|
}
|
||||||
|
|
||||||
func requiredStringOption(sub *discordgo.ApplicationCommandInteractionDataOption, name string) string {
|
func requiredStringOption(sub *discordgo.ApplicationCommandInteractionDataOption, name string) string {
|
||||||
for _, opt := range sub.Options {
|
return stringOption(sub.Options, name)
|
||||||
|
}
|
||||||
|
|
||||||
|
// stringOption looks up a named string-valued option in a flat options slice -
|
||||||
|
// used directly for flat commands like /speak, and via requiredStringOption for
|
||||||
|
// commands nested one level under a subcommand.
|
||||||
|
func stringOption(opts []*discordgo.ApplicationCommandInteractionDataOption, name string) string {
|
||||||
|
for _, opt := range opts {
|
||||||
if opt.Name == name {
|
if opt.Name == name {
|
||||||
return opt.StringValue()
|
return opt.StringValue()
|
||||||
}
|
}
|
||||||
@ -456,6 +547,12 @@ func requiredStringOption(sub *discordgo.ApplicationCommandInteractionDataOption
|
|||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// isSubcommandOption reports whether opt represents a subcommand/subcommand-group
|
||||||
|
// node, as opposed to a plain value option (e.g. a flat command's own arguments).
|
||||||
|
func isSubcommandOption(opt *discordgo.ApplicationCommandInteractionDataOption) bool {
|
||||||
|
return opt.Type == discordgo.ApplicationCommandOptionSubCommand || opt.Type == discordgo.ApplicationCommandOptionSubCommandGroup
|
||||||
|
}
|
||||||
|
|
||||||
func optionalUint32Option(sub *discordgo.ApplicationCommandInteractionDataOption, name string) *uint32 {
|
func optionalUint32Option(sub *discordgo.ApplicationCommandInteractionDataOption, name string) *uint32 {
|
||||||
for _, opt := range sub.Options {
|
for _, opt := range sub.Options {
|
||||||
if opt.Name == name {
|
if opt.Name == name {
|
||||||
@ -468,6 +565,9 @@ func optionalUint32Option(sub *discordgo.ApplicationCommandInteractionDataOption
|
|||||||
|
|
||||||
func focusedOptionValue(data discordgo.ApplicationCommandInteractionData) string {
|
func focusedOptionValue(data discordgo.ApplicationCommandInteractionData) string {
|
||||||
for _, sub := range data.Options {
|
for _, sub := range data.Options {
|
||||||
|
if sub.Focused {
|
||||||
|
return sub.StringValue()
|
||||||
|
}
|
||||||
if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup {
|
if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup {
|
||||||
for _, nested := range sub.Options {
|
for _, nested := range sub.Options {
|
||||||
for _, opt := range nested.Options {
|
for _, opt := range nested.Options {
|
||||||
@ -488,6 +588,9 @@ func focusedOptionValue(data discordgo.ApplicationCommandInteractionData) string
|
|||||||
|
|
||||||
func focusedOptionName(data discordgo.ApplicationCommandInteractionData) string {
|
func focusedOptionName(data discordgo.ApplicationCommandInteractionData) string {
|
||||||
for _, sub := range data.Options {
|
for _, sub := range data.Options {
|
||||||
|
if sub.Focused {
|
||||||
|
return sub.Name
|
||||||
|
}
|
||||||
if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup {
|
if sub.Type == discordgo.ApplicationCommandOptionSubCommandGroup {
|
||||||
for _, nested := range sub.Options {
|
for _, nested := range sub.Options {
|
||||||
for _, opt := range nested.Options {
|
for _, opt := range nested.Options {
|
||||||
|
|||||||
108
discord-bot/internal/adapters/primary/discord/handler_test.go
Normal file
108
discord-bot/internal/adapters/primary/discord/handler_test.go
Normal file
@ -0,0 +1,108 @@
|
|||||||
|
package discord
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/bwmarrin/discordgo"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestStringOption(t *testing.T) {
|
||||||
|
opts := []*discordgo.ApplicationCommandInteractionDataOption{
|
||||||
|
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice Shower"},
|
||||||
|
{Name: "text", Type: discordgo.ApplicationCommandOptionString, Value: "hello"},
|
||||||
|
}
|
||||||
|
|
||||||
|
if got := stringOption(opts, "speaker"); got != "Rice Shower" {
|
||||||
|
t.Fatalf("stringOption(speaker) = %q, want %q", got, "Rice Shower")
|
||||||
|
}
|
||||||
|
if got := stringOption(opts, "missing"); got != "" {
|
||||||
|
t.Fatalf("stringOption(missing) = %q, want empty", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIsSubcommandOption(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
typ discordgo.ApplicationCommandOptionType
|
||||||
|
want bool
|
||||||
|
}{
|
||||||
|
{name: "subcommand", typ: discordgo.ApplicationCommandOptionSubCommand, want: true},
|
||||||
|
{name: "subcommand group", typ: discordgo.ApplicationCommandOptionSubCommandGroup, want: true},
|
||||||
|
{name: "plain string option", typ: discordgo.ApplicationCommandOptionString, want: false},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
got := isSubcommandOption(&discordgo.ApplicationCommandInteractionDataOption{Type: tt.typ})
|
||||||
|
if got != tt.want {
|
||||||
|
t.Fatalf("isSubcommandOption(%v) = %v, want %v", tt.typ, got, tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFocusedOptionNameAndValue(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
data discordgo.ApplicationCommandInteractionData
|
||||||
|
wantName string
|
||||||
|
wantValue string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "flat command - top-level option focused",
|
||||||
|
data: discordgo.ApplicationCommandInteractionData{
|
||||||
|
Name: "speak",
|
||||||
|
Options: []*discordgo.ApplicationCommandInteractionDataOption{
|
||||||
|
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice", Focused: true},
|
||||||
|
{Name: "text", Type: discordgo.ApplicationCommandOptionString, Value: "hi"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
wantName: "speaker",
|
||||||
|
wantValue: "Rice",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "nested subcommand option focused (regression)",
|
||||||
|
data: discordgo.ApplicationCommandInteractionData{
|
||||||
|
Name: "ai",
|
||||||
|
Options: []*discordgo.ApplicationCommandInteractionDataOption{
|
||||||
|
{
|
||||||
|
Name: "model",
|
||||||
|
Type: discordgo.ApplicationCommandOptionSubCommandGroup,
|
||||||
|
Options: []*discordgo.ApplicationCommandInteractionDataOption{
|
||||||
|
{
|
||||||
|
Name: "set",
|
||||||
|
Type: discordgo.ApplicationCommandOptionSubCommand,
|
||||||
|
Options: []*discordgo.ApplicationCommandInteractionDataOption{
|
||||||
|
{Name: "name", Type: discordgo.ApplicationCommandOptionString, Value: "llama", Focused: true},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
wantName: "name",
|
||||||
|
wantValue: "llama",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "nothing focused",
|
||||||
|
data: discordgo.ApplicationCommandInteractionData{
|
||||||
|
Name: "speak",
|
||||||
|
Options: []*discordgo.ApplicationCommandInteractionDataOption{
|
||||||
|
{Name: "speaker", Type: discordgo.ApplicationCommandOptionString, Value: "Rice"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
wantName: "",
|
||||||
|
wantValue: "",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
if got := focusedOptionName(tt.data); got != tt.wantName {
|
||||||
|
t.Fatalf("focusedOptionName() = %q, want %q", got, tt.wantName)
|
||||||
|
}
|
||||||
|
if got := focusedOptionValue(tt.data); got != tt.wantValue {
|
||||||
|
t.Fatalf("focusedOptionValue() = %q, want %q", got, tt.wantValue)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -207,6 +207,25 @@ func RegisterCommands(s *discordgo.Session, guildID string) error {
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
Name: "speak",
|
||||||
|
Description: "Speak text in your current voice channel",
|
||||||
|
Options: []*discordgo.ApplicationCommandOption{
|
||||||
|
{
|
||||||
|
Type: discordgo.ApplicationCommandOptionString,
|
||||||
|
Name: "speaker",
|
||||||
|
Description: "Speaker voice",
|
||||||
|
Required: true,
|
||||||
|
Autocomplete: true,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Type: discordgo.ApplicationCommandOptionString,
|
||||||
|
Name: "text",
|
||||||
|
Description: "Text to speak",
|
||||||
|
Required: true,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
if _, err := s.ApplicationCommandBulkOverwrite(appID, guildID, commands); err != nil {
|
if _, err := s.ApplicationCommandBulkOverwrite(appID, guildID, commands); err != nil {
|
||||||
|
|||||||
131
discord-bot/internal/adapters/primary/discord/voice.go
Normal file
131
discord-bot/internal/adapters/primary/discord/voice.go
Normal file
@ -0,0 +1,131 @@
|
|||||||
|
package discord
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
|
||||||
|
"github.com/bwmarrin/discordgo"
|
||||||
|
"github.com/jonas747/dca"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ErrNotInVoiceChannel is returned when the invoking user isn't in a voice channel.
|
||||||
|
var ErrNotInVoiceChannel = errors.New("join a voice channel first")
|
||||||
|
|
||||||
|
// resolveVoiceChannel returns the voice channel the given guild member currently
|
||||||
|
// occupies, or ErrNotInVoiceChannel if they aren't in one.
|
||||||
|
func resolveVoiceChannel(s *discordgo.Session, guildID, userID string) (string, error) {
|
||||||
|
vs, err := s.State.VoiceState(guildID, userID)
|
||||||
|
if err != nil {
|
||||||
|
return "", ErrNotInVoiceChannel
|
||||||
|
}
|
||||||
|
return vs.ChannelID, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// speak joins the invoking user's current voice channel, transcodes audio (as
|
||||||
|
// returned by tts-gateway, e.g. AAC) to Opus via ffmpeg/dca, streams it, then
|
||||||
|
// leaves. This is Discord-specific playback plumbing that the transport-agnostic
|
||||||
|
// app layer has no business knowing about - see internal/app/command.go's
|
||||||
|
// HandleSpeak for the RPC half of this feature.
|
||||||
|
func speak(ctx context.Context, s *discordgo.Session, guildID, userID string, audio []byte, mimeType string) error {
|
||||||
|
log := logger.FromContext(ctx).With("mime_type", mimeType)
|
||||||
|
|
||||||
|
channelID, err := resolveVoiceChannel(s, guildID, userID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
vc, err := s.ChannelVoiceJoin(guildID, channelID, false, true)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("join voice channel: %w", err)
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
if err := vc.Disconnect(); err != nil {
|
||||||
|
log.Error("voice disconnect failed", "error", err.Error())
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
if err := waitForVoiceReady(ctx, vc); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// dca shells out to ffmpeg, which needs a seekable input for containers like the
|
||||||
|
// AAC/M4A tts-gateway returns - its own moov atom sits at the end of the file (see
|
||||||
|
// tts-gateway/internal/adapters/secondary/ffmpeg/ffmpeg.go), so a non-seekable pipe
|
||||||
|
// input would fail. Write to a temp file first, mirroring that same adapter's idiom
|
||||||
|
// on the output side.
|
||||||
|
tmpPath, err := writeTempAudio(audio)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer os.Remove(tmpPath)
|
||||||
|
|
||||||
|
encodeSession, err := dca.EncodeFile(tmpPath, dca.StdEncodeOptions)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("encode audio to opus: %w", err)
|
||||||
|
}
|
||||||
|
defer encodeSession.Cleanup()
|
||||||
|
|
||||||
|
if err := vc.Speaking(true); err != nil {
|
||||||
|
return fmt.Errorf("set speaking state: %w", err)
|
||||||
|
}
|
||||||
|
defer func() {
|
||||||
|
if err := vc.Speaking(false); err != nil {
|
||||||
|
log.Error("clear speaking state failed", "error", err.Error())
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
done := make(chan error, 1)
|
||||||
|
dca.NewStream(encodeSession, vc, done)
|
||||||
|
|
||||||
|
select {
|
||||||
|
case err := <-done:
|
||||||
|
if err != nil && err != io.EOF {
|
||||||
|
return fmt.Errorf("stream audio: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
case <-ctx.Done():
|
||||||
|
return ctx.Err()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// writeTempAudio persists synthesized audio to a temp file for ffmpeg to read from.
|
||||||
|
func writeTempAudio(audio []byte) (string, error) {
|
||||||
|
tmpFile, err := os.CreateTemp("", "discord-bot-speak-*")
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("create temp audio file: %w", err)
|
||||||
|
}
|
||||||
|
tmpPath := tmpFile.Name()
|
||||||
|
|
||||||
|
if _, err := tmpFile.Write(audio); err != nil {
|
||||||
|
_ = tmpFile.Close()
|
||||||
|
os.Remove(tmpPath)
|
||||||
|
return "", fmt.Errorf("write temp audio file: %w", err)
|
||||||
|
}
|
||||||
|
if err := tmpFile.Close(); err != nil {
|
||||||
|
os.Remove(tmpPath)
|
||||||
|
return "", fmt.Errorf("close temp audio file: %w", err)
|
||||||
|
}
|
||||||
|
return tmpPath, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// waitForVoiceReady polls until the voice connection's UDP socket is ready to
|
||||||
|
// send, or ctx is done - discordgo has no blocking/channel-based signal for this.
|
||||||
|
func waitForVoiceReady(ctx context.Context, vc *discordgo.VoiceConnection) error {
|
||||||
|
ticker := time.NewTicker(50 * time.Millisecond)
|
||||||
|
defer ticker.Stop()
|
||||||
|
for {
|
||||||
|
if vc.Ready {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-ticker.C:
|
||||||
|
case <-ctx.Done():
|
||||||
|
return fmt.Errorf("wait for voice connection ready: %w", ctx.Err())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
73
discord-bot/internal/adapters/primary/discord/voice_test.go
Normal file
73
discord-bot/internal/adapters/primary/discord/voice_test.go
Normal file
@ -0,0 +1,73 @@
|
|||||||
|
package discord
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/bwmarrin/discordgo"
|
||||||
|
)
|
||||||
|
|
||||||
|
func newTestSession(t *testing.T, guildID string, voiceStates []*discordgo.VoiceState) *discordgo.Session {
|
||||||
|
t.Helper()
|
||||||
|
state := discordgo.NewState()
|
||||||
|
if err := state.GuildAdd(&discordgo.Guild{ID: guildID, VoiceStates: voiceStates}); err != nil {
|
||||||
|
t.Fatalf("GuildAdd() error = %v", err)
|
||||||
|
}
|
||||||
|
return &discordgo.Session{State: state}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveVoiceChannel(t *testing.T) {
|
||||||
|
const guildID = "guild-1"
|
||||||
|
const userID = "user-1"
|
||||||
|
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
voiceStates []*discordgo.VoiceState
|
||||||
|
userID string
|
||||||
|
wantChannel string
|
||||||
|
wantErr error
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "user in a voice channel",
|
||||||
|
voiceStates: []*discordgo.VoiceState{
|
||||||
|
{GuildID: guildID, UserID: userID, ChannelID: "channel-1"},
|
||||||
|
},
|
||||||
|
userID: userID,
|
||||||
|
wantChannel: "channel-1",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "user not in any voice channel",
|
||||||
|
voiceStates: nil,
|
||||||
|
userID: userID,
|
||||||
|
wantErr: ErrNotInVoiceChannel,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "user in a different guild's voice channel is not found",
|
||||||
|
voiceStates: []*discordgo.VoiceState{
|
||||||
|
{GuildID: guildID, UserID: "someone-else", ChannelID: "channel-1"},
|
||||||
|
},
|
||||||
|
userID: userID,
|
||||||
|
wantErr: ErrNotInVoiceChannel,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
session := newTestSession(t, guildID, tt.voiceStates)
|
||||||
|
|
||||||
|
got, err := resolveVoiceChannel(session, guildID, tt.userID)
|
||||||
|
if tt.wantErr != nil {
|
||||||
|
if !errors.Is(err, tt.wantErr) {
|
||||||
|
t.Fatalf("resolveVoiceChannel() error = %v, want %v", err, tt.wantErr)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("resolveVoiceChannel() error = %v", err)
|
||||||
|
}
|
||||||
|
if got != tt.wantChannel {
|
||||||
|
t.Fatalf("resolveVoiceChannel() = %q, want %q", got, tt.wantChannel)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
124
discord-bot/internal/adapters/secondary/ttsgateway/client.go
Normal file
124
discord-bot/internal/adapters/secondary/ttsgateway/client.go
Normal file
@ -0,0 +1,124 @@
|
|||||||
|
package ttsgateway
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/tls"
|
||||||
|
"crypto/x509"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.nik4nao.com/nik/home-services/discord-bot/internal/logger"
|
||||||
|
ttsv1 "gitea.nik4nao.com/nik/home-services/gen/tts/v1"
|
||||||
|
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
|
||||||
|
"google.golang.org/grpc"
|
||||||
|
"google.golang.org/grpc/credentials"
|
||||||
|
"google.golang.org/grpc/credentials/insecure"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Client implements the app's TTS driven port over gRPC.
|
||||||
|
type Client struct {
|
||||||
|
conn *grpc.ClientConn
|
||||||
|
client ttsv1.TTSServiceClient
|
||||||
|
log *slog.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
// New constructs a gRPC client for the internal tts-gateway service.
|
||||||
|
func New(ctx context.Context, addr, tlsDir string, log *slog.Logger) (*Client, error) {
|
||||||
|
transportCreds := insecure.NewCredentials()
|
||||||
|
if tlsDir != "" {
|
||||||
|
creds, err := loadTransportCredentials(tlsDir)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("load mTLS credentials: %w", err)
|
||||||
|
}
|
||||||
|
transportCreds = creds
|
||||||
|
}
|
||||||
|
|
||||||
|
conn, err := grpc.NewClient(
|
||||||
|
addr,
|
||||||
|
grpc.WithTransportCredentials(transportCreds),
|
||||||
|
grpc.WithStatsHandler(otelgrpc.NewClientHandler()),
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("dial tts-gateway: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &Client{
|
||||||
|
conn: conn,
|
||||||
|
client: ttsv1.NewTTSServiceClient(conn),
|
||||||
|
log: log,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close closes the underlying gRPC connection.
|
||||||
|
func (c *Client) Close() error {
|
||||||
|
if err := c.conn.Close(); err != nil {
|
||||||
|
return fmt.Errorf("close tts-gateway client: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Synthesize forwards a speech synthesis request to tts-gateway. noise_scale/
|
||||||
|
// noise_scale_w/length_scale are left unset so tts-gateway applies its own defaults.
|
||||||
|
func (c *Client) Synthesize(ctx context.Context, speakerName, text string) ([]byte, string, error) {
|
||||||
|
start := time.Now()
|
||||||
|
log := logger.FromContext(ctx).With("grpc.method", "TTSService/Synthesize")
|
||||||
|
resp, err := c.client.Synthesize(ctx, &ttsv1.SynthesizeRequest{
|
||||||
|
SpeakerName: speakerName,
|
||||||
|
Text: text,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
log.Error("grpc call failed",
|
||||||
|
"duration_ms", time.Since(start).Milliseconds(),
|
||||||
|
"error", err.Error(),
|
||||||
|
)
|
||||||
|
return nil, "", fmt.Errorf("synthesize speech: %w", err)
|
||||||
|
}
|
||||||
|
log.Debug("grpc call completed", "duration_ms", time.Since(start).Milliseconds())
|
||||||
|
return resp.GetAudio(), resp.GetMimeType(), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListSpeakers returns the speaker roster from tts-gateway, optionally filtered by search.
|
||||||
|
func (c *Client) ListSpeakers(ctx context.Context, search string) ([]string, error) {
|
||||||
|
start := time.Now()
|
||||||
|
log := logger.FromContext(ctx).With("grpc.method", "TTSService/ListSpeakers")
|
||||||
|
resp, err := c.client.ListSpeakers(ctx, &ttsv1.ListSpeakersRequest{Search: search})
|
||||||
|
if err != nil {
|
||||||
|
log.Error("grpc call failed",
|
||||||
|
"duration_ms", time.Since(start).Milliseconds(),
|
||||||
|
"error", err.Error(),
|
||||||
|
)
|
||||||
|
return nil, fmt.Errorf("list tts-gateway speakers: %w", err)
|
||||||
|
}
|
||||||
|
log.Debug("grpc call completed", "duration_ms", time.Since(start).Milliseconds())
|
||||||
|
return append([]string(nil), resp.GetSpeakerNames()...), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadTransportCredentials(tlsDir string) (credentials.TransportCredentials, error) {
|
||||||
|
cert, err := tls.LoadX509KeyPair(
|
||||||
|
filepath.Join(tlsDir, "tls.crt"),
|
||||||
|
filepath.Join(tlsDir, "tls.key"),
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("load client key pair: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
caPEM, err := os.ReadFile(filepath.Join(tlsDir, "ca.crt"))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read server CA: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rootCAs := x509.NewCertPool()
|
||||||
|
if !rootCAs.AppendCertsFromPEM(caPEM) {
|
||||||
|
return nil, fmt.Errorf("append server CA: invalid PEM")
|
||||||
|
}
|
||||||
|
|
||||||
|
return credentials.NewTLS(&tls.Config{
|
||||||
|
Certificates: []tls.Certificate{cert},
|
||||||
|
RootCAs: rootCAs,
|
||||||
|
ServerName: "tts-gateway.home-services.svc.cluster.local",
|
||||||
|
MinVersion: tls.VersionTLS13,
|
||||||
|
}), nil
|
||||||
|
}
|
||||||
@ -23,11 +23,12 @@ type CommandApp struct {
|
|||||||
ai driven.AIGateway
|
ai driven.AIGateway
|
||||||
models *modelstore.Store
|
models *modelstore.Store
|
||||||
validator *modelvalidator.Validator
|
validator *modelvalidator.Validator
|
||||||
|
tts driven.TTSGateway
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewCommandApp constructs the Discord command application service.
|
// NewCommandApp constructs the Discord command application service.
|
||||||
func NewCommandApp(ha driven.HAGateway, ai driven.AIGateway, models *modelstore.Store, validator *modelvalidator.Validator) *CommandApp {
|
func NewCommandApp(ha driven.HAGateway, ai driven.AIGateway, models *modelstore.Store, validator *modelvalidator.Validator, tts driven.TTSGateway) *CommandApp {
|
||||||
return &CommandApp{ha: ha, ai: ai, models: models, validator: validator}
|
return &CommandApp{ha: ha, ai: ai, models: models, validator: validator, tts: tts}
|
||||||
}
|
}
|
||||||
|
|
||||||
// HandleLightList formats discovered lights into a monospace-friendly response.
|
// HandleLightList formats discovered lights into a monospace-friendly response.
|
||||||
@ -279,6 +280,33 @@ func (a *CommandApp) HandleAIModelList(ctx context.Context) (string, error) {
|
|||||||
return strings.Join(lines, "\n"), nil
|
return strings.Join(lines, "\n"), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// HandleSpeak synthesizes speech via tts-gateway for the invoking Discord adapter
|
||||||
|
// to play back in a voice channel; the app layer stays transport-agnostic and
|
||||||
|
// leaves voice-channel playback itself to the Discord adapter.
|
||||||
|
func (a *CommandApp) HandleSpeak(ctx context.Context, speakerName, text string) ([]byte, string, error) {
|
||||||
|
audio, mimeType, err := a.tts.Synthesize(ctx, speakerName, text)
|
||||||
|
if err != nil {
|
||||||
|
return nil, "", fmt.Errorf("handle speak: %w", err)
|
||||||
|
}
|
||||||
|
return audio, mimeType, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// AutocompleteSpeakers returns the full tts-gateway speaker roster for the /speak
|
||||||
|
// command; handleAutocomplete's existing substring filter narrows it client-side,
|
||||||
|
// same as the light/switch/ac autocompletes above.
|
||||||
|
func (a *CommandApp) AutocompleteSpeakers(ctx context.Context) ([]Choice, error) {
|
||||||
|
speakers, err := a.tts.ListSpeakers(ctx, "")
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("autocomplete speakers: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
choices := make([]Choice, 0, len(speakers))
|
||||||
|
for _, speaker := range speakers {
|
||||||
|
choices = append(choices, Choice{Label: speaker, Value: speaker})
|
||||||
|
}
|
||||||
|
return choices, nil
|
||||||
|
}
|
||||||
|
|
||||||
// AutocompleteAIModels returns model names for the /ai model set command.
|
// AutocompleteAIModels returns model names for the /ai model set command.
|
||||||
func (a *CommandApp) AutocompleteAIModels(ctx context.Context) ([]Choice, error) {
|
func (a *CommandApp) AutocompleteAIModels(ctx context.Context) ([]Choice, error) {
|
||||||
models, err := a.validator.Known(ctx)
|
models, err := a.validator.Known(ctx)
|
||||||
|
|||||||
@ -35,6 +35,11 @@ type mockAIGateway struct {
|
|||||||
listModelsFunc func(ctx context.Context) ([]string, error)
|
listModelsFunc func(ctx context.Context) ([]string, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type mockTTSGateway struct {
|
||||||
|
synthesizeFunc func(ctx context.Context, speakerName, text string) ([]byte, string, error)
|
||||||
|
listSpeakersFunc func(ctx context.Context, search string) ([]string, error)
|
||||||
|
}
|
||||||
|
|
||||||
func (m *mockHAGateway) ListLights(ctx context.Context) ([]driven.Light, error) {
|
func (m *mockHAGateway) ListLights(ctx context.Context) ([]driven.Light, error) {
|
||||||
if m.listLightsFunc == nil {
|
if m.listLightsFunc == nil {
|
||||||
return nil, nil
|
return nil, nil
|
||||||
@ -147,8 +152,22 @@ func (m *mockAIGateway) ListModels(ctx context.Context) ([]string, error) {
|
|||||||
return m.listModelsFunc(ctx)
|
return m.listModelsFunc(ctx)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (m *mockTTSGateway) Synthesize(ctx context.Context, speakerName, text string) ([]byte, string, error) {
|
||||||
|
if m.synthesizeFunc == nil {
|
||||||
|
return nil, "", nil
|
||||||
|
}
|
||||||
|
return m.synthesizeFunc(ctx, speakerName, text)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *mockTTSGateway) ListSpeakers(ctx context.Context, search string) ([]string, error) {
|
||||||
|
if m.listSpeakersFunc == nil {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
return m.listSpeakersFunc(ctx, search)
|
||||||
|
}
|
||||||
|
|
||||||
func newTestCommandApp(ha *mockHAGateway, ai *mockAIGateway) *CommandApp {
|
func newTestCommandApp(ha *mockHAGateway, ai *mockAIGateway) *CommandApp {
|
||||||
return NewCommandApp(ha, ai, modelstore.New(), modelvalidator.New(ai, time.Minute))
|
return NewCommandApp(ha, ai, modelstore.New(), modelvalidator.New(ai, time.Minute), &mockTTSGateway{})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCommandAppHandleLightList(t *testing.T) {
|
func TestCommandAppHandleLightList(t *testing.T) {
|
||||||
@ -1081,7 +1100,7 @@ func TestCommandAppHandleAIQuery(t *testing.T) {
|
|||||||
}
|
}
|
||||||
return "Turning on Kitchen.", "llama3:latest", nil
|
return "Turning on Kitchen.", "llama3:latest", nil
|
||||||
},
|
},
|
||||||
}, store, modelvalidator.New(&mockAIGateway{}, time.Minute))
|
}, store, modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{})
|
||||||
|
|
||||||
got, err := app.HandleAIQuery(context.Background(), "turn on kitchen")
|
got, err := app.HandleAIQuery(context.Background(), "turn on kitchen")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@ -1092,6 +1111,116 @@ func TestCommandAppHandleAIQuery(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestCommandAppHandleSpeak(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
speakerName string
|
||||||
|
text string
|
||||||
|
audio []byte
|
||||||
|
mimeType string
|
||||||
|
synthErr error
|
||||||
|
wantErr string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "synthesize success",
|
||||||
|
speakerName: "Rice Shower",
|
||||||
|
text: "おはようございます",
|
||||||
|
audio: []byte{0x00, 0x01, 0x02},
|
||||||
|
mimeType: "audio/aac",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "synthesize error propagated",
|
||||||
|
speakerName: "Unknown Speaker",
|
||||||
|
text: "hello",
|
||||||
|
synthErr: errors.New("boom"),
|
||||||
|
wantErr: "handle speak: boom",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
var gotSpeaker, gotText string
|
||||||
|
app := NewCommandApp(&mockHAGateway{}, &mockAIGateway{}, modelstore.New(), modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{
|
||||||
|
synthesizeFunc: func(ctx context.Context, speakerName, text string) ([]byte, string, error) {
|
||||||
|
gotSpeaker, gotText = speakerName, text
|
||||||
|
if tt.synthErr != nil {
|
||||||
|
return nil, "", tt.synthErr
|
||||||
|
}
|
||||||
|
return tt.audio, tt.mimeType, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
|
audio, mimeType, err := app.HandleSpeak(context.Background(), tt.speakerName, tt.text)
|
||||||
|
if tt.wantErr != "" {
|
||||||
|
if err == nil || err.Error() != tt.wantErr {
|
||||||
|
t.Fatalf("HandleSpeak() error = %v, want %q", err, tt.wantErr)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("HandleSpeak() error = %v", err)
|
||||||
|
}
|
||||||
|
if gotSpeaker != tt.speakerName || gotText != tt.text {
|
||||||
|
t.Fatalf("Synthesize() called with (%q, %q), want (%q, %q)", gotSpeaker, gotText, tt.speakerName, tt.text)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(audio, tt.audio) || mimeType != tt.mimeType {
|
||||||
|
t.Fatalf("HandleSpeak() = (%#v, %q), want (%#v, %q)", audio, mimeType, tt.audio, tt.mimeType)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCommandAppAutocompleteSpeakers(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
speakers []string
|
||||||
|
listErr error
|
||||||
|
want []Choice
|
||||||
|
wantErr string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "maps speaker names to choices",
|
||||||
|
speakers: []string{"Rice Shower", "Special Week"},
|
||||||
|
want: []Choice{
|
||||||
|
{Label: "Rice Shower", Value: "Rice Shower"},
|
||||||
|
{Label: "Special Week", Value: "Special Week"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "ListSpeakers error",
|
||||||
|
listErr: errors.New("boom"),
|
||||||
|
wantErr: "autocomplete speakers: boom",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
app := NewCommandApp(&mockHAGateway{}, &mockAIGateway{}, modelstore.New(), modelvalidator.New(&mockAIGateway{}, time.Minute), &mockTTSGateway{
|
||||||
|
listSpeakersFunc: func(ctx context.Context, search string) ([]string, error) {
|
||||||
|
if tt.listErr != nil {
|
||||||
|
return nil, tt.listErr
|
||||||
|
}
|
||||||
|
return tt.speakers, nil
|
||||||
|
},
|
||||||
|
})
|
||||||
|
|
||||||
|
got, err := app.AutocompleteSpeakers(context.Background())
|
||||||
|
if tt.wantErr != "" {
|
||||||
|
if err == nil || err.Error() != tt.wantErr {
|
||||||
|
t.Fatalf("AutocompleteSpeakers() error = %v, want %q", err, tt.wantErr)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("AutocompleteSpeakers() error = %v", err)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(got, tt.want) {
|
||||||
|
t.Fatalf("AutocompleteSpeakers() = %#v, want %#v", got, tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestCommandAppHandleAIModelSet(t *testing.T) {
|
func TestCommandAppHandleAIModelSet(t *testing.T) {
|
||||||
app := newTestCommandApp(&mockHAGateway{}, &mockAIGateway{
|
app := newTestCommandApp(&mockHAGateway{}, &mockAIGateway{
|
||||||
listModelsFunc: func(ctx context.Context) ([]string, error) {
|
listModelsFunc: func(ctx context.Context) ([]string, error) {
|
||||||
|
|||||||
@ -11,6 +11,7 @@ type Config struct {
|
|||||||
GuildID string
|
GuildID string
|
||||||
HAGatewayAddr string
|
HAGatewayAddr string
|
||||||
AIGatewayAddr string
|
AIGatewayAddr string
|
||||||
|
TTSGatewayAddr string
|
||||||
TLSDir string
|
TLSDir string
|
||||||
OTELEndpoint string
|
OTELEndpoint string
|
||||||
LogLevel string
|
LogLevel string
|
||||||
@ -34,6 +35,7 @@ func Load() (*Config, error) {
|
|||||||
GuildID: os.Getenv("GUILD_ID"),
|
GuildID: os.Getenv("GUILD_ID"),
|
||||||
HAGatewayAddr: addr,
|
HAGatewayAddr: addr,
|
||||||
AIGatewayAddr: getenvDefault("AI_GATEWAY_ADDR", "ai-gateway.home-services.svc.cluster.local:50052"),
|
AIGatewayAddr: getenvDefault("AI_GATEWAY_ADDR", "ai-gateway.home-services.svc.cluster.local:50052"),
|
||||||
|
TTSGatewayAddr: getenvDefault("TTS_GATEWAY_ADDR", "tts-gateway.home-services.svc.cluster.local:50053"),
|
||||||
TLSDir: os.Getenv("TLS_DIR"),
|
TLSDir: os.Getenv("TLS_DIR"),
|
||||||
OTELEndpoint: os.Getenv("OTEL_ENDPOINT"),
|
OTELEndpoint: os.Getenv("OTEL_ENDPOINT"),
|
||||||
LogLevel: getenvDefault("LOG_LEVEL", "info"),
|
LogLevel: getenvDefault("LOG_LEVEL", "info"),
|
||||||
|
|||||||
9
discord-bot/internal/core/ports/driven/tts.go
Normal file
9
discord-bot/internal/core/ports/driven/tts.go
Normal file
@ -0,0 +1,9 @@
|
|||||||
|
package driven
|
||||||
|
|
||||||
|
import "context"
|
||||||
|
|
||||||
|
// TTSGateway exposes the text-to-speech synthesis API used by the Discord bot.
|
||||||
|
type TTSGateway interface {
|
||||||
|
Synthesize(ctx context.Context, speakerName, text string) (audio []byte, mimeType string, err error)
|
||||||
|
ListSpeakers(ctx context.Context, search string) ([]string, error)
|
||||||
|
}
|
||||||
90
handoff.md
Normal file
90
handoff.md
Normal file
@ -0,0 +1,90 @@
|
|||||||
|
# tts-gateway Handoff
|
||||||
|
|
||||||
|
Status as of 2026-07-25: Phases 0–6 of `TTS_GATEWAY_PLAN.md` are done, and the deployed service is
|
||||||
|
confirmed working in production (real `grpcurl` call through `kubectl port-forward`, real GPU
|
||||||
|
inference, real playable audio). This file is a punch list for picking the remaining work up in a
|
||||||
|
fresh session — it doesn't re-explain things that are already documented elsewhere; it points to
|
||||||
|
where.
|
||||||
|
|
||||||
|
**Update (later same day):** items 1 and 3 below are now done — model artifact distribution is a
|
||||||
|
versioned image + k8s `initContainer` (see `tts-gateway/README.md`), and `discord-bot` has a
|
||||||
|
working `/speak` command (see `TTS_GATEWAY_PLAN.md` Phase 6). Only item 2 (mTLS) remains, left
|
||||||
|
untouched per an explicit decision to verify these two first. The two "what's left" entries below
|
||||||
|
are kept as-written for their historical reasoning/decision trail rather than rewritten in place.
|
||||||
|
|
||||||
|
## Where things stand
|
||||||
|
|
||||||
|
- `tts-gateway` (Go) + `tts-sidecar` (Python/libtorch) are both built, containerized, and running
|
||||||
|
in Kubernetes (`home-services` namespace).
|
||||||
|
- CI (`.gitea/workflows/ci.yaml`) builds and pushes both images on every push to `main`, gated by
|
||||||
|
path-filtered per-service rebuilds and using registry-based Docker layer caching.
|
||||||
|
- Full history — the Phase 0 ONNX-export/g2p investigation, why there's a Python sidecar instead
|
||||||
|
of pure Go, the exact `japanese_cleaners` pipeline verification — is in `TTS_GATEWAY_PLAN.md`
|
||||||
|
and `tmp/reference/uma-tts-api/spike/FINDINGS.md`. Read those before re-deriving anything; the
|
||||||
|
ONNX export failure in particular took several iterations to characterize correctly and isn't
|
||||||
|
worth re-investigating from scratch.
|
||||||
|
|
||||||
|
## What's left
|
||||||
|
|
||||||
|
1. **Model artifact distribution isn't automated.** The k8s Deployment
|
||||||
|
(`~/repo/homelab/manifests/home-services/tts-gateway.yaml` — separate repo, see below)
|
||||||
|
`hostPath`-mounts `/data/tts-gateway` on the `nik-gpu` node into the sidecar container.
|
||||||
|
Someone has to manually place `G_790000.pth` + `uma.json` there before the pod goes Ready — no
|
||||||
|
init-container download, no PVC, nothing automated. `TTS_GATEWAY_PLAN.md`'s Phase 5 flagged
|
||||||
|
this as an open decision; it's still open, just worked around. If asked to fix it: options are
|
||||||
|
a PVC populated by an init-container download step, baking the checkpoint into a private image
|
||||||
|
layer, or similar — weigh against the checkpoint being ~455MB and not something to casually put
|
||||||
|
in a git-tracked Dockerfile context.
|
||||||
|
|
||||||
|
2. **mTLS is off.** `tts-gateway.yaml` comments out the `TLS_DIR` env var and its volume mount,
|
||||||
|
for plaintext `grpcurl` testing from outside the cluster during initial rollout — a live TODO,
|
||||||
|
not a permanent decision. Every other service in this repo assumes internal-network-or-mTLS as
|
||||||
|
its only access boundary (`CLAUDE.md`'s "Configuration Notes"). Re-enabling: uncomment the env
|
||||||
|
var + mount in that manifest, and set up a `tts-gateway-tls` secret the same way
|
||||||
|
`ha-gateway-sealed.yaml`/`ha-gateway-secret.sh` do it for ha-gateway (tts-gateway doesn't
|
||||||
|
currently have its own `-secret.sh`/`-sealed.yaml` pair — it'll need one, since unlike
|
||||||
|
ha-gateway/discord-bot it has no other secrets today, so this would be its first).
|
||||||
|
|
||||||
|
3. **Phase 6 (discord-bot integration) not started.** A `/speak`-style Discord slash command
|
||||||
|
calling `tts-gateway`, following the existing pattern — `discord-bot` already has gRPC clients
|
||||||
|
for `ha-gateway` and `ai-gateway` at `discord-bot/internal/adapters/secondary/{gateway,aigateway}`;
|
||||||
|
a new `internal/adapters/secondary/ttsgateway` client would follow the same shape. Needs a
|
||||||
|
product decision first: keep AAC (parity with what's already implemented) or switch to Opus
|
||||||
|
(Discord's native voice codec) — deliberately deferred in `TTS_GATEWAY_PLAN.md`'s open
|
||||||
|
questions until this exact moment, not yet decided.
|
||||||
|
|
||||||
|
## Things worth knowing before touching this
|
||||||
|
|
||||||
|
- **`tts-gateway` and `tts-sidecar` share one Kubernetes Pod**, not two separate Deployments — the
|
||||||
|
Go gateway reaches the sidecar over `localhost:50054`. This mirrors the `--network host` setup
|
||||||
|
documented in `tts-gateway/README.md` for local Docker testing. Don't split them into separate
|
||||||
|
Deployments/Services without also rethinking `INFERENCE_SIDECAR_ADDR` and the networking model.
|
||||||
|
- **The k8s manifests live in a different repo**: `~/repo/homelab/manifests/home-services/` — not
|
||||||
|
in this repo at all. `CLAUDE.md` now has an "Infrastructure / Deployment" section documenting
|
||||||
|
this; check there (and that repo directly) before assuming something isn't deployed just because
|
||||||
|
this repo has no manifests for it.
|
||||||
|
- **`open_jtalk`'s tokenization has an intentional bug that must be preserved.** The Go text
|
||||||
|
normalizer (`tts-gateway/internal/adapters/secondary/jtalk`) maps the cleaned phoneme string to
|
||||||
|
symbol IDs character-by-character, not phoneme-by-phoneme, because that's what the checkpoint
|
||||||
|
was actually trained on (`text/symbols.py`'s "wrong tokens" comment in the original reference
|
||||||
|
implementation). Don't "fix" this without retraining the model — see the comments in `jtalk.go`
|
||||||
|
and `tts-gateway/README.md`.
|
||||||
|
- **ONNX export doesn't work for this checkpoint, and it's not a quick fix.** Read
|
||||||
|
`tmp/reference/uma-tts-api/spike/FINDINGS.md` before re-attempting it — the model's
|
||||||
|
data-dependent output length (predicted phoneme durations determine audio length at runtime)
|
||||||
|
defeats `torch.export`'s guard system in a way that would need an unknown number of per-layer
|
||||||
|
`torch._check` hints to resolve, not a single targeted change.
|
||||||
|
- **CI's build cache needs one "priming" run per image.** If a build job still looks slow after
|
||||||
|
the caching change in `ci.yaml`, check whether a `:buildcache` tag actually exists yet in the
|
||||||
|
registry for that specific image — the first run after the cache was added has nothing to pull
|
||||||
|
from and builds fully fresh.
|
||||||
|
|
||||||
|
## Where to look for more detail
|
||||||
|
|
||||||
|
- `TTS_GATEWAY_PLAN.md` (repo root) — original plan, feasibility analysis, phase breakdown, open
|
||||||
|
product questions (codec, concurrency, auth).
|
||||||
|
- `tmp/reference/uma-tts-api/spike/FINDINGS.md` — Phase 0 spike results with full reasoning.
|
||||||
|
- `tts-gateway/README.md` — service-level docs: gRPC API, config reference, how to run/test
|
||||||
|
locally and on nik-gpu.
|
||||||
|
- `~/repo/homelab/manifests/home-services/tts-gateway.yaml` — actual production config (separate
|
||||||
|
repo, maintained independently — check it directly rather than trusting a stale summary).
|
||||||
@ -67,8 +67,33 @@ The server also registers gRPC health checks and reflection.
|
|||||||
| `PORT` | `50054` | HTTP listen port |
|
| `PORT` | `50054` | HTTP listen port |
|
||||||
|
|
||||||
Neither the checkpoint nor the hparams file is committed to git - both must
|
Neither the checkpoint nor the hparams file is committed to git - both must
|
||||||
be supplied at runtime (see below). Model artifact distribution (bind mount
|
be supplied at runtime (see below). In production, they're baked into a
|
||||||
vs. build-time download vs. registry) is still an open decision.
|
small versioned image (`tts-gateway/model/Dockerfile`) that a k8s
|
||||||
|
`initContainer` copies into a shared volume before `tts-sidecar` starts -
|
||||||
|
see "Model artifact distribution (production)" below. For local/manual
|
||||||
|
testing on nik-gpu, a plain bind mount (as shown below) is simpler.
|
||||||
|
|
||||||
|
### Model artifact distribution (production)
|
||||||
|
|
||||||
|
CI can never build the model image - the checkpoint (~455MB) and hparams
|
||||||
|
aren't committed to git, and nik-gpu's local disk at `/data/tts-gateway` is
|
||||||
|
the only place they exist. Build and push it manually, directly on nik-gpu
|
||||||
|
(not via `docker --context nik-gpu` from elsewhere - the build context is
|
||||||
|
gathered client-side, so it would look for `/data/tts-gateway` on the wrong
|
||||||
|
machine), whenever the checkpoint or config change:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ssh nik-gpu
|
||||||
|
docker build -f ~/repo/home-service/tts-gateway/model/Dockerfile \
|
||||||
|
-t gitea.nik4nao.com/nik/tts-model:g790000-v1 /data/tts-gateway
|
||||||
|
docker push gitea.nik4nao.com/nik/tts-model:g790000-v1
|
||||||
|
```
|
||||||
|
|
||||||
|
Bump the tag (`v2`, `v3`, ...) rather than overwriting one - non-`:latest`
|
||||||
|
tags default to `imagePullPolicy: IfNotPresent`, so a node that already
|
||||||
|
pulled a tag won't re-pull an overwritten one. The k8s Deployment
|
||||||
|
(`~/repo/homelab/manifests/home-services/tts-gateway.yaml`) references the
|
||||||
|
tag explicitly in its `model-init` init container.
|
||||||
|
|
||||||
## Running And Testing On nik-gpu
|
## Running And Testing On nik-gpu
|
||||||
|
|
||||||
@ -188,8 +213,8 @@ sidecar/ # Python/libtorch inference servic
|
|||||||
- Concurrency is unbounded on the Go side, but the sidecar's model forward
|
- Concurrency is unbounded on the Go side, but the sidecar's model forward
|
||||||
pass is inherently single-threaded per GPU; heavy concurrent load will
|
pass is inherently single-threaded per GPU; heavy concurrent load will
|
||||||
just queue at the sidecar.
|
just queue at the sidecar.
|
||||||
- Model artifact distribution isn't finalized - the sidecar currently
|
- Model artifact distribution to production is a manually-built-and-pushed
|
||||||
expects a bind-mounted checkpoint/config, not something the image ships
|
versioned image (see above), not something CI can automate - the
|
||||||
with.
|
checkpoint only ever existed on nik-gpu's local disk.
|
||||||
- No Discord-bot integration yet (a `/speak`-style command calling this
|
- `discord-bot`'s `/speak` command calls this service and plays the result
|
||||||
service is a deliberate follow-up, not yet built).
|
in a Discord voice channel (`discord-bot/internal/adapters/primary/discord/voice.go`).
|
||||||
|
|||||||
7
tts-gateway/model/Dockerfile
Normal file
7
tts-gateway/model/Dockerfile
Normal file
@ -0,0 +1,7 @@
|
|||||||
|
# One-time (or only-when-the-checkpoint-changes) manually built and pushed by a human on
|
||||||
|
# nik-gpu - CI can never build this: the checkpoint and hparams aren't committed to git
|
||||||
|
# (~455MB, and nik-gpu's local disk at /data/tts-gateway is the only place they exist).
|
||||||
|
# Build CONTEXT must be that directory, not a checkout of this repo - see
|
||||||
|
# tts-gateway/README.md for the exact commands.
|
||||||
|
FROM busybox:1.36
|
||||||
|
COPY G_790000.pth uma.json /models/
|
||||||
Loading…
x
Reference in New Issue
Block a user