feat(tts): StepFun voice selection via CharacterDesigner + provider-aware beat-audio

Make homepage cards and live sessions produce sound when the server is
configured for StepFun TTS, instead of silently failing (the prebaked
Xiaomi voice was useless on a StepFun server, and wasted ~220KB/beat in
Fast Origin Transfer).

Three coordinated changes:

1. CharacterDesigner now picks a StepFun preset voice id directly from the
   32-entry catalog in the SAME LLM call that designs the character — zero
   extra latency, LLM-grade match quality. The Xiaomi prompt path is
   byte-identical to history (verified programmatically) so cache hit rate
   and voice quality are preserved. pickStepfunVoiceId (keyword scorer)
   remains the fallback for orphan speakers / invalid LLM picks.

2. The 32-preset catalog moves to lib/tts-client/stepfun-voices.json as the
   single source of truth, shared by the scorer, the CharacterDesigner
   prompt, /api/tts-provider, and the offline enrich script.

3. A new GET /api/tts-provider endpoint lets the client probe the server's
   TTS provider at /play mount. fetchBeatAudio then shapes its request body:
   on a StepFun server it sends the lightweight stepfunVoiceId /
   voiceDescription and omits the ~220KB Xiaomi reference audio (FOT saving
   ~13MB per protagonist per session on prebaked cards). requestBeatAudio
   re-provisions on a provider mismatch before synth, so audio never goes
   silent on a cross-provider replay or mid-session provider flip.

New type fields are all optional and backward-compatible: Character.stepfunVoiceId,
BeatAudioRequest.voiceDescription/characterName/stepfunVoiceId, voice made
optional. AGENTS.md updated for the new route, type fields, dependency map,
and StepFun voice-selection flow.
This commit is contained in:
yuanzonghao
2026-06-15 12:49:25 +08:00
parent da191dd7a2
commit ca73a41a0b
15 changed files with 754 additions and 90 deletions
+11 -10
View File
@@ -17,18 +17,19 @@ export async function POST(req: Request) {
return NextResponse.json({ error: "Invalid JSON" }, { status: 400 });
}
// Accept either provider's voice shape — xiaomi carries referenceAudioBase64,
// stepfun carries voiceId. We only check the discriminator + the line text;
// shape-specific validation lives in each provider's synth function.
// Voice is now optional — when the server runs StepFun, the client omits
// the ~220KB Xiaomi reference audio and sends stepfunVoiceId /
// voiceDescription instead (saves Fast Origin Transfer bandwidth). The
// engine's resolveVoice re-provisions on a provider mismatch. We only
// require the beat text + SOMETHING to synthesize from.
const VALID_TTS_PROVIDERS = ["xiaomi", "stepfun"];
if (
!body.beat?.id ||
!body.beat?.line ||
!body.voice?.provider ||
!VALID_TTS_PROVIDERS.includes(body.voice.provider)
) {
const hasVoice =
!!body.voice?.provider && VALID_TTS_PROVIDERS.includes(body.voice.provider);
const hasFallback =
!!body.stepfunVoiceId || !!body.voiceDescription;
if (!body.beat?.id || !body.beat?.line || (!hasVoice && !hasFallback)) {
return NextResponse.json(
{ error: "beat.id, beat.line and voice.provider (xiaomi|stepfun) are required" },
{ error: "beat.id and beat.line are required, plus either voice.provider (xiaomi|stepfun) or stepfunVoiceId/voiceDescription" },
{ status: 400 },
);
}
+25
View File
@@ -0,0 +1,25 @@
import type { TtsProviderResponse } from "@infiplot/types";
import { inferTtsProvider } from "@infiplot/tts-client";
import { NextResponse } from "next/server";
import { loadEngineConfig } from "@/lib/config";
import { requireUser } from "@/lib/supabase/guard";
export const runtime = "nodejs";
// GET /api/tts-provider — tells the client which TTS provider the server is
// configured for, so the play page can shape /api/beat-audio request bodies
// accordingly (skip the ~220KB Xiaomi reference audio when the server runs
// StepFun → saves Fast Origin Transfer bandwidth; the response itself is a
// few dozen bytes). Runs once at /play mount; same auth as other routes so
// the provider (a server-config fact, not user data) isn't leaked publicly.
// BYO client TTS (clientTts:true) takes precedence and bypasses this signal.
export async function GET() {
const auth = await requireUser();
if (auth instanceof NextResponse) return auth;
const cfg = loadEngineConfig();
const provider = cfg.tts ? inferTtsProvider(cfg.tts) : null;
const body: TtsProviderResponse = { provider };
return NextResponse.json(body);
}