feat(tts): StepFun voice selection via CharacterDesigner + provider-aware beat-audio
Make homepage cards and live sessions produce sound when the server is configured for StepFun TTS, instead of silently failing (the prebaked Xiaomi voice was useless on a StepFun server, and wasted ~220KB/beat in Fast Origin Transfer). Three coordinated changes: 1. CharacterDesigner now picks a StepFun preset voice id directly from the 32-entry catalog in the SAME LLM call that designs the character — zero extra latency, LLM-grade match quality. The Xiaomi prompt path is byte-identical to history (verified programmatically) so cache hit rate and voice quality are preserved. pickStepfunVoiceId (keyword scorer) remains the fallback for orphan speakers / invalid LLM picks. 2. The 32-preset catalog moves to lib/tts-client/stepfun-voices.json as the single source of truth, shared by the scorer, the CharacterDesigner prompt, /api/tts-provider, and the offline enrich script. 3. A new GET /api/tts-provider endpoint lets the client probe the server's TTS provider at /play mount. fetchBeatAudio then shapes its request body: on a StepFun server it sends the lightweight stepfunVoiceId / voiceDescription and omits the ~220KB Xiaomi reference audio (FOT saving ~13MB per protagonist per session on prebaked cards). requestBeatAudio re-provisions on a provider mismatch before synth, so audio never goes silent on a cross-provider replay or mid-session provider flip. New type fields are all optional and backward-compatible: Character.stepfunVoiceId, BeatAudioRequest.voiceDescription/characterName/stepfunVoiceId, voice made optional. AGENTS.md updated for the new route, type fields, dependency map, and StepFun voice-selection flow.
This commit is contained in:
+11
-10
@@ -17,18 +17,19 @@ export async function POST(req: Request) {
|
||||
return NextResponse.json({ error: "Invalid JSON" }, { status: 400 });
|
||||
}
|
||||
|
||||
// Accept either provider's voice shape — xiaomi carries referenceAudioBase64,
|
||||
// stepfun carries voiceId. We only check the discriminator + the line text;
|
||||
// shape-specific validation lives in each provider's synth function.
|
||||
// Voice is now optional — when the server runs StepFun, the client omits
|
||||
// the ~220KB Xiaomi reference audio and sends stepfunVoiceId /
|
||||
// voiceDescription instead (saves Fast Origin Transfer bandwidth). The
|
||||
// engine's resolveVoice re-provisions on a provider mismatch. We only
|
||||
// require the beat text + SOMETHING to synthesize from.
|
||||
const VALID_TTS_PROVIDERS = ["xiaomi", "stepfun"];
|
||||
if (
|
||||
!body.beat?.id ||
|
||||
!body.beat?.line ||
|
||||
!body.voice?.provider ||
|
||||
!VALID_TTS_PROVIDERS.includes(body.voice.provider)
|
||||
) {
|
||||
const hasVoice =
|
||||
!!body.voice?.provider && VALID_TTS_PROVIDERS.includes(body.voice.provider);
|
||||
const hasFallback =
|
||||
!!body.stepfunVoiceId || !!body.voiceDescription;
|
||||
if (!body.beat?.id || !body.beat?.line || (!hasVoice && !hasFallback)) {
|
||||
return NextResponse.json(
|
||||
{ error: "beat.id, beat.line and voice.provider (xiaomi|stepfun) are required" },
|
||||
{ error: "beat.id and beat.line are required, plus either voice.provider (xiaomi|stepfun) or stepfunVoiceId/voiceDescription" },
|
||||
{ status: 400 },
|
||||
);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
import type { TtsProviderResponse } from "@infiplot/types";
|
||||
import { inferTtsProvider } from "@infiplot/tts-client";
|
||||
import { NextResponse } from "next/server";
|
||||
import { loadEngineConfig } from "@/lib/config";
|
||||
import { requireUser } from "@/lib/supabase/guard";
|
||||
|
||||
export const runtime = "nodejs";
|
||||
|
||||
// GET /api/tts-provider — tells the client which TTS provider the server is
|
||||
// configured for, so the play page can shape /api/beat-audio request bodies
|
||||
// accordingly (skip the ~220KB Xiaomi reference audio when the server runs
|
||||
// StepFun → saves Fast Origin Transfer bandwidth; the response itself is a
|
||||
// few dozen bytes). Runs once at /play mount; same auth as other routes so
|
||||
// the provider (a server-config fact, not user data) isn't leaked publicly.
|
||||
// BYO client TTS (clientTts:true) takes precedence and bypasses this signal.
|
||||
export async function GET() {
|
||||
const auth = await requireUser();
|
||||
if (auth instanceof NextResponse) return auth;
|
||||
|
||||
const cfg = loadEngineConfig();
|
||||
const provider = cfg.tts ? inferTtsProvider(cfg.tts) : null;
|
||||
|
||||
const body: TtsProviderResponse = { provider };
|
||||
return NextResponse.json(body);
|
||||
}
|
||||
Reference in New Issue
Block a user