- StepFun voice selection: CharacterDesigner picks a preset voiceId from the
32-entry catalog (zero extra LLM call); pickStepfunVoiceId remains as fallback.
- Prebaked homepage cards enriched with stepfunVoiceId (147 characters, gemini model).
- /api/tts-provider endpoint + client probe: skip the ~220KB Xiaomi reference
audio when the server runs StepFun (saves Fast Origin Transfer bandwidth).
- Server-side resolveVoice normalization: re-provisions on provider mismatch.
- Removed hardcoded 1.2x speech playback speed (was for slow MiMo voice).
- Hardened voice-provider validation per PR-agent review.
Xiaomi path prompt is byte-identical to history (prompt-cache-preserving).
Critical: play-page bootstrap infinite loop when AUTH_ENABLED and no
resume snapshot. The refactor changed the gate from
`if (AUTH_ENABLED && hasSnapshot)` to `if (AUTH_ENABLED)`, so any
snapshot-less /play entry (the common case — normal card/preset/custom
start) entered the async branch, got null from consumeResumeSnapshot,
bumped retryBootstrap, and re-ran the effect forever. Restored the
peek-before-await: only enter the async resume branch when a snapshot
actually exists; otherwise fall straight through to normal bootstrap.
Verified via control-flow simulation across all three paths (no
snapshot / snapshot + signed in / snapshot + not signed in).
Major: homepage auto-started a game after a bare OAuth login. Routing
persistPendingStart through AuthModal.onBeforeOAuth fired it for every
OAuth redirect, including bare logins via UserChip / StyleModal
onRequireAuth (where pendingAction is null and the user only wanted to
sign in). Guarded the snapshot on `pendingAction === "start"` so only
the mid-start flow persists; bare logins no longer resurrect the form
and auto-start on return.
Extract the page-agnostic resume primitives into lib/authResume.ts:
- isAuthed() — single login check (was duplicated in app/page.tsx)
- writeResumeSnapshot(key, primary, fallbacks) — quota-safe sessionStorage
write with ordered lighter-payload fallbacks (was hand-rolledTry/catch
in both pages)
- consumeResumeSnapshot<T>(key) — consume-once resume gate that verifies
the user is signed in before returning the snapshot, else clears it
Both pages now share this plumbing while keeping their own snapshot shapes
and restore side effects (home: form fields + start(); play: Session +
restorePlayResume + deferred action replay).
Unify the persist trigger: home previously snapshotted eagerly inside
start() before opening the modal, while play snapshotted in
AuthModal.onBeforeOAuth at redirect time. Move home to the same
onBeforeOAuth trigger so both pages persist at the single OAuth-redirect
instant — the eager-snapshot special case is gone, and OTP (no redirect)
keeps its in-place onSuccess resume on both pages.
Net: -21 lines. Behavior preserved for OTP; OAuth resume now consistent.
Google/GitHub OAuth is a full-page round-trip that unmounts the app and
destroys the in-memory Session (the server is stateless). Returning to
/play?card=m0 re-bootstrapped from the first-act JSON, restarting the
story from scene 1 — the user lost all progress. OTP login kept state
in-memory (no redirect) and was unaffected.
Mirror the homepage 89a5c54 OAuth state-loss fix: snapshot the exact
scene/beat/visited-beats/orientation/image into sessionStorage just
before the redirect, then restore it on mount after the round-trip
(verified signed in). Re-resolve the remote image URL to a fresh blob
(blob: URLs are revoked on unmount). The pending action that hit the
401 (choice / freeform / background-click) is replayed once the restored
state commits, so the player lands exactly where they were headed.
Quota fallback drops the user-uploaded style-reference image (~100KB)
and retries; voices are kept (continuity over rare quota miss). Failure
to restore (corrupt snapshot / not signed in) relinquishes the bootstrap
slot and falls back to normal card/preset/custom start instead of a
blank loading screen.
AuthModal gains an optional onBeforeOAuth callback fired synchronously
before signInWithOAuth navigates away (sessionStorage.setItem is sync).
Two follow-ups from pr-agent review of #79:
1. director.ts voicePromises built a Character WITHOUT stepfunVoiceId, so
on a StepFun server the client (which omits the voice payload to save
FOT) echoed back only voiceDescription — and the server re-scored via
pickStepfunVoiceId every beat instead of honoring the LLM pick. The
whole "CharacterDesigner picks a preset id" mechanism was effectively
bypassed on live StepFun sessions (it only worked for prebaked cards,
which carry stepfunVoiceId in their JSON). Persist stepfunVoiceId onto
the Character so the client→server round-trip keeps the LLM selection.
2. fetchBeatAudio's null-provider branch (probe pending) required
speaker.voice and silently dropped a stepfun-only speaker. Accept any
synthesizable source (voice | stepfunVoiceId | voiceDescription) so a
slow getTtsProvider probe can't drop audio during the first scene's
fetch window. The server resolveVoice normalizes regardless of which
fields arrive.
Make homepage cards and live sessions produce sound when the server is
configured for StepFun TTS, instead of silently failing (the prebaked
Xiaomi voice was useless on a StepFun server, and wasted ~220KB/beat in
Fast Origin Transfer).
Three coordinated changes:
1. CharacterDesigner now picks a StepFun preset voice id directly from the
32-entry catalog in the SAME LLM call that designs the character — zero
extra latency, LLM-grade match quality. The Xiaomi prompt path is
byte-identical to history (verified programmatically) so cache hit rate
and voice quality are preserved. pickStepfunVoiceId (keyword scorer)
remains the fallback for orphan speakers / invalid LLM picks.
2. The 32-preset catalog moves to lib/tts-client/stepfun-voices.json as the
single source of truth, shared by the scorer, the CharacterDesigner
prompt, /api/tts-provider, and the offline enrich script.
3. A new GET /api/tts-provider endpoint lets the client probe the server's
TTS provider at /play mount. fetchBeatAudio then shapes its request body:
on a StepFun server it sends the lightweight stepfunVoiceId /
voiceDescription and omits the ~220KB Xiaomi reference audio (FOT saving
~13MB per protagonist per session on prebaked cards). requestBeatAudio
re-provisions on a provider mismatch before synth, so audio never goes
silent on a cross-provider replay or mid-session provider flip.
New type fields are all optional and backward-compatible: Character.stepfunVoiceId,
BeatAudioRequest.voiceDescription/characterName/stepfunVoiceId, voice made
optional. AGENTS.md updated for the new route, type fields, dependency map,
and StepFun voice-selection flow.
Reverts the regressions from b63b694 on the server-fallback path:
P0 — fetchBeatAudio non-BYO branch was a bare return; every non-BYO
user got silent playback regardless of server TTS config. Re-connect
to /api/beat-audio with the beatAudioAbortRef signal, count 204/!ok
as silence strikes, create a blob URL on success.
P1 — stripVoicesForTransport + mergeCharactersPreserveVoice were
deleted, so the server-fallback path re-sent ~160KB
referenceAudioBase64 per character on every request AND lost voices
for already-known characters after scene 1. Re-add both, applied
ONLY on the server-fallback branches in engineClient.ts (BYO
client-direct path untouched).
P3 — the aborted-before-store blob URL race had no revoke, leaking
one blob URL per cancelled synth. Re-add the else-if revoke.
P2 — handleSettingsSaved ignored ttsConfigured, so a BYO key entered
mid-session only took effect after a page reload. Re-add the ref/state
refresh + audio re-prefetch. Also restore the silence-nudge UI
(silenceStrikes counter, SILENCE_NUDGE_THRESHOLD, dismissible pill
beside the mute toggle) that surfaces BYO-key guidance when the
shared server key is being rate-limited.
Verified live: /api/beat-audio now returns 200 (was 0 calls under
the bug); audio plays after synth completes.
Resolve conflicts: keep login_success alongside the new play_error /
play_visibility_lost analytics events; fold auth retry into the play-page
catch blocks so 401s open the login modal and are NOT tracked as play_error.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- proxy: await getUser() so refreshed session cookies land on the response
- callback: gate on AUTH_ENABLED, reject non-relative next (open redirect)
- page: snapshot + resume form and style image across the OAuth redirect;
require login before the style-image vision parse
- play: wire authResolveRef so login retries the action that hit 401;
dismissing the modal no longer re-fires it
- server: wrap cookie setAll in try/catch for read-only contexts
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Track play_error and play_visibility_lost events via Umami to
distinguish mobile vs desktop failure modes. Each error event
captures orientation, connection type, visibility state, elapsed
time bucket, and error classification — all categorical, no free
text. Includes postJson "HTTP \d+" status parsing for the new
engineClient dual-path architecture.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Introduce user registration/login gated behind optional NEXT_PUBLIC_SUPABASE_*
env vars (leave blank to disable — app behaves exactly as before). Adds
proxy.ts for automatic cookie session refresh, requireUser() API route
guards on all 7 compute-consuming routes, AuthModal (Google/GitHub OAuth +
6-digit email OTP), UserChip header component, and login_success analytics
event. Identity is fully decoupled from Session/engine — no type changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Keep the "transitioning" overlay visible until the <img> element's
bitmap is fully decoded, so the user never sees progressive paint
or a blank flash between scenes.
- Add onImageReady callback to PlayCanvas (<img onLoad> + decode())
- Delay setPhase("ready") until decode resolves (3s timeout fallback)
- Applied to all 4 scene entry paths: prebaked card, live /api/start,
performSceneTransition, and recorded replay transition
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a user has not configured their own model keys in localStorage,
engine calls now automatically route through /api/* server routes
instead of throwing "模型配置未设置". This lets Vercel deploys with
server-side environment variables work out of the box.
- Add lib/engineClient.ts as a unified client-side routing layer:
checks localStorage for BYO config, falls back to POST /api/start,
/api/scene, /api/vision, /api/classify-freeform, /api/insert-beat
- Update app/play/page.tsx to use engineClient instead of direct
engine imports; remove buildEngineConfig()
- Update app/page.tsx style-image parsing to also fall back to
/api/parse-style-image when no local model config exists
Signed-off-by: zhi <zhi@peropero.net>
Walk every speaking beat at export time, reuse current scene's beatAudioMap,
and synth the rest via BYO TTS or /api/beat-audio with concurrency 4. Show a
progress toast on the play page while collecting.
Gallery export keeps audio in a sidecar localStorage key so the first paint
is not blocked by JSON.parse-ing several MB of base64; the gallery lazy-loads
it after the first scene image, then plays per-beat audio with a mute toggle
persisted to localStorage. .infiplot share files embed audioByBeatId in the
doc itself (v2); on import the data URIs survive scene swaps and feed back
into the per-beat audio map so replayers hear the original voices for free.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Hash the lowercased description (matching the case-insensitive scoring)
so the same archetype text picks the same preset regardless of case.
- Thread the character name through provisionVoice -> stepfunProvision as
the hash salt, so two characters that share archetype keywords spread
across the top-N candidate presets instead of collapsing on one voice.
Xiaomi path is unaffected (voicedesign mints a unique clip per call).
Adds a ref-based mutex so concurrent /api/story-pack requests and
duplicate file downloads cannot be triggered by rapid clicking.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Content-Length pre-check to story-pack and story-unpack routes
to reject oversized payloads before buffering the body
- Suppress internal error details in story-unpack catch (was leaking
e.message to the client)
- Strengthen sceneIndex validation: require non-negative integer
- Guard against undefined storyState when replaying shared stories
- Fix prefetch regression: remove currentBeat?.id from useEffect deps
that was re-triggering all change-scene prefetches on every beat
- Fix double detach: use else-if so the second replay detach guard
doesn't fire redundantly after the first already detached
- Align client file-size limit by format (.json 12MB, .infiplot 13MB)
- Move "载入剧情" import button next to "开始" with hover tooltip
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Restrict PR Agent workflow to trusted collaborators on PR comments only,
fix UTF-8 byte counting in gallery-pack, correct portrait-to-landscape
fallback orientation, track inserted freeform beats in visitedBeatIds,
allow clearing stored TTS key, and guard empty-string fuzzy match in
style selector.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Merge vision-click toggle into the shared SettingsModal alongside
player name and TTS key configuration. Remove standalone TtsKeyModal.
Add settings gear button to PlayCanvas dialogue card and header.
Fix fullscreen settings modal not rendering in immersive mode.
Voice toggle uses standard CategorySelect dropdown matching other
tab bar options.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Player name: stored in localStorage, injected into Architect/Writer/InsertBeat
prompts so NPCs address the player by name, displayed in dialogue UI
- Freeform input: compact button at choice nodes expands to text input, LLM
classifier routes to insert-beat (interactive NPC response) or change-scene
- SettingsModal: unified panel merging player name, voice toggle (with
collapsible TTS key section), replacing the old TtsKeyModal
- Insert-beat upgrade: prompt now requires NPC reaction when characters are
present, shared by both freeform and Vision paths
- IME guard: isComposing check on freeform input to prevent CJK mid-composition
submission
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a "导出图集" action at the bottom-right of the play canvas that
snapshots the current session into localStorage and opens
/gallery#id=<id> in a new tab — the original play page keeps running
untouched. In parallel, sends the doc to /api/gallery-pack and
downloads the result as a binary .infiplot file the player can send
to a friend.
The snapshot pulls in:
- Every visited scene's image + beat graph + recorded visit trail
- All AI-prefetched alternate scenes (a new resolvedPrefetchesRef in
PlayInner captures each prefetch as it resolves, so abandoned
branches the engine already paid to generate are kept)
- Character names + basePortraitUrl (voice base64 / styleReference
are stripped — they aren't needed for replay)
/gallery is a no-network interactive replay:
- Per-beat advance and per-choice navigation. Picked choices are
highlighted; unpicked choices are clickable when an alternate was
prefetched, greyed otherwise.
- Stack-based navigation for stepping into branches with one-tap
"返回主线" to collapse back to the main path.
- Top-bar batch download for scene images (including unique
AI-prefetched branch scenes, deduped against the main path) and
character portraits. Fetched with a per-file AbortController + 20s
timeout in a small concurrency pool, then clicked serially.
Prevents one slow CDN response from stranding the busy button.
- In-progress hint banner reminding the player to allow the
browser's "multiple downloads" prompt.
- F-key fullscreen with a top toolbar that auto-retracts after the
initial glance and pops back down on cursor approach.
- Per-scene dialogue panel (fa-clock-rotate-left, matching the
in-game history affordance).
- "导入分享文件" entry on the empty/error state — accepts a friend's
.infiplot, posts to /api/gallery-unpack, renders the decrypted doc.
Share-file format (.infiplot):
- AES-256-GCM via Web Crypto (portable to Cloudflare Workers).
- Layout: 4-byte magic "IFPL" + 1-byte version + 12-byte nonce +
ciphertext (includes 16-byte auth tag).
- Key derived from GALLERY_SECRET via SHA-256.
- GCM's auth tag gives tamper-detection for free; any flip in the
ciphertext/nonce surfaces as "文件校验失败" — same error as wrong-key,
so the distinction can't leak server config.
- Stateless: server keeps no record of issued files.
- GALLERY_SECRET unset → /api/gallery-pack returns 503, the play page
silently skips the share-file download, local view still works.
Rotating the secret invalidates every previously-issued file.
Retention: trimGalleryExports keeps only the 2 most recent localStorage
docs; older ones are evicted before each write so quota stays flat
regardless of how many times the player exports. Share files live on
the player's own disk — no retention concern.
Adds 'gallery_export' to the analytics event schema (scene_count only —
no free text).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mobile users clicking preset story cards now get portrait (9:16) scene
images instead of landscape. Previously card paths hardcoded orientation
to "landscape"; now they respect detectOrientation() and load from
firstact-portrait/ with graceful fallback to landscape.
- Add --portrait and --only flags to prebake-firstacts.mjs
- Add --portrait flag to localize-firstact-images.mjs
- Fix prebake STYLE_MAP extraction (moved to lib/options.ts)
- Generate 60 portrait firstact JSONs + firstscene webp assets
- Remove hardcoded "landscape" in play page card path
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix @AGETNTS.md → @AGENTS.md typo in CLAUDE.md
- Remove ref read inside useMemo (React anti-pattern causing one-frame stale data)
- Simplify buildDialogueHistory to read visitedBeatIds directly from session.history,
which also fixes incorrect scene-ID matching when the same ID appears multiple times
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The BYO (Bring Your Own) API key configuration for LLM and image
generation will be re-implemented via Cloudflare Workers. Remove
the client-side implementation to prepare for that migration.
TTS (text-to-speech) BYO key support is intentionally preserved.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three transport-only optimizations that cut per-session Vercel FOT by ~50-60%:
P0 — Server strips voice.referenceAudioBase64 from already-known characters
in /api/scene and /api/insert-beat responses (defense-in-depth).
P1 — Client strips all voice data from session before sending to
/api/scene, /api/vision, and /api/insert-beat. Voices are retained locally
and re-merged from responses via mergeCharactersPreserveVoice(). The engine
only needs character names + visualDescriptions for scene generation.
P3 — /api/beat-audio returns binary audio (Response with Content-Type)
instead of JSON-wrapped base64, saving ~33% encoding overhead. Client
converts to blob URLs; PlayCanvas accepts a single audioSrc prop.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(security): harden BYO API header against SSRF and input abuse
- Add lib/validateUrl.ts with HTTPS-only + public-IP enforcement,
provider allowlist, IPv6 rejection, and userinfo-in-URL blocking.
- Add lib/byoHeaders.ts — single source of truth for client-side BYO
header construction (deduplicates app/page.tsx & app/play/page.tsx).
- config.ts: validate BYO endpoints via isPublicUrl(), cap header at
2 KB, truncate apiKey/model strings, sanitize log output.
- fetchWithRetry: default redirect to "manual" to block 302-to-intranet.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(security): address Copilot review — trim endpoint, strip control chars, drop unused import
- safeEndpoint: trim whitespace before URL validation
- safeString: strip ASCII control characters to prevent header injection
- play/page.tsx: remove unused BYO_STORAGE_KEY import
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Set the session orientation in an isomorphic layout effect so portrait
phones don't flash the landscape loading chrome for a frame before the
bootstrap effect runs. State still inits to "landscape" for SSR-safety;
the correction now lands before first paint (no-op on landscape devices).
Addresses Copilot review on PR #31.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Thread orientation (portrait|landscape) from client through API, engine,
and image gen. Portrait devices render 1024x1792 (9:16) full-bleed scenes;
desktop/landscape keeps 1792x1024 (16:9). Adds cover-aware click→image
coordinate mapping, session-locked orientation, a shared coerceOrientation
helper, and a choices overflow cap in portrait.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Public users share one server TTS key, so Xiaomi's per-key RPM/TPM limits
cause silent playback under concurrency. This adds an OPTIONAL path: a user
can store their own Xiaomi MiMo key in the browser and synthesize voice
client-side against Xiaomi's CORS-open endpoints. The key lives only in
localStorage and is never sent to or logged by our server; the shared server
key still serves everyone who does not opt in.
- components/TtsKeyModal.tsx: shared key modal (key-family + region picker),
reused by both the home and play pages
- app/play/page.tsx: silence nudge moved beside the mute toggle; modal opens
in place instead of redirecting to the home page
- app/page.tsx: home page consumes the shared modal + readStoredTtsConfig
- lib/clientTtsConfig.ts, lib/ttsPresets.ts: browser config + region presets
- app/api/{start,scene,insert-beat}: thread per-request voice; lib/types update
- docs/xiaomi-tts-key.md + README note
Verified with tsc --noEmit (exit 0).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Addresses two GitHub Copilot review comments on PR #24:
- preloadImage cleared the 20s timeout in onload, before awaiting
img.decode(), leaving the decode phase unguarded — a hung decode could
keep the promise pending forever and stall the play loop. Move
clearTimeout into a single idempotent done() so the timeout stays armed
through decode() too, matching the stated "timeouts resolve quietly"
intent.
- .env.example said to leave BOTH proxy vars blank, but shipped
NEXT_PUBLIC_IMAGE_PROXY_ALLOWED_HOSTS=im.runware.ai. Only
NEXT_PUBLIC_IMAGE_PROXY_URL gates the feature; the allowlist is inert
until the URL is set. Corrected the wording, kept the self-documenting
default value.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Address the Copilot review on #26.
#1 The game_start / art_style_select payload fields were typed as bare
`string`, so free text could still slip through despite the "content-free
by construction" claim. Add lib/options.ts as the single source of truth
for the selector option sets (`as const` → literal-union types), have the
home OPTS render from those arrays, and type the analytics fields from the
derived unions (gender/art_style/plot_style/pacing/style) plus a template
type for `card`. Free text now fails to compile; no casts at call sites.
#2 The /play heartbeat scheduled its 30s interval unconditionally. Gate the
effect on the same NEXT_PUBLIC_UMAMI_* env used for script injection, so
nothing is scheduled when the tracker is off (visibility check kept — a
hidden tab still never emits).
#3 choice_select no longer emits a -1 choice_index: skip the event when the
index can't be resolved instead of polluting the index distribution.
Verified with tsc (exit 0) and a throwaway negative test: free text in any
of the six fields raises TS2322, valid enum/template values compile.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
yuanzonghao
Zonghao Yuan
baizhi958216
baizhi958216
DESKTOP-I1T6TF3\Q