Walk every speaking beat at export time, reuse current scene's beatAudioMap,
and synth the rest via BYO TTS or /api/beat-audio with concurrency 4. Show a
progress toast on the play page while collecting.
Gallery export keeps audio in a sidecar localStorage key so the first paint
is not blocked by JSON.parse-ing several MB of base64; the gallery lazy-loads
it after the first scene image, then plays per-beat audio with a mute toggle
persisted to localStorage. .infiplot share files embed audioByBeatId in the
doc itself (v2); on import the data URIs survive scene swaps and feed back
into the per-beat audio map so replayers hear the original voices for free.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replace the auto-generated kyoani / shinkai style thumbnails with hand-picked
reference frames. Source PNGs were center-cropped to square and re-encoded as
512x512 WEBP (~41KB each) to match the existing thumbnail format. Bumps the
shared cache-buster from v5 to v6 so existing browsers fetch the new files.
The home-page file-import button accepts .infiplot story files. The
tooltip now spells out the file type so users distinguish it from
"开始剧情"/"载入预设" affordances on the same screen.
- Validate voice.provider against known whitelist (xiaomi|stepfun) in
beat-audio route to return a clear 400 instead of falling through
- Move single-char pronouns (他/她) to weak-signal fallback in
detectGender to avoid false positives on compounds like 其他
- Update .env.example with StepFun configuration examples
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Hash the lowercased description (matching the case-insensitive scoring)
so the same archetype text picks the same preset regardless of case.
- Thread the character name through provisionVoice -> stepfunProvision as
the hash salt, so two characters that share archetype keywords spread
across the top-N candidate presets instead of collapsing on one voice.
Xiaomi path is unaffected (voicedesign mints a unique clip per call).
Add StepFun step-tts-mini / step-tts-2 / stepaudio-2.5-tts as an alternate
TTS provider alongside Xiaomi MiMo. Auto-detected from TTS_BASE_URL host
(contains `stepfun.com` → StepFun; otherwise → MiMo), mirroring how the
image client infers Runware from `*.runware.ai`.
CharacterVoice becomes a discriminated union on `provider`:
- xiaomi: { referenceAudioBase64, mimeType } — unchanged
- stepfun: { voiceId, model, mimeType } — preset voice ID + chosen model
Provision dispatches on the current cfg's base URL; synthesis dispatches
on the voice's own `provider` tag so a session with mixed voices (e.g. a
provider switch mid-development) routes each beat through the correct
protocol. xiaomiSynthesize now guards against being called with a non-
xiaomi voice, surfacing the bug as a clear runtime error instead of a
TypeScript narrow violation at the access site.
StepFun has no voicedesign equivalent — only preset voices + voice
cloning from a reference audio upload. Cloning would require an extra
asset per character, so v1 maps the LLM's Chinese voiceDescription to one
of the 32 published preset IDs via gender + age + tone keyword scoring,
with a deterministic hash spread across the top-3 candidates so multiple
characters with similar descriptions don't collapse onto the identical
preset. lineDelivery is accepted but not yet propagated to StepFun's
voice_label.emotion / .style fields — left as a follow-up.
beat-audio route validation relaxed from `voice.referenceAudioBase64`
(xiaomi-shaped) to `voice.provider` (shape-agnostic), so stepfun voices
pass the gate; provider-specific shape errors still surface from the
synth function.
Observed latency on InfiPlot's dev loop: StepFun step-tts-mini median
~2.3s per beat with 0% timeouts across the test session, vs MiMo's
median ~8s with the long tail tripping the existing 15s synth budget
on roughly 2 of 3 beats. Pricing: step-tts-mini ¥0.9/万字符 (~¥0.14
per typical 50-beat session) vs MiMo TTS currently free under the
Token Plan creator incentive.
AGENTS.md provider matrix updated to describe both providers and the
discriminated-union dispatch.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a ref-based mutex so concurrent /api/story-pack requests and
duplicate file downloads cannot be triggered by rapid clicking.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add Content-Length pre-check to story-pack and story-unpack routes
to reject oversized payloads before buffering the body
- Suppress internal error details in story-unpack catch (was leaking
e.message to the client)
- Strengthen sceneIndex validation: require non-negative integer
- Guard against undefined storyState when replaying shared stories
- Fix prefetch regression: remove currentBeat?.id from useEffect deps
that was re-triggering all change-scene prefetches on every beat
- Fix double detach: use else-if so the second replay detach guard
doesn't fire redundantly after the first already detached
- Align client file-size limit by format (.json 12MB, .infiplot 13MB)
- Move "载入剧情" import button next to "开始" with hover tooltip
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Handle downloadImagesAsZip return value and surface errors to user
- Fix inferImageExtension garbage output for data URIs without semicolons
- Scale blob URL revocation delay for large zip files (>5MB → 60s)
- Cap uniqueZipPath dedup loop at 10k iterations with timestamp fallback
- Support relative URLs in inferImageExtension via base URL
- Handle svg+xml MIME subtype correctly
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Restrict PR Agent workflow to trusted collaborators on PR comments only,
fix UTF-8 byte counting in gallery-pack, correct portrait-to-landscape
fallback orientation, track inserted freeform beats in visitedBeatIds,
allow clearing stored TTS key, and guard empty-string fuzzy match in
style selector.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Merge vision-click toggle into the shared SettingsModal alongside
player name and TTS key configuration. Remove standalone TtsKeyModal.
Add settings gear button to PlayCanvas dialogue card and header.
Fix fullscreen settings modal not rendering in immersive mode.
Voice toggle uses standard CategorySelect dropdown matching other
tab bar options.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add 14 new painting styles sourced from preset story card generation
scripts: Dunhuang fresco, Persian miniature, Byzantine mosaic, stained
glass, vaporwave, vector illustration, low poly, pop art, glitch art,
papercut, steampunk, xianxia fantasy, dark fairytale, and urban fantasy.
Reorder all 36 styles into logical visual categories (anime → cinematic
→ Eastern traditional → Western traditional → genre → digital → handcraft)
for easier browsing. Update "auto" thumbnail to a 3×3 composite grid and
"custom" thumbnail to a paintbrush-on-canvas concept image.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Player name: stored in localStorage, injected into Architect/Writer/InsertBeat
prompts so NPCs address the player by name, displayed in dialogue UI
- Freeform input: compact button at choice nodes expands to text input, LLM
classifier routes to insert-beat (interactive NPC response) or change-scene
- SettingsModal: unified panel merging player name, voice toggle (with
collapsible TTS key section), replacing the old TtsKeyModal
- Insert-beat upgrade: prompt now requires NPC reaction when characters are
present, shared by both freeform and Vision paths
- IME guard: isComposing check on freeform input to prevent CJK mid-composition
submission
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a "导出图集" action at the bottom-right of the play canvas that
snapshots the current session into localStorage and opens
/gallery#id=<id> in a new tab — the original play page keeps running
untouched. In parallel, sends the doc to /api/gallery-pack and
downloads the result as a binary .infiplot file the player can send
to a friend.
The snapshot pulls in:
- Every visited scene's image + beat graph + recorded visit trail
- All AI-prefetched alternate scenes (a new resolvedPrefetchesRef in
PlayInner captures each prefetch as it resolves, so abandoned
branches the engine already paid to generate are kept)
- Character names + basePortraitUrl (voice base64 / styleReference
are stripped — they aren't needed for replay)
/gallery is a no-network interactive replay:
- Per-beat advance and per-choice navigation. Picked choices are
highlighted; unpicked choices are clickable when an alternate was
prefetched, greyed otherwise.
- Stack-based navigation for stepping into branches with one-tap
"返回主线" to collapse back to the main path.
- Top-bar batch download for scene images (including unique
AI-prefetched branch scenes, deduped against the main path) and
character portraits. Fetched with a per-file AbortController + 20s
timeout in a small concurrency pool, then clicked serially.
Prevents one slow CDN response from stranding the busy button.
- In-progress hint banner reminding the player to allow the
browser's "multiple downloads" prompt.
- F-key fullscreen with a top toolbar that auto-retracts after the
initial glance and pops back down on cursor approach.
- Per-scene dialogue panel (fa-clock-rotate-left, matching the
in-game history affordance).
- "导入分享文件" entry on the empty/error state — accepts a friend's
.infiplot, posts to /api/gallery-unpack, renders the decrypted doc.
Share-file format (.infiplot):
- AES-256-GCM via Web Crypto (portable to Cloudflare Workers).
- Layout: 4-byte magic "IFPL" + 1-byte version + 12-byte nonce +
ciphertext (includes 16-byte auth tag).
- Key derived from GALLERY_SECRET via SHA-256.
- GCM's auth tag gives tamper-detection for free; any flip in the
ciphertext/nonce surfaces as "文件校验失败" — same error as wrong-key,
so the distinction can't leak server config.
- Stateless: server keeps no record of issued files.
- GALLERY_SECRET unset → /api/gallery-pack returns 503, the play page
silently skips the share-file download, local view still works.
Rotating the secret invalidates every previously-issued file.
Retention: trimGalleryExports keeps only the 2 most recent localStorage
docs; older ones are evicted before each write so quota stays flat
regardless of how many times the player exports. Share files live on
the player's own disk — no retention concern.
Adds 'gallery_export' to the analytics event schema (scene_count only —
no free text).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Eliminate the dual code path (raw fetch vs AI SDK) for text and vision.
All providers now go through createLanguageModel() + generateText(),
removing chatOpenAiCompatible/analyzeOpenAiCompatible, the manual Usage
type, summarizeUsage, and responseFormat plumbing from 8 call sites.
Key fix: @ai-sdk/openai v3 defaults to the Responses API (/responses);
DeepSeek only supports Chat Completions, so we use .chat() explicitly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mobile users clicking preset story cards now get portrait (9:16) scene
images instead of landscape. Previously card paths hardcoded orientation
to "landscape"; now they respect detectOrientation() and load from
firstact-portrait/ with graceful fallback to landscape.
- Add --portrait and --only flags to prebake-firstacts.mjs
- Add --portrait flag to localize-firstact-images.mjs
- Fix prebake STYLE_MAP extraction (moved to lib/options.ts)
- Generate 60 portrait firstact JSONs + firstscene webp assets
- Remove hardcoded "landscape" in play page card path
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add isComposing guard to the homepage prompt textarea so CJK users
no longer accidentally submit while composing. Also show a subtle
"Enter 发送 · Shift+Enter 换行" hint when the input has content.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When user picks "自动", the client sends styleGuide="auto" to the
server. The orchestrator then runs a lightweight style-selector LLM
call in parallel with the Architect — both only depend on worldSetting,
so there is zero added latency. The selector picks the best-matching
preset from STYLE_MAP based on genre, mood, and setting.
Also moves STYLE_MAP from page.tsx to lib/options.ts so it can be
shared between client and server.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Remove unused `isAuto` variable after magic-wand button removal
- Add focus-visible ring to style cards for keyboard accessibility
- Update DEFAULT_STYLE comment to match actual fallback (吉卜力)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rework custom style view: fixed modal height to match grid view, move
upload and preset-import controls to bottom toolbar alongside cancel/save,
textarea fills remaining space. Add bordered style to cancel button,
improve disabled save button visibility, remove per-card magic-wand
customize button, and add placeholder hint about English prompts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix @AGETNTS.md → @AGENTS.md typo in CLAUDE.md
- Remove ref read inside useMemo (React anti-pattern causing one-frame stale data)
- Simplify buildDialogueHistory to read visitedBeatIds directly from session.history,
which also fixes incorrect scene-ID matching when the same ID appears multiple times
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Rewrite all 20 STYLE_MAP prompts with precise art terminology (sfumato,
feibai, bokashi, broken-color, etc.) and richer color/texture descriptions.
KyoAni prompt now references Beyond the Boundary and Sound Euphonium;
Ghibli references Spirited Away and Howl's Moving Castle. Regenerate all
style thumbnails using a two-step pipeline: DeepSeek picks an optimal
visual-novel scene per style, then Runware renders it. Add cache-busting
query param (thumbV) to thumbnail URLs. Include gen-style-thumbs.ts script
for future regeneration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Redesign the painting-style picker inspired by Pollo AI: widen modal to
1400px, show styles as square thumbnail cards in a 4-column grid with
name labels below, add ember glow hover effect, and split custom-style
editing into its own view. Simplify style names (e.g. "京阿尼细腻日常" →
"京阿尼"), add 22 .webp preview thumbnails, and remove the per-preset
override mechanism in favor of a cleaner grid + custom flow.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Code-level `export const maxDuration = 60` and vercel.json `functions`
block were overriding the dashboard's 300s setting, causing ~100 504
timeouts per day on /api/scene and /api/start. Removing them lets each
Vercel plan use its own default (60s Hobby, 300s Pro) without breaking
self-deployers.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The BYO (Bring Your Own) API key configuration for LLM and image
generation will be re-implemented via Cloudflare Workers. Remove
the client-side implementation to prepare for that migration.
TTS (text-to-speech) BYO key support is intentionally preserved.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three transport-only optimizations that cut per-session Vercel FOT by ~50-60%:
P0 — Server strips voice.referenceAudioBase64 from already-known characters
in /api/scene and /api/insert-beat responses (defense-in-depth).
P1 — Client strips all voice data from session before sending to
/api/scene, /api/vision, and /api/insert-beat. Voices are retained locally
and re-merged from responses via mergeCharactersPreserveVoice(). The engine
only needs character names + visualDescriptions for scene generation.
P3 — /api/beat-audio returns binary audio (Response with Content-Type)
instead of JSON-wrapped base64, saving ~33% encoding overhead. Client
converts to blob URLs; PlayCanvas accepts a single audioSrc prop.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(security): harden BYO API header against SSRF and input abuse
- Add lib/validateUrl.ts with HTTPS-only + public-IP enforcement,
provider allowlist, IPv6 rejection, and userinfo-in-URL blocking.
- Add lib/byoHeaders.ts — single source of truth for client-side BYO
header construction (deduplicates app/page.tsx & app/play/page.tsx).
- config.ts: validate BYO endpoints via isPublicUrl(), cap header at
2 KB, truncate apiKey/model strings, sanitize log output.
- fetchWithRetry: default redirect to "manual" to block 302-to-intranet.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(security): address Copilot review — trim endpoint, strip control chars, drop unused import
- safeEndpoint: trim whitespace before URL validation
- safeString: strip ASCII control characters to prevent header injection
- play/page.tsx: remove unused BYO_STORAGE_KEY import
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Set the session orientation in an isomorphic layout effect so portrait
phones don't flash the landscape loading chrome for a frame before the
bootstrap effect runs. State still inits to "landscape" for SSR-safety;
the correction now lands before first paint (no-op on landscape devices).
Addresses Copilot review on PR #31.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Thread orientation (portrait|landscape) from client through API, engine,
and image gen. Portrait devices render 1024x1792 (9:16) full-bleed scenes;
desktop/landscape keeps 1792x1024 (16:9). Adds cover-aware click→image
coordinate mapping, session-locked orientation, a shared coerceOrientation
helper, and a choices overflow cap in portrait.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Harden the BYO-mode signal at the API boundary (start/scene/insert-beat):
only clientTts === true drops server TTS, so a stray truthy non-boolean can't
silently disable it. Add a non-blocking prefix hint in TtsKeyModal that warns
when the pasted key prefix (tp-/sk-) mismatches the selected key type — a
mismatch hits the wrong endpoint and plays silently, the symptom BYO fixes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Public users share one server TTS key, so Xiaomi's per-key RPM/TPM limits
cause silent playback under concurrency. This adds an OPTIONAL path: a user
can store their own Xiaomi MiMo key in the browser and synthesize voice
client-side against Xiaomi's CORS-open endpoints. The key lives only in
localStorage and is never sent to or logged by our server; the shared server
key still serves everyone who does not opt in.
- components/TtsKeyModal.tsx: shared key modal (key-family + region picker),
reused by both the home and play pages
- app/play/page.tsx: silence nudge moved beside the mute toggle; modal opens
in place instead of redirecting to the home page
- app/page.tsx: home page consumes the shared modal + readStoredTtsConfig
- lib/clientTtsConfig.ts, lib/ttsPresets.ts: browser config + region presets
- app/api/{start,scene,insert-beat}: thread per-request voice; lib/types update
- docs/xiaomi-tts-key.md + README note
Verified with tsc --noEmit (exit 0).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
DESKTOP-I1T6TF3\Q
Zonghao Yuan
yuanzonghao
baizhi958216