- Model support: text/vision now OpenAI-compatible only (Claude/Gemini
via their OpenAI-compatible endpoints); drop stale native-protocol copy
and the anthropic/google rows from the *_PROVIDER table
- TTS: document StepFun (step-tts-2, paid, better quality) alongside
Xiaomi MiMo (free) across zh/en/ja; update Vercel deploy envDescription
- Claude: note that direct Anthropic endpoints lack caching, recommend
gateway for full experience
- Cloudflare: preserve existing one-click deploy (compat work in progress)
Three follow-ups to ef3b579 (OpenAI SDK migration) and ebe39ef (canvas frame):
- .env.example / config.ts / AGENTS.md: anthropic & google native protocols
were removed with the Vercel AI SDK, but .env.example and AGENTS.md still
advertised them. Rewrite the docs to point Claude/Gemini at their
OpenAI-compatible endpoints (api.anthropic.com/v1,
generativelanguage.googleapis.com/v1beta/openai), drop the dead Gemini
"Nano Banana" image example, sync AGENTS.md (text/vision protocol list,
image protocol list, the "OpenAI/Gemini via AI SDK" reference note), and
append a short hint in readProvider() error message guiding
anthropic/google users to openai_compatible instead of a bare rejection.
- chat.ts: drop the unsafe `as { prompt_tokens_details?: ... }` cast; read
cached_tokens straight off the SDK's CompletionUsage type. Add a comment
noting the OpenAI usage object reports cache reads only (no cache-write
count), so the create cost the old AI SDK path logged is unrecoverable.
- PlayCanvas.tsx: revert <img key={imageUrl}> to key={imageUrl.slice(-48)}.
The gpt-image/mock paths emit multi-MB data URIs; using the full string as
React's reconciliation key adds avoidable diff overhead during the frequent
re-renders. Matches the existing <audio> element's key convention.
Validation: pnpm typecheck passes. (pnpm lint fails on a pre-existing Next 16
`next lint` CLI issue, identical on staging — unrelated to this change.)
Introduce user registration/login gated behind optional NEXT_PUBLIC_SUPABASE_*
env vars (leave blank to disable — app behaves exactly as before). Adds
proxy.ts for automatic cookie session refresh, requireUser() API route
guards on all 7 compute-consuming routes, AuthModal (Google/GitHub OAuth +
6-digit email OTP), UserChip header component, and login_success analytics
event. Identity is fully decoupled from Session/engine — no type changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
IMAGE_TIMEOUT_MS sets a per-attempt hard deadline (AbortSignal.timeout);
IMAGE_HEDGE_MS races a second identical scene-paint request when the
first is still pending past the threshold. Both default to OFF when
unset, preserving historical behavior for self-hosted deploys.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Validate voice.provider against known whitelist (xiaomi|stepfun) in
beat-audio route to return a clear 400 instead of falling through
- Move single-char pronouns (他/她) to weak-signal fallback in
detectGender to avoid false positives on compounds like 其他
- Update .env.example with StepFun configuration examples
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a "导出图集" action at the bottom-right of the play canvas that
snapshots the current session into localStorage and opens
/gallery#id=<id> in a new tab — the original play page keeps running
untouched. In parallel, sends the doc to /api/gallery-pack and
downloads the result as a binary .infiplot file the player can send
to a friend.
The snapshot pulls in:
- Every visited scene's image + beat graph + recorded visit trail
- All AI-prefetched alternate scenes (a new resolvedPrefetchesRef in
PlayInner captures each prefetch as it resolves, so abandoned
branches the engine already paid to generate are kept)
- Character names + basePortraitUrl (voice base64 / styleReference
are stripped — they aren't needed for replay)
/gallery is a no-network interactive replay:
- Per-beat advance and per-choice navigation. Picked choices are
highlighted; unpicked choices are clickable when an alternate was
prefetched, greyed otherwise.
- Stack-based navigation for stepping into branches with one-tap
"返回主线" to collapse back to the main path.
- Top-bar batch download for scene images (including unique
AI-prefetched branch scenes, deduped against the main path) and
character portraits. Fetched with a per-file AbortController + 20s
timeout in a small concurrency pool, then clicked serially.
Prevents one slow CDN response from stranding the busy button.
- In-progress hint banner reminding the player to allow the
browser's "multiple downloads" prompt.
- F-key fullscreen with a top toolbar that auto-retracts after the
initial glance and pops back down on cursor approach.
- Per-scene dialogue panel (fa-clock-rotate-left, matching the
in-game history affordance).
- "导入分享文件" entry on the empty/error state — accepts a friend's
.infiplot, posts to /api/gallery-unpack, renders the decrypted doc.
Share-file format (.infiplot):
- AES-256-GCM via Web Crypto (portable to Cloudflare Workers).
- Layout: 4-byte magic "IFPL" + 1-byte version + 12-byte nonce +
ciphertext (includes 16-byte auth tag).
- Key derived from GALLERY_SECRET via SHA-256.
- GCM's auth tag gives tamper-detection for free; any flip in the
ciphertext/nonce surfaces as "文件校验失败" — same error as wrong-key,
so the distinction can't leak server config.
- Stateless: server keeps no record of issued files.
- GALLERY_SECRET unset → /api/gallery-pack returns 503, the play page
silently skips the share-file download, local view still works.
Rotating the secret invalidates every previously-issued file.
Retention: trimGalleryExports keeps only the 2 most recent localStorage
docs; older ones are evicted before each write so quota stays flat
regardless of how many times the player exports. Share files live on
the player's own disk — no retention concern.
Adds 'gallery_export' to the analytics event schema (scene_count only —
no free text).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- TEXT/VISION: add native Anthropic & Google Gemini paths via Vercel AI SDK,
selectable through TEXT_PROVIDER / VISION_PROVIDER (default openai_compatible)
- IMAGE: expand to openai (gpt-image) / google (Nano Banana) via AI SDK
alongside the existing Runware task-array and OpenAI-compatible REST paths
- normalizeBaseUrl: tolerate URLs with/without /v1 (or /chat/completions);
append the per-protocol version segment only for bare hosts
- config: readProvider() reads *_PROVIDER; types: ProviderProtocol + provider?
- deps: @ai-sdk/anthropic, @ai-sdk/google; docs in .env.example + README
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Addresses two GitHub Copilot review comments on PR #24:
- preloadImage cleared the 20s timeout in onload, before awaiting
img.decode(), leaving the decode phase unguarded — a hung decode could
keep the promise pending forever and stall the play loop. Move
clearTimeout into a single idempotent done() so the timeout stays armed
through decode() too, matching the stated "timeouts resolve quietly"
intent.
- .env.example said to leave BOTH proxy vars blank, but shipped
NEXT_PUBLIC_IMAGE_PROXY_ALLOWED_HOSTS=im.runware.ai. Only
NEXT_PUBLIC_IMAGE_PROXY_URL gates the feature; the allowlist is inert
until the URL is set. Corrected the wording, kept the self-documenting
default value.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Instrument the play flow with 9 content-free custom events (game_start,
art_style_select, style_image_upload, scene_reached, choice_select,
vision_click, tts_toggle, fullscreen_toggle, play_heartbeat) to measure
retention, engagement depth and session duration.
Privacy is enforced by construction, not convention:
- lib/analytics.ts types each event with a discriminated union, so a
payload has no slot for free text — prompts, world guides, uploaded
images and vision output can never reach analytics (compile-time
guarantee, not a comment).
- track() no-ops without window.umami and never throws into the app.
- coarse 30s heartbeat fires only while the tab is visible.
- script stays gated on NEXT_PUBLIC_UMAMI_* env (blank → no script),
honours Do-Not-Track, and locks to an exact data-domains allowlist.
- one-line on-site disclosure with a link, shown only when tracking is on.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
b805b1d routed every scene <img> through fetch → Blob → createObjectURL to
kill QUIC progressive-paint, but in doing so added an *unconditional*
dependency on a CORS-adding proxy. That breaks the default deployment:
im.runware.ai sends no Access-Control-Allow-Origin, so a direct
fetch().blob() throws and the scene image silently fails to load for anyone
who hasn't stood up the Cloudflare Worker.
Restore the pre-b805b1d behavior as the *default* and make the proxy
strictly opt-in:
- Direct path (no env set): preloadImage() warms the HTTP cache + decodes,
then <img> uses the original https://im.runware.ai URL — as before
b805b1d. No fetch().blob(), no CORS dependency: a fresh clone just works.
- Proxy path (NEXT_PUBLIC_IMAGE_PROXY_URL set): fetch the proxied URL →
Blob → createObjectURL, exactly as b805b1d, gaining the QUIC-immune
HTTP/2 edge + atomic paint.
shouldProxy(url) gates the two paths: proxy only when a base is configured
AND the host is in NEXT_PUBLIC_IMAGE_PROXY_ALLOWED_HOSTS (default
im.runware.ai). data: / non-http / unknown-host URLs always take the direct
path. blobUrlCache + revoke logic is unchanged and safe for both paths
(revoke is a no-op on non-blob: URLs).
The Cloudflare Worker moves out of this repo into a standalone, one-click-
deployable project (infiplot-image-proxy) so the optional infra isn't
carried by every clone; .env.example and the READMEs link to it.
restore: preloadImage() helper deleted by b805b1d
add: NEXT_PUBLIC_IMAGE_PROXY_ALLOWED_HOSTS (default im.runware.ai)
remove: worker/ (moved to standalone repo)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Symptom: in Chrome on certain networks the scene <img> renders row-by-row
from top to bottom — "层层加载" — instead of appearing atomically.
Root cause (confirmed via DevTools):
- Chrome opportunistically opens HTTP/3 (QUIC) to im.runware.ai.
- QUIC streams to Runware sometimes error mid-transfer:
net::ERR_QUIC_PROTOCOL_ERROR
HTTP-level status stays 200 (response headers received), but bytes are
truncated. The browser paints whatever PNG bytes it has so far → visible
row-by-row decode.
- The earlier preloadImage()+decode() trick can't fix this — neither
HTTP-cache reuse nor sync decode helps when the bytes themselves were
never fully delivered.
Two-tier fix:
1. Client: fetch → Blob → URL.createObjectURL() (app/play/page.tsx)
- <img src> only ever points to a blob: URL whose bytes are 100%
resident in the JS heap. No network-backed src = no possibility of
progressive paint.
- Module-level blobUrlCache keys by original URL so speculative
prefetch + the eventual commit share one fetch.
- Old blobs are URL.revokeObjectURL()'d on scene swap + unmount to
release memory.
2. Network: optional Cloudflare Worker proxy (worker/)
- Browser ↔ Worker is HTTP/2 over CF edge (extremely stable).
- Worker ↔ Runware is a server-to-server fetch (no QUIC fragility,
Cloudflare's backbone handles transit).
- Worker buffers the full upstream response → client never sees a
half-stream.
- Bonus: CF edge cache (cacheEverything, 1y TTL) on Runware UUIDs;
Access-Control-Allow-Origin: * so client fetch() can't hit CORS.
- Hardened: only proxies im.runware.ai, only GET/HEAD/OPTIONS, all
other hosts/methods → 403/405.
Wired via NEXT_PUBLIC_IMAGE_PROXY_URL (inlined at build). Empty → no proxy
→ direct fetch (which still uses the blob path, just exposed to QUIC).
──────────────────────────────────────────────────────────────────────
Deploy steps (one-time, do this AFTER pulling this commit):
1. Install wrangler globally:
npm i -g wrangler
2. Log in to Cloudflare (opens browser for OAuth):
wrangler login
3. From the worker/ directory, deploy:
cd worker
wrangler deploy
wrangler will print the deployed URL, e.g.
https://infiplot-image-proxy.<your-cf-username>.workers.dev
4. Paste that URL into .env.local for local dev:
NEXT_PUBLIC_IMAGE_PROXY_URL=https://infiplot-image-proxy.<...>.workers.dev
…and into Vercel project settings (Environment Variables) for prod.
NEXT_PUBLIC_ vars are inlined at build time, so the URL bakes into
the bundle on the next deploy/dev-server restart.
5. Restart dev server (pnpm dev) so the new env baked in. Generate a
scene; Network tab should show requests going to *.workers.dev
instead of im.runware.ai, no ERR_QUIC_PROTOCOL_ERROR, image renders
atomically.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Cookieless, env-gated page-view tracking via Umami. The <Analytics />
component injects the script only when NEXT_PUBLIC_UMAMI_SRC and
NEXT_PUBLIC_UMAMI_WEBSITE_ID are both set, so local dev and forks send
nothing to our instance. Adds .env.example docs (section 6) and a
homepage footer privacy disclosure. No Cookie consent banner needed.
Flatten the pnpm monorepo (apps/web + packages/*) into a single web package at the repo root.
- Move app/lib/components/scripts/public to root; drop apps/web and packages/* wrappers
- Rewrite tsconfig paths (@infiplot/*) to ./lib/*; turbopack.root = __dirname
- Update Vercel (no root-directory) and Cloudflare (pnpm build:cf at root) deploy paths
- Regenerate pnpm-lock.yaml to drop stale workspace importers
- Bump engines.node to >=22 to match wrangler
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Engine
- Split /api/vision out from /api/interact so client can drive
prefetch + cache lookup independently of click interpretation
- Image client switched to chat-completions+modalities API (OpenRouter/
provider style), supporting markdown image URL responses
- annotateClick now resizes to 768w before composite to keep vision
payloads small and avoid CDN timeouts
- Prompts updated to mention "JSON" in user messages (required by
Gemini's strict JSON mode)
- Shared fetchWithRetry helper: 2 retries for chat/image, 0 for vision
(with 60s hard timeout)
Client
- Parallel prefetch of all three choice branches on each new frame
- Effect deliberately excludes phase from deps so user-click doesn't
abort in-flight prefetches
- Cache hit/miss/free-form fallback handled in handleClick
- PlayCanvas reads img naturalWidth/Height and adapts container to
whatever aspect AI returns (no more cropped third choice)
- max-width raised to 560px, max-height calc(100dvh - 200px)
Misc
- README env-path corrected to apps/web/.env.local
- users.md: BGM/TTS idea note
- .env.example moved into apps/web alongside next config
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
yuanzonghao
Zonghao Yuan
DESKTOP-I1T6TF3\Q