fix(play): scene image renders progressively from top → CF Worker proxy
Symptom: in Chrome on certain networks the scene <img> renders row-by-row
from top to bottom — "层层加载" — instead of appearing atomically.
Root cause (confirmed via DevTools):
- Chrome opportunistically opens HTTP/3 (QUIC) to im.runware.ai.
- QUIC streams to Runware sometimes error mid-transfer:
net::ERR_QUIC_PROTOCOL_ERROR
HTTP-level status stays 200 (response headers received), but bytes are
truncated. The browser paints whatever PNG bytes it has so far → visible
row-by-row decode.
- The earlier preloadImage()+decode() trick can't fix this — neither
HTTP-cache reuse nor sync decode helps when the bytes themselves were
never fully delivered.
Two-tier fix:
1. Client: fetch → Blob → URL.createObjectURL() (app/play/page.tsx)
- <img src> only ever points to a blob: URL whose bytes are 100%
resident in the JS heap. No network-backed src = no possibility of
progressive paint.
- Module-level blobUrlCache keys by original URL so speculative
prefetch + the eventual commit share one fetch.
- Old blobs are URL.revokeObjectURL()'d on scene swap + unmount to
release memory.
2. Network: optional Cloudflare Worker proxy (worker/)
- Browser ↔ Worker is HTTP/2 over CF edge (extremely stable).
- Worker ↔ Runware is a server-to-server fetch (no QUIC fragility,
Cloudflare's backbone handles transit).
- Worker buffers the full upstream response → client never sees a
half-stream.
- Bonus: CF edge cache (cacheEverything, 1y TTL) on Runware UUIDs;
Access-Control-Allow-Origin: * so client fetch() can't hit CORS.
- Hardened: only proxies im.runware.ai, only GET/HEAD/OPTIONS, all
other hosts/methods → 403/405.
Wired via NEXT_PUBLIC_IMAGE_PROXY_URL (inlined at build). Empty → no proxy
→ direct fetch (which still uses the blob path, just exposed to QUIC).
──────────────────────────────────────────────────────────────────────
Deploy steps (one-time, do this AFTER pulling this commit):
1. Install wrangler globally:
npm i -g wrangler
2. Log in to Cloudflare (opens browser for OAuth):
wrangler login
3. From the worker/ directory, deploy:
cd worker
wrangler deploy
wrangler will print the deployed URL, e.g.
https://infiplot-image-proxy.<your-cf-username>.workers.dev
4. Paste that URL into .env.local for local dev:
NEXT_PUBLIC_IMAGE_PROXY_URL=https://infiplot-image-proxy.<...>.workers.dev
…and into Vercel project settings (Environment Variables) for prod.
NEXT_PUBLIC_ vars are inlined at build time, so the URL bakes into
the bundle on the next deploy/dev-server restart.
5. Restart dev server (pnpm dev) so the new env baked in. Generate a
scene; Network tab should show requests going to *.workers.dev
instead of im.runware.ai, no ERR_QUIC_PROTOCOL_ERROR, image renders
atomically.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -56,6 +56,16 @@ TTS_SPEECH_MODEL=mimo-v2.5-tts
|
||||
# Text/story/voice still run normally. Great for iterating on TTS.
|
||||
MOCK_IMAGE=false
|
||||
|
||||
# ---- 5b. Image proxy (Cloudflare Worker, optional) -----------------
|
||||
# Chrome's direct fetch of im.runware.ai is unreliable on some networks
|
||||
# (ERR_QUIC_PROTOCOL_ERROR mid-stream → partial bytes → <img> renders
|
||||
# progressively from top to bottom). Routing the fetch through a CF Worker
|
||||
# (see worker/) avoids the QUIC fragility and adds edge caching + CORS.
|
||||
# Empty → no proxy → direct fetch (fine when the network behaves).
|
||||
# NEXT_PUBLIC_ vars are inlined at BUILD time — set in Vercel project settings.
|
||||
# Deploy the Worker per worker/wrangler.toml, then paste the workers.dev URL:
|
||||
NEXT_PUBLIC_IMAGE_PROXY_URL=
|
||||
|
||||
# ---- 6. Analytics · Umami (optional — leave blank to disable) ------
|
||||
# Privacy-friendly, cookieless page-view stats — no Cookie consent banner.
|
||||
# Cloud: sign up at https://cloud.umami.is, add your site, copy its ID into
|
||||
|
||||
+119
-42
@@ -40,33 +40,91 @@ const MUTED_STORAGE_KEY = "infiplot:muted";
|
||||
const IMAGE_PRELOAD_TIMEOUT_MS = 20000;
|
||||
|
||||
// ──────────────────────────────────────────────────────────────────────
|
||||
// Image preload — decode the Runware URL in memory before committing to
|
||||
// React state, so when the <img> mounts, the browser cache is warm and
|
||||
// rendering is instant. Without this the user sees a blank canvas during
|
||||
// the Runware-CDN download (~1-3s) after /api/scene returns.
|
||||
// Image fetch → blob URL — bulletproof against browser progressive paint.
|
||||
//
|
||||
// Data URIs (MOCK_IMAGE mode) and prefetched-then-cached real URLs both
|
||||
// resolve fast / instantly. Errors and timeouts resolve quietly — better
|
||||
// to render a broken-image than to hang the play loop indefinitely.
|
||||
// Why not a plain <img src={cdnUrl}>: Runware CDN returns weak cache headers
|
||||
// (every <img> mount issues a fresh GET — confirmed in DevTools, status 200
|
||||
// not "from disk cache"), so the Image() preload + decode() trick can warm
|
||||
// HTTP cache but the actual <img> still streams bytes from network and
|
||||
// paints row-by-row as they arrive.
|
||||
//
|
||||
// Fix: fetch the bytes ourselves, materialize a blob: URL pointing at the
|
||||
// fully-local copy, and only set the <img src> to that blob: URL. The <img>
|
||||
// never sees a network-backed src, so there is no "字节还在路上" middle state
|
||||
// and no progressive paint is possible. Trade-off: callers MUST revoke the
|
||||
// blob URL when swapping it out, or the bytes leak in JS heap.
|
||||
//
|
||||
// Failure mode: on network error / timeout we fall back to the original CDN
|
||||
// URL so the <img> still attempts to render (with possible progressive paint
|
||||
// — same as pre-fix behavior, never worse).
|
||||
//
|
||||
// Data URIs (MOCK_IMAGE mode) are already local; passed through unchanged.
|
||||
// ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
function preloadImage(url: string): Promise<void> {
|
||||
return new Promise<void>((resolve) => {
|
||||
const img = new Image();
|
||||
const done = () => resolve();
|
||||
const timer = setTimeout(done, IMAGE_PRELOAD_TIMEOUT_MS);
|
||||
img.onload = () => {
|
||||
clearTimeout(timer);
|
||||
// .decode() forces the bitmap to be fully decoded before we proceed —
|
||||
// without it, a slow decode could still cause a flash on first paint.
|
||||
img.decode().then(done, done);
|
||||
};
|
||||
img.onerror = () => {
|
||||
clearTimeout(timer);
|
||||
done();
|
||||
};
|
||||
img.src = url;
|
||||
});
|
||||
// Optional Cloudflare Workers proxy in front of Runware. Reason: Chrome's
|
||||
// direct fetch of im.runware.ai sometimes hits ERR_QUIC_PROTOCOL_ERROR
|
||||
// mid-stream, leaving the browser with partial PNG bytes that render
|
||||
// progressively. The Worker re-fetches Runware server-to-server (no QUIC
|
||||
// fragility) and serves the bytes over HTTP/2 — atomic and reliable.
|
||||
//
|
||||
// Inlined by Next.js at build time. Empty / unset → fall back to direct
|
||||
// fetch of the original URL (works fine when Runware's CDN cooperates,
|
||||
// and on browsers/networks where QUIC isn't flaky).
|
||||
const IMAGE_PROXY_BASE = (
|
||||
process.env.NEXT_PUBLIC_IMAGE_PROXY_URL ?? ""
|
||||
).replace(/\/$/, "");
|
||||
|
||||
function proxiedImageUrl(originalUrl: string): string {
|
||||
if (!IMAGE_PROXY_BASE) return originalUrl;
|
||||
// Data URIs (MOCK_IMAGE) are already local; proxy is irrelevant.
|
||||
if (originalUrl.startsWith("data:")) return originalUrl;
|
||||
// Only proxy real Runware CDN URLs — keeps the Worker's whitelist tight
|
||||
// and dodges the proxy hop for any other origin we might add later.
|
||||
if (!originalUrl.startsWith("https://im.runware.ai/")) return originalUrl;
|
||||
return `${IMAGE_PROXY_BASE}/?url=${encodeURIComponent(originalUrl)}`;
|
||||
}
|
||||
|
||||
async function fetchImageAsBlobUrl(url: string): Promise<string> {
|
||||
if (url.startsWith("data:")) return url;
|
||||
// Cache keys (blobUrlCache) stay on the original Runware URL — the proxy
|
||||
// is an internal fetch detail, callers shouldn't need to think about it.
|
||||
const fetchUrl = proxiedImageUrl(url);
|
||||
const ctrl = new AbortController();
|
||||
const timer = setTimeout(() => ctrl.abort(), IMAGE_PRELOAD_TIMEOUT_MS);
|
||||
try {
|
||||
const r = await fetch(fetchUrl, { signal: ctrl.signal });
|
||||
if (!r.ok) return url;
|
||||
const blob = await r.blob();
|
||||
return URL.createObjectURL(blob);
|
||||
} catch {
|
||||
return url;
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
}
|
||||
|
||||
// Module-level cache so speculative prefetches and the eventual commit share
|
||||
// the same in-flight fetch — no double-download per scene. Keyed by the
|
||||
// ORIGINAL CDN URL (the blob: URL it resolves to is the value). Persists for
|
||||
// the page's lifetime; entries are explicitly revoked when the scene swaps.
|
||||
const blobUrlCache = new Map<string, Promise<string>>();
|
||||
|
||||
function getOrCreateBlobUrl(originalUrl: string): Promise<string> {
|
||||
let p = blobUrlCache.get(originalUrl);
|
||||
if (!p) {
|
||||
p = fetchImageAsBlobUrl(originalUrl);
|
||||
blobUrlCache.set(originalUrl, p);
|
||||
}
|
||||
return p;
|
||||
}
|
||||
|
||||
function revokeBlobUrlFor(originalUrl: string): void {
|
||||
const p = blobUrlCache.get(originalUrl);
|
||||
if (!p) return;
|
||||
blobUrlCache.delete(originalUrl);
|
||||
p.then((u) => {
|
||||
if (u.startsWith("blob:")) URL.revokeObjectURL(u);
|
||||
}).catch(() => {});
|
||||
}
|
||||
|
||||
// ──────────────────────────────────────────────────────────────────────
|
||||
@@ -164,11 +222,11 @@ function prefetchScenePath(
|
||||
}
|
||||
const data = (await res.json()) as SceneResponse;
|
||||
|
||||
// Warm the browser's HTTP + image-decode cache for this URL so when the
|
||||
// player eventually picks this choice and we render the <img>, it's
|
||||
// instant. Don't await — let the bytes stream in the background; the
|
||||
// transition path will await its own preloadImage() before committing.
|
||||
void preloadImage(data.imageUrl);
|
||||
// Kick off the blob fetch for this URL so when the player eventually
|
||||
// picks this choice, transitioning is a no-op cache lookup instead of a
|
||||
// fresh CDN download. Don't await — let it run in the background; the
|
||||
// transition path awaits the same cached promise via getOrCreateBlobUrl.
|
||||
void getOrCreateBlobUrl(data.imageUrl);
|
||||
|
||||
// Recursive: if the resulting scene has exactly one change-scene exit,
|
||||
// it is a must-pass node — prefetch its child too.
|
||||
@@ -288,6 +346,10 @@ function PlayInner() {
|
||||
const currentSceneRef = useRef<Scene | null>(null);
|
||||
const currentBeatRef = useRef<Beat | null>(null);
|
||||
const visitedBeatsRef = useRef<string[]>([]);
|
||||
// Original (CDN) URL of the currently-rendered scene image. Used as the key
|
||||
// to revoke its blob: URL when the scene swaps. We track the ORIGINAL URL,
|
||||
// not the blob URL, because blobUrlCache is keyed by original URL.
|
||||
const lastImageOriginalUrlRef = useRef<string | null>(null);
|
||||
|
||||
const currentBeat = useMemo<Beat | null>(() => {
|
||||
if (!currentScene || !currentBeatId) return null;
|
||||
@@ -580,10 +642,11 @@ function PlayInner() {
|
||||
|
||||
fetchStart
|
||||
.then(async (data) => {
|
||||
// Decode the Runware image in memory before committing to state, so
|
||||
// the <img> renders instantly when it mounts (same rationale as the
|
||||
// performSceneTransition path).
|
||||
await preloadImage(data.imageUrl);
|
||||
// Pull the full image bytes into a local blob: URL before committing
|
||||
// to state. The <img> then mounts pointed at a fully-local blob, which
|
||||
// the browser paints atomically — no row-by-row "层层加载".
|
||||
const blobUrl = await getOrCreateBlobUrl(data.imageUrl);
|
||||
lastImageOriginalUrlRef.current = data.imageUrl;
|
||||
|
||||
const initial: Session = {
|
||||
id: data.sessionId,
|
||||
@@ -604,7 +667,7 @@ function PlayInner() {
|
||||
setSession(initial);
|
||||
setCurrentScene(data.scene);
|
||||
setCurrentBeatId(data.scene.entryBeatId);
|
||||
setImageUrl(data.imageUrl);
|
||||
setImageUrl(blobUrl);
|
||||
// beatAudioMap is populated lazily by the per-beat fetch effect once
|
||||
// currentScene becomes non-null (see fetchBeatAudio).
|
||||
setPhase("ready");
|
||||
@@ -639,6 +702,9 @@ function PlayInner() {
|
||||
// stop paying for background scene/image generation. Empty deps → fires only
|
||||
// on unmount; it must NOT run on scene transitions, which rely on
|
||||
// consumeChoice keeping the re-rooted survivor prefetches alive.
|
||||
// Also revoke any surviving blob: URLs so their bytes can be GC'd — the
|
||||
// module-level blobUrlCache outlives the component but its entries should
|
||||
// not survive the page navigation that unmounts us.
|
||||
useEffect(() => {
|
||||
const pool = poolRef.current;
|
||||
const beatAborts = beatAudioAbortRef.current;
|
||||
@@ -646,6 +712,9 @@ function PlayInner() {
|
||||
clearPool(pool);
|
||||
for (const c of beatAborts.values()) c.abort();
|
||||
beatAborts.clear();
|
||||
for (const [originalUrl] of blobUrlCache) {
|
||||
revokeBlobUrlFor(originalUrl);
|
||||
}
|
||||
};
|
||||
}, []);
|
||||
|
||||
@@ -672,13 +741,21 @@ function PlayInner() {
|
||||
const base = sessionRef.current;
|
||||
if (!base) throw new Error("Session lost mid-transition");
|
||||
|
||||
// Wait for the browser to download + decode the Runware-hosted image
|
||||
// BEFORE committing it to state, so the <img> renders instantly when it
|
||||
// mounts. For prefetched scenes the preloadImage call inside
|
||||
// prefetchScenePath has already warmed the cache, so this resolves
|
||||
// almost immediately. For cold transitions we trade an extra ~1-3s of
|
||||
// "transitioning" overlay for an image-pop-in-from-blank flash.
|
||||
await preloadImage(result.imageUrl);
|
||||
// Pull full image bytes into a local blob: URL before committing. For
|
||||
// prefetched scenes the speculative getOrCreateBlobUrl in
|
||||
// prefetchScenePath already has this in flight (often resolved), so
|
||||
// this is a near-instant cache lookup. For cold transitions we eat the
|
||||
// CDN download time under the "transitioning" overlay — same cost as
|
||||
// before, but the <img> never sees a network-backed src and therefore
|
||||
// can't paint progressively.
|
||||
const blobUrl = await getOrCreateBlobUrl(result.imageUrl);
|
||||
// Revoke the previous scene's blob (no longer rendered) to release JS
|
||||
// heap. New scene's original URL takes its place as "current".
|
||||
const priorOriginal = lastImageOriginalUrlRef.current;
|
||||
if (priorOriginal && priorOriginal !== result.imageUrl) {
|
||||
revokeBlobUrlFor(priorOriginal);
|
||||
}
|
||||
lastImageOriginalUrlRef.current = result.imageUrl;
|
||||
|
||||
const closedHistory = base.history.map((h, i, arr) =>
|
||||
i === arr.length - 1
|
||||
@@ -701,7 +778,7 @@ function PlayInner() {
|
||||
setSession(newSession);
|
||||
setCurrentScene(result.scene);
|
||||
setCurrentBeatId(result.scene.entryBeatId);
|
||||
setImageUrl(result.imageUrl);
|
||||
setImageUrl(blobUrl);
|
||||
// beatAudioMap reset + per-beat fetches kicked off by the scene effect.
|
||||
setLastExitLabel(exitLabel);
|
||||
setPhase("ready");
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
// ─────────────────────────────────────────────────────────────────────────
|
||||
// InfiPlot — Runware image proxy (Cloudflare Worker)
|
||||
//
|
||||
// Why this exists:
|
||||
// Chrome's direct fetch of `im.runware.ai` images sometimes fails with
|
||||
// `ERR_QUIC_PROTOCOL_ERROR` — HTTP/3 stream errors mid-transfer leave the
|
||||
// browser holding a partial PNG, which it renders progressively
|
||||
// (the "层层从上往下" visible-decode glitch). Routing the fetch through
|
||||
// this Worker fixes it in two ways:
|
||||
//
|
||||
// 1. Browser ↔ Worker is HTTP/2 over Cloudflare's edge — extremely
|
||||
// stable, no QUIC fragility.
|
||||
// 2. Worker ↔ Runware is a server-to-server fetch (Cloudflare's
|
||||
// backbone) — also reliable, and the Worker buffers the full
|
||||
// response before streaming it back, so the client never gets
|
||||
// partial bytes mid-stream.
|
||||
//
|
||||
// Bonus side-effects:
|
||||
// - CORS: Worker adds `Access-Control-Allow-Origin: *` so the client's
|
||||
// `fetch()` → blob URL path works regardless of Runware's policy.
|
||||
// - Edge cache: same Runware UUID re-fetched twice in 24h hits the CF
|
||||
// edge cache, sub-50ms response from anywhere in the world.
|
||||
//
|
||||
// Hardening:
|
||||
// - Only proxies `im.runware.ai` (open proxies invite abuse + quota burn).
|
||||
// - Only accepts GET / HEAD / OPTIONS.
|
||||
// ─────────────────────────────────────────────────────────────────────────
|
||||
|
||||
const ALLOWED_HOST = "im.runware.ai";
|
||||
|
||||
const corsHeaders = {
|
||||
"Access-Control-Allow-Origin": "*",
|
||||
"Access-Control-Allow-Methods": "GET, HEAD, OPTIONS",
|
||||
"Access-Control-Max-Age": "86400",
|
||||
};
|
||||
|
||||
export default {
|
||||
async fetch(req) {
|
||||
if (req.method === "OPTIONS") {
|
||||
return new Response(null, { headers: corsHeaders });
|
||||
}
|
||||
if (req.method !== "GET" && req.method !== "HEAD") {
|
||||
return new Response("method not allowed", { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
const reqUrl = new URL(req.url);
|
||||
const target = reqUrl.searchParams.get("url");
|
||||
if (!target) {
|
||||
return new Response("missing ?url=", { status: 400, headers: corsHeaders });
|
||||
}
|
||||
|
||||
let targetUrl;
|
||||
try {
|
||||
targetUrl = new URL(target);
|
||||
} catch {
|
||||
return new Response("malformed ?url=", { status: 400, headers: corsHeaders });
|
||||
}
|
||||
if (targetUrl.hostname !== ALLOWED_HOST) {
|
||||
return new Response(`only ${ALLOWED_HOST} is allowed`, {
|
||||
status: 403,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
// Fetch upstream. `cf.cacheEverything: true` tells the CF edge to cache
|
||||
// by URL even though Runware's own cache headers are weak — so a second
|
||||
// hit on the same UUID lands in edge memory rather than re-touching
|
||||
// Runware. 1y TTL: image UUIDs are immutable, the bytes never change.
|
||||
const upstream = await fetch(targetUrl.toString(), {
|
||||
cf: { cacheTtl: 31536000, cacheEverything: true },
|
||||
});
|
||||
|
||||
// Stream the body through (no buffering — CF Workers' Response can take
|
||||
// a ReadableStream directly). Rebuild headers to add CORS + strong cache
|
||||
// hints, preserve content-type / content-length from upstream.
|
||||
const headers = new Headers(corsHeaders);
|
||||
headers.set(
|
||||
"Content-Type",
|
||||
upstream.headers.get("content-type") ?? "image/png",
|
||||
);
|
||||
headers.set("Cache-Control", "public, max-age=31536000, immutable");
|
||||
const len = upstream.headers.get("content-length");
|
||||
if (len) headers.set("Content-Length", len);
|
||||
|
||||
return new Response(upstream.body, {
|
||||
status: upstream.status,
|
||||
headers,
|
||||
});
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,16 @@
|
||||
# Cloudflare Worker — Runware image proxy.
|
||||
# See worker/src/index.js for what it does and why.
|
||||
#
|
||||
# Deploy:
|
||||
# 1. `npm i -g wrangler` (one-time)
|
||||
# 2. `wrangler login` (one-time, OAuth flow in browser)
|
||||
# 3. From this directory: `wrangler deploy`
|
||||
# 4. wrangler prints the deployed URL, e.g.
|
||||
# https://infiplot-image-proxy.<your-cf-username>.workers.dev
|
||||
# 5. Set NEXT_PUBLIC_IMAGE_PROXY_URL=<that URL> in .env.local for dev
|
||||
# and in Vercel project settings for prod.
|
||||
|
||||
name = "infiplot-image-proxy"
|
||||
main = "src/index.js"
|
||||
compatibility_date = "2025-01-01"
|
||||
workers_dev = true
|
||||
Reference in New Issue
Block a user