Merge pull request #24 from zonghaoyuan/fix/optional-image-proxy
fix(play): make scene-image proxy opt-in (default direct-connect)
This commit is contained in:
Zonghao Yuan
GitHub
+21
-8
@@ -56,15 +56,28 @@ TTS_SPEECH_MODEL=mimo-v2.5-tts
|
||||
# Text/story/voice still run normally. Great for iterating on TTS.
|
||||
MOCK_IMAGE=false
|
||||
|
||||
# ---- 5b. Image proxy (Cloudflare Worker, optional) -----------------
|
||||
# Chrome's direct fetch of im.runware.ai is unreliable on some networks
|
||||
# (ERR_QUIC_PROTOCOL_ERROR mid-stream → partial bytes → <img> renders
|
||||
# progressively from top to bottom). Routing the fetch through a CF Worker
|
||||
# (see worker/) avoids the QUIC fragility and adds edge caching + CORS.
|
||||
# Empty → no proxy → direct fetch (fine when the network behaves).
|
||||
# NEXT_PUBLIC_ vars are inlined at BUILD time — set in Vercel project settings.
|
||||
# Deploy the Worker per worker/wrangler.toml, then paste the workers.dev URL:
|
||||
# ---- 5b. Image proxy (Cloudflare Worker, OPTIONAL) -----------------
|
||||
# Leave NEXT_PUBLIC_IMAGE_PROXY_URL blank (the default) and the browser
|
||||
# fetches images directly from the provider — exactly as the app worked
|
||||
# before this proxy existed. The ALLOWED_HOSTS value below is inert until
|
||||
# a proxy URL is set, so you're completely unaffected; skip this section.
|
||||
#
|
||||
# Why you might want it: Chrome's direct fetch of im.runware.ai is unreliable
|
||||
# on some networks (ERR_QUIC_PROTOCOL_ERROR mid-stream → partial bytes →
|
||||
# <img> paints progressively top-to-bottom). Routing the fetch through a tiny
|
||||
# Cloudflare Worker re-fetches server-to-server (no QUIC fragility) and serves
|
||||
# over HTTP/2 — atomic paint, plus edge caching + CORS.
|
||||
#
|
||||
# Deploy your own in ~1 min (one-click "Deploy to Cloudflare" button):
|
||||
# https://github.com/zonghaoyuan/infiplot-image-proxy
|
||||
# Then paste the workers.dev URL it prints below. NEXT_PUBLIC_ vars are
|
||||
# inlined at BUILD time — set them in Vercel/Cloudflare project settings.
|
||||
NEXT_PUBLIC_IMAGE_PROXY_URL=
|
||||
# Hostnames the proxy is allowed to fetch (comma-separated). Default covers
|
||||
# Runware's CDN. If your IMAGE_BASE_URL points at another provider, add that
|
||||
# provider's image host here so its URLs take the proxy path too. Anything
|
||||
# not listed stays on the direct fetch. Only matters when the URL above is set.
|
||||
NEXT_PUBLIC_IMAGE_PROXY_ALLOWED_HOSTS=im.runware.ai
|
||||
|
||||
# ---- 6. Analytics · Umami (optional — leave blank to disable) ------
|
||||
# Privacy-friendly, cookieless page-view stats — no Cookie consent banner.
|
||||
|
||||
@@ -155,6 +155,10 @@ Where to set them (see `.env.example` for the exact shape):
|
||||
|
||||
With the recommended trio, each scene's cost comes mainly from the image generation model. The FLUX.2 [klein] 9B KV image is roughly **\$0.00078** per scene (1792×1024, 4 steps, sub-second); the text model uses `deepseek-v4-flash`, so text costs are negligible by comparison. Tapping through a scene's beats is free. To keep transitions instant, the engine also pre-generates scenes you might pick but ultimately don't — so real spend runs somewhat higher than the scenes you actually see.
|
||||
|
||||
**4. Image proxy (optional)**
|
||||
|
||||
By default the browser fetches images directly from the provider — no setup needed; leave `NEXT_PUBLIC_IMAGE_PROXY_URL` blank and you're completely unaffected. You only want this if you hit progressive "top-to-bottom" image loading (Chrome's `ERR_QUIC_PROTOCOL_ERROR` on some networks paints partial PNGs row by row): deploy a tiny Cloudflare Worker that re-fetches images server-side and serves them atomically over HTTP/2. One-click deploy at **[infiplot-image-proxy](https://github.com/zonghaoyuan/infiplot-image-proxy)**, then paste the `workers.dev` URL it prints into `NEXT_PUBLIC_IMAGE_PROXY_URL`.
|
||||
|
||||
---
|
||||
|
||||
## Roadmap
|
||||
|
||||
@@ -154,6 +154,10 @@ InfiPlot は 4 種類のモデルプロバイダと通信します。**テキス
|
||||
|
||||
推奨の 3 点セットでは、各シーンのコストは主に画像生成モデルによるものです。FLUX.2 [klein] 9B KV の画像は 1 シーンあたり概ね **$0.00078**（1792×1024、4 ステップ、サブ秒）。テキストモデルは `deepseek-v4-flash` を使用するため、テキストコストは比較になりません。シーン内のビートをタップしていくのは無料です。切り替えを一瞬に保つため、エンジンは選ぶ可能性はあるが最終的に選ばないシーンも先行生成します —— そのため実際の支出は、あなたが実際に見るシーン数よりやや高くなります。
|
||||
|
||||
**4. 画像プロキシ（オプション）**
|
||||
|
||||
デフォルトではブラウザが画像プロバイダーに直接アクセスするため、設定は不要です —— `NEXT_PUBLIC_IMAGE_PROXY_URL` を空欄のままにすれば、まったく影響ありません。画像が「上から順に」表示される現象（一部のネットワークで Chrome の `ERR_QUIC_PROTOCOL_ERROR` により PNG が行ごとに描画される）に遭遇した場合のみ必要です。小さな Cloudflare Worker をデプロイすると、画像をサーバー側で再取得し HTTP/2 で一括返却します。ワンクリックデプロイは **[infiplot-image-proxy](https://github.com/zonghaoyuan/infiplot-image-proxy)** を参照し、出力された `workers.dev` の URL を `NEXT_PUBLIC_IMAGE_PROXY_URL` に設定してください。
|
||||
|
||||
---
|
||||
|
||||
## Roadmap
|
||||
|
||||
@@ -154,6 +154,10 @@ InfiPlot 会与四类模型供应商通信。**文本（Text）和视觉（Visio
|
||||
|
||||
使用推荐的三件套时，每一幕场景的开销主要来自图像生成模型。FLUX.2 [klein] 9B KV 的图像大约 **$0.00078** 一张（1792×1024，4 步，亚秒级）；文本模型使用 `deepseek-v4-flash` 时，成本极低。逐拍点过一个场景是免费的。为了让切换瞬间完成，引擎还会预测式地生成那些你可能选、但最终可能没选的场景 —— 所以真实花费会比你实际看到的场景数略高一些。
|
||||
|
||||
**4. 图片代理（可选）**
|
||||
|
||||
默认浏览器直连图片供应商，无需任何配置 —— 留空 `NEXT_PUBLIC_IMAGE_PROXY_URL` 即可，完全不受影响。只有当你遇到图片「层层加载」（Chrome 在某些网络下 `ERR_QUIC_PROTOCOL_ERROR` 导致 PNG 逐行渲染）时才需要它：部署一个极小的 Cloudflare Worker，把图片改为服务端转发 + HTTP/2 原子返回。一键部署见 **[infiplot-image-proxy](https://github.com/zonghaoyuan/infiplot-image-proxy)**，然后把它给出的 `workers.dev` 地址填进 `NEXT_PUBLIC_IMAGE_PROXY_URL`。
|
||||
|
||||
---
|
||||
|
||||
## Roadmap
|
||||
|
||||
+95
-41
@@ -41,59 +41,113 @@ const MUTED_STORAGE_KEY = "infiplot:muted";
|
||||
const IMAGE_PRELOAD_TIMEOUT_MS = 20000;
|
||||
|
||||
// ──────────────────────────────────────────────────────────────────────
|
||||
// Image fetch → blob URL — bulletproof against browser progressive paint.
|
||||
// Two ways an <img> gets its pixels, picked per-URL by shouldProxy():
|
||||
//
|
||||
// Why not a plain <img src={cdnUrl}>: Runware CDN returns weak cache headers
|
||||
// (every <img> mount issues a fresh GET — confirmed in DevTools, status 200
|
||||
// not "from disk cache"), so the Image() preload + decode() trick can warm
|
||||
// HTTP cache but the actual <img> still streams bytes from network and
|
||||
// paints row-by-row as they arrive.
|
||||
// 1. DIRECT (default — no proxy configured): preload the URL with an
|
||||
// Image() + decode() so the HTTP cache is warm and the bitmap decoded
|
||||
// before React commits, then hand the ORIGINAL URL to <img>. This is the
|
||||
// long-standing behavior; deployers who set no env var get exactly this
|
||||
// and are completely unaffected by the proxy machinery below.
|
||||
//
|
||||
// Fix: fetch the bytes ourselves, materialize a blob: URL pointing at the
|
||||
// fully-local copy, and only set the <img src> to that blob: URL. The <img>
|
||||
// never sees a network-backed src, so there is no "字节还在路上" middle state
|
||||
// and no progressive paint is possible. Trade-off: callers MUST revoke the
|
||||
// blob URL when swapping it out, or the bytes leak in JS heap.
|
||||
// 2. PROXY (opt-in — NEXT_PUBLIC_IMAGE_PROXY_URL set, host allow-listed):
|
||||
// fetch the bytes through the Cloudflare Worker (which adds CORS and
|
||||
// serves over stable HTTP/2), await the FULL body via .blob(), materialize
|
||||
// a blob: URL over that local copy, and hand THAT to <img>. The <img>
|
||||
// never sees a network-backed src, so there's no "字节还在路上" middle
|
||||
// state and no progressive paint.
|
||||
// Why it matters: Chrome's direct fetch of im.runware.ai sometimes hits
|
||||
// ERR_QUIC_PROTOCOL_ERROR mid-stream, leaving partial PNG bytes that
|
||||
// paint row-by-row. The Worker re-fetches server-to-server (no QUIC
|
||||
// fragility) and serves over HTTP/2 — atomic and reliable. Trade-off:
|
||||
// callers MUST revoke the blob URL when swapping it out (revokeBlobUrlFor)
|
||||
// or the bytes leak in the JS heap.
|
||||
//
|
||||
// Failure mode: on network error / timeout we fall back to the original CDN
|
||||
// URL so the <img> still attempts to render (with possible progressive paint
|
||||
// — same as pre-fix behavior, never worse).
|
||||
//
|
||||
// Data URIs (MOCK_IMAGE mode) are already local; passed through unchanged.
|
||||
// Data URIs (MOCK_IMAGE mode) are already local; passed through unchanged
|
||||
// on both paths. blobUrlCache is keyed by the ORIGINAL URL either way.
|
||||
// ──────────────────────────────────────────────────────────────────────
|
||||
|
||||
// Optional Cloudflare Workers proxy in front of Runware. Reason: Chrome's
|
||||
// direct fetch of im.runware.ai sometimes hits ERR_QUIC_PROTOCOL_ERROR
|
||||
// mid-stream, leaving the browser with partial PNG bytes that render
|
||||
// progressively. The Worker re-fetches Runware server-to-server (no QUIC
|
||||
// fragility) and serves the bytes over HTTP/2 — atomic and reliable.
|
||||
//
|
||||
// Inlined by Next.js at build time. Empty / unset → fall back to direct
|
||||
// fetch of the original URL (works fine when Runware's CDN cooperates,
|
||||
// and on browsers/networks where QUIC isn't flaky).
|
||||
// Direct-path preload: decode the URL in memory before committing to React
|
||||
// state, so when the <img> mounts the cache is warm and first paint is
|
||||
// instant. Errors / timeouts resolve quietly — better a broken <img> than a
|
||||
// hung play loop. (im.runware.ai sends no CORS header, so we can't fetch()
|
||||
// its bytes here; warming + decoding is the most the direct path can do.)
|
||||
function preloadImage(url: string): Promise<void> {
|
||||
return new Promise<void>((resolve) => {
|
||||
const img = new Image();
|
||||
let timer: ReturnType<typeof setTimeout>;
|
||||
// Single exit: clear the timeout and resolve. resolve() is idempotent, so
|
||||
// whichever path fires first (load+decode, error, timeout) wins.
|
||||
const done = () => {
|
||||
clearTimeout(timer);
|
||||
resolve();
|
||||
};
|
||||
// Armed across BOTH network load and decode, so a hung decode still
|
||||
// resolves quietly — better a broken <img> than a stuck play loop.
|
||||
timer = setTimeout(done, IMAGE_PRELOAD_TIMEOUT_MS);
|
||||
img.onload = () => {
|
||||
// .decode() forces the bitmap to be fully decoded before we proceed —
|
||||
// without it, a slow decode could still cause a flash on first paint.
|
||||
img.decode().then(done, done);
|
||||
};
|
||||
img.onerror = done;
|
||||
img.src = url;
|
||||
});
|
||||
}
|
||||
|
||||
// Opt-in Cloudflare Workers proxy (deploy your own — see the link in README).
|
||||
// Inlined by Next.js at build time. Empty / unset → no proxy → every URL takes
|
||||
// the direct path above, exactly as if this feature didn't exist.
|
||||
const IMAGE_PROXY_BASE = (
|
||||
process.env.NEXT_PUBLIC_IMAGE_PROXY_URL ?? ""
|
||||
).replace(/\/$/, "");
|
||||
|
||||
// Hostnames eligible for the proxy. Default: Runware's CDN only. Deployers who
|
||||
// point IMAGE_BASE_URL at another provider can opt that provider's image host
|
||||
// in via NEXT_PUBLIC_IMAGE_PROXY_ALLOWED_HOSTS (comma-separated). Inlined at
|
||||
// build time. Anything not on this list stays on the direct path.
|
||||
const IMAGE_PROXY_ALLOWED_HOSTS = (
|
||||
process.env.NEXT_PUBLIC_IMAGE_PROXY_ALLOWED_HOSTS ?? "im.runware.ai"
|
||||
)
|
||||
.split(",")
|
||||
.map((h) => h.trim().toLowerCase())
|
||||
.filter(Boolean);
|
||||
|
||||
// Route a URL through the proxy only when a proxy is configured AND it's a
|
||||
// remote http(s) image on an allow-listed host. data: URIs (MOCK_IMAGE) are
|
||||
// already local; malformed URLs and any other origin fall through to direct.
|
||||
function shouldProxy(originalUrl: string): boolean {
|
||||
if (!IMAGE_PROXY_BASE) return false;
|
||||
if (originalUrl.startsWith("data:")) return false;
|
||||
try {
|
||||
const { protocol, hostname } = new URL(originalUrl);
|
||||
if (protocol !== "https:" && protocol !== "http:") return false;
|
||||
return IMAGE_PROXY_ALLOWED_HOSTS.includes(hostname.toLowerCase());
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function proxiedImageUrl(originalUrl: string): string {
|
||||
if (!IMAGE_PROXY_BASE) return originalUrl;
|
||||
// Data URIs (MOCK_IMAGE) are already local; proxy is irrelevant.
|
||||
if (originalUrl.startsWith("data:")) return originalUrl;
|
||||
// Only proxy real Runware CDN URLs — keeps the Worker's whitelist tight
|
||||
// and dodges the proxy hop for any other origin we might add later.
|
||||
if (!originalUrl.startsWith("https://im.runware.ai/")) return originalUrl;
|
||||
return `${IMAGE_PROXY_BASE}/?url=${encodeURIComponent(originalUrl)}`;
|
||||
}
|
||||
|
||||
async function fetchImageAsBlobUrl(url: string): Promise<string> {
|
||||
if (url.startsWith("data:")) return url;
|
||||
// Cache keys (blobUrlCache) stay on the original Runware URL — the proxy
|
||||
// is an internal fetch detail, callers shouldn't need to think about it.
|
||||
const fetchUrl = proxiedImageUrl(url);
|
||||
|
||||
// Direct path (default): warm the cache + decode, hand back the original
|
||||
// URL. No fetch() — im.runware.ai has no CORS, so fetch().blob() would throw.
|
||||
if (!shouldProxy(url)) {
|
||||
await preloadImage(url);
|
||||
return url;
|
||||
}
|
||||
|
||||
// Proxy path (opt-in): fetch through the Worker and materialize a blob: URL.
|
||||
// On error / timeout fall back to the original URL so <img> still tries
|
||||
// (possible progressive paint — same as the direct path, never worse).
|
||||
const ctrl = new AbortController();
|
||||
const timer = setTimeout(() => ctrl.abort(), IMAGE_PRELOAD_TIMEOUT_MS);
|
||||
try {
|
||||
const r = await fetch(fetchUrl, { signal: ctrl.signal });
|
||||
const r = await fetch(proxiedImageUrl(url), { signal: ctrl.signal });
|
||||
if (!r.ok) return url;
|
||||
const blob = await r.blob();
|
||||
return URL.createObjectURL(blob);
|
||||
@@ -660,9 +714,9 @@ function PlayInner() {
|
||||
|
||||
fetchStart
|
||||
.then(async (data) => {
|
||||
// Pull the full image bytes into a local blob: URL before committing
|
||||
// to state. The <img> then mounts pointed at a fully-local blob, which
|
||||
// the browser paints atomically — no row-by-row "层层加载".
|
||||
// Resolve to a paintable src before committing to state. Proxy path:
|
||||
// a fully-local blob: URL the browser paints atomically (no row-by-row
|
||||
// "层层加载"). Direct path (default): the preloaded original URL.
|
||||
const blobUrl = await getOrCreateBlobUrl(data.imageUrl);
|
||||
lastImageOriginalUrlRef.current = data.imageUrl;
|
||||
|
||||
@@ -764,9 +818,9 @@ function PlayInner() {
|
||||
// prefetched scenes the speculative getOrCreateBlobUrl in
|
||||
// prefetchScenePath already has this in flight (often resolved), so
|
||||
// this is a near-instant cache lookup. For cold transitions we eat the
|
||||
// CDN download time under the "transitioning" overlay — same cost as
|
||||
// before, but the <img> never sees a network-backed src and therefore
|
||||
// can't paint progressively.
|
||||
// CDN download / preload time under the "transitioning" overlay. Proxy
|
||||
// path: the <img> then gets a fully-local blob (no progressive paint);
|
||||
// direct path (default): the preloaded original URL.
|
||||
const blobUrl = await getOrCreateBlobUrl(result.imageUrl);
|
||||
// Revoke the previous scene's blob (no longer rendered) to release JS
|
||||
// heap. New scene's original URL takes its place as "current".
|
||||
|
||||
@@ -1,90 +0,0 @@
|
||||
// ─────────────────────────────────────────────────────────────────────────
|
||||
// InfiPlot — Runware image proxy (Cloudflare Worker)
|
||||
//
|
||||
// Why this exists:
|
||||
// Chrome's direct fetch of `im.runware.ai` images sometimes fails with
|
||||
// `ERR_QUIC_PROTOCOL_ERROR` — HTTP/3 stream errors mid-transfer leave the
|
||||
// browser holding a partial PNG, which it renders progressively
|
||||
// (the "层层从上往下" visible-decode glitch). Routing the fetch through
|
||||
// this Worker fixes it in two ways:
|
||||
//
|
||||
// 1. Browser ↔ Worker is HTTP/2 over Cloudflare's edge — extremely
|
||||
// stable, no QUIC fragility.
|
||||
// 2. Worker ↔ Runware is a server-to-server fetch (Cloudflare's
|
||||
// backbone) — also reliable, and the Worker buffers the full
|
||||
// response before streaming it back, so the client never gets
|
||||
// partial bytes mid-stream.
|
||||
//
|
||||
// Bonus side-effects:
|
||||
// - CORS: Worker adds `Access-Control-Allow-Origin: *` so the client's
|
||||
// `fetch()` → blob URL path works regardless of Runware's policy.
|
||||
// - Edge cache: same Runware UUID re-fetched twice in 24h hits the CF
|
||||
// edge cache, sub-50ms response from anywhere in the world.
|
||||
//
|
||||
// Hardening:
|
||||
// - Only proxies `im.runware.ai` (open proxies invite abuse + quota burn).
|
||||
// - Only accepts GET / HEAD / OPTIONS.
|
||||
// ─────────────────────────────────────────────────────────────────────────
|
||||
|
||||
const ALLOWED_HOST = "im.runware.ai";
|
||||
|
||||
const corsHeaders = {
|
||||
"Access-Control-Allow-Origin": "*",
|
||||
"Access-Control-Allow-Methods": "GET, HEAD, OPTIONS",
|
||||
"Access-Control-Max-Age": "86400",
|
||||
};
|
||||
|
||||
export default {
|
||||
async fetch(req) {
|
||||
if (req.method === "OPTIONS") {
|
||||
return new Response(null, { headers: corsHeaders });
|
||||
}
|
||||
if (req.method !== "GET" && req.method !== "HEAD") {
|
||||
return new Response("method not allowed", { status: 405, headers: corsHeaders });
|
||||
}
|
||||
|
||||
const reqUrl = new URL(req.url);
|
||||
const target = reqUrl.searchParams.get("url");
|
||||
if (!target) {
|
||||
return new Response("missing ?url=", { status: 400, headers: corsHeaders });
|
||||
}
|
||||
|
||||
let targetUrl;
|
||||
try {
|
||||
targetUrl = new URL(target);
|
||||
} catch {
|
||||
return new Response("malformed ?url=", { status: 400, headers: corsHeaders });
|
||||
}
|
||||
if (targetUrl.hostname !== ALLOWED_HOST) {
|
||||
return new Response(`only ${ALLOWED_HOST} is allowed`, {
|
||||
status: 403,
|
||||
headers: corsHeaders,
|
||||
});
|
||||
}
|
||||
|
||||
// Fetch upstream. `cf.cacheEverything: true` tells the CF edge to cache
|
||||
// by URL even though Runware's own cache headers are weak — so a second
|
||||
// hit on the same UUID lands in edge memory rather than re-touching
|
||||
// Runware. 1y TTL: image UUIDs are immutable, the bytes never change.
|
||||
const upstream = await fetch(targetUrl.toString(), {
|
||||
cf: { cacheTtl: 31536000, cacheEverything: true },
|
||||
});
|
||||
|
||||
// Stream the body through (no buffering — CF Workers' Response can take
|
||||
// a ReadableStream directly). Rebuild headers to add CORS + strong cache
|
||||
// hints, preserve content-type / content-length from upstream.
|
||||
const headers = new Headers(corsHeaders);
|
||||
headers.set(
|
||||
"Content-Type",
|
||||
upstream.headers.get("content-type") ?? "image/png",
|
||||
);
|
||||
headers.set("Cache-Control", "public, max-age=31536000, immutable");
|
||||
const len = upstream.headers.get("content-length");
|
||||
if (len) headers.set("Content-Length", len);
|
||||
|
||||
return new Response(upstream.body, {
|
||||
status: upstream.status,
|
||||
headers,
|
||||
});
|
||||
},
|
||||
};
|
||||
@@ -1,16 +0,0 @@
|
||||
# Cloudflare Worker — Runware image proxy.
|
||||
# See worker/src/index.js for what it does and why.
|
||||
#
|
||||
# Deploy:
|
||||
# 1. `npm i -g wrangler` (one-time)
|
||||
# 2. `wrangler login` (one-time, OAuth flow in browser)
|
||||
# 3. From this directory: `wrangler deploy`
|
||||
# 4. wrangler prints the deployed URL, e.g.
|
||||
# https://infiplot-image-proxy.<your-cf-username>.workers.dev
|
||||
# 5. Set NEXT_PUBLIC_IMAGE_PROXY_URL=<that URL> in .env.local for dev
|
||||
# and in Vercel project settings for prod.
|
||||
|
||||
name = "infiplot-image-proxy"
|
||||
main = "src/index.js"
|
||||
compatibility_date = "2025-01-01"
|
||||
workers_dev = true
|
||||
Reference in New Issue
Block a user