Storage Architecture¶

This page documents how benchy/ persists data across the four browser storage mechanisms, why each mechanism was chosen, and the known gaps. It supersedes the prototype's single-key localStorage approach (mlbench_runs JSON array, documented in prototype/index.md).

Overview¶

All IndexedDB access goes through a dedicated object store (createStore('mlbench-db', 'mlbench-store') in src/state/idb.ts) so app data cannot collide with other origins' keys, and every persistence path degrades gracefully (in-memory fallback) when storage is unavailable (private browsing, quota pressure).

Model Byte Cache¶

Design¶

The Cache API layer guarantees that a second load against the same model URL costs zero network time, so download time never contaminates measured inference:

1. Preflight (prepareModels() in downloadManager.ts) downloads every selected model once with byte-level progress (ReadableStream + Content-Length) and writes the bytes to the Cache API. "Start Benchmark" stays disabled until every selected model is cached.
2. Adapters create sessions from cached ArrayBuffers — not URLs:
3. ONNX: ort.InferenceSession.create(modelBytes, ...) (src/core/runtimes/onnx.ts)
4. LiteRT: loadAndCompile(modelBytes, { accelerator }) (src/core/runtimes/litert.ts)
5. Worker path is cache-first too — src/workers/inference.worker.ts routes through downloadModelWithProgress(), so offloaded inference never re-downloads (Cache API is available in dedicated workers via WorkerGlobalScope.caches).

Why Cache API and not IndexedDB for model bytes?¶

Both share the origin storage quota and eviction policy (see MDN: Storage quotas). The split follows each API's intended use:

• Cache API stores HTTP Request/Response pairs — model files are exactly that, and the API makes URL-keyed lookup a one-liner (cache.match(url)).
• IndexedDB stores structured, queryable records — benchmark runs are exactly that.

Self-caching runtimes (no explicit cache needed)¶

• Transformers.js manages its own browser cache via the Cache API under the transformers-cache key (env.useBrowserCache defaults to on when the Cache API is available; confirmed against the transformers.js source).
• MediaPipe Tasks fetches modelAssetPath through its own loading path with standard HTTP caching.

Known Gap: TF.js and ML5 Bypass the Model Cache¶

Impact: For these two adapters, a cold browser HTTP cache means the model download happens inside the measured load() phase. This does not corrupt inference metrics — load time is reported separately and excluded from inference statistics — but loadTimeMs for TF.js/ML5 is not comparable with the cache-first ONNX/LiteRT path when the HTTP cache is cold.

Mitigations in place:

• All model URLs are same-origin (/models/tfjs/...), so standard HTTP caching applies and warm loads are fast.
• The download preflight warms the HTTP cache for these URLs even though it cannot hand bytes to the runtime.

Why not fixed: A proper fix requires a custom TF.js IOHandler that reassembles the model JSON + shard set from cached bytes (the graph-model format is a JSON manifest plus binary shard files, not a single blob). The effort is significant and the benchmark impact is nil, since load time is already excluded from inference stats.

Future Hardening (documented, not implemented)¶

Cache Invalidation¶

The Cache API store (mlbench-models-v1) is keyed by URL only — there is no versioning or busting logic. If vendored model bytes ever change (e.g., a new pinned version in scripts/vendor-models.mjs with a different sha), a browser that still holds the old bytes under the same URL would keep serving them from the cache.

Current risk: low. Model URLs are version-pinned paths under /models/... maintained by the vendor script, and the sha manifest (public/models/*) is committed — but the Cache API layer does not consult that manifest. If model artifacts are ever updated in place, either:

• bump the cache name (mlbench-models-v1 → v2) to force a cold cache, or
• key cache entries by content sha instead of URL.

Storage Persistence¶

Both the Cache API and IndexedDB live under the origin storage quota and are evictable under disk pressure ("best-effort" mode). A long offline demo could lose cached models or run history if the browser evicts origin data.

Optional hardening: call navigator.storage.persist() once at startup to request persistent storage, which opts the origin out of eviction under pressure (the browser may still show a permission prompt or grant based on engagement heuristics). modelCacheSizeEstimate() in modelCache.ts already reads navigator.storage.estimate(); a persistence request would complement it. Not implemented because eviction has not been observed in practice and the API's grant behavior varies by browser.

Prototype vs benchy¶