Runtime & Backend Selection¶
2026-05-06 — Decision log from prototype planning
Runtimes to include (4)¶
| Runtime | API Level | Philosophy |
|---|---|---|
| TensorFlow.js | Tensor ops | "Build it yourself" |
| ONNX Runtime Web | Session + tensors | "Manage inference yourself" |
| Transformers.js | Pipeline (one-liner) | "Just give me a model" |
| MediaPipe Tasks | Task-specific API | "Just give me data" |
These four runtimes span the full low-level → high-level abstraction spectrum, which aligns with the thesis criterion Funktionalität / Integration.
Runtimes dropped¶
keras.js — treated as a research dead end. Last commit ~2018, Keras 1.x era. Superseded by TensorFlow.js. Its deprecation and lack of modern backend support is itself a valid thesis finding under the criterion Verbreitung / Community.
Backend support¶
| Backend | TF.js | ONNX | Transformers.js | MediaPipe |
|---|---|---|---|---|
| CPU (pure JS) | alias→WASM | |||
| WASM | ||||
| WebGL | ||||
| WebGPU | ||||
| WebNN |
Key decisions¶
- CPU vs WASM are genuinely distinct in TF.js: CPU runs pure JS TF ops; WASM runs compiled C++ ops via WebAssembly with SharedArrayBuffer threads. This is one of the most interesting benchmark dimensions.
- ONNX Runtime Web supports WebGL and WebNN — official docs confirm both.
- Transformers.js kept despite internally wrapping ONNX Runtime Web — it represents a distinct API level (pipeline abstraction, HuggingFace ecosystem) and is relevant to the Integration criterion.
- Transformers.js API does not expose WebGL/WebNN even though the underlying ORT supports them. Only
device: "webgpu" | "wasm"are available. - MediaPipe is WASM+XNNPACK only — CPU-only, no GPU backends. Still valuable as Google's task-optimized approach for real-time applications.
Model distribution plan¶
Two CDN sources for all model artifacts:
| Source | Used by | Model |
|---|---|---|
| TF Hub (via jsDelivr) | TF.js | MobileNet v2 |
| HuggingFace Hub | ONNX, Transformers.js | MobileNet v2 (ONNX quantized) |
| Google Storage | MediaPipe | EfficientNet-Lite0 |
Future: a unified ModelsProvider will centralize model URLs and metadata.