Thesis¶
Structure, outline, research questions, and comparison criteria for the thesis.
Overview¶
This thesis compares web frameworks for running ML models in the browser — evaluating 3 runtimes (TensorFlow.js, ONNX Runtime Web, LiteRT.js) and 3 frameworks (Transformers.js, MediaPipe Tasks, ML5.js) across performance metrics and developer experience.
Thesis Structure¶
Part I: Introduction & Background¶
- Problem statement: Why browser-based ML matters
- Research questions (RQ1, RQ2, RQ3)
- Browser ML landscape overview
Part II: Methodology¶
- Comparison criteria (5 dimensions)
- Benchmark design (workload, model, input)
- Measurement procedure (6 metrics)
- Experimental environment
Part III: Implementation¶
- Prototype architecture
- Runtime adapter pattern
- Backend and capability detection
- Benchmark result schema
Part IV: Results & Discussion¶
- Runtime capability results
- Quantitative results (load time, inference, memory)
- Qualitative assessment (ease of integration, documentation, community)
- Cross-browser-hardware analysis
- Trade-offs and recommendations
Comparison Criteria¶
| Criterion | Description | Measurement |
|---|---|---|
| Compatibility | Browser support, backend availability | Runtime detection, COMPAT matrix |
| Functionality | Supported tasks, model formats, API surface | Documentation analysis |
| Performance | Load time, inference latency, memory | performance.now(), performance.memory |
| Adoption & Maintenance | Release frequency, community, docs | GitHub activity, documentation review |
| Integration Effort | Setup complexity, API ergonomics | Implementation experience |
Research Questions¶
RQ1: Framework and Runtime Capabilities¶
Which machine learning frameworks, runtimes, model formats, and browser backends are suitable for executing pretrained models directly in the browser?
RQ2: Performance and Resource Behavior¶
How do selected browser machine learning approaches differ in model loading time, cold inference latency, warm inference latency, memory behavior, and backend availability?
RQ3: Developer Experience and Integration¶
What trade-offs do the selected approaches create for developers integrating browser-based machine learning into web applications?
Important Distinction: Runtimes vs Frameworks¶
| Type | Examples | Characteristics |
|---|---|---|
| Runtimes | TensorFlow.js, ONNX Runtime Web, LiteRT.js | Low-level inference engines, tensor operations, manual preprocessing |
| Frameworks | Transformers.js, MediaPipe Tasks, ML5.js | High-level APIs, pipeline abstractions, automatic preprocessing |
Transformers.js uses ONNX Runtime Web internally. The comparison includes both to span the full abstraction spectrum from low-level tensor operations to high-level pipeline APIs.
Prototype Limitations¶
Thesis Scope
The thesis discusses the prototype as an early exploration tool. The following limitations are documented:
- ONNX Runtime Web — re-enabled via HuggingFace CDN (previously disabled due to GitHub LFS issues)
- Memory measurement limited —
performance.memoryis Chromium-only - Single task — Only image classification implemented
- No server persistence — Results stored in localStorage
- COOP/COEP requirement — Non-technical users need a server for headers
These limitations are addressed in the successor application (benchy/): results moved to IndexedDB (via idb-keyval) and model bytes are cached in the Cache API so inference sessions are created from cached bytes with zero network activity — see Storage Architecture. Server-side persistence remains future work (see Components).