planned-app/¶

The planned production React application for benchmarking browser ML frameworks. This app will address the limitations of the early prototypes and provide a proper benchmarking environment.

Purpose¶

The main app will be a production-grade benchmarking platform that:

• Hosts models server-side with authentication
• Runs benchmarks across multiple frameworks and backends
• Stores results persistently for comparison and analysis
• Provides a polished UI for configuring and running benchmarks

Planned Architecture¶

graph TD
    A[React + Vite App] --> B[Server API]
    B --> C[Model Registry]
    B --> D[Result Storage]
    B --> E[Auth Layer]
    A --> F[Benchmark Engine]
    F --> G[Runtime Adapters]
    G --> H[TF.js]
    G --> I[ONNX Runtime Web]
    G --> J[LiteRT.js]
    G --> K[Transformers.js]
    G --> L[MediaPipe Tasks]
    G --> M[ML5.js]

    style A fill:#3b82f6,color:#fff
    style B fill:#ef4444,color:#fff
    style F fill:#22c55e,color:#fff

Key Features (Planned)¶

1. Server-Side Model Hosting¶

Models will be hosted on the server and proxied to the browser with authentication:

// Server endpoint
app.get('/api/models/:framework/:filename', authMiddleware, (req, res) => {
  const modelPath = `./models/${req.params.framework}/${req.params.filename}`;
  res.sendFile(modelPath);
});

This solves: - ONNX model access (resolved via HuggingFace CDN for prototype; server proxy planned for production) - Model version control - Rate limiting and abuse prevention - Fair comparison (same model source for all frameworks)

2. Persistent Result Storage¶

Implemented (browser-local): Results are stored in IndexedDB (via idb-keyval, dedicated mlbench-db store) instead of localStorage — removing the ~5–10 MB ceiling and surviving page reloads without a server. Model bytes are cached separately in the Cache API so inference sessions are created from cached ArrayBuffers with zero network activity during measurement.

Planned (server-side, future work): a server database for cross-device result sharing:

interface BenchmarkResult {
  id: string;
  userId: string;
  timestamp: Date;
  runtime: string;
  model: string;
  backend: string;
  metrics: {
    loadTimeMs: number;
    coldInferMs: number;
    warmInferMs: number;
    memoryDeltaMb: number;
  };
}

3. Multi-User Support¶

The app will support multiple users running benchmarks:

• User authentication and authorization
• Personal benchmark history
• Shared result comparison
• Role-based access (admin, researcher, viewer)

4. Comprehensive Framework Support¶

All six adapters will be fully functional:

Technology Choices¶

Frontend¶

• React — Component-based UI
• Vite — Fast build tooling
• TypeScript — Type safety
• Tailwind CSS — Styling (TBD)

Backend¶

No backend in the current implementation. Server-side persistence and auth remain future work (parked — see research/plans/impl-plan-benchy.md item #1).

Benchmarking¶

• Custom adapters — Runtime integration (one adapter per runtime/framework)
• performance.now() — Timing measurements (no external benchmark engine; Tinybench was evaluated and not adopted)

Migration from Prototype¶

The prototype's adapter pattern will be reused:

// Reusable from prototype
interface MLAdapter {
  load(): Promise<void>;
  infer(imgEl: HTMLImageElement): Promise<{ label: string; score: number }>;
  getBackend(): string;
}

The main differences:

1. Models served same-origin (/models/..., vendored) instead of CDN
2. Authentication required for benchmark access (not yet implemented)
3. Results stored in IndexedDB instead of localStorage (server database remains future work)
4. Build step (Vite) for production optimization

Timeline¶

Connection to thesis¶

The planned app architecture informs:

• Chapter 3 (Methodology): Discussion of server-side model hosting for fair comparison
• Chapter 3a (Implementation): Limitations of prototype vs planned app
• Chapter 5 (Discussion): Recommendations for production benchmarking
• Chapter 6 (Conclusion): Future work items