NPU/WebNN Benchmarking¶

Status: Research Complete

1. What is WebNN?¶

The Web Neural Network (WebNN) API is a W3C web standard that enables high-performance neural network inference directly in the browser, with access to hardware-specific accelerators:

Backend	Hardware	Platform
CPU	Any CPU	All browsers
GPU	Integrated/discrete GPU	Chrome/Edge
NPU	Neural Processing Unit	Chrome/Edge (Windows, macOS, Android)

Key Differences from WebGPU¶

Feature	WebGPU	WebNN
Purpose	General GPU compute	ML-specific inference
API Level	Low-level (shaders)	High-level (graphs)
Hardware Access	GPU only	CPU, GPU, NPU
Power Efficiency	High (general compute)	Optimized for ML (low power)
Operator Support	Manual implementation	Built-in ML operators
Browser Support	Chrome 113+, Firefox (flag)	Chrome 127+ (flag), Edge

2. NPU Hardware in Your System¶

AMD RX 6650 XT¶

The AMD RX 6650 XT does NOT have a dedicated NPU. NPUs are found in:

Vendor	NPU Name	Found In
Intel	Intel AI Boost (Meteor Lake+)	Core Ultra 7/9
AMD	XDNA (Ryzen AI)	Ryzen 7040+
Qualcomm	Hexagon NPU	Snapdragon X Elite
Apple	Neural Engine	M1+

However: Chrome 148 reports WebNN/NPU support via navigator.ml because it can route to GPU via DirectML on Windows. The "NPU" backend in Chrome may actually map to GPU on systems without dedicated NPU hardware.

3. WebNN API Usage¶

Basic Usage with ONNX Runtime Web¶

// 1. Import ORT with WebNN support
import * as ort from 'onnxruntime-web/all';

// 2. Create session with WebNN execution provider
const session = await ort.InferenceSession.create(modelPath, {
  executionProviders: [
    {
      name: 'webnn',
      deviceType: 'npu',  // 'cpu' | 'gpu' | 'npu'
      powerPreference: 'default',  // 'default' | 'low-power' | 'high-performance'
    },
  ],
});

// 3. Run inference
const input = new ort.Tensor('float32', inputData, [1, 3, 224, 224]);
const output = await session.run({ input });

Device Type Selection¶

// Auto-detect best backend
async function getBestDevice() {
  if (!navigator.ml) return 'wasm';  // Fallback

  // Check NPU availability
  try {
    const context = await navigator.ml.createContext({ deviceType: 'npu' });
    context.destroy();
    return 'npu';
  } catch {}

  // Check GPU availability
  try {
    const context = await navigator.ml.createContext({ deviceType: 'gpu' });
    context.destroy();
    return 'gpu';
  } catch {}

  return 'cpu';
}

4. WebNN Benchmarking Approach¶

Phase 1: Capability Detection¶

async function detectWebNNCapabilities() {
  const caps = {
    webnnSupported: !!navigator.ml,
    deviceTypes: [],
    operators: null,
  };

  if (!navigator.ml) return caps;

  // Test each device type
  for (const deviceType of ['cpu', 'gpu', 'npu']) {
    try {
      const ctx = await navigator.ml.createContext({ deviceType });
      caps.deviceTypes.push(deviceType);
      ctx.destroy();
    } catch {}
  }

  return caps;
}

Phase 2: Model Benchmarking¶

async function benchmarkWebNN(modelUrl, deviceType, iterations = 100) {
  const session = await ort.InferenceSession.create(modelUrl, {
    executionProviders: [{
      name: 'webnn',
      deviceType: deviceType,
      powerPreference: 'high-performance',
    }],
  });

  const input = createInputTensor();  // Pre-allocated

  // Warmup
  for (let i = 0; i < 10; i++) {
    await session.run({ input });
  }

  // Benchmark
  const times = [];
  for (let i = 0; i < iterations; i++) {
    const start = performance.now();
    await session.run({ input });
    times.push(performance.now() - start);
  }

  return {
    mean: times.reduce((a, b) => a + b) / times.length,
    min: Math.min(...times),
    max: Math.max(...times),
    p50: times.sort((a, b) => a - b)[Math.floor(times.length / 2)],
    p99: times.sort((a, b) => a - b)[Math.floor(times.length * 0.99)],
  };
}

Phase 3: Comparison Matrix¶

Test each combination:

Model	CPU	GPU (WebGPU)	GPU (WebNN)	NPU (WebNN)
MobileNet v2	✅	✅	✅	?
EfficientNet-Lite0	✅	✅	✅	?
SqueezeNet 1.0	✅	✅	✅	?

5. WebNN Operator Support¶

Current Status (as of 2026)¶

Operator Category	Support Level	Notes
Convolution	✅ Full	Core operator
Pooling	✅ Full	Max, Average, Global
Activation	✅ Full	ReLU, Sigmoid, Tanh
Batch Normalization	✅ Full	Common in MobileNet
Softmax	✅ Full	Output layer
Element-wise	✅ Full	Add, Multiply, etc.
Reshape/Transpose	✅ Full	Layout conversion
Attention	⚠️ Partial	Transformer models
Custom operators	❌ Limited	May fall back to WASM

Impact on Benchmarking¶

For image classification models (MobileNet, EfficientNet, SqueezeNet): - All required operators are supported - WebNN should provide full hardware acceleration - No operator fallback expected

For transformer/LLM models: - Some operators may fall back to WASM - Mixed execution paths reduce performance gains - Test with operator coverage check

6. Implementation Plan¶

Step 1: Add WebNN Detection to ml-browser-check¶

// Add to ml-browser-check/server.js capability detection
async function detectWebNN() {
  if (!navigator.ml) return { supported: false };

  const devices = {};
  for (const deviceType of ['cpu', 'gpu', 'npu']) {
    try {
      const ctx = await navigator.ml.createContext({ deviceType });
      devices[deviceType] = true;
      ctx.destroy();
    } catch {
      devices[deviceType] = false;
    }
  }

  return { supported: true, devices };
}

Step 2: Add WebNN Benchmark to prototype¶

Add WebNN as a new execution provider in the ORT adapter:

// In ONNXAdapter
async load() {
  // ... existing code ...

  if (this.backendPref === 'webnn-npu') {
    executionProviders = [{
      name: 'webnn',
      deviceType: 'npu',
      powerPreference: 'high-performance',
    }, 'wasm'];
  } else if (this.backendPref === 'webnn-gpu') {
    executionProviders = [{
      name: 'webnn',
      deviceType: 'gpu',
    }, 'wasm'];
  }
}

Step 3: Create WebNN Benchmark Suite¶

const WEBNN_BENCHMARK_SUITE = {
  name: 'webnn-acceleration',
  models: ['mobilenet-v2', 'efficientnet-lite0', 'squeezenet-1.0'],
  devices: ['cpu', 'gpu', 'npu'],
  iterations: 100,
  warmup: 10,
};

7. Expected Results¶

Based on research and hardware specs:

Backend	Expected Latency	Notes
WebNN CPU	50-200ms	Similar to WASM
WebNN GPU	10-30ms	Via DirectML, comparable to WebGPU
WebNN NPU	5-15ms	If dedicated NPU available
WebGPU	6-20ms	Current fastest on Chrome

For AMD RX 6650 XT (no dedicated NPU): - WebNN GPU via DirectML should be competitive with WebGPU - WebNN "NPU" may fall back to GPU on this hardware - Main benefit: power efficiency, not raw speed

8. References¶

WebNN API Spec: https://www.w3.org/TR/webnn/
ONNX Runtime WebNN EP: https://onnxruntime.ai/docs/tutorials/web/ep-webnn.html
Microsoft WebNN Tutorial: https://learn.microsoft.com/en-us/windows/ai/directml/webnn-overview
WebNN Testing Guide: https://webnn.io/en/learn/get-started/testing
Intel WebNN Demo: https://www.intel.com/content/www/us/en/developer/videos/fast-hardware-agnostic-ai-web-apps-with-webnn.html