How do I compare WebGPU and WASM fairly?

Use the same device, browser version, model, voice, text, and settings. Measure model loading separately from first audio and warm synthesis, and publish the raw runs and runtime configuration.

What segment length is best for a TTS benchmark?

There is no universal best length. Use identical text and boundaries for both runs, then choose segments that represent the real workflow.

Why are cold starts different from warm runs?

First use may include model download, cache lookup, and runtime initialization. Report those separately from synthesis after the model is ready.

Browser TTS Benchmark Guide: WebGPU vs. WebAssembly

This is a measurement guide, not a report of a controlled benchmark. No current device-by-device raw results are published here, so this page makes no speed-ratio or browser-parity claim. It explains how to compare the runtimes on your own target devices and what to record before drawing a conclusion.

What this page can and cannot claim

A benchmark result is useful only when another person can see how it was produced. This page does not have a maintained set of raw timing runs for named devices, browser versions, model files, and runtime settings. For that reason, the earlier speed multipliers, browser comparisons, and fixed hardware descriptions have been removed. Treat the sections below as a protocol for collecting local evidence, not as measured results for every visitor.

Kokoro Web checks whether the browser worker can obtain a WebGPU adapter and then selects a runtime; when no adapter is available, it uses the WebAssembly path. The interface reports the selected device. An adapter check identifies an available capability, but it does not measure how fast that runtime will be on a particular device.

Define the timing you want to compare

“Fast” can refer to several different parts of the experience. A first visit may spend time downloading model weights. A later visit may read cached files but still need to initialize the inference runtime. During generation, a user may care most about the time until the first audio chunk, while a batch workflow may care about the time until the entire file is ready. Record these as separate measurements instead of combining them into one latency number.

Cold start: time from opening the tool to the model becoming ready, with network and cache state recorded.
Warm start: time to readiness when the required model files are already cached.
First audio: time from the generation action to the first playable or streamed audio.
Completion: time until the complete output is ready for download.
Quality and stability: pronunciation, pauses, artifacts, memory pressure, and whether the run completes without an error.

Keep the comparison controlled

Use one device and keep the browser version, operating system, power mode, model, voice, speed, text, and segment boundaries the same. Close unrelated workloads and record whether the machine is plugged in. Do not compare WebGPU on a recent laptop with WASM on a different computer and attribute the whole difference to the runtime.

Run the same sample under cold-cache and warm-cache conditions. Repeat each condition enough times to see whether timing is stable, and publish the individual observations along with the median and a spread such as the 90th percentile. Keep failed and unusually slow runs in the record; explain exclusions rather than silently removing them. A short sample should test responsiveness, while a representative long script should reveal memory or recovery issues.

Record runtime details with every result

For each report, include the device model and memory, operating system, browser and version, date, model identifier or revision, voice, speed, text length, segment boundaries, runtime shown by the tool, and whether the model was cached. Also state whether network download time is included and how you measured audio completion. These details let readers decide whether the result applies to their own setup.

Keep the generated audio with the timing record. Listen to both outputs at matched playback loudness and compare names, punctuation, pauses, and the end of each segment. A faster render is not an improvement if it changes the output quality or fails more often. If you publish a comparison, include the raw timing table and the exact test text so another person can repeat it.

Choose a workflow for the target devices

WebGPU is a secure-context browser API and is not available in every browser or device. Check the current WebGPU browser documentation and test the actual devices that matter to your audience. When the API is unavailable, a WASM path can provide a compatibility route; when it is available, measure the application rather than assuming that a GPU-backed run is automatically faster.

For narration, split text at sentence or thought boundaries when separate files make editing and recovery easier. Keep the same boundaries in a runtime comparison. There is no universal number of seconds per segment: the right unit depends on the model, device memory, desired pauses, and whether the workflow streams partial audio or exports one finished file.

Benchmark reporting checklist

Publish the test device, OS, browser version, model revision, voice, and generation settings.
Report model download, initialization, first-audio, and full-completion times separately.
Separate cold-cache runs from warm-cache runs, and include repeated raw observations.
Use identical text and segment boundaries; keep the audio outputs for quality review.
Report failures and explain any excluded runs.

Until such a dataset is available, the responsible conclusion is limited: runtime availability and performance depend on the full device, browser, model, and workload combination. Use the selected-device indicator to understand which path ran, then measure your own target environment before making a speed claim.

Author: Kokoro Web Team • Last updated 2026-10-15