Browser AI Felt Fine on Desktop. Then Mobile Froze the UI.

The first version worked. That was the dangerous part.

On a desktop browser, loading an ONNX model and calling session.run() directly from the app felt acceptable. The result arrived, the feature looked finished, and it was easy to move on. Then the same flow met a less powerful phone and the interface stopped feeling like a product.

Buttons responded late. Progress states struggled to render. A heavy inference run could make the page feel frozen even when the model eventually returned the right output.

Local AI is not a good user experience if the interface disappears while the model is thinking.

The model was local, but the work was still blocking the experience

Running AI in the browser removes a server round trip and can keep private inputs on the device. It does not automatically make the workload cheap. Model initialization, tensor preparation, WebAssembly work, and inference can still compete with the interface when too much happens on the main thread.

Desktop hardware can hide that mistake. Mobile hardware exposes it quickly. Even when total inference time is reasonable, a blocked interface makes the wait feel longer because the user loses feedback and control.

The useful boundary was a Web Worker

The fix was not a new loading animation. The inference lifecycle needed its own execution lane.

We moved ONNX Runtime loading, model session creation, and session.run() into a Web Worker. The main thread became responsible for the interface and a small message protocol. The worker became responsible for the heavy model work.

• The UI creates a client and asks the worker to preload a model.
• The worker loads ONNX Runtime and keeps one model session cached.
• Inputs cross the boundary as serializable tensor-shaped data.
• The worker runs inference and sends serialized output tensors back.
• Progress, errors, timeouts, and disposal travel through the same protocol.

The model did not become smaller and inference did not become free. The important change was that the page could keep painting, updating status, and responding to the user while the worker was busy.

Why the worker lives in public

The reusable version uses a classic worker and importScripts() to load the ONNX Runtime bundle. That makes stable static URLs important.

A bundler-managed worker created with new URL(..., import.meta.url) can be elegant, but framework tooling may treat development and production differently. Hashed output paths or injected development code can become especially awkward when the worker expects classic-worker syntax.

Serving the worker, ONNX Runtime files, WebAssembly binaries, and model from known public URLs is less clever. It is also easier to debug. For this plumbing, boring is a feature.

The small client wrapper matters as much as the worker

Raw postMessage calls are simple until several requests, progress events, errors, and cleanup rules exist at the same time. The starter wraps that protocol in a promise-based OnnxWorkerClient, so preload() and run() feel like ordinary asynchronous functions.

There is also an optional React hook that exposes status, progress, error, preload, run, and dispose without tying the core worker client to React. A plain JavaScript app can use the client directly.

What the starter includes

We pulled the reusable plumbing out of Lumli and published it as a small MIT-licensed starter. It intentionally does not include a model, image preprocessing, or product UI.

• A classic ONNX worker that loads the runtime and caches one session.
• A promise-based client around the worker message protocol.
• An optional React hook for application state.
• A minimal demo component.
• Protocol tests for requests, responses, errors, and disposal.

Open the ONNX Web Worker Starter on GitHub

A worker is not a performance miracle

Moving inference off the UI thread protects responsiveness; it does not remove memory limits, slow model initialization, large downloads, or expensive preprocessing. Mobile browsers still have tighter constraints than desktop machines.

For image models, reducing the input before inference is often the highest-value optimization. Crop to the useful region, resize to the dimensions the model actually needs, avoid unnecessary copies, and test with the real execution provider and real device class.

The practical goal is not to make browser AI sound effortless. It is to give each part of the product the right job: the interface stays responsive, the worker handles inference, and the model remains small enough for the device you expect people to use.

Why we published the plumbing

This pattern took longer to make reliable than it should have. Once it worked, keeping it buried inside one product felt wasteful.

The repository is deliberately small enough to read in one sitting and incomplete in the useful way: bring your own model, inputs, preprocessing, and UI. The worker boundary and client protocol are already there, so the next browser-AI experiment can start closer to the interesting problem.