1. Why Run ML Inference in the Browser?

Deploying a trained Python model typically requires a backend server (such as FastAPI or Flask), container orchestration, ongoing compute costs, and network round-trip latency. Free tiers on PaaS platforms also introduce cold-start delays when containers spin down after inactivity.
PaaS platforms like Render, Railway, or Streamlit Cloud host your backend app for you, and “cold starts” happen when a free server goes to sleep and takes several seconds to wake up on the next request.

WebAssembly (WASM) is a portable binary code format that lets languages like C, C++, and Rust run inside a web browser at near-native speed alongside JavaScript.

While WebAssembly (WASM) is a general-purpose compilation target used both server-side (via runtimes like Wasmtime or edge workers) and client-side, executing inference directly inside the user’s browser eliminates the backend server entirely:

  • Zero Server Compute Cost: The model is served as a static binary asset from a CDN (a globally distributed network of servers that delivers static files quickly to users based on their location) or static host (such as GitHub Pages). The client device supplies the CPU or GPU cycles.
  • Zero Inference Round-Trip Latency: Once the model downloads, predictions run locally in milliseconds without HTTP requests.
  • Data Privacy: Sensitive user inputs never leave the client device.

It’s up to you to figure out if there’s a use case for your situation.

Why Tabular Tree Models Fit Browser Execution

Deep neural networks often require tens or hundreds of megabytes of weights and heavy matrix multiplication kernels. By contrast, decision trees and gradient-boosted ensembles (scikit-learn, XGBoost, LightGBM) compile down to compact arrays of node thresholds, feature indices, and leaf values.

Executing a tree ensemble is deterministic branching and floating-point arithmetic. Even on mobile CPUs, a compiled tree model with dozens of estimators evaluates in single-digit milliseconds, making tabular models ideal candidates for browser-based inference. In simple terms, it’s just a bunch of if/then conditions.

2. Evaluating the Landscape: Browser Execution Options

There are 3 main ways to run tree-based machine learning models inside a web browser.

2.1. Native Framework Compilations (e.g., @wlearn/xgboost)

Libraries like @wlearn/xgboost compile the upstream C++ engine to WebAssembly via Emscripten (a compiler toolchain that translates C and C++ codebases into WebAssembly), exposing both fit() and predict() APIs in JavaScript.

  • Best for: Browser applications that must train or fine-tune models client-side.
  • Trade-off: Shipping training logic increases bundle size and couples your frontend to a single framework’s runtime.

2.2. Pure Code Transpilation (m2cgen)

Tools such as m2cgen (Model to Code Generator) transpile trained scikit-learn or XGBoost models into zero-dependency JavaScript, C, or Rust source code consisting of nested if/else statements.

  • Best for: Small models where external runtime dependencies are unacceptable.
  • Trade-off: Large ensembles generate tens of megabytes of raw source text that slow down browser parsing. Furthermore, as of November 2026, m2cgen has not been updated in 4 years, making it unsuitable for production systems that rely on active security maintenance and modern library versions.

2.3. Standardized Graph + Runtime (ONNX + ONNX Runtime Web)

ONNX is a cross-platform inference standard used across cloud servers, mobile apps, and edge devices. For the browser, Microsoft maintains ONNX Runtime Web (onnxruntime-web), which executes .onnx computation graphs via WebAssembly and WebGPU.

  • Best for: Browser-based inference with pre-trained models across frameworks.
  • Trade-off: Requires an explicit conversion step in your Python pipeline, but decouples training from inference and leverages actively maintained C++/WASM SIMD kernels (Single Instruction, Multiple Data: a CPU feature that lets a single instruction process multiple numbers in parallel, speeding up math operations).
Approach Client Training Runtime Dependency Maintenance & Ecosystem Browser Fit
Native WASM (@wlearn/xgboost) Yes Framework-specific WASM Tied to wrapper updates In-browser training
Code Gen (m2cgen) No None (Pure JS / C / Rust) Stale (unmaintained) Tiny, legacy-only models
ONNX Runtime Web No onnxruntime-web (WASM/WebGPU) Actively maintained (Microsoft/ONNX) Pre-trained model inference

3. Understanding ONNX & ONNX Runtime

Before looking at browser execution internals, it helps to separate the file specification from the execution engine.

ONNX vs. ONNX Runtime

  • ONNX (Open Neural Network Exchange): An open-standard binary format (.onnx, serialized via Protocol Buffers. Protocol Buffers are Google’s compact binary format for serializing structured data, similar to JSON but smaller and faster to parse) that represents a machine learning model as a directed acyclic computation graph. Instead of Python objects or framework-specific weights, it stores standardized operators (such as Add, MatMul, or TreeEnsembleClassifier) and typed tensor shapes (a tensor is simply a multi-dimensional array with a fixed data type, essentially the ML framework equivalent of a NumPy ndarray).
  • ONNX Runtime (ORT): A cross-platform, C++-backed execution engine maintained by Microsoft. When loading an .onnx graph, ORT performs graph-level optimizations (such as operator fusion and constant folding. These two mean combining consecutive math steps into one and pre-computing static values at load time) and delegates execution to hardware-specific backends called Execution Providers (EPs), such as CUDA/TensorRT on NVIDIA GPUs, CoreML on Apple devices, DirectML on Windows, or WebAssembly and WebGPU in browsers.

Why Frameworks Do Not Export ONNX Natively

Deep learning frameworks like PyTorch represent models as explicit tensor computation graphs, allowing them to provide native exporters (torch.onnx.export()).

Traditional tabular libraries (scikit-learn, XGBoost, LightGBM) do not build computation graphs:

  • scikit-learn stores estimators as Python and Cython objects. Its native persistence format (pickle or joblib) serializes live Python memory, which requires an identical Python and scikit-learn runtime to deserialize.
  • XGBoost serializes boosted trees into its own internal binary or JSON tree structures.

Rather than adding deployment-specific exporters to every training library, the ONNX ecosystem maintains dedicated compiler libraries that inspect trained objects and translate their decision logic into standard ONNX operators:

  • skl2onnx: The official converter for scikit-learn estimators and pipelines.
  • onnxmltools: The companion converter supporting XGBoost, LightGBM, CatBoost (CatBoost actually includes a built-in native exporter to onnx via its standard save_model() method), SparkML, and CoreML. For scikit-learn and TensorFlow, onnxmltools acts as a unified wrapper that delegates directly to skl2onnx and tf2onnx under the hood.

So if you have to get familiar with 1 tool, get familiar with onnxmltools.

4. Under the Hood: Execution Providers, Fallbacks & Browser Compatibility

When ONNX Runtime Web (onnxruntime-web) initializes in a browser or Node.js environment, it abstracts hardware differences through Execution Providers (EPs):

  • WebAssembly (wasm): The universal CPU baseline. It executes compiled C++ inference kernels inside the browser’s WebAssembly sandbox and works across every modern desktop and mobile browser, as well as Node.js (single-threaded).
  • WebGPU (webgpu): The modern GPU compute backend, designed for heavy parallel matrix workloads (such as vision models and LLMs). Currently supported in Chromium-based browsers on Windows, macOS, and Android, but unavailable on other browsers/OSes, and you can’t run it on Node.js directly.
  • WebGL (webgl): The legacy shader-based GPU backend. It is currently in maintenance mode in favour of WebGPU.
  • WebNN (webnn): An emerging W3C standard exposing OS-level neural hardware accelerators, currently behind experimental browser flags in Chromium.

Browser Compatibility Matrix

According to the official onnxruntime-web support matrix, backend availability breaks down as follows:

Execution Provider Chrome / Edge (Win & macOS) Chrome / Edge (Android) Safari / Chrome (iOS) Firefox (Desktop) Node.js
WebAssembly (wasm) ✔️ ✔️ ✔️ ✔️ ✔️ (Single-thread)
WebGPU (webgpu) ✔️ (Chromium 113+) ✔️ (Chromium 121+) ❌ ❌ ❌
WebGL (webgl) ✔️ (Maintenance) ✔️ (Maintenance) ✔️ (Maintenance) ✔️ (Maintenance) ❌
WebNN (webnn) ✔️ (Experimental Flag) ❌ ❌ ❌ ❌

Automatic Fallback & Runtime Probing

When creating an InferenceSession (See https://onnxruntime.ai/docs/api/js/interfaces/InferenceSessionFactory.html#create and https://onnxruntime.ai/docs/api/js/interfaces/InferenceSession.SessionOptions.html), you pass an ordered list of preferred Execution Providers:

const session = await ort.InferenceSession.create("model.onnx", {
  executionProviders: ["webgpu", "wasm"]
});

ONNX Runtime Web evaluates this list from left to right. If the client device or browser lacks WebGPU support (for example, on iOS Safari or Firefox), it catches the initialization failure internally and falls back to wasm. For classical tree ensembles (TreeEnsembleClassifier), CPU WebAssembly is typically optimal anyway because tree traversal is branch-heavy rather than matrix-multiplication bound.

Once the wasm provider is selected, the runtime performs two hardware and environment checks:

[ONNX Runtime Web Initializes]
       │
       ├──► 1. Execution Provider Check (Left-to-Right)
       │       ├── WebGPU supported & requested? ──► Execute on GPU
       │       └── Otherwise ──────────────────────► Fall back to WebAssembly (CPU)
       │
       ├──► 2. Threading Check (SharedArrayBuffer + crossOriginIsolated)
       │       ├── COOP/COEP headers present? ─────► Multi-threaded WASM (Web Workers)
       │       └── Missing headers (Static Hosts) ─► Fall back to Single-threaded WASM
       │
       └──► 3. SIMD Check (128-bit Vector Instructions)
               ├── Browser supports WASM SIMD? ────► Execute SIMD-accelerated WASM
               └── Legacy pre-2021 browser? ───────► Fails (modern CDN builds require SIMD)

Static Hosting & Mobile Browser Caveats

  1. Multi-Threading on Static Hosts: Parallel WASM execution requires SharedArrayBuffer (a JavaScript feature that lets multiple background threads share memory safely), which browsers only enable when the server sends Cross-Origin-Opener-Policy: same-origin and Cross-Origin-Embedder-Policy: require-corp headers. Static hosts like GitHub Pages do not emit these headers by default, so ONNX Runtime Web automatically falls back to single-threaded execution (ort.env.wasm.numThreads = 1). More context: following the Spectre CPU security vulnerabilities, browsers only enable it if the web server sends strict isolation HTTP headers (COOP and COEP), which static hosts like GitHub Pages do not allow you to configure.
  2. Explicit wasmPaths Resolution: When loading ort.min.js from a CDN, the runtime dynamically fetches companion .mjs glue code and .wasm binaries. Explicitly setting ort.env.wasm.wasmPaths to the CDN dist/ directory prevents path-resolution hangs on nested static routes.
  3. Privacy-Hardened Mobile Browsers: Standard mobile browsers (Chrome on Android, Safari on iOS) support WASM SIMD out of the box. However, privacy-hardened browsers such as Firefox Focus restrict WebAssembly JIT compilation (Just-In-Time compilation, where the browser translates WebAssembly bytecode into native machine code right before running it) and block external worker/CDN scripts, which can prevent the runtime from initializing.

5. Practical Walkthrough: Biotech Venture Catalyst Case Study

To illustrate how this works in practice, let’s look at how I added client-side inference to Biotech Venture Catalyst (live static demo).

The Architectural Split: Feature Engineering vs. Model Graph

In Python, the training pipeline (train.py) combines two stages:

  1. Raw input parsing (pandas datetime parsing, string splitting, regex keyword counts) into a Python dictionary.
  2. Vectorization (DictVectorizer) followed by a RandomForestClassifier (n_estimators=50, max_depth=10).

Trying to compile Python regular expressions and flexible date parsing into ONNX graph operators is brittle and poorly supported in browser runtimes. Instead, we split responsibilities cleanly:

  • JavaScript handles feature extraction and vectorization: Mirrors the exact Python rules in ~40 lines of vanilla JS and maps keys against a static metadata.json vocabulary exported from DictVectorizer.
  • ONNX handles the tree ensemble: Only the trained RandomForestClassifier is compiled into model.onnx, accepting a flat Float32Array (a fixed-length JavaScript array of 32-bit floating-point numbers, matching NumPy’s float32 dtype) input tensor.

Step 1: Exporting the Classifier & Vocabulary (export_onnx.py)

Using skl2onnx, we extract the ordered feature names from DictVectorizer and compile the RandomForestClassifier into an ONNX graph.

A critical detail is passing zipmap: False. By default, skl2onnx appends a ZipMap operator to classifiers, which outputs an array of JavaScript Map objects rather than a standard tensor. Disabling zipmap ensures ONNX Runtime Web returns a flat Float32Array ([prob_class_0, prob_class_1]):

#!/usr/bin/env python

import json
import os
import pickle

from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType

os.makedirs("docs", exist_ok=True)

with open("model.bin", "rb") as f:
    pipeline = pickle.load(f)

dv = pipeline.named_steps["vectorizer"]
rf = pipeline.named_steps["classifier"]

# 1. Save the exact ordered feature vocabulary from DictVectorizer
feature_names = dv.get_feature_names_out().tolist()
with open("docs/metadata.json", "w") as f:
    json.dump({"feature_names": feature_names}, f, indent=2)

# 2. Export only the RandomForestClassifier expecting a 1D float vector
initial_type = [("float_input", FloatTensorType([None, len(feature_names)]))]
onnx_model = convert_sklearn(
    rf,
    initial_types=initial_type,
    options={id(rf): {"zipmap": False}},
)

with open("docs/model.onnx", "wb") as f:
    f.write(onnx_model.SerializeToString())

print(f"Exported docs/model.onnx ({len(feature_names)} input features) and docs/metadata.json")

Step 2: Running Inference in the Browser (docs/index.html)

On the static GitHub Pages frontend, init() configures WASM paths via CDN, loads metadata.json, and initializes the InferenceSession. When the user clicks predict, vectorize() maps JavaScript features into the Float32Array layout required by model.onnx.
The complete client-side JavaScript implementation for initializing the ONNX runtime, vectorizing input records, and running browser-based inference can be inspected directly in the repository: https://github.com/k-candidate/biotech-venture-catalyst/blob/main/docs/index.html.

Because model.onnx is under 1 MB and contains 50 shallow trees, initialization takes a fraction of a second and inference completes in single-digit milliseconds, producing identical probabilities to the FastAPI and Streamlit Python backends.

6. Adapting the Pattern for Different Use Cases

The Biotech Venture Catalyst implementation uses a small scikit-learn Random Forest running on the default CDN build. If your production requirements differ, you can adapt the setup in three main ways:

1. Optimizing Bundle Size for Mobile & Low-Bandwidth Clients

Loading ort.min.js includes support for WebGPU and WebGL, which causes the runtime to fetch large .wasm binaries (~30 MB for JSEP (JavaScript Execution Provider: the internal bridge ONNX Runtime Web uses to talk to WebGPU and WebGL) /WebGPU-enabled builds) even if you only intend to run on CPU.

  • Adaptation: If you only target CPU execution via executionProviders: ["wasm"], import the CPU-only build (ort.wasm.min.js) instead and set ort.env.wasm.numThreads = 1. This removes the GPU shader compilers and reduces the WebAssembly payload size by roughly half, lowering peak browser RAM usage on older mobile devices.

Btw, you can see the list of what you have available here: https://cdn.jsdelivr.net/npm/onnxruntime-web@1.31.1/dist/.

2. Using Gradient-Boosted Libraries (XGBoost, LightGBM, CatBoost)

If your model is trained with a gradient-boosted library rather than scikit-learn, skl2onnx will not convert the estimator directly.

  • Adaptation: Use onnxmltools (convert_xgboost, convert_lightgbm, or convert_catboost) during your Python export step. Note that onnxmltools requires feature names to follow a strict f0, f1, ..., fn naming convention before conversion (unlike skl2onnx, which accepts arbitrary strings). Once exported, the resulting .onnx graph uses the exact same TreeEnsembleClassifier operator, meaning your browser JavaScript code requires zero changes.

3. Embedding Preprocessing Inside the ONNX Graph

In our case study, custom regexes and flexible date parsing were ported to JavaScript because those string operations do not map cleanly to ONNX operators.

  • Adaptation: If your pipeline relies strictly on standard tabular transformations (such as SimpleImputer, StandardScaler, MinMaxScaler, or OneHotEncoder inside a scikit-learn ColumnTransformer), you can pass the entire Pipeline directly to skl2onnx. skl2onnx compiles those mathematical preprocessing steps directly into the .onnx computation graph, eliminating the need for a separate metadata.json file or manual vectorization logic in JavaScript.