Private distributed AI, directly in the browser

RallyCompute is a browser-based distributed AI runtime. It divides an open model into pieces (shards) and executes those shards across a group of computers working together directly in the browser using WebGPU.

Probably. But as the guy who stuffed, GPT-2 into a spreadsheet (Spreadsheets-are-all-you-need) and VanillaJS, I seem to have a deep attraction to impractically implemented inference. The distributed-systems background just means I can now do it across several machines at once.

RallyCompute can connect workers across the public internet when WebRTC can establish a direct connection. The current preview has no TURN relay, so some restrictive NAT, carrier-grade NAT, and firewall combinations may not connect. Even when it does connect the latency will be high going over the public internet. For the best speed and reliability, keep workers on the same low-latency LAN when possible. Broader network compatibility is on the roadmap.

Current model routes include SmolLM2 360M Instruct and Gemma 4 12B, both using Q4_K_M quantization. More models are coming—contact us if there is a specific model you want to run.

The coding workflow reads only the files you select. Proposed changes are staged in browser storage and require approval before writing. Selected context is processed by the trusted computers participating in your room.

RallyCompute's primary advantages are control, privacy, and using existing compute. Faster? No. Cheaper? Possibly.

Participating machines need a Chrome browser and WebGPU-capable hardware. The amount of machine will depend on your selected model and each machine's memory limits for WebGPU. RallyCompute will measure your machine's WebGPU limits.