Invariants and load runs

irtio simulate points a crowd of real clients at a room that is already running, plays it for a few seconds, and prints a pass or fail line for each built-in invariant. Every bot is a real @irtio/client session over a real socket, with the same write batching, prediction, and reconnect logic a browser gets. The invariants are checked against the bytes on the wire rather than against the SDK’s own view of them, so a bug this finds is a bug a player would hit.

This is the check that answers questions in-process tests cannot: does the room stay inside its bandwidth budget with twenty people in it, does a delta ever carry something to a client that should not see it, does the correction rate stay sane under real timing. It exits 1 on any violation, which makes it usable as a gate.

Running it

Start the room, then point the simulator at it.

npx irtio dev              # in one terminal
npx irtio simulate         # in another

The defaults are 5 bots for 10 seconds against ws://localhost:7070.

Bot 0 joins first and creates the room when no --room was given, and the rest join the code it came back with. Without that, twenty bots would create twenty empty rooms and every invariant would pass while nothing was tested.

What it needs from your project

The command loads irtio/schema.ts (or irtio/schema.js, or schema.ts) and bundles it the way irtio dev does. That module has to export a schema as schema or as its default export, and the error message says so when it does not.

Everything the default bots do is derived from that schema: which collections a client may own, what each field’s declared type will accept, which server RPCs take no return value. That is what lets the command work on a room it has never seen.

Two things follow from this that will surprise you if you do not expect them:

• No schema module means a relay run. The command falls back to a schema-less relay simulation, where bots broadcast small payloads on a timer, and it says so in the report. That is the honest thing to do for a relay room, and it is a clear signal that you ran it from the wrong directory.
• A physics room needs irtio/world.ts. With a shared world-builder module exporting gravity, and optionally setup, bodies, and intents, the bots predict physics locally exactly as a browser client does, and body-field corrections are judged as real mispredictions in world units. Without it the bots interpolate instead, those corrections count as body sync, and the report tells you the numbers mean less than they look like they do. The command also builds the world twice and compares the engine’s own snapshots byte for byte. A builder that reads Math.random(), a clock, or module state fails the run there, before its nondeterminism can be misread as bad netcode.

The invariants

misprediction and snaps have no cap by default, so they report their measured peak and mean without ever failing. They only become gates when you set a threshold yourself: pass --misprediction-max <units> or --snaps-max <n> on the CLI, or the equivalent mispredictionMagnitudeMax / snapsMax through spawnBots. correction-storm is the one invariant of the three that is not opt-in — it fails on its own default of 5 corrections/s per bot, and --corrections-max (or correctionsPerSecMax through spawnBots) raises it rather than waives it.

handler-error is the one invariant with a budget rather than a hard zero, because a room that refuses an illegal RPC is working correctly. Raise handlerErrorsMax for a run that expects some rejections.

Reading the report

The command prints the bot count, duration, endpoint, and room code, then one line per invariant, then a summary.

  ok   schema-validity   every snapshot and delta decoded
  ok   visibility-leak   no collection reached a role that cannot see it
  ok   bandwidth         peak 9120 B/s in per bot, budget 128000 B/s
  ok   handler-error     0 error reply/frame(s), tolerated 0
  ok   correction-storm  184 correction(s), peak 3/s per bot, threshold 5/s
  ok   misprediction     peak 2.4 units, mean 0.7 over 184 correction(s)
  ok   snaps             no snaps (every correction replayed within the resim window)
  ok   disconnects       every bot stayed connected

  612 frames/s · in 9.1 kB/s/bot · out 1.2 kB/s/bot · 184 corrections · misprediction mean 0.7 max 2.4 · 0 snap(s) · convergence 34 ms (p50 of 412)
  trace: .irtio/sim/trace-2026-08-27T09-14-02-113Z.json

  every invariant held across 20 bots.

The numbers worth watching:

• Convergence is the headline. It is the median milliseconds from one bot writing a value to another bot’s view showing it, sampled from real writes crossing the seam. When it says “convergence not sampled”, nothing converged during the run, which usually means the bots never owned anything to write.
• Bytes in per bot is what one player’s connection costs. Compare it to the budget you intend to hold, not to the 128,000 B/s default, which is a ceiling rather than a target. If your room filters by area of interest, a budget near the default would pass even if filtering silently regressed to sending everything.
• Corrections counts the server overruling a client. A steady trickle is normal in a room with a strict validate. A storm means either your validator is too tight for what your client predicts, or the client is predicting something the server computes differently.
• Snaps counts corrections that arrived too late to replay and had to jump the client’s state instead. Those are the visible ones. Zero is the number you want.

A failing run names each broken invariant and its first few offenders in the same line, and the process exits 1.

Making the bots cheat

--cheat inverts what the default bots do. Instead of small steps around a sensible value, they write the far end of an integer range and five-order-of-magnitude jumps on a float: type-valid, play-illegal, exactly what a modified client sends. Every one of those should draw a CORRECT from your room.

npx irtio simulate --bots 10 --cheat

If a cheat run draws no corrections at all, that is not a clean bill of health, and the command says so. It means the room accepted every out-of-range and teleporting write the bots threw at it, which is what the permissive validate from irtio init does until you write rules into it. See Server authority for how to write those rules.

Writing your own bots

The default script wanders. It cannot know that your world is 800 units wide or that a player has to cross a boundary for the thing you want to measure to happen. When you need a specific behaviour, write a script and drive it with spawnBots from @irtio/bots, which is what the command uses underneath.

A script is one async function per bot. It gets a seeded random generator, a wait that returns early when the run ends, and the bot’s real client room.

// irtio/bot.ts
import type { BotScript } from '@irtio/bots';

import type { schema } from './schema.js';

const WIDTH = 800;
const HEIGHT = 500;
const SPEED = 12;

/**
 * Sweeps each bot across the world on its own fixed heading, bouncing off the edges. Every bot
 * covers ground on every run, which a random walk cannot promise.
 */
export const sweep: BotScript<typeof schema> = async (bot) => {
  if (bot.role !== 'player') return;

  // The golden angle, so no two bots trace the same path whatever the bot count.
  const heading = bot.index * 2.399963229728653 + 0.31;
  let dx = Math.cos(heading) * SPEED;
  let dy = Math.sin(heading) * SPEED;

  while (!bot.stopped) {
    const me = bot.room.state.players.get(bot.id);
    if (me) {
      if (me.x + dx < 0 || me.x + dx > WIDTH) dx = -dx;
      if (me.y + dy < 0 || me.y + dy > HEIGHT) dy = -dy;
      me.x += dx;
      me.y += dy;
      bot.room.flush();
    }
    await bot.wait(50);
  }
};

Then a test that spawns them, waits for the run to finish, and asserts on the report. This one needs a room to talk to, so it skips itself when there is nothing running. The CI page shows how to start one first.

// irtio/simulate.test.ts
import { spawnBots } from '@irtio/bots';
import { describe, expect, it } from 'vitest';

import { sweep } from './bot.js';
import { schema } from './schema.js';

const url = process.env.IRTIO_URL ?? '';

describe.skipIf(url === '')('20 bots on one room', () => {
  it('holds every invariant inside a tighter bandwidth budget', async () => {
    const runner = await spawnBots(20, {
      url,
      schema,
      role: 'player',
      name: (i) => `bot-${i}`,
      script: sweep,
      durationMs: 6_000,
      // A quarter of the global default. This room filters by area of interest, so the
      // default ceiling would pass even if filtering stopped working.
      budgetBytesPerSec: 32_000,
    });

    // `done()` waits for the scripts to run their full duration. `stop()` on its own would
    // end the run immediately and report on whatever had happened so far.
    await runner.done();
    const report = await runner.stop();

    for (const invariant of report.invariants) {
      expect(invariant.ok, `${invariant.name}: ${invariant.detail}`).toBe(true);
    }
    expect(report.convergenceLagMs).toBeLessThan(200);

    await runner.trace.save('.irtio/sim/trace.json');
  }, 60_000);
});

spawnBots takes the same three threshold options the CLI’s flags set — mispredictionMagnitudeMax, snapsMax, and correctionsPerSecMax — plus two the CLI does not expose: budgetBytesPerSec and handlerErrorsMax. Setting the first two of those five is how you turn misprediction and snaps from reports into gates, the same as --misprediction-max and --snaps-max do from the command line. It also takes seed (bot i uses seed + i, so a run replays exactly), role and name as a value or a function of the bot index, flushMs, and rpc implementations shared by every bot.

runner.report() is safe to call mid-run, which is how you assert on a snapshot of the numbers without stopping the bots. runner.scriptErrors holds anything a script threw, and runner.done() rejects with the first one.

Scripted or random

Reach for the random script, which is what irtio simulate runs by default, when you want broad coverage of a room you are still shaping, or when you want the cheat mode. It touches every ownable collection and every void RPC without you naming them, and it keeps working when you add a field.

Reach for a scripted bot when the property you are measuring depends on where the bots go. Bandwidth under area-of-interest filtering is the clearest case: a random walk might never cross a cell boundary, so the run would report a number that says nothing. Anything with a spawn point, a race, or a specific sequence of RPCs wants a script too.

One detail of the random script is worth knowing. Numeric writes are a small mean-reverting walk around the value the bot first saw, not a uniform draw over the type’s range. A field’s type says what is representable, never what is legal: f32 knows nothing about your 1000-unit world or your 50-unit speed limit. A bot that teleported every tick would be corrected every tick and the correction-storm invariant would tell you nothing. On a physics collection the honest script writes intent fields only, in [-1, 1], and leaves body fields to the simulation. Cheat mode inverts both of those on purpose.

Reading a trace

Every run writes a trace, and --trace <path> chooses where. It is JSON: one entry per frame per bot, in time order, headers only.

{
  "startedAt": 1756284842113,
  "bots": 20,
  "dropped": 0,
  "entries": [
    { "at": 12, "bot": 0, "dir": "out", "type": 1, "frame": "HELLO", "bytes": 48 },
    { "at": 31, "bot": 0, "dir": "in", "type": 2, "frame": "WELCOME", "bytes": 412,
      "note": "joined 7KQ2 as c1/player" },
    { "at": 1084, "bot": 3, "dir": "in", "type": 8, "frame": "CORRECT", "bytes": 37,
      "note": "corrected players (clientTick 208)" }
  ]
}

at is milliseconds since the run started, and startedAt turns that back into wall time. dir is out for client to server and in for server to client. note is set when a frame is worth a word: a decode failure, an error code, an RPC name, a correction’s client tick.

When an invariant goes red, find the first offending frame in the report line, then find its timestamp in the trace and read the frames around it. The interesting part is almost never the frame itself, it is what the bot sent in the fifty milliseconds before it.

One caveat: the trace is a fixed-size ring of 4096 frames per bot. A long or busy run overwrites its own beginning, and dropped counts how many entries went. A non-zero dropped means you are reading a tail, not the whole run, so shorten --seconds or reduce --bots when you need the start.

Limits worth knowing

A cold, freshly-deployed project fails its first run on E_STARTING. --url and --key point the run at a deployed project instead of irtio dev. A tenant that was asleep answers the first joins with E_STARTING, a non-fatal error frame, and handler-error tolerates zero of those:

FAIL handler-error  1 error reply/frame(s), tolerated 0; bot 0: E_STARTING: your game server is starting

That is the invariant working as written, not a bug in your room — it cannot tell a waking tenant from a room answering badly, and there is no flag to exempt it. Warm the tenant first, by connecting once from a browser or by running a short throwaway simulate, so the run you measure starts against a project that is already up. See Reference: errors for E_STARTING.

The per-IP connection cap is 120 new connections per minute, and it cannot be raised for a deployed project. A run of more bots than that from one address is refused with E_RATE_LIMITED. Run against local irtio dev for a bigger count, or split the bots across several source addresses.

Live runs check role visibility, not spatial visibility. A bot connects over a socket and has no server truth to compare against, so visibility-leak here judges whether a frame named a collection outside the bot’s role view. Rooms using visibility: 'spatial-grid' need toHaveNoVisibilityLeaks in a testRoom test, which does have real authority to compare against and does catch spatial leaks. Keep that assertion even when a bot run covers the same room.

Hundreds of bots on one machine measure the machine. A run of twenty real client sessions is real load on a shared CI box. Runs in the hundreds are worth doing, but treat them as a local or staging exercise and read the numbers as a profile rather than a pass or fail.

A physics room is judged more loosely by default. Contact-heavy moments produce one correction opportunity per tick, so a busy physics room on slow hardware can burst past the cursor-tuned correction threshold without anything being wrong. Raise correctionsPerSecMax for those rooms rather than accepting a red run as noise.

Next steps

• Testing in CI for wiring both kinds of test into a push
• Simulated players for the in-process harness
• Reference: CLI for every command and flag