Testing in CI

An irtio project has two kinds of test, and they cost different things to run. Room tests are plain unit tests: no server, no sockets, a whole suite in well under a second. Bot simulations open real client sessions against a running room, so they need a process to talk to and they cost real seconds and real CPU.

Both belong on every push. The difference is how you size them.

Running them locally

There is no irtio test command. Room tests are ordinary tests, so your test runner runs them.

npx vitest run                      # every room test
npx vitest run irtio/room.test.ts   # one file
npx vitest                          # watch mode while you work

The simulation needs a room to talk to, in a second terminal:

npx irtio dev
npx irtio simulate --bots 20 --seconds 10

A GitHub Actions workflow

Two jobs, because they need different things. The room tests need nothing but Node. The simulation needs a room process alive while it runs.

# .github/workflows/ci.yml
name: ci
on:
  push:
    branches: [main]
  pull_request:

jobs:
  room-tests:
    name: room tests
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: pnpm
      - run: pnpm install --frozen-lockfile
      - run: pnpm exec tsc --noEmit
      - run: pnpm exec vitest run

  simulate:
    name: 20 bots for 10 seconds
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: pnpm
      - run: pnpm install --frozen-lockfile

      - name: Start the room
        run: nohup pnpm exec irtio dev --no-watch > dev.log 2>&1 &

      - name: Wait for it to accept connections
        run: |
          for i in $(seq 1 30); do
            curl -sf http://localhost:7070/__irt/state.json > /dev/null && exit 0
            sleep 1
          done
          echo "irtio dev never came up:" >&2
          cat dev.log >&2
          exit 1

      # Exits 1 on any invariant violation, which is what makes this a gate.
      - run: pnpm exec irtio simulate --bots 20 --seconds 10

      - name: Keep the evidence when it fails
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: simulate-trace
          path: |
            .irtio/sim/
            dev.log

--no-watch stops the dev server watching your room file’s directory, which is wasted work on a runner that never edits anything. /__irt/state.json is the dev server’s live state inspector and answers as soon as it is listening, so it works as a readiness probe before any room exists.

Upload the trace on failure. A red invariant tells you which frame broke the rule, and the trace is the only thing that tells you what led up to it. See Invariants and load runs for how to read one.

Sizing the simulation

Twenty real client sessions is real load on a shared two-core runner. That is roughly the pressure you want to prove the room survives, but it means the report describes the runner as much as the room. Treat convergence and bytes per second from a CI run as a regression signal against previous CI runs, not as an absolute.

Two hard edges before you raise the bot count:

• A server refuses more than 120 connections per IP per minute by default. A run in the hundreds opens every socket from one address in a few seconds and gets turned away. Locally, a harness that starts the server itself raises it in process with startDev({ limits: { connectionsPerIpPerMin } }). Against a deployed project it is per-project configuration: see connectionsPerIpPerMin. Raise it for the run, or split the bots across several source addresses.
• A physics room burns more CPU per bot. Contact-heavy moments produce a correction opportunity every tick, so a busy physics room on a slow runner can trip the default correction-storm threshold (5 corrections/s per bot) with nothing actually wrong. Raise it with --corrections-max <perSec> on the CLI, or correctionsPerSecMax through spawnBots, for those rooms rather than getting used to a red run.
• A queued run needs a control plane. --queue puts the bots through the real match API, and the matchmaker lives on the control plane rather than on the game server, so irtio dev has no queue to join and the flag is refused without --control. In CI that means either pointing at a real control plane (a staging project, with its per-address match limit in mind: ten in a burst and thirty a minute, so twenty tickets from one runner will be throttled) or starting a test plane in process, which is what the platform’s own matchmaking tests do. An ordinary run needs no control plane at all, which is why a queued run is worth its own job rather than the default.

Determinism

Room tests run on a fake clock, so nothing races a real timer. The room’s own random source is seeded: room.random() replays from the seed you pass to testRoom, and so does the latency model. A test that passes once passes every time.

When one does behave differently between runs, the cause is almost always the room reaching around the runtime. Use room.now instead of Date.now() and room.random() instead of Math.random() and it becomes reproducible. The same rule applies harder to a shared physics world builder. irtio simulate builds the world twice and fails the run when the two builds differ, because a nondeterministic builder produces “mispredictions” that have nothing to do with netcode. On the engines that have a snapshot it compares those byte for byte; on the ones that do not it steps both builds a second and compares every body’s position and velocity.

Bot simulations run over real sockets on a real clock, so they are not deterministic in the same way. Their seeds make the bots’ behaviour replay identically, which is enough to compare two runs, but timing-sensitive thresholds still want headroom. That is why the misprediction and snap invariants report rather than fail by default.

What to cover

Spatial visibility is the one thing only a room test can check. A bot has no server truth to compare its view against, so a live run checks role visibility and stops there. If your room uses visibility: 'spatial-grid', the toHaveNoVisibilityLeaks assertion in a room test is the only coverage you have.

Gating a deploy

irtio deploy needs a credential, and CI cannot get one from irtio login, which needs a browser and a person. Put a credentials file in place and set IRT_CONTROL_URL, then make the deploy job depend on both test jobs so nothing ships past a red invariant.

  deploy:
    needs: [room-tests, simulate]
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    env:
      IRT_CONTROL_URL: https://irt.io
      IRT_CREDENTIALS_FILE: ${{ runner.temp }}/credentials.json
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: pnpm
      - run: pnpm install --frozen-lockfile
      - name: Restore the credential
        run: |
          umask 077
          printf '%s' "$IRT_CREDENTIALS" > "$IRT_CREDENTIALS_FILE"
        env:
          IRT_CREDENTIALS: ${{ secrets.IRT_CREDENTIALS }}
      - run: pnpm exec irtio deploy

Deploying a room covers the credential handling and what a deploy actually does.

Related

• Simulated players for the in-process harness
• Invariants and load runs for what the simulation checks