irt.io

Rooms

Proximity voice

Give rooms voice where each listener hears the closest speakers it can see.

Rooms can have voice chat in which each listener hears the speakers near it. The room decides who hears whom; @irt/client/voice joins the voice server and plays what the room allows.

How it works

  • Voice is audio only and runs on a voice server (LiveKit) in each region, outside your VMs. It runs no tenant code.
  • A project turns voice on with its voice setting. Each connection then gets a voice token from the router after welcome, in a voice message (type 7) with the voice server’s URL. The voice identity is the player ID, stable across reconnects.
  • Who hears whom comes from the room. With voice: {} in defineRoom, a listener hears the players whose entities it can see under the room’s interest, ranked by the interest limit’s priority, capped at maxSpeakers. An entity belongs to the player in its owner string field, unless you pass owner. audible(room, listener) replaces the rule, for teams, radio channels or a whole room on one call.
  • Each tick the room works out every connection’s speakers (who it hears) and listeners (who hears it) and sends them, only when they change and at most 4 times a second, in a peers message (type 9).
  • The client subscribes only to its speakers, and sets its own microphone track’s permissions to its listeners, so a modified client can’t overhear anyone and can only expose its own voice. A subscribe that arrives before a speaker’s permission is retried when the permission comes.
  • By default each speaker plays through its own <audio> element, with volume falling linearly to silence at range. spatial: true pans speakers in 3D with Web Audio, played through one <audio> element, because Chrome’s echo cancellation ignores audio played straight from Web Audio.

Room

// room.ts
import { defineRoom, defineSchema, entity, grid, type RoomEntity } from "@irt/room";

export const Avatar = entity("avatar", { owner: "string", x: "f32", y: "f32" });
export const schema = defineSchema({ entities: [Avatar] });

type A = RoomEntity<typeof Avatar>;

export default defineRoom({
  schema,
  interest: {
    ...grid({ cellSize: 16, radius: 1, dims: "xy", around: (_room, c) => c.data.avatar }),
    limit: {
      max: 64,
      priority: (_room, c, e) => -Math.hypot((e as A).x - (c.data.avatar as A).x, (e as A).y - (c.data.avatar as A).y),
    },
  },
  voice: { maxSpeakers: 6 },
  onJoin(room, conn) {
    conn.data.avatar = [...room.entities.all(Avatar)].find((e) => e.owner === conn.player) ?? room.entities.create(Avatar, { owner: conn.player, x: 0, y: 0 });
  },
  onPlayerLeave(room, player) {
    for (const e of room.entities.all(Avatar)) if (e.owner === player) room.entities.delete(e);
  },
});

Voice options on defineRoom({ voice }):

Option Default Meaning
maxSpeakers 8, or 256 with audible Speakers each listener hears, at most 256. The project’s voice.maxSpeakers setting is a ceiling: a room gets the smaller of the two
owner(entity) its owner string field The player an entity speaks for
audible(room, listener) speakers it can see Player IDs the listener hears, replacing the default rule

Whole room on one call

Party games and calls want everyone to hear everyone, whatever they can see. Return every connection’s player from audible; the listener’s own ID is dropped for you:

// call-room.ts
import { defineRoom, defineSchema } from "@irt/room";

export const schema = defineSchema({});

export default defineRoom({
  schema,
  voice: { audible: (room) => [...room.connections.values()].map((c) => c.player) },
});

With audible the room’s own cap defaults to 256, and the project’s voice.maxSpeakers setting, 64 by default, caps it, so a call of up to 65 players needs no setting. On the client, leave out position so every speaker plays at full volume; see Client.

Muting a player

room.muteVoice(player) mutes a player in this room, for good or until { expiresAt } in wall-clock milliseconds; room.unmuteVoice(player) lifts it, and room.voiceMutes() lists them. The room leaves a muted player out of every listener’s speakers, whatever audible or the default rule says, so no client the room drives subscribes to it, and its own client sees no listeners. Mutes are kept in room state, like bans, so they last through hibernation and a room host restart.

calls: {
  mute: (room, conn, args: { player: string }) => {
    if (conn.player === room.creator) room.muteVoice(args.player);
  },
},

The voice server enforces a mute too: the player can’t publish while it lasts, and its voice tokens can’t either, so neither a modified client nor a reconnect gets around it. A player on the voice server is muted at once, and one who joins it with a token from before the mute is muted as it joins; should the voice server miss either, within a minute. When the mute ends, Voice turns the player’s microphone back on if it was on, unless the player muted it with setMuted. The server API’s voice/mute is a one-off mute of the player’s tracks, which its own client can undo.

Client

// client.ts
import { connect } from "@irt/client";
import { joinVoice } from "@irt/client/voice";
import { Avatar, schema } from "./room.js";

const room = await connect({ router: "wss://eu.irt.io", project: "my-game", room: "plaza", schema });

const voice = joinVoice(room, {
  position(player) {
    const a = [...room.entities.all(Avatar)].find((e) => e.owner === player);
    return a && { x: a.x, y: a.y };
  },
  range: 24,
});

document.querySelector("#talk")!.addEventListener("click", () => void voice.enableMic());
document.querySelector("#mute")!.addEventListener("click", () => void voice.setMuted(!voice.muted));
function joinVoice(room: RoomClient, opts?: VoiceOptions): Voice;

interface VoiceOptions {
  position?(player: string): { x: number; y: number; z?: number } | undefined;
  spatial?: boolean; // default false
  range?: number; // default 20
}

class Voice {
  readonly project: string;                      // the room connection's project
  readonly muted: boolean;                       // this player's mute, from setMuted
  readonly micOn: boolean;                       // enableMic called and not muted
  readonly participants: ReadonlySet<string>;    // other players on the voice server
  readonly speaking: ReadonlySet<string>;        // players speaking now, this one included
  readonly mutedPlayers: ReadonlySet<string>;    // other players on voice sending no audio
  on(event: "participants" | "speaking" | "mutedPlayers", fn: (players: ReadonlySet<string>) => void): () => void;
  on(event: "micError", fn: (error: Error) => void): () => void;
  enableMic(): Promise<void>;
  setMuted(muted: boolean): Promise<void>;
  disconnect(): Promise<void>;

  // Player controls; see below
  volume(player: string): number;                // 0 to 1, default 1
  setVolume(player: string, volume: number, keep?: boolean): void;
  readonly blocked: ReadonlySet<string>;
  setBlocked(player: string, blocked: boolean): void;
  devices(): Promise<{ inputs: VoiceDevice[]; outputs: VoiceDevice[] }>;  // VoiceDevice: { id, label }
  readonly input: string | undefined;            // chosen microphone; undefined for the default
  readonly output: string | undefined;           // chosen speaker; undefined for the default
  readonly canSetOutput: boolean;                // the browser has setSinkId
  setInput(deviceId: string): Promise<void>;
  setOutput(deviceId: string): Promise<void>;
  readonly level: number;                        // microphone input level, 0 to 1
  readonly pushToTalk: boolean;
  setPushToTalk(on: boolean): Promise<void>;
  talk(down: boolean): Promise<void>;
}
  • position places players for volume and panning, and a speaker it returns undefined for is silent. Leave it out and every speaker plays at full volume, unpanned, as on a call; spatial and range then do nothing.
  • participants is the set of other players’ IDs on the voice server, whether or not the room has this player hear them: everyone a voice chat list should show. A player in the room who hasn’t joined voice, or whose voice dropped, isn’t in it. The participants event fires with the new set each time a player joins or leaves voice.
  • speaking is the set of player IDs the voice server hears talking now, this player’s included, so it covers speakers this client doesn’t hear too. The speaking event fires with the new set each time it changes.
  • enableMic while the room mutes this player publishes nothing until the mute ends, then turns the microphone on. Called before voice has joined, it turns the microphone on when it does.
  • micError fires when the microphone fails to start, as when the player refuses permission, whether from enableMic or on joining. The microphone is then off again, so micOn is false and the next enableMic asks again; enableMic also rejects with the error. An error named NotAllowedError is a refusal.
  • mutedPlayers is the other players on the voice server sending no audio: muted with setMuted, or not yet past enableMic. The mutedPlayers event fires with the new set; voice.muted stays this player’s own mute. A player who isn’t on the voice server is in neither set.
  • The three sets empty when voice drops and fill again after it rejoins, so keep the same listeners across reconnects; there’s nothing to rebind. voice.livekit is replaced on each reconnect, so listeners on it would need binding again.

Player controls

These are the player’s own settings, which the project shows in its UI; the UI kit’s <irt-voice-settings> and <irt-voice-chat> are built on them.

  • setVolume(player, volume) sets how loud this player hears another, from 0 to 1, multiplied into the distance attenuation on both the <audio> and the spatial path. It can’t go above 1: an <audio> element’s volume is capped there, and gain past it would need Web Audio, which Chrome’s echo cancellation ignores. With keep false it isn’t saved, for a slider being dragged; save the value it stops at.
  • setBlocked(player, true) is a local mute: the client never subscribes to a blocked player, whatever the room picks, so a blocked player costs no bandwidth. Other players still hear them. To silence a player for everyone, the room calls muteVoice.
  • Volumes and blocks are kept in localStorage per project by player ID, so a player blocked in one room stays blocked in the next. Where the browser blocks storage they last for the page.
  • devices() lists microphones and speakers. Labels are empty until the page has microphone permission, so list them again after enableMic. setInput and setOutput switch devices, and a device chosen before voice joins is used when it does. Speakers can only be chosen where the browser has setSinkId, which canSetOutput reports; elsewhere devices() lists none and setOutput throws.
  • level is the microphone’s input level from 0 to 1, 0 while it isn’t sending. Read it on animation frames for a meter.
  • setPushToTalk(true) keeps the microphone muted except between talk(true) and talk(false). Voice doesn’t bind keys; the page does, and should ignore key presses in text fields:
// push-to-talk.ts
import { connect } from "@irt/client";
import { joinVoice } from "@irt/client/voice";

const room = await connect({ router: "wss://eu.irt.io", project: "my-party", room: "lobby" });
const voice = joinVoice(room);
await voice.setPushToTalk(true);

const typing = (e: KeyboardEvent) => e.target instanceof HTMLInputElement || e.target instanceof HTMLTextAreaElement;
addEventListener("keydown", (e) => {
  if (e.code === "KeyV" && !e.repeat && !typing(e)) void voice.talk(true);
});
addEventListener("keyup", (e) => {
  if (e.code === "KeyV") void voice.talk(false);
});
document.querySelector("#mic")!.addEventListener("click", () => void voice.enableMic());

Indicators

// indicators.ts
import { connect } from "@irt/client";
import { joinVoice } from "@irt/client/voice";

const room = await connect({ router: "wss://eu.irt.io", project: "my-party", room: "lobby" });
const voice = joinVoice(room);

function badge(player: string): string {
  if (voice.mutedPlayers.has(player)) return "muted";
  return voice.speaking.has(player) ? "speaking" : "";
}

function show(): void {
  const onVoice = [...room.presence].filter((p) => voice.participants.has(p.player));
  console.log(onVoice.map((p) => `${p.player} ${badge(p.player)}`));
}

for (const event of ["participants", "speaking", "mutedPlayers"] as const) voice.on(event, show);

The voice follows the room connection: it leaves the voice server when the connection drops, joins again with the new token after the reconnect, and ends when the connection closes for good. Without the project’s voice setting the router sends no token, and joinVoice does nothing. room.peers ({ speakers, listeners }) and the peers event are on every RoomClient, so a project can show who’s talking without @irt/client/voice, which is a separate entry so projects without voice don’t load livekit-client.

livekit-client is most of a voice game’s bundle, about 570 KB minified. Import @irt/client/voice dynamically, once the game has connected or when the player opts in, and the scaffold’s build-web.mjs, which bundles with esbuild’s splitting, puts it in a chunk of its own that loads only then:

// lazy-voice.ts
import { connect } from "@irt/client";

const room = await connect({ router: "wss://eu.irt.io", project: "my-party", room: "lobby" });

const { joinVoice } = await import("@irt/client/voice");
const voice = joinVoice(room);
document.querySelector("#talk")!.addEventListener("click", () => void voice.enableMic());

A build script of your own needs splitting: true, format: "esm" and outdir rather than outfile for this; without splitting, esbuild inlines the dynamic import into the main bundle. Keep enableMic in a click handler of its own, after the import has resolved, since browsers grant the microphone only from a user gesture.

Testing

Test clients expose the room’s decision, so proximity voice is testable without a voice server:

// voice.test.ts
import { testRoom } from "@irt/testing";
import { expect, it } from "vitest";
import room, { Avatar } from "./room.js";

it("hears nearby players", async () => {
  const t = testRoom(room);
  const alice = await t.connect({ player: "alice" });
  const bob = await t.connect({ player: "bob" });
  await t.host.tick();
  expect(alice.speakers).toEqual(["bob"]);
  expect(bob.listeners).toEqual(["alice"]);
  const far = [...t.live.entities.all(Avatar)].find((a) => a.owner === "bob")!;
  far.x = 500;
  await t.host.tick(10);
  expect(alice.speakers).toEqual([]);
});

Turning it on

Voice is a project setting:

Setting Default Meaning
voice.enabled false Connections get a voice token
voice.maxSpeakers 64 1 to 64. A ceiling on every room’s speakers per listener: a room gets the smaller of this and its own maxSpeakers, whose default of 8 is unchanged for proximity voice. A change applies to each room from its next wake

Declare them in irt.config.ts:

settings: { voice: { enabled: true } },

Locally, voice is off unless you run a LiveKit server and point irt dev at it; peers messages arrive either way, so the room’s rule can be checked without one:

docker run -d --rm --network host livekit/livekit-server:v1.9.1 --dev --bind 127.0.0.1
IRT_LIVEKIT_URL=ws://127.0.0.1:7880 npx @irt/cli dev

IRT_LIVEKIT_KEY and IRT_LIVEKIT_SECRET default to LiveKit’s dev mode devkey and secret. The token irt dev mints has the grants the router’s does: identity the player ID, LiveKit room {project}/{room}, publishing only the microphone, and subscribing.

Common mistakes

  • Leaving out voice in defineRoom. Without it the room sends no peers, and every client hears no one.
  • An interest that shows everyone everything. With no interest, every connection sees every entity, so a listener hears the first maxSpeakers players rather than the nearest.
  • A priority that ranks far entities higher. Priority is “higher is more important”; use the negative of the distance.
  • Entities without an owner. The default rule only counts entities with an owner string field naming a player; pass owner otherwise.
  • Calling enableMic outside a click. Browsers only grant the microphone from a user gesture.
  • Expecting voice to keep a room awake. Rooms hibernate 30 seconds after their last connection closes, whatever voice is doing.

View this page as Markdown