Multiplayer architecture and operations

Structure custody uses a root version-one migration across building anchors/footprints, frozen district sites, selected structure work and construction-cart receipts. Restore rejects mismatched geometry, changed addressed work sites and overlapping non-cleared buildings. A wholly unmarked older family gains location metadata without advancing construction or spending again; the normal pre-client migration commit makes it durable. District scheduling also rechecks its saved footprint before placing a paid order. See technical schema and boundaries and the primary living world design.

The one-world scale assessment evaluates the planned thousands-player, hundreds-faction workload and recommends the later PostgreSQL/regional-worker architecture. The mechanisms below describe the current implementation; movement workers and historical load results do not establish distributed living-economy capacity. The single-process/file-storage stage remains a prototype foundation rather than a scale certification.

Optimization coverage

Protocol version 3

Finite mineral deposits retain numeric runtime keys plus versioned axial addresses. Startup validates the complete generated and established frontier deposit ledger; missing or forged seams cannot be filled in from defaults. A complete older ledger with wholly absent address metadata gains coordinates without changing its remaining mineral, revisions or provenance. Road, quarry, livestock and watershed journeys now also retain checked axial routes in live jobs and returned receipts, with exact site custody and completed road edges where relevant. Household migration version nine adds the frontier source, active/retained family routes and death/relief points, including their community-mortality and caravan copies. A wholly unmarked older household family upgrades without moving people or changing its economy; partial or conflicting addresses reject. Both recall paths reverse numeric and axial routes together. After all constructor validation, the gateway compares the restored snapshot with the loaded snapshot and durably commits every additive migration before creating the WebSocket server, including when the world already has its persistent instance ID. Event-only restart before the next periodic snapshot retains these upgrades and prior economic custody. These are additive snapshot schemas; movement protocol and geographic compatibility rules are unchanged. See the technical reference for per-family markers and the restricted pre-deposit version-one boundary.

Control messages are small JSON objects. The first client message must be:

{"type":"hello","protocol":3,"token":"optional reconnect token","name":"Traveler"}

The server replies with welcome, including a random reconnect token, numeric entity ID, authoritative position, world seed, generator version, rates, chunk size, and persisted mutations. Other controls are presence, mutation, profile, ping, and pong.

welcome.world.instanceId identifies this persisted hosted world independently of its terrain seed. world.id and world.generatorNamespaceId retain the earlier deterministic seed namespace for compatibility. The health response exposes the same fields. Fresh instances use a random 128-bit ID, stored with worldInstanceIdentityVersion: 1 in both the manifest and living snapshot. Startup adopts an existing matching pair or migrates a compatible save only when both fields are absent in both records; partial, malformed or conflicting identities reject. New and migrated identities are committed before clients are accepted, with an event timestamp after the loaded snapshot boundary so immediate replay keeps the ID. A copied identified save remains the same logical world; the explicit offline fork command below assigns a new instance with ancestry; protocol 3 provides command/session binding. This additive protocol data does not replace the seed, generator and geography-checksum compatibility checks.

Every later JSON request includes session: {version:1, worldInstanceId, actorId, epoch} copied from welcome. The epoch is fresh on every hello. The gateway checks the active token/player/socket at receipt and again when queued commands or private reads execute. Resuming elsewhere revokes queued old-session work. The replacement welcome waits behind prior commits or rollback before publishing its initial private overview; already executing work finishes its atomic commit and remains recoverable through actor-private status queries in the same world. JSON replies echo the current binding. WORLD_INSTANCE_MISMATCH and STALE_SESSION reject foreign or obsolete envelopes before domain execution. Movement snapshots and their presence/entity baselines also wait for welcome. No epoch is saved in receipts. Protocol-2 clients must reload/update.

Movement input is {"type":"input","seq":42,"mask":2,"session":{"version":1,"worldInstanceId":"<welcome instance>","actorId":1,"epoch":"<welcome epoch>"}}. Mask bits are north 1, east 2, south 4, and west 8. Inputs are intents, never trusted coordinates.

Snapshot frames are little-endian binary:

Unchanged entities are omitted. Every two seconds, a full interest-set repair snapshot is sent. Leaving-interest removals are explicit presence controls.

Ownership and transfer

Each chunk hashes to exactly one worker index. The gateway routes inputs to the current owner. When a simulated position enters a chunk owned by another worker, the gateway removes the entity from the old worker and adds its latest authoritative state to the new worker. Only the owning worker's state batch is accepted.

The gateway maintains a read-only mirror for interest routing and network encoding. That mirror cannot author gameplay outcomes.

Persistence and recovery

Generator 6 expands the world to 2,048 × 1,100 cells (2,252,800 total), ten times generation 5. It retains five large continents, three smaller continents, varied islands, and approximately 71% water by spherical surface area. The default seed is GONZ-CONTINENTS-01; checksum is b407e9b3. Save files below are relative to <DATA_DIRECTORY>/world-v6/ (default .data/world-v6/). Generation-5 files remain intact in world-v5/, and older files remain in their former directories. Changing width changes numeric cell IDs; earlier saves are not restored onto incompatible IDs or geography. The expanded map starts with fresh player and settlement state; no previous snapshot or event log is deleted. Checksum and starting-region searches stream the world with bounded memoization, and radius queries scan only their local bounds. The existing 32-bit position encoding covers the expanded coordinates. See current expanded world.

Active movement never blocks on storage. Player upserts and world mutations enter an in-memory event batch. Every two seconds, player events append to one of sixteen player partitions and mutations append to their 16×16 chunk partition under events/. These background writes remain best-effort between checkpoints. Living-world transactions append separately to events/world.ndjson in the shared write queue; only a successful append publishes the candidate manifest/state and its coupled player to checkpoint memory. Unrelated player/chunk write errors cannot reject a successful world transaction. The gateway acknowledges a world order after this append and rolls its proposed simulation state back when the append rejects.

world-snapshot.json is replaced through a temporary file and rename every thirty seconds. Payload capture occurs inside the write queue after preceding commits succeed or fail. Each event receives a monotonic timestamp greater than both the previous event and checkpoint watermark. A snapshot reserves its watermark before awaiting disk writes; later updates remain replayable even if the wall clock is unchanged or moves backward. The checkpoint watermark advances only after a successful rename. Active positions checkpoint every ten seconds and all connected players are saved during graceful shutdown. A failed final snapshot rejects shutdown and exits with status 1.

readPersistedWorld materializes the snapshot and streams journals without creating directories, timers or source files. Startup validates the wrapper, event types, timestamp boundaries, matching manifest/state instances and one-to-one token/player identities before publishing loaded state. The gateway prepares the owned store without creating world files, validates the full domain state, activates directories and durably commits any migration before completing setup and activating background saves. If restoration or migration fails, startup abort drains the write queue without a checkpoint; a still-running embedding process cannot later save the rejected state. Missing storage is an empty world; malformed existing storage is an error. The legacy root journal precedes sorted partition files. Newer-than-snapshot world records retain physical append order, including older records whose wall clock moved backward. Player/mutation records retain order within a file; across partitions their event timestamps choose the newer value, with explicit player updates preferred over an embedded world profile at an equal timestamp. Partitioned metadata supersedes the earlier legacy root format even across clock rollback. All nonempty journal lines must parse, including a damaged trailing line; automatic truncation or repair is unavailable. Existing legacy events at or before their snapshot watermark retain the historical skip rule.

Each writable store first claims a canonical local world directory through exclusive creation of the sibling .<world-name>.writer-lock-v1 file. The claim records a version, random nonce, PID, hostname, purpose and acquisition time. Canonical paths govern both the claim and every persistence read/write; existing-world and missing-parent junction aliases converge. Write operations verify ownership; normal release checks the retained file identity and nonce. A second cooperating server is refused. Preparation and activation are separate so domain validation can fail without changing source files. Setup abort drains without checkpointing; close rejects new writes, waits for every partition append and the final checkpoint, and releases even if that checkpoint fails. Final player state is captured before socket teardown, and delayed close handlers cannot write after release. Gateway closure waits for WebSockets and terminates stragglers after one second; server shutdown waits for HTTP closure with the same bounded connection fallback. Signals during startup wait for initialization, and a failed bind cleans up the world.

A process crash leaves its claim behind. Every existing claim, including empty or malformed metadata, refuses startup; the server never steals one by age or PID. For operator recovery, verify that the recorded process has exited and that no server is using the world directory, then remove only the exact sibling claim named in the startup error and restart. Never remove a live owner's claim. Save files and journals remain in place. This local protocol covers normal paths and junction aliases, not every mapped-drive/UNC spelling, a network filesystem or distributed lease fencing.

These checks establish local ownership, queued-write consistency and fail-closed loading. The offline fork workflow below uses these foundations; a damaged-journal repair workflow remains planned. Append completion and atomic rename are not an fsync or power-loss durability guarantee.

Reconnect tokens are 192-bit random values. Only their SHA-256 hashes are persisted. A reconnect replaces an older socket using that identity and restores entity ID, name, color, and last durable position.

For multi-host deployment, implement the same persistence methods over a transactional database or append-only stream, partition mutations by world/chunk, and replace worker-thread messages with a regional broker. Zone ownership must remain exclusive; connection gateways may then remain stateless apart from their live sockets and delta baselines.

Offline world forks

Stop the source server and wait for clean shutdown, then pass the exact existing world directory and a new destination world directory:

npm run world:fork -- --source ".data/world-v6" --destination ".data-fork/world-v6"

Both arguments are required. The command accepts no overwrite/force option. It refuses any existing destination, including an empty directory, file or link, any source/destination overlap and any active or stale writer claim. It holds source and destination claims throughout validation and publication. To host the example child, set DATA_DIRECTORY to .data-fork (the parent of world-v6) using the existing server configuration; if hosting both instances, use different ports. A normal file copy retains the parent's identity, whereas this command creates a new world instance. This is an offline operation with no in-game control.

The source must already have matching instance identity and current valid domain state. An existing snapshot must use the complete supported version-one envelope with no unknown top-level fields. A complete current event-only world also works: replay is materialized into a child checkpoint without checkpointing the source. Unidentified or migration-requiring saves are refused; migrate a compatible save by starting and cleanly stopping the current server before retrying. Domain equality is checked at the saved clock; forking performs no catch-up, grants or rebalance. The command returns JSON containing both canonical paths, both instance IDs and the new provenance.

The new canonical checkpoint preserves every gameplay field, reconnect token hash, player position/profile, map mutation and retained command receipt. It preserves paid work and cargo in transit. Raw parent event logs are not copied. The only domain changes are the new instance pair and matching versioned ancestry in manifest and state: root instance, immediate parent, generation, source revision, semantic SHA-256 digest and fork time. Descendant forks retain the original root. The digest covers canonical materialized state, not file timestamps, and is evidence rather than a cryptographic authenticity signature. Native worlds remain unmarked. Startup and commits reject malformed, changed or missing ancestry and a source revision ahead of the child. Existing charter/generator namespaces remain unchanged.

The saved reconnect identity works in the child; protocol 3 nevertheless rejects requests carrying the parent instance. City clears private state and pending recovery when switching worlds, and must obtain fresh reviews. The child can answer its own actor-private queries for copied retained receipts. The worlds evolve independently with no merge or synchronization, and no repeated founding grant.

A validated sibling staging directory holds the complete checkpoint. Before creating the destination, the command publishes a complete sibling .<world-name>.fork-pending-v1 marker, then exclusively creates the destination and uses a no-replace hard link for its snapshot. It verifies the installed digest and both claims before removing the marker. A failure cleans up only its verified own files and empty directories. If cleanup cannot finish, stage and marker remain and every ordinary startup refuses that destination, even if its writer claim was removed. Local hard-link support is required; an unsupported filesystem fails safely.

If interrupted, first verify that the publisher process has exited. Remove only the exact verified-dead writer claims under the ownership rules above so the unchanged source can be opened again. Keep the incomplete target, sibling stage and pending marker for inspection, and retry into another new destination. Do not remove the pending marker to start a partial child: there is no automatic repair or resume command. A crash before marker publication can leave an orphan stage but has not exposed the destination. A failure reported after successful publication explicitly names the published destination; inspect that result before retrying. Append/link completion does not guarantee power-loss durability or distributed fencing.

Fifty-eight focused tests pass across fork/provenance/CLI/crash handling and the storage, migration, identity and real-gateway prerequisites. The command-earned case preserves a physically travelling supply team and receipt through two generations, rejects parent-session frames, restores the copied account and leaves all source tree bytes and modification times unchanged. An actual forced process exit before snapshot publication verifies that an incomplete destination remains blocked after dead-claim recovery. These checks do not certify arbitrary network or mapped-drive alias behavior or the full gameplay suite.

Capacity and failure policy

• Incoming WebSocket frames are capped at 16 KiB; client binary messages are rejected.
• Inputs are token-window limited per connection and connections are limited per source IP.
• Each client receives at most MAX_VISIBLE_ENTITIES, with the local and closest chunks first, so concentrated crowds have a fixed bandwidth ceiling.
• A socket above 256 KB buffered output skips snapshots until it drains.
• Distant subscribed chunks update at half the nearby rate.
• Simulation workers report current and maximum tick duration to /health and /metrics.
• Static assets stream when large; small compressible resources are memory-cached and served with validators.

Alert on sustained tick duration above half the tick interval, increasing backpressure drops, rejected connections, or snapshot bytes per player outside the tested budget.