Skip to content

Native Gridworld Substrates

Coins, Chemistry, Commons Harvest, Territory, and Gift Refinements are local PettingZoo ParallelEnv implementations. The curated three live under src/environments/substrates/; archived Chemistry and Gift Refinements live under src/environments/archive/substrates/ but still use the shared _structured_observations.py there, which dispatches on class name and so needs no edits for the move. They were synchronized from nightly/meltingpot-pz at commit 7d7af52, but this repository has no runtime dependency on DeepMind Melting Pot, Shimmy, dm_env, or Lab2D.

_structured_observations.py (with _stamina_bar.py, which it imports) was re-synchronized with meltingpot-pz 270fb1a on 2026-09-26, after an audit of every substrate for structured observations that alias states RGB or upstream distinguish (meltingpot-pz docs/AI/structured_observation_audit.md). The local Coins colour fix below is now upstream, so the file is again an exact copy (reformatted). For the substrates here:

  • Commons Harvest, Territory, Gift Refinements: a BEAMS grid for the zap, claim and gift beams RGB draws -- who just acted, and so who is on cooldown; in Gift Refinements, who gave a gift.
  • Territory: CLAIMS, wet or dry; a wet claim cannot pay yet.
  • Chemistry: AVATAR_VESICLES, other agents' carried compounds, which take part in neighbouring reactions.
  • All, in global mode: a SELF grid marking the viewer's own avatar.

The substrate dynamics here were unchanged by the audit. Observation spaces gained keys, so flattened observation sizes changed: results from before the re-sync are not directly comparable with results after it.

For the curated suite this made safe_harvest_4 (global mode, 1729 -> 2593 flattened dims) and role_claim_2 (global mode, 2647 -> 3970) stale. The Coins suite runs in local mode (fully_observable=False), and a seeded random rollout gave bit-identical observations before and after the re-sync, so it was not rerun. Both stale suites were rerun on 2026-09-26 with the unchanged campaign arguments (env.sh <env> amd96 bundles=15 feasible=True walltime=96:00:00 num_runs=5 calculate_satisfaction=True, Slurm jobs 8731-8746). The superseded algorithm directories were moved to /vol/gpudata/oja24-pltlf/stale-before-09458fb/, outside exports/, so suite discovery would not find them. They were deleted on 2026-10-10 (see paper data archive).

They had to be moved rather than resumed in place: the partial-resume fingerprint hashes the resolved config, not the observation space, so a resumed run would have reused the stale checkpoints. Any later change to observations needs the same treatment.

Egocentric full window (opt-in, 2026-10-02)

The Coins, Commons Harvest and Territory cores and the structured module were re-synchronized with meltingpot-pz e360624 (and the structured module with e0ff9bf, whose environment copies share the focal index cache), which adds a "focal" observation mode (with the shared window helper as _views.py): an agent-centred window of side 2 * max(rows, cols) - 1, covering the whole map from any position and orientation, so nearby objects stay on fixed inputs as in the local window while nothing is hidden. Structured focal grids are gathered from one world-frame pass per viewer.

It is off by default here. make_coins_env, make_harvest_env and make_territory_env take egocentric=False; egocentric=True with fully_observable=True selects focal (native_observation_mode in gridworlds/native.py), and in RGB mode returns the agent's own focal frame instead of the world-frame state(). It has no effect on local views. A seeded random rollout of every factory in local and global mode (including archived Chemistry and Gift Refinements) gave bit-identical observations and rewards before and after this sync, so no existing result is stale.

The experiment configs deliberately do not expose it yet: a new shared config key enters both the resolved config and the spec's extra_config_keys, and the partial-resume fingerprint hashes both, so every in-flight run would stop resuming even with the option off. To use it, add "egocentric" to the domain's shared keys and extra_config_keys, pass egocentric=bool(config["egocentric"]) in its env_factory, and treat the runs as a new campaign. Archived Chemistry and Gift Refinements cores were not updated and support only local and global.

Layering

  • src/environments/substrates/ owns dynamics, structured observations, events, rendering, stochastic termination, and variant configuration.
  • src/environments/gridworlds/<domain>/env.py provides the stable factory and optional flat-object or RGB observation adapters.
  • src/environments/gridworlds/<domain>/mix_in.py turns native rewards, events, infos, and lifecycle state into temporal atoms.
  • src/environments/gridworlds/<domain>/constraints.py attaches one SpecPair per agent through BoolRewardWrapper.

Archived domains use the same layering under src/environments/archive/. When re-syncing from meltingpot-pz, copy upstream chemistry/ and gift_refinements/ into archive/substrates/ and rewrite their absolute imports from src.environments.substrates.<domain> to src.environments.archive.substrates.<domain>.

The domain factories accept the historical upstream substrate names. Coins exposes a procedurally generated fixed two-player game, Chemistry supports four metabolic-cycle variants, Harvest supports open, closed, and partnership, Territory supports open, rooms, and inside_out, and Gift Refinements exposes its fixed six-player variant.

Rendering And Observation Contract

  • render_mode=None returns compact structured gymnasium.spaces.Dict observations. Reset, step, and render() do not rasterize an RGB frame.
  • flatten_observations=True in that mode installs NativeObjectObservationWrapper, which normalizes and flattens structured fields into the Box vector required by the trainers.
  • render_mode="rgb_array" and render_mode="human" wrap the substrate with NativeRGBObservationWrapper and return normalized RGB Box observations.
  • state() remains an explicit on-demand global uint8 RGB frame on the base substrate.

Experiment adapters request render_mode=None, flatten_observations=True so training avoids rasterization while still producing the Box values consumed by the neural trainers and temporal wrapper. Inspection callers can leave flattening disabled and interpret categorical values with the base environment's read-only object_observation_ids mapping. Play notebooks use the separate RGB path.

See Native Gridworld Observations for shape, preprocessing, and side-channel details.

Events And Lifecycle

Native step infos carry event dictionaries directly. events_from_infos(...) deduplicates copies shared across per-agent infos before label mixins inspect them. native_lifecycle(...) walks the wrapper chain and distinguishes the configured step-limit timeout from a native stochastic termination.

Both policy adapters preserve relevant scalar side channels in namespaced info keys and derive collective reward as the current sum of per-agent rewards. Label mixins must use these native contracts; do not restore reactive subscriptions or compatibility-wrapper unwrapping from the removed integration.