Skip to content

Environments

The repository includes a repeated matrix game, a Pursuit gridworld, and five native PettingZoo substrate families. Every objective wrapper supplies exactly one finite-trace temporal formula per agent.

Factory functions and constraint wrappers are documented in the Environments API.

Observation Modes

The native Coins, Chemistry, Harvest, Territory, and Gift Refinements factories have three useful policy/inspection modes:

  • render_mode=None returns compact structured observations without rasterizing frames.
  • render_mode=None, flatten_observations=True normalizes and concatenates the structured fields into a flat Box policy vector.
  • render_mode="rgb_array" or "human" returns normalized RGB Box observations. Fully observable world views and four-times spatial pooling are the defaults; local 22 x 22 x 3 views are optional.

Experiment helpers select the flat object mode, so training avoids RGB generation while satisfying the trainers' Box contract. Play and inspection notebooks use RGB. The temporal wrapper appends the current monitor-state id to either adapted observation.

Coins

Locations:

src/environments/substrates/coins/
src/environments/gridworlds/coins/

Coins is a procedurally generated, fixed two-player social dilemma. Each player has a colour. Collecting either coin rewards the collector, but collecting the partner's colour penalizes the partner.

The own-coin-first objective is:

(!player_i_mismatched_coin_collected) U player_i_matched_coin_collected

The active OwnCoinFirst_2_mo.py notebook uses a 256-step horizon, regrow_rate=0.005, 200,000 environment steps, local observations, a validated collector-local reward, gamma=1.0, and IPPO lr=1e-4. Its curated cluster ID is own_coin_first_2.

Stag Hunt

Locations:

src/environments/matrix/stag_hunt/
notebooks/matrix/stag_hunt/

Stag Hunt is a fixed two-player repeated matrix game. Actions are Hare and Stag, with default payoffs (3, 3) for both hare, (4, 4) for both stag, and (0, 3) for stag against hare.

The primary Sustained Stag objective is:

F (both_stag & X (both_stag & G !unmirrored))

Handshake and Both Modes alternatives live under the notebook alt/ folder.

Pursuit

Locations:

src/environments/gridworlds/pursuit/
notebooks/gridworlds/pursuit/

Pursuit contains asymmetric guard and intruder objectives over collisions, goal reachability, and gap movement. Active training notebooks are Classic_2_mo.py, Capture_2_mo.py, Classic_3_mo.py, and Capture_3_mo.py. Capture-3 defaults to its validated formula-aligned guard/intruder training reward; the other three default to objective reward.

Example atoms include goal_reached, collision_ever, guard_intruder_collision, and intruder_in_gap.

Chemistry

Locations:

src/environments/substrates/chemistry/
src/environments/gridworlds/chemistry/
notebooks/gridworlds/chemistry/

Supported variants are the two- and three-metabolic-cycle layouts, with optional distractors. Current notebooks use chemistry__two_metabolic_cycles with four players.

Main objectives:

  • Food: F player_i_food_metabolized
  • FoodThenXY: F (player_i_food_metabolized_ever & player_i_xy_metabolized)
  • Python-only middle constraint FoodThenSharedXY: F (player_i_food_metabolized_ever & xy_metabolized)

Food is the easier reachability smoke benchmark; FoodThenXY keeps a personal two-stage dependency.

Harvest

Locations:

src/environments/substrates/commons_harvest/
src/environments/gridworlds/harvest/
notebooks/gridworlds/harvest/

The factory supports open, closed, and partnership variants. The active four-player benchmark uses commons_harvest__partnership.

Main objectives:

  • Harvest: F player_i_apple_collected
  • SafeHarvest: (G !player_i_zapped) & (F player_i_apple_collected)

SafeHarvest_4_mo.py is active and defaults to its validated agent-local safe-apple reward with undiscounted, annealed IPPO. The simpler Harvest notebook is under alt/.

Territory

Locations:

src/environments/substrates/territory/
src/environments/gridworlds/territory/
notebooks/gridworlds/territory/

The factory supports rooms, open, and inside_out. The active benchmark is RoleClaim_2_mo.py, a first-sole-claim race:

player_0: (!player_1_resource_claimed) U
          (player_0_resource_claimed & !player_1_resource_claimed)
player_1: (!player_0_resource_claimed) U
          (player_1_resource_claimed & !player_0_resource_claimed)

The first sole claimant wins, while a simultaneous first claim is a tie and satisfies neither player. The mutually exclusive payoffs make this a contested rational-synthesis game. Claim_4 and the brittle global-safety PeacefulClaim_4 comparison are under alt/.

Gift Refinements

Locations:

src/environments/substrates/gift_refinements/
src/environments/gridworlds/gift_refinements/
notebooks/gridworlds/gift_refinements/

Gift Refinements is fixed at six players and has nine discrete actions.

Main objectives:

  • Consume: F player_i_tokens_consumed
  • RefinedThenConsume: F (player_i_refined_gift_received_ever & player_i_tokens_consumed)

Consume is an easy reachability smoke task. RefinedThenConsume requires a personal refined-gift event before consumption. Neither is in the curated all-environment launcher: random play already solves Consume, while the strongest Refined optimisation profile could not meet the p >= 0.5 gate. Both notebooks remain runnable by direct path. PR2-IQL scales poorly here because its joint-action critic has 9**6 outputs per agent.

Labeling Rule Of Thumb

Prefer agent-specific atoms when each learner should earn its own satisfaction:

player_i_apple_collected
player_i_tokens_consumed
player_i_resource_claimed

Use aggregate atoms only when a formula intentionally describes team state or a shared milestone. NashConv, equilibrium, and AlphaRank are never environment atoms; they are post-training properties of a learned policy profile.