Skip to content

Territory Experiments

Territory uses the native PettingZoo implementation under src/environments/substrates/territory/. The stable factory is make_territory_env(...).

Construction And Labels

The factory accepts territory__rooms, territory__open, and territory__inside_out. Experiment adapters flatten structured categorical fields without rendering; visual mode uses the shared native RGB adapter. territory.ready_to_shoot and territory.collective_reward remain in infos for label computation.

TerritoryLabelMixin exposes:

  • lifecycle atoms: timeout, terminal, stochastic_termination;
  • aggregate and per-agent resource-claim atoms with sticky forms;
  • collective claim atoms derived from the collective-reward side channel;
  • all_zappers_ready and any_zapper_cooling_down.

A per-agent claim is the substrate's explicit claimed_resource event. It is not inferred from the delayed stochastic native payout: positive reward can arrive many frames after ownership changes and is therefore not a sound claim timestamp. Merely firing the claim beam without changing ownership does not count. Zappers count as ready only when every preserved readiness value is at least 0.999.

Objectives

  • TerritoryClaimConstraint: F player_i_resource_claimed
  • TerritoryPeacefulClaimConstraint: (G all_zappers_ready) & (F player_i_resource_claimed)
  • TerritoryRoleClaimConstraint, restricted to player_0 and player_1: each player must make the first claim on a step where the opponent does not also claim. A simultaneous first-claim step is a tie and satisfies neither objective; claims after the first claim step cannot change the winner. The two satisfaction events are mutually exclusive.

Claim_4 is an easy label/environment check. PeacefulClaim_4 is archived because one zap irreversibly fails every terminal-only monitor and often creates all-zero learning data. RoleClaim_2 is the active coupled benchmark.

RoleClaim Training Choice

RoleClaim_2_mo.py uses a 256-step horizon and gamma=1.0 across IPPO, IQL, and Nash-Q. The shorter horizon gives each 256-step rollout a complete finite trace instead of producing only about 200 episodes in a 200,000-step run. It is an intentional trainability choice; the native stochastic termination regime does not begin until step 1000.

The handcrafted role reward is equal and opposite: the first sole claimant receives +1 and the opponent -1, while a simultaneous first-claim tie pays zero. The reward resolves once per episode and ignores all later claims. This keeps shaped learning contested and agent-specific; frozen-policy evaluation still uses the two PLTLF race objectives rather than shaped returns.

Before the 2026-08 reward-optimisation pass, the player-1 formula was merely a reordering of the player-0 formula and the shaping reward was shared. Those historical artifacts are not evidence for the current competitive task and must not be mixed with fresh results.

Always read final satisfaction with NashConv. An all-zero profile can look approximately converged when short best-response training also finds no success.

Notebooks And Artifacts

  • Active: notebooks/gridworlds/territory/RoleClaim_2_mo.py
  • Alternatives: notebooks/gridworlds/territory/alt/Claim_4_mo.py and alt/PeacefulClaim_4_mo.py

Artifacts live under exports/gridworlds/territory/<objective>/<agents>/.