Skip to content

Iterated Stag Hunt Matrix Environment

The repeated two-player Stag Hunt environment lives under src/environments/matrix/stag_hunt.

Environment contract

StagHuntMatrix follows the same PettingZoo-style surface used elsewhere in the repo:

  • observation_space(agent) / state_space
  • action_space(agent)
  • state()
  • num_cells()
  • channel_names()
  • action_names()
  • get_state() / set_state()
  • human and rgb_array rendering

It is intentionally fixed to num_agents=2 in v1.

Stage game

Actions are:

  • Hare = 0
  • Stag = 1

Stage payoffs are:

  • (Hare, Hare) -> (hare_payoff, hare_payoff)
  • (Stag, Stag) -> (stag_payoff, stag_payoff)
  • (Stag, Hare) -> (stag_fail_payoff, hare_payoff)
  • (Hare, Stag) -> (hare_payoff, stag_fail_payoff)

Defaults are:

  • stag_payoff = 4.0
  • hare_payoff = 3.0
  • stag_fail_payoff = 0.0
  • max_moves = 20

The base env truncates only at the horizon. It does not have earlier terminal states.

Observation design

The env does not include an explicit time bit.

The binary observation/state channels are:

  • player_0_hare
  • player_0_stag
  • player_1_hare
  • player_1_stag
  • unmirrored

The first four channels encode the last joint action. The last channel is true on off-diagonal outcomes.

We intentionally skipped adding a handcrafted time feature because the repo's BoolRewardWrapper already augments observations with reward-monitor state when RMConfig.temporally_extended=True (the default). That means formulas using nested X can still expose progress/history to the policy through the DFA state without duplicating time in the base env.

Temporal propositions and wrappers

The stag-hunt label mixin exposes these shared propositions:

  • both_hare
  • both_stag
  • unmirrored
  • mirrored
  • player_0_hare
  • player_0_stag
  • player_1_hare
  • player_1_stag

mirrored means both players selected the same action, and unmirrored means the action pair was off-diagonal. These names avoid overloading game-theoretic coordination, where "coordinated" can imply successful coordination on a strategically good convention rather than merely matching actions.

The first experiment sketches are symmetric across both players:

  • SHHandshakeConstraint Formula: F (both_hare & X both_stag) & G !unmirrored
  • SHSustainedStagConstraint Formula: F (both_stag & X (both_stag & G !unmirrored))
  • SHBothModesConstraint Formula: F both_hare & F both_stag & G !unmirrored

These goals are intended to create interesting coordination pressure without hard-coding a single equilibrium trace.

SHSustainedStagConstraint intentionally allows unmirrored attempts before the sustained stag pair is reached. Once the trace has a both_stag step followed by another both_stag, the inner G !unmirrored requires the rest of the episode to remain mirrored.

Shared label helper

src/environments/matrix/stag_hunt/wrappers/mix_in.py also exposes shared_stag_hunt_labels(...).

That helper converts a (player_0_action, player_1_action) pair plus optional env info into the shared atom valuation used by the wrappers. The wrapper mixin now delegates to it, and the interactive play notebook reuses the same helper for its atom overlay. That keeps the manual-play overlay aligned with the monitor labels instead of duplicating slightly different logic in the notebook.

Interactive Play Notebook

There is a dedicated play notebook at:

  • notebooks/matrix/stag_hunt/play_mo.py

It mirrors the lightweight pygame loop used by the pursuit play notebook, but with Stag Hunt-specific controls:

  • A / S for player 0 (Hare / Stag)
  • K / L for player 1 (Hare / Stag)
  • R to reset
  • T to toggle the atom overlay

The notebook can launch either the base repeated game or one of the temporal wrappers (sustained_stag, handshake, both_modes). The base renderer still shows the matrix and cumulative stage payoffs, while the notebook overlay shows the current objective, last outcome, latest external reward signal, pending joint action selections, and optionally the currently true atoms.

The primary training target is:

  • notebooks/matrix/stag_hunt/SustainedStag_2_mo.py

The secondary Stag Hunt experiment definitions live under alt/ because they are useful sketches but not current primary run targets:

  • notebooks/matrix/stag_hunt/alt/Handshake_2_mo.py
  • notebooks/matrix/stag_hunt/alt/BothModes_2_mo.py

Generic Temporal-Game Experiment Helper

The former pursuit-only orchestration has been extracted into src/experiments/temporal_game_experiment.py.

Public shape

The generic spec is:

TemporalGameExperimentSpec(title, env_factory, success_threshold=0.5, extra_config_keys=())

env_factory can optionally accept:

  • seed
  • num_agents
  • config

The helper captures the resolved notebook config and passes config through when the factory declares it. This is how the stag-hunt notebooks keep max_moves visible in their notebook config while still using a generic runner.

extra_config_keys lets env-specific top-level notebook keys participate in the helper's config validation. Stag hunt currently uses:

  • ("max_moves",)

Shared responsibilities

The generic helper owns:

  • config validation and coercion
  • training orchestration across the 10 PLTLf variants and five handcrafted baselines
  • checkpoint bundles and validated partial-run resume manifests
  • timings sidecar updates
  • optional NashConv evaluation
  • optional final-policy satisfaction evaluation
  • optional AlphaRank evaluation
  • NashConv-driven one-shot auto-resume
  • comparison-run relabeling and handcrafted endpoint data for shared plotting

src/experiments/pursuit_experiment.py is now a compatibility layer that adapts PursuitExperimentSpec(wrapper_factory=...) into the generic env-factory form.

AlphaRank

AlphaRank support is part of the generic helper from the start.

Per completed training run, the helper stores one strategy snapshot per agent. When calculate_alpharank=True, it evaluates the cross-play empirical game over those saved run strategies, records a single alpharank timing stage, and returns both:

  • alpharank_result
  • alpharank_summary

Each algorithm subdirectory follows the generic artifact contract:

  • alpharank.npy
  • alpharank_report.json

Reward-wrapper note

While wiring the new env tests, the repo's BoolRewardWrapper needed a small fix: on terminal/truncated steps, some envs clear env.agents before the wrapper assigns final monitor rewards. The wrapper now falls back to obs.keys() when env.agents is empty, so terminal rewards are still written on the last step.