Skip to content

Harvest Experiments

Commons Harvest is a native PettingZoo gridworld under src/environments/substrates/commons_harvest/. Use make_harvest_env(...) from src/environments/gridworlds/harvest/env.py.

Construction And Observations

The factory accepts commons_harvest__open, commons_harvest__closed, and commons_harvest__partnership. The benchmark overrides the upstream-sized layout to four players. Experiment adapters flatten structured observations without rasterization; visual mode uses HarvestRGBObservationWrapper and preserves namespaced readiness and collective-reward side channels in infos.

Labels And Objectives

HarvestLabelMixin treats positive native reward as apple collection and negative native reward as the legacy hidden-role punishment. It also maps the native zap event's one-based target index to per-agent zapped atoms. All three families expose aggregate, per-agent, and sticky *_ever forms, plus timeout, terminal, and stochastic_termination.

  • CHHarvestConstraint: F player_i_apple_collected
  • CHSafeHarvestConstraint: (G !player_i_zapped) & (F player_i_apple_collected)

CHSafeHarvestConstraint is the active notebook-facing benchmark. The plain Harvest objective remains useful for environment and label smoke checks.

The earlier safety formula used punishment_received. That proposition cannot become true in the benchmark's default partnership roles: native avatars have role "none", while the hidden tile only penalizes "putative_cooperator". Frozen random-policy evaluation consequently reached about 0.98 for the two inside-spawn agents without any punishment. Zap avoidance uses an actual adversarial event and makes other agents' policies strategically relevant.

Handcrafted Safe Harvest training pays an agent once for its first apple while still unzapped. Its first zap pays -1; if an apple was already paid, this cancels that earlier +1. Later apples cannot restore a failed safety formula. No reward is copied between learners.

The active notebook defaults IPPO to this handcrafted reward with gamma=gae_lambda=1 and linear learning-rate annealing. This is the selected three-seed optimisation profile: fixed player 1 reached mean satisfaction 0.5987 with seed minimum 0.585, versus its 0.582 uniform-random baseline. Frozen evaluation still constructs an objective-mode environment and checks the unchanged safety/collection formula.

Horizon

SafeHarvest_4_mo.py uses max_cycles=1000. The native substrate only begins checking stochastic termination at step 1000 and otherwise truncates at the configured limit, so shorter runs deliberately test a different finite-horizon problem.

Experiment Surface

run_harvest_experiments(...) supports the same 15 algorithm selections as the generic temporal-game suite. The active training notebook is:

  • notebooks/gridworlds/harvest/SafeHarvest_4_mo.py

The simpler notebook is archived at notebooks/gridworlds/harvest/alt/Harvest_4_mo.py. Artifacts are stored under exports/gridworlds/harvest/<objective>/4/<algorithm>/.