Skip to content

Workflow

Most experiments follow the same loop whether they run in matrix games or Melting Pot gridworlds.

1. Pick A Benchmark

Choose an environment and objective pair from the notebook tree:

  • notebooks/matrix/stag_hunt/ for the repeated matrix game.
  • notebooks/gridworlds/pursuit/ for guard and intruder pursuit tasks.
  • notebooks/gridworlds/chemistry/ for Melting Pot chemistry tasks.
  • notebooks/gridworlds/harvest/ for commons harvest tasks.
  • notebooks/gridworlds/territory/ for resource-claim tasks.
  • notebooks/gridworlds/gift_refinements/ for gift and consume tasks.

Each training notebook constructs a temporal-game experiment spec and calls a shared suite helper.

The curated Python entrypoints are listed in the Experiments API and Environments API.

2. Configure Algorithms

The notebook config has top-level switches such as:

{
    "run_ippo": True,
    "run_ippo_cer": True,
    "run_iql": False,
    "run_nashq": False,
    "calculate_conv": True,
    "calculate_satisfaction": False,
}

Algorithm-specific hyperparameters live in nested blocks such as ippo, iql, and nashq.

3. Train

The suite helper resolves config, creates the wrapped environment, and runs selected algorithms across configured seeds. Each algorithm writes its own subdirectory under save_dir, for example:

exports/gridworlds/territory/role_claim/2/ippo/
exports/gridworlds/territory/role_claim/2/ippo_cer/

Long multi-seed runs can opt into partial resume. When enabled, completed prefixes are validated and reused if the same config is launched again.

4. Verify

Verification can run as part of training or in later analysis.

  • NashConv estimates unilateral deviation incentive.
  • AlphaRank ranks learned policies through response graph dynamics.
  • Final satisfaction estimates each agent's objective satisfaction probability with confidence intervals.

Verification is always performed against the objective reward path, even when a handcrafted baseline trained on shaped rewards.

See the Verification API for the report and estimator helpers.

5. Compare

The experiment helpers return per-algorithm results and cross-algorithm comparison histories. Notebook plotting cells write:

  • per-algorithm satisfaction curves,
  • cross-algorithm average-agent summaries,
  • best-agent summaries,
  • per-agent summaries,
  • legends for exported plots.

Use the generated JSON reports for exact numbers and the plots for trend inspection.

6. Diagnose

Low NashConv does not automatically mean the project succeeded. A profile where everyone has zero satisfaction can also have low measured deviation incentive if short best-response training cannot find a better policy.

For each run, inspect:

  • objective satisfaction curves,
  • final satisfaction reports,
  • NashConv baseline and best-response probabilities,
  • per-agent formulas and labels,
  • whether episode horizons make the finite trace too sparse.