Skip to content

Chemistry Experiments

Chemistry uses the native PettingZoo substrates in src/environments/substrates/chemistry/. The stable factory is make_chemistry_env(...) in src/environments/gridworlds/chemistry/env.py.

Construction

The factory supports these historical substrate names:

  • chemistry__two_metabolic_cycles
  • chemistry__two_metabolic_cycles_with_distractors
  • chemistry__three_metabolic_cycles
  • chemistry__three_metabolic_cycles_with_plentiful_distractors

It forwards the requested player count, seed, horizon, and global/local observation choice to the native parallel environment. render_mode=None returns structured observations, and flatten_observations=True adapts them for training without rendering. Rendered callers use ChemistryRGBObservationWrapper, a compatibility subclass of the shared RGB adapter.

Labels

ChemistryLabelMixin exposes:

  • lifecycle atoms: timeout, terminal, stochastic_termination;
  • aggregate metabolic atoms and sticky *_ever forms;
  • per-agent food and XY atoms and sticky forms;
  • per-agent, per-reaction progress atoms for the six R1/R2/R3 vesicle reactions in the X and Y cycles;
  • reaction atoms: metabolize_food1, metabolize_food2, metabolize_xy.

Food and XY ownership is reward-derived: food rewards are at least 1 and less than 10, while XY rewards are at least 10. Native event dictionaries distinguish Food1 from Food2. The compatibility decoder still accepts older serialized event shapes for saved or injected traces, but live environments provide native event mappings through info["events"].

Lifecycle labeling combines native _step_count/max_episode_steps state with PettingZoo termination and truncation dictionaries. A truncation is a timeout; a terminal episode that is not a timeout is a stochastic termination.

Objectives

  • ChemFoodRewardConstraint: F player_i_food_metabolized
  • ChemXYOnlyConstraint: F player_i_xy_metabolized
  • ChemFoodThenSharedXYConstraint: F (player_i_food_metabolized_ever & xy_metabolized)
  • ChemFoodThenXYConstraint: F (player_i_food_metabolized_ever & player_i_xy_metabolized)

Use agent-specific goals for rational-synthesis experiments. An aggregate goal such as F food_metabolized credits every learner for one agent's event and can hide free-riding.

Food is an easy reachability smoke task. FoodThenXY keeps the harder personal two-stage dependency. Neither is in the curated launcher after objective, shaped, discount, representation, exploration, IPPO, and PR2 screens all failed p >= 0.5. The notebooks and Python APIs remain available for diagnostics. FoodThenSharedXY is available as a middle constraint in Python but currently has no checked-in training notebook.

In handcrafted training mode, the Food wrappers use bounded reaction-progress shaping. Each agent can receive +0.1 once per episode for each of its own six X/Y cycle vesicle reactions (R1x through R3y), then +1 for its own target food event. Food-Then-XY keeps the same progress signals and pays its own food and later XY milestones. Ground reactions are never attributed to a learner, and another learner's vesicle event is never copied. The six intermediate payments total at most 0.6, below the target milestone, which limits reward cycling while giving PPO credit before the rare terminal event. Final satisfaction evaluation always uses the unchanged objective monitor regardless of the training reward mode.

Experiment Surface

src/experiments/chemistry_experiment.py exposes the complete base, CER, PR2, Nash-Q, and handcrafted suite through run_chemistry_experiments(...). The direct-run notebooks are:

  • notebooks/gridworlds/chemistry/Food_4_mo.py
  • notebooks/gridworlds/chemistry/FoodThenXY_4_mo.py

The easy notebook uses four players, a 250-step finite horizon, 200,000 environment steps, and IPPO/IQL base-plus-CER defaults. Artifacts live under exports/gridworlds/chemistry/<objective>/4/<algorithm>/.