Reward screen: 2026-08-10¶

Purpose¶

This is the first reward-optimisation pass over every curated environment. It screens IPPO_CER and PR2_IPPO_CER before changing objectives or tuning environment-specific hyperparameters.

Configuration¶

The screen runs on the DoC CPU Slurm cluster, partition amd48, through the CPU-specific ~/py_cpu.sh runner. Every algorithm-environment pair is a separate job. Each job uses:

num_runs=2
num_env_steps=250000
calculate_conv=False
calculate_alpharank=False
calculate_satisfaction=True
satisfaction_eval_episodes=500
nashconv_auto_resume=False
resume_partial_runs=False

NashConv is deferred until a profile has useful satisfaction probability. Outputs are isolated under:

exports/optimisations/reward-screen-20260810/<environment>/<algorithm>/

The exact launcher shape was:

SUBMIT_DELAY_SECONDS=0 JOB_SCRIPT="$HOME/py_cpu.sh" \
  bash ~/scripts/pltlf/env.sh <environment> amd48 \
  algorithms=ippo_cer,pr2_ippo_cer bundles=2 \
  num_runs=2 num_env_steps=250000 \
  calculate_conv=False calculate_alpharank=False \
  calculate_satisfaction=True satisfaction_eval_episodes=500 \
  nashconv_auto_resume=False resume_partial_runs=False \
  save_dir=exports/optimisations/reward-screen-20260810/<environment>

Job manifest¶

An additional replacement-candidate screen uses the same configuration:

Matrix Stag Hunt also has the one non-PPO comparison justified by the task's small tabular state space:

A reward-discount variant was started for the hardest 1000-step terminal task:

This changes only return discounting. Objective-mode training emits a terminal 0/1 satisfaction bit, so the baseline gamma=0.99 weights a reward 1000 steps away by about 0.99^1000 = 4.3e-5. The variant preserves the same objective, threshold, budgets, and seeds, and writes to exports/o/rtc-g1/.

The same controlled discount comparison was later extended to the other objective-reward tasks whose notebook default is gamma=0.99:

These jobs change only ippo.gamma to 1.0. This is especially material for the 1000-step Harvest and Claim horizons, where an undiscounted terminal satisfaction bit avoids the baseline 0.99^1000 attenuation. Food has a 250-step horizon and provides the shorter-horizon control.

The original Claim PR2 submission 6673 remained resource-pending on amd24 without starting. It was cancelled and replaced by identical job 6678 on amd48; no seed or artifact was produced by 6673.

Two formula-decomposition reward variants run alongside the objective-reward screen:

These variants set reward_mode=handcrafted only during training. They retain agent-indexed rewards, do not credit an agent for another agent's completion, and use the unchanged PLTLF formula under objective mode for every frozen- policy satisfaction rollout. They are therefore optimisation variants, not alternative success criteria. Refined PR2 keeps the measured 16 GB allocation; the other three use 8 GB.

Initial jobs 6636--6639 stopped after 38 seconds, before environment creation, because the notebooks did not expose the experiment layer's existing reward_mode setting through their CLI config. The notebook surface was fixed with an explicit reward_mode="objective" default and coverage across every training notebook. These failed attempts contain no training result and are retained in the manifest rather than silently reused.

Replacement jobs after the interface fix:

The six-agent PR2 variants spend multiple CPU-hours in XLA compilation before their first update and require about 9 GB at compile peak. A purported shorter-unroll matrix was submitted with ippo.num_steps=256 for dense handcrafted training rewards:

The PR2 jobs request 16 GB based on the measured compilation footprint; IPPO uses the default allocation. A later source-history audit found that the notebooks' PPO default was already num_steps=256, so this is not an optimisation-horizon comparison. Job 6701 duplicates 6665, and jobs 6703/6704 duplicate 6642/6643 except for their output paths; 6702 is the first Consume PR2 execution but does not represent a distinct unroll setting. Job 6703's exact duplicate result is not counted as independent evidence. Still-running PR2 duplicate 6704 was cancelled at 21:22:41; authoritative job 6643 continues. Satisfaction in every case is evaluated on the unchanged 512-step Consume or 1,000-step Refined-Then-Consume objective-mode environment.

The objective wrapper audit showed that objective mode deliberately pays only the terminal satisfaction bit. To compare this sparse signal with existing agent-local domain rewards, an additional shaped-reward matrix was submitted after the runner fix:

These jobs use reward_mode=handcrafted for training and the same two seeds, 250,000 steps, and 500 evaluation episodes as the primary screen. Coins uses its explicit own-coin/avoid-interference reward. Food uses a new objective- aligned helper that pays each agent once for its own food-metabolism label and ignores the native XY payoff. The initial Safe Harvest, Claim, and Consume variants exposed native per-agent rewards whose signs/events were thought to define their corresponding labels. The later baseline audits below invalidate the Safe Harvest and Consume interpretation. No reward is copied across learners. As in the earlier shaped variants, final satisfaction is evaluated by a separately reconstructed objective-mode environment with the original PLTLF formulas and threshold. Consume only starts an IPPO variant because its objective IPPO run was already promising and six-agent PR2 incurs a measured 9 GB compilation footprint.

Initial shaped submissions 6648--6656 exposed deployment drift in the CPU runner: the checked-in version had the filename bound but lacked the deployed copy's JAX_PLATFORMS=cpu safeguard and per-project setup lock. All nine were cancelled at zero environment steps after the CUDA-plugin traceback appeared. The safeguards were restored upstream in nightly/setup commit 7b0e47a, the combined runner was deployed, and replacement jobs 6657--6665 started without the device-probe error. The cancelled attempts contain no training result.

Food attempts 6659 and 6660 were then cancelled at zero environment steps when the native-reward audit identified the unrelated +10 XY payoff. Commit 30fd9ae added the one-time, own-food-only shaping reward with unit coverage; replacement jobs 6666 and 6667 run that exact helper. Other replacement IDs remain unchanged.

Early training from both Chemistry tasks continued to show zero learner return under terminal-only or one-shot milestone rewards. The native substrate also emits agent-owned vesicle-reaction events for the six X/Y cycle stages. Commit 2860674 exposes those stages as per-agent labels and bounds their shaping: each distinct own-agent stage pays +0.1 once per episode, so all intermediate credit totals at most 0.6, while the own target milestone still pays +1. Unowned ground reactions and other agents' events pay nothing. This avoids coordinates, map state, and seed knowledge while providing semantic subgoal credit. A controlled two-seed screen keeps every other setting unchanged:

The jobs still evaluate the original personal PLTLF formulas in a reconstructed objective-mode environment; intermediate shaping atoms are not added to the success condition.

Coin attempts 6657 and 6658 used the historical prosocial helper, whose repeated partner-harm penalty is outside the selected until formula; their large negative learner returns made the mismatch visible before either seed completed. Commit 2124c8b replaced that helper with a one-shot automaton- aligned signal (+1 for matching first, -1 for mismatching first, then resolved). The old jobs were cancelled at 100,000/250,000 steps with no frozen- policy report. Jobs 6679 and 6680 restart the same two seeds and budget without a resumable completed seed, using the corrected reward.

The one-shot aligned IPPO job completed at (0.450, 0.478) and (0.542, 0.506) across its two seeds, for means (0.496, 0.492). The winner changes by seed and the aggregate matches the (0.484, 0.498) random baseline, so this variant is rejected. One decisive learner reward per 256-step episode is a visible credit bottleneck. Commit 7e3f407 adds a denser collector-local comparison: every matching collection pays its collector +1, every mismatching collection pays that collector -1, and partner-harm labels still pay nothing. Repeating colour-preference credit cannot change the frozen first-collection objective. The otherwise identical jobs are:

One-shot PR2 completes at (0.328, 0.336) and (0.010, 0.002), for means (0.169, 0.169), and is rejected. Repeated-reward IPPO completes at (0.512, 0.526) and (0.494, 0.494), for means (0.503, 0.510); that is still the uniform-random result rather than evidence of optimisation. Repeated-reward PR2 completes at (0.400, 0.000) and (0.346, 0.358), for means (0.373, 0.179), and also fails the gate. A controlled IPPO-only follow-up, job 6737 at exports/o/coin-e0/, changes only ippo.ent_coef from 0.01 to 0.0 while keeping the repeated own-colour reward, two seeds, and all screen/evaluation budgets. This tests whether entropy pressure is masking colour preference without changing observations, dynamics, or the success formula.

The zero-entropy job completes cleanly in 03:07:44 at (0.022, 0.012) and (0.428, 0.470), for fixed-agent means (0.225, 0.241). Removing exploration therefore worsens the aggregate and does not clear the gate; job 6737 is rejected.

Job 6765 adds an orthogonal stability control at exports/o/coin-a/: it restores the default 0.01 entropy coefficient and changes only ippo.anneal_lr to true. It otherwise uses the same dense collector-local reward, global observation, two contiguous 250,000-step seeds, and 500 frozen- policy episodes. This tests whether constant late PPO updates erase an earlier colour preference; success is still the unchanged own-coin-first formula.

Job 6765 completes cleanly in 03:06:03 at (0.464, 0.494) and (0.520, 0.508), for means (0.492, 0.501). This is indistinguishable from the (0.484, 0.498) random baseline, so learning-rate annealing is rejected. The local-observation profile remains the only Coin candidate in long validation.

The global Coin policy input is an absolute categorical grid, whereas its movement actions are avatar-relative. The substrate already provides an agent-centric local observation that contains the learner's own colour and nearby coin colours. Commit a375407 exposes the existing fully_observable setting through the Coins experiment config so this representation can be screened as a hyperparameter. Local observation changes neither hidden state nor dynamics: it removes distant global cells and rotates the visible grid with the avatar. The experiment config now exposes this setting, and the follow-up keeps the repeated own-colour reward and all other optimisation/evaluation settings fixed. IPPO-CER job 6751 runs the two predeclared 250,000-step seeds with 500 frozen-policy episodes each and writes to exports/o/coin-local/. It completes cleanly in 02:54:54 at (0.522, 0.478) and (0.516, 0.462), giving fixed-agent means (0.519, 0.470). Player 0 is stable and clears both 0.5 and its 0.484 random baseline, but the small +0.035 margin and mean marginal interval [0.4583, 0.5797] are screen evidence only. The profile is promoted unchanged to job 6768 at exports/o/coin-lv/, with three seeds, 1,000,000 steps, and 1,000 frozen- formula episodes per seed.

Validation completes at (0.493, 0.465), (0.496, 0.473), and (0.494, 0.492). Fixed-agent means are (0.4943, 0.4767), so neither role clears the 0.5 gate; player 0 is only +0.0103 above its 0.484 random baseline. Its mean marginal interval is [0.4514, 0.5373] and simultaneous interval is [0.4475, 0.5411]. Job 6768 completes cleanly in 16:23:08; the 250k trend is rejected. Two orthogonal long controls retain local observations, dense collector-local reward, three contiguous 1,000,000-step seeds, and 1,000 frozen episodes: job 6772 (exports/o/coin-la/) changes only ippo.anneal_lr=True, while job 6773 (exports/o/coin-ll/) changes only ippo.lr=1e-4. Both run as one Slurm task with two allocated CPUs and 8 GB. They test late-update policy erosion without changing dynamics, formulas, or seed selection.

The controls remain close to the decision boundary. Annealing gives (0.498, 0.495) and (0.484, 0.510), for provisional fixed-agent means (0.491, 0.5025). The lower learning rate gives (0.486, 0.502) and (0.521, 0.511), for provisional means (0.5035, 0.5065). These margins are only a few successes per 1,000 episodes, so neither is a selection. Both predeclared third seeds continue; the campaign compares the fixed-agent three-seed estimates rather than selecting whichever role happens to win one near-tied evaluation.

The third seeds resolve the comparison. Annealing gives (0.500, 0.504), for final means (0.494, 0.503) and a player-1 minimum of 0.495. It completes cleanly in 17:28:49. Fixed lr=1e-4 gives (0.484, 0.517), for final means (0.497, 0.510); player 1 clears 0.5 in every seed at (0.502, 0.511, 0.517) and exceeds its 0.498 random baseline. Its marginal mean interval is [0.4671, 0.5529] and simultaneous interval is [0.4632, 0.5568], so the evidence meets the requested point-estimate criterion but is deliberately reported as narrow rather than statistically separated from 0.5. Job 6773 completes cleanly in 17:17:46. The local-observation, dense handcrafted, fixed-1e-4 IPPO-CER profile is selected over annealing on mean, worst seed, and consistency with the gate.

The OwnCoinFirst_2_mo.py visible defaults now encode this selected training profile: local observations, reward_mode="handcrafted", and IPPO learning rate 1e-4. The formula, 256-step trace, regrowth rate, and objective-mode frozen evaluator are unchanged; CLI overrides can still recover every control.

Chemistry, Commons Harvest, and Gift Refinements use the same absolute-global grid with avatar-relative actions. Their experiment configs now expose the same existing fully_observable switch. Setting it false gives each learner only its rotated local field plus its existing private scalar state; it does not add privileged information or centralise execution. Local-mode controls retain each task's formulas, maps, horizon, agent-local shaping, algorithms, and contiguous seeds. They compare representation only against the matched global-observation reward screen.

The flattened base-policy dimensions change from global to local as follows: Coin 1446 -> 606, Chemistry 1402 -> 486, Harvest 1729 -> 485, and Gift Refinements 2920 -> 488. These counts exclude the monitor-state suffix added by the objective wrapper. The controls therefore test a smaller, action- aligned input rather than a larger source of privileged state.

Every control uses two 250,000-step seeds and 500 frozen-policy episodes per seed. All five local-mode jobs, including Coin 6751, started with two CPUs and one scheduler task.

Chemistry's first higher-entropy opposed-cycle checkpoint discovered assigned reaction credit (return_mean=0.1 at 25,000 steps), while its matched default-entropy IPPO and PR2 jobs remained at zero through 50,000 steps. Food job 6766 therefore applies the same one-factor control at exports/o/food-e5/: local observations, bounded progress shaping, two seeds, 250,000 steps, and 500 formula evaluations are unchanged, while ippo.ent_coef increases from 0.01 to 0.05. Food satisfaction remains F player_i_food_metabolized; learner return is only an early exploration diagnostic and cannot promote the profile.

A 1,000-step Chemistry horizon was considered as a final sparsity control. The local fixed-random diagnostic was stopped without a result when its first 100-episode block proved too slow for an interactive process. No 1,000-step training job is launched: without a completed chance baseline, extending the trace could silently make reachability easy and would not meet the campaign's nontriviality rule.

Refined-Then-Consume has an additional local IPPO control, job 6760 at exports/o/ref-local-g1/, that changes only gamma and gae_lambda from 0.99/0.95 to 1.0/1.0. Its refinement and consumption milestones can be hundreds of steps apart; the undiscounted setting tests long-range credit without changing either event, their agent ownership, or the satisfaction formula. It retains the same two seeds, 250,000 steps, 256-step updates, and 500 evaluation episodes.

The first objective-IPPO seed (job 6626) evaluates at (0.174, 0.212, 0.218, 0.090, 0.212, 0.162). This is a real increase over the corresponding random range of 0.076--0.102, but the best agent remains far below 0.5. The result is retained as optimisation evidence, not promoted; its second predeclared seed and the discount/handcrafted controls continue.

The objective-IPPO second seed is (0.156, 0.218, 0.208, 0.156, 0.150, 0.152). Fixed-agent means are (0.165, 0.215, 0.213, 0.123, 0.181, 0.157), so job 6626 is rejected. Slurm reports FAILED after 20:12:34 only because the old runner cannot write its overlong final Marimo filename; both frozen reports and checkpoints were already written and are valid.

Handcrafted IPPO job 6642 gives (0.046, 0.038, 0.026, 0.060, 0.054, 0.070) and (0.154, 0.168, 0.138, 0.296, 0.294, 0.192). Fixed-agent means are (0.100, 0.103, 0.082, 0.178, 0.174, 0.131), so even its best role is far below 0.5 and the profile is rejected. It completes cleanly in 21:24:43. Purported shorter-unroll job 6703 completes in 20:25:57 with exactly the same per-seed values because 256 was already the notebook default; it is a duplicate execution, not replication under a distinct hyperparameter.

The first local-observation handcrafted seed (job 6755) is lower still, at (0.020, 0.034, 0.022, 0.042, 0.026, 0.026). Every role is below its random baseline; the second predeclared seed continues, but the local representation provides no seed-0 rescue signal.

Its second seed is (0.148, 0.076, 0.038, 0.118, 0.078, 0.054). The best fixed-agent mean is only 0.084; job 6755 completes cleanly in 13:05:44 and is rejected. Local observations alone are not the source of the stronger combined local/undiscounted result.

Combining local observations with undiscounted gamma=gae_lambda=1 changes that result materially: job 6760 seed 0 gives (0.308, 0.288, 0.164, 0.320, 0.174, 0.208). The best role remains below 0.5, but the broad improvement over the 0.076--0.102 random range is the strongest Refined seed-0 signal so far. Seed 1 regresses to (0.156, 0.228, 0.234, 0.240, 0.230, 0.220); its best fixed-agent mean is only 0.280, so the 250,000-step profile is rejected. Because this environment takes hours per frozen evaluation, the unchanged profile had already started a long-horizon scale check as job 6774 at exports/o/ref-lv/: three contiguous 1,000,000-step seeds and 1,000 formula episodes per seed, one Slurm task, two CPUs, and 8 GB. This is not a one-seed promotion; retaining the environment requires the long evidence to clear the 0.5 gate.

Long seed 0 gives (0.145, 0.131, 0.109, 0.128, 0.145, 0.164). Its best role is only 0.164, below the strongest 250,000-step seed and far below the gate; scaling has therefore supplied no rescue evidence. Long seed 1 is similarly low at (0.165, 0.176, 0.145, 0.119, 0.165, 0.140). Even if seed 2 gave one fixed role perfect p=1, the best possible three-seed mean would be below 0.437; the long profile is mathematically unable to clear the aggregate gate. Job 6774 was therefore stopped before seed 2 at 2-01:44:29.

The three remaining PR2 screens were also stopped at 70% of seed-0 training after 2-16:50:53--2-17:32:30. The objective, undiscounted-objective, and handcrafted profiles had no sustained terminal-return signal and would each have required several additional node-days before even beginning their second seed. This early stop is not seed selection: no PR2 frozen result was counted as success or failure. It follows completed failures for objective IPPO, undiscounted IPPO, handcrafted IPPO, local handcrafted IPPO, and combined local/undiscounted IPPO, plus the two conclusive long seeds. No viable profile reaches 0.5, so refined_then_consume_6 is removed from the curated launcher; its notebook and Python APIs remain available by direct path. The final all-environments suite has 8 tasks.

Undiscounted terminal-objective job 6633 completes cleanly in 20:21:26 at (0.156, 0.136, 0.124, 0.104, 0.116, 0.178) and (0.174, 0.102, 0.180, 0.152, 0.138, 0.168). Its best fixed-agent mean is only 0.173, so changing discounting without the local shaped-reward combination is rejected.

Two fixed 500-episode uniform-random diagnostics exposed tasks that must not be credited merely for clearing 0.5:

• Safe Harvest, environment seeds 0--499 and action RNG seed 20260810, gives per-agent satisfaction 0.978, 0.980, 0.094, 0.088. No agent receives the nominal punishment in any episode. Source inspection confirms why: all native avatars default to role "none", but the hidden -10 tile only affects "putative_cooperator". The original safety conjunct is inert.
• Consume, using the same fixed seeds/RNG and its 512-step horizon, gives 0.912, 0.926, 0.948, 0.932, 0.918, 0.930. This agent-indexed reachability task is an environment smoke check rather than an optimisation benchmark. In contrast, Refined-Then-Consume at its 1,000-step horizon gives only 0.076, 0.102, 0.086, 0.090, 0.084, 0.096.

The same fixed seed/RNG protocol confirms that the Chemistry objectives are not chance-level shortcuts. Food gives per-agent probabilities 0.054, 0.062, 0.044, 0.052 over its 250-step horizon, while Food-Then-XY produces zero successes for all four agents in 500 episodes. These values justify retaining both tasks and using the aligned decomposition rewards to address sparse credit rather than changing their success criteria.

A separate 100-episode diagnostic counted agent-owned vesicle progress under the same fixed random policy. Summed across the six R1/R2/R3 X-or-Y atoms, the per-agent agent-stage episode-hit counts were 19, 19, 21, 22; individual atoms ranged from zero to nine episode hits. Food events were 6, 5, 4, 5, and XY events remained all-zero. Thus reaction progress is observable and nontrivial rather than ubiquitous. If the food targets remain unlearnable, these data identify a defensible agent-local objective boundary for a later replacement task; no running task has been credited against that weaker boundary.

ChemOpposedCyclesConstraint and OpposedCycles_4_mo.py provide that bounded replacement candidate without weakening to a shared objective. Players 0 and 2 have F player_i_reaction_r3x; players 1 and 3 have F player_i_reaction_r3y. An R3x reaction produces iy, which inhibits the Y cycle, while R3y produces the X inhibitor ix; the two objective families therefore impose opposing externalities. Satisfaction remains agent-owned. Handcrafted training credit pays each assigned R1/R2 stage 0.2 once and the assigned R3 target 1.0 once, for a maximum 1.4 per agent and no reward for the other branch or another learner's event. Its 500-episode fixed-random baseline gives (0.036, 0.038, 0.034, 0.028), so R3 completion is neither an automatic nor a chance-level pass. Local-observation IPPO-CER job 6758 and PR2-IPPO-CER job 6759 therefore screen two 250,000-step seeds with 500 frozen- policy episodes each at exports/o/opp-local/. The candidate is not counted as a retained task unless a screen and long validation justify replacing Food-Then-XY.

Because both initial opposed-cycle learners still report zero return at the first 25,000-step checkpoint, job 6763 is a predeclared exploration control: it changes only ippo.ent_coef from 0.01 to 0.05, while retaining local observations, handcrafted credit, two contiguous seeds, 250,000 steps, and 500 objective-mode evaluation episodes. It writes to exports/o/opp-e5/. The higher entropy coefficient is not a weaker success condition; it tests whether premature policy concentration prevents discovery of the rare assigned reactions whose random baseline is only 0.028--0.038.

Initial submissions 6756 and 6757 exited in one second with code 66 and zero environment steps. The shared runner checks that a notebook exists before its normal repository pull, so the newly added path was not visible in the older cluster checkout. A clean fast-forward made the file visible; replacements 6758 and 6759 are the only valid opposed-cycle jobs.

Default-entropy IPPO job 6758 completes cleanly in 07:16:35 at (0.004, 0, 0, 0) and all-zero. It is below every role's (0.036, 0.038, 0.034, 0.028) random baseline and is rejected. The PR2 and predeclared higher-entropy IPPO controls remain before deciding whether the candidate can replace Food-Then-XY.

Higher-entropy IPPO job 6763 also fails. It completes cleanly in 06:53:17 at (0.010, 0.002, 0.016, 0.004) and (0.010, 0.014, 0.016, 0.044), for fixed-agent means (0.010, 0.008, 0.016, 0.024). Every mean is below the corresponding random baseline, and none approaches 0.5. Only the running PR2 control remains before the replacement candidate is closed.

PR2 job 6759 closes the candidate all-zero in both seeds and completes cleanly in 07:55:47. Default IPPO, higher-entropy IPPO, and PR2 all fail their random baselines and the 0.5 gate, so Opposed Cycles cannot replace Food-Then-XY. Both food_4 and food_then_xy_4 are removed from the curated launcher under the task's removal allowance. Their notebooks and explicit direct paths remain available; the all-environments suite drops from 11 to 9 tasks pending the separate Refined-Then-Consume decision.

The primary objective screens complete without a promotable result. Food IPPO gives (0.046, 0.056, 0.094, 0.008) and (0.012, 0, 0.044, 0.044), for a best fixed-agent mean of 0.069; Food PR2 is all-zero twice. Food-Then-XY is all-zero for both algorithms and both seeds. Jobs 6616--6619 are marked FAILED after 07:42:48--08:27:41 only because the old runner's final Marimo filename is too long; both checkpoints and two-run reports were already written and are valid. Every primary profile is rejected because it remains far below 0.5.

Three earlier PR2 controls subsequently complete both seeds and are rejected. Food's one-shot own-food shaping (job 6667) is all-zero in both seeds; its undiscounted objective reward (job 6669) gives (0.002, 0, 0, 0) then all-zero; and Food-Then-XY milestone shaping (job 6641) is all-zero in both seeds. Their clean wall times are 08:41:39, 08:39:24, and 09:22:00, respectively. The local/high-entropy IPPO controls remain relevant because their training logs at least show agent-owned reaction discovery, but no learner-return checkpoint can substitute for frozen formula evaluation.

The corresponding earlier IPPO jobs also complete and are rejected. Food-Then-XY milestone shaping (job 6640) is all-zero twice. Food one-shot shaping (job 6666) gives (0, 0.044, 0.038, 0.038) and (0.056, 0.006, 0.058, 0.032); undiscounted objective reward (job 6668) gives (0.016, 0.052, 0.022, 0.062) and (0.006, 0, 0.014, 0.008). Neither Food profile improves on the per-agent random baseline in aggregate, and all are far below the gate. Their clean wall times are 08:42:39, 07:58:35, and 08:04:27.

The denser six-stage progress reward does not rescue Food-Then-XY under IPPO: job 6725 completes cleanly in 07:43:06 and is all-zero in both seeds. It is rejected. Its PR2 counterpart, job 6726, also completes all-zero in both seeds and cleanly exits in 08:26:10. Every original Food-Then-XY reward/algorithm profile is therefore rejected; the separate opposed-cycle candidate retains its predeclared seeds.

The final original-task representation control, local-observation IPPO job 6753, is also all-zero in both seeds and completes cleanly in 06:58:40. Changing representation cannot rescue Food-Then-XY. Every original objective, reward, algorithm, discount, and observation profile screened in this campaign is now rejected; only the explicitly separate opposed-cycle replacement candidate remains to close.

Food's progress-reward IPPO job 6723 also completes cleanly in 07:54:01, at (0.038, 0.004, 0.046, 0.058) and (0.028, 0.016, 0.050, 0.014). Its fixed- agent means are at or below the random baseline and nowhere near 0.5, so this profile is rejected as well. The matching PR2 job 6724 is all-zero in both seeds and completes cleanly in 08:35:18. Every original Food profile is now rejected; only the predeclared local-observation and higher-entropy controls remain, and neither can be promoted without its own frozen-policy evidence.

The local-observation progress-reward control (job 6752) is all-zero in both seeds and completes cleanly in 06:54:13. The action-aligned representation does not rescue Food. Only the predeclared higher-entropy local control remains before the curated task's removal decision.

The final higher-entropy local control, job 6766, completes cleanly in 06:48:54 at (0.052, 0.032, 0.022, 0.010) and (0.014, 0.014, 0.014, 0.024). Its fixed-agent means (0.033, 0.023, 0.018, 0.017) are all below the corresponding random baselines and far below 0.5. Food has no viable profile after objective, one-shot and progress rewards, discount, both primary algorithms, global/local observations, and entropy controls. It is therefore removed from the curated launcher under the task's explicit removal allowance; the direct notebook remains available for diagnostics.

Plain Consume is therefore removed from the curated launcher under the task's explicit removal allowance; its notebook remains runnable by direct path. Jobs 6624, 6625, 6665, 6701, and 6702 were cancelled without a completed seed, and none can count as optimisation evidence.

Safe Harvest is repaired rather than removed. Its objective now uses the native per-agent zap target event:

(G !player_i_zapped) & (F player_i_apple_collected)

This remains agent-indexed and general-sum because any other player can make an agent fail by zapping it. The final 500-episode fixed random baseline is 0.570, 0.582, 0.032, 0.028; an earlier 200-episode pilot also found 0.965/0.975 for the weaker "apple before first zap" alternative. The global safety formula is retained because it leaves room to demonstrate optimisation above random while keeping the required best-agent point estimate near the 0.5 gate. Its aligned shaped reward pays the first safe apple +1, pays the first zap -1, cancels a previously paid apple on later failure, and never credits one learner for another learner's event. Pre-fix jobs 6620, 6621, 6661, 6662, 6670, and 6671 were cancelled without a completed seed and are invalid for the revised objective.

The fresh zap-avoidance screen uses:

All four jobs retain two seeds, 250,000 steps, and 500 objective-mode frozen- policy episodes per seed. Both pairs use gamma=1.0 and gae_lambda=1.0 over the existing 1,000-step unroll. This is essential for shaped alignment too: an early apple +1 and a later zap -1 must cancel without discounting or GAE attenuation. Jobs 6705--6708 were cancelled at zero environment steps when this late-failure credit issue was caught; they produced no report.

Seed 0 now separates the reward variants. Objective IPPO gives (0.580, 0.434, 0.024, 0.018), which matches the 0.570 best-role random baseline. Aligned-handcrafted IPPO gives (0.814, 0.366, 0.050, 0.060): player 0 clears both the 0.5 gate and its baseline by 0.244. This is provisional optimisation evidence, not a promotion from a selected seed; both predeclared second seeds remain mandatory before choosing a validation profile.

The aligned-handcrafted IPPO second seed gives (0.634, 0.580, 0.018, 0.052). Player 0 therefore passes in both predeclared seeds and has a fixed- agent mean of 0.724, versus its 0.570 random baseline. Its two-seed mean marginal interval is [0.6633, 0.7847] and simultaneous interval is [0.6528, 0.7952]. Job 6712 completes cleanly in 09:15:45. This profile is promoted unchanged to validation job 6769 at exports/o/harv-v/: three seeds, 1,000,000 steps, and 1,000 frozen objective-mode episodes per seed. Slurm confirms one task, two allocated CPUs, and 8 GB RAM on amd48 despite the site wrapper's known misleading submission warning.

Long-validation seed 0 gives (0.602, 0.577, 0.014, 0.034). Both safe-apple roles clear 0.5 while the two opposed roles remain low; player 0 is also above its 0.570 same-role random baseline. This is a valid first seed, not final selection: seeds 1 and 2 continue under the same 1,000,000-step and 1,000-episode protocol.

Seed 1 gives (0.372, 0.591, 0.036, 0.048). The successful safe role changes: player 0 falls below 0.5 while player 1 clears it. Two-seed fixed-agent means are (0.487, 0.584, 0.025, 0.041); player 1 is only +0.002 above its 0.582 same-role random baseline. Seed 2 is therefore decisive for both the requested gate and evidence of optimisation beyond chance.

Seed 2 gives (0.650, 0.493, 0.139, 0.012). Final fixed-agent means are (0.5413, 0.5537, 0.0630, 0.0313). Player 1 has the best mean; its marginal interval [0.5107, 0.5966] and simultaneous interval [0.5033, 0.6040] both clear 0.5. Job 6769 completes cleanly in 1-22:40:35, so the profile meets the formal success gate. However, neither safe-role mean improves on its same-role random baseline (0.570, 0.582), so this is not yet accepted as a reward-optimisation win.

Two three-seed, 1,000,000-step stability controls retain global observations, the aligned handcrafted reward, gamma=gae_lambda=1, unchanged formulas, and 1,000 frozen episodes: job 6810 (exports/o/harv-ha/) enables linear learning- rate annealing, while job 6811 (exports/o/harv-hl/) fixes lr=1e-4. Both start as one Slurm task with two CPUs on amd48. A control must clear 0.5 and beat the corresponding random role before it can replace the formal-but- chance-level validation profile.

The first two annealed-control seeds give (0.554, 0.616, 0.031, 0.037) and (0.550, 0.585, 0.022, 0.031). Its fixed-agent means are therefore (0.552, 0.6005, 0.0265, 0.034): player 1 is +0.0185 above its 0.582 same-role random baseline. The corresponding fixed-1e-4 seeds give (0.690, 0.545, 0.030, 0.015) and (0.561, 0.524, 0.056, 0.027), with fixed-agent means (0.6255, 0.5345, 0.043, 0.021): player 0 is +0.0555 above its 0.570 baseline. Both predeclared seed-2 runs continue because the controls currently improve different fixed roles; no profile is selected from this two-seed checkpoint.

Seed 2 gives (0.629, 0.595, 0.030, 0.022) with annealing and (0.510, 0.608, 0.025, 0.026) with fixed 1e-4. The final annealed fixed-agent means are (0.5777, 0.5987, 0.0277, 0.0300). Player 1 clears 0.5 in every seed at (0.616, 0.585, 0.595), beats its 0.582 same-role random baseline in every seed, and has marginal mean interval [0.5557, 0.6416] and simultaneous mean interval [0.5483, 0.6490]. The fixed-1e-4 means are (0.587, 0.559, 0.037, 0.0227); its best fixed role has a lower mean and a 0.510 worst seed. The annealed profile is selected on best-agent mean, worst seed, and per-seed baseline consistency. SafeHarvest_4_mo.py now defaults to the aligned handcrafted reward, gamma=gae_lambda=1, and linear IPPO learning-rate annealing. Objective formulas and frozen evaluation are unchanged.

Jobs 6810 and 6811 complete with exit code zero in 1-23:15:19 and 1-23:02:21, respectively. Each used one Slurm task, two allocated CPUs, and 8 GB on amd48. Transient live-notebook refreshes reported home-filesystem quota errors during evaluation, but both logs end successfully after writing all checkpoints, three-run JSON reports, arrays, and final notebooks; scheduler and report state, rather than those optional live refreshes, is authoritative.

The terminal-objective IPPO control (job 6710) also completes cleanly in 09:13:38; its second seed is (0.294, 0.690, 0.068, 0.026). Its fixed-agent means are (0.437, 0.562, 0.046, 0.022), below the corresponding random baselines (0.570, 0.582, 0.032, 0.028) for both roles near the gate. It is rejected in favour of the shaped profile on predeclared aggregate evidence, not because either individual seed was selected after inspection.

Both PR2 controls are rejected as well. Terminal-objective job 6711 completes in 09:37:37 at (0.508, 0.460, 0, 0) and (0.494, 0, 0, 0). Its player-0 mean of 0.501 nominally clears the point gate by 0.001 but is below the same role's 0.570 random baseline. Aligned-handcrafted job 6713 completes in 09:40:10 at (0, 0.016, 0, 0) and (0.484, 0, 0, 0), with no fixed-agent mean above 0.242. Safe Harvest's complete two-algorithm screen therefore selects shaped IPPO on its stable 0.724 aggregate, not merely on a threshold crossing.

The local-observation shaped IPPO control (job 6754) completes in 06:59:45 at (0.222, 0.356, 0.030, 0.410) and (0.530, 0.140, 0.044, 0). Its fixed- agent means are (0.376, 0.248, 0.037, 0.205), so no role clears 0.5 and the profile is rejected. This closes the representation comparison in favour of the selected global-observation shaped IPPO profile.

All 24 jobs entered RUNNING state immediately. Slurm resolved each request to one task, two allocated CPUs, and 8 GB RAM. Initial logs confirmed the current checkout, shared PLTLF environment, CPU JAX backend, and notebook/CLI arguments. The Marimo runner warned that some live snapshot filenames were too long; experiment artifacts are written independently under the save directory, so this is monitored as a presentation-artifact issue rather than assumed to invalidate training.

The warning is caused by the runner incorporating every CLI override into the adjacent notebook artifact name. Experiment reports are written before the final notebook export, but the overlong final name can still make Slurm report an otherwise-complete job as failed. Validation launches therefore use a short exports/o/... save path and omit overrides that already match notebook defaults; the JSON report and checkpoint manifest, rather than the auxiliary notebook export, remain the completion evidence.

The shared cluster runner was subsequently fixed in nightly/setup commit 204ef5c: stems longer than 180 characters now keep a readable prefix plus a 16-hex SHA-256 suffix. The updated py_cpu.sh and common helper were deployed to cpucluster with the previous files retained as before-204ef5c backups. Jobs already running keep the runner state from their launch, while later jobs produce bounded, collision-resistant final and live artifact names.

The original Refined-Then-Consume PR2_IPPO_CER job 6627 reached the 8 GB limit and lost its training worker, leaving only the waiting Marimo parent. After inspecting the allocation and node process tree, it was cancelled. Attempt 6630 showed that SBATCH_MEM=16G was ignored by the local wrapper and was cancelled before training. Replacement job 6631 was submitted with an explicit submit --mem=16G request; Slurm confirmed ReqTRES mem=16G.

Prior evidence used for triage¶

The canonical exports predate explicit learned-policy satisfaction evaluation, so their IPPO_CER and PR2_IPPO_CER satisfaction_report.json files contain zero evaluated runs. Their final 64 training episodes nevertheless provide a useful, non-certifying signal:

The paper repository additionally reports frozen-profile estimates of 0.573 for the classic intruder, 0.979 for the capture intruder, 1.000 for Sustained Stag, and 0.609 averaged over Safe Harvest agents. Its strongest non- handcrafted Territory result is only 0.344 with PR2_IQL_CER. These values are historical evidence, not substitutes for the new pass.

Territory Claim_4 was submitted alongside the primary matrix because it already provides four agent-indexed F player_i_resource_claimed objectives in a contested substrate. It is a candidate replacement for Role Claim if the primary task remains both unsuccessful and unsuitable as a general-sum satisfaction game.

Source inspection then established that historical RoleClaim_2 was not a valid candidate at all: its player-1 formula was only a reordered copy of player 0's formula, and its handcrafted reward was shared. Commit 4959d62 first revised the task into an opposed claim/abstention contest:

player_0: (F player_0_resource_claimed) & (G !player_1_resource_claimed)
player_1: (F player_1_resource_claimed) & (G !player_0_resource_claimed)

The interim shaped reward was equal-and-opposite (+1 to the claimant, -1 to the opponent, once per player's first claim). The old common-objective jobs 6622 and 6623 were cancelled after one and zero completed training seeds, respectively, and none of their results can count. An interim two-seed, 250,000-step screen used:

Claim_4 stayed a diagnostic candidate rather than silently replacing the benchmark with an easier four-player reachability objective. The later event- semantics diagnostic below superseded the interim claim/abstention formulation before any fresh seed completed.

A fixed 50-episode uniformly random diagnostic (environment seeds 0--49, action RNG seed 20260810, 256-step horizon) initially produced p_hat=0.0 for both corrected Role Claim objectives. The later label audit established that this diagnostic measured delayed payout rather than the substrate's claimed_resource event, so the value is retained only as pre-fix provenance and is not a valid baseline for the corrected atom.

All four corrected 256-step runs still had zero learner return at the 50,000-step checkpoint. Because the historical task also failed at that horizon, a controlled full-trace variant was started rather than waiting for a likely all-zero terminal signal:

These jobs preserve the corrected formulas, seeds, 250,000-step budget, and 500-episode evaluator. They set max_cycles=1000, ippo.num_steps=1000, gamma=1.0, and gae_lambda=1.0, so every PPO rollout contains one complete finite trace and the terminal bit can propagate without temporal discount or GAE attenuation. The original 256-step matrix continues in parallel; this is a horizon/credit-assignment comparison, not a replacement chosen after seeing final seed outcomes.

The ensuing source audit found a more fundamental issue before any of these Territory jobs completed a seed: TerritoryLabelMixin derived player_i_resource_claimed from positive native reward. Territory ownership changes immediately through an explicit claimed_resource event, whereas the native payout begins only after a 25-frame delay and then occurs stochastically at rate 0.01. A payout is neither a reliable claim timestamp nor guaranteed to identify the current ownership transition. The label now consumes the deduplicated native event and maps its one-based player_index to the agent; positive reward without that event does not satisfy the claim atom.

This semantic correction invalidates every Territory screen launched before the fix: candidate Claim jobs 6628, 6629, 6663, 6664, 6672, and 6678, plus competitive Role Claim jobs 6674--6677 and 6681--6684. They are cancelled and retained in the manifest rather than interpreted as failed learning. Fresh fixed-seed diagnostics and screens use the event-defined proposition. Claim-4 remains diagnostic only: an easy raw claim event is not by itself evidence that it is an acceptable replacement for the contested Role objective.

The first event-defined claim/abstention replacements were:

All four were cancelled at zero environment steps after the event-defined random diagnostic completed. Across 500 fixed episodes (environment seeds 0--499, action RNG seed 20260810), both agents claimed at least once in every episode. The interim objectives therefore remained p_hat=0.0 for both players for a structural reason: two agents who each rationally seek a claim make the other's permanent abstention implausible.

The curated task is consequently revised to a first-sole-claim race:

player_0: (!player_1_resource_claimed) U
          (player_0_resource_claimed & !player_1_resource_claimed)
player_1: (!player_0_resource_claimed) U
          (player_1_resource_claimed & !player_0_resource_claimed)

A first step containing only one player's claim satisfies that player and irreversibly fails the opponent; a simultaneous first-claim step is a tie and satisfies neither. The shaped reward now resolves on that same step: +1/-1 for a sole claimant/opponent and zero for a tie, with later claims ignored. This preserves mutually exclusive, competitive payoffs while removing the irrational requirement that a losing agent abstain forever.

The same 500-episode fixed random diagnostic gives player 0 p_hat=0.490, player 1 p_hat=0.486, ties 0.024, and no episode without a claim. These values are the chance baseline for promotion: a learned result that merely rounds to 0.5 is not treated as reward optimisation evidence.

The fresh two-seed race screen uses:

All four logs confirm checkout e9d9d75, one task, two CPUs, the unchanged 250,000-step/two-seed budget, and 500 frozen-policy episodes per seed.

Objective-IPPO completes at (0.416, 0.414) and (0.890, 0.098), giving fixed- agent means (0.653, 0.256). Player 0 therefore exceeds both the 0.5 gate and its 0.490 random baseline in aggregate, although the wide seed spread must be resolved by long validation. Shaped IPPO flips its winner across seeds: (0.648, 0.142) then (0.236, 0.592), whose fixed-agent means (0.442, 0.367) fail the gate. The objective-IPPO profile is promoted to job 6744 at exports/o/role-v/, with three seeds, 1,000,000 steps, and 1,000 evaluation episodes per seed. Validation seed 0 gives (0.999, 0.000), decisively above player 0's 0.490 random baseline. Seeds 1 and 2 give (0.859, 0.101) and (0.977, 0.006). Player 0's final mean is 0.945, its minimum seed is 0.859, its marginal mean interval is [0.9021, 0.9673], and its simultaneous interval is [0.8982, 0.9686]. Job 6744 completes cleanly in 20:09:13; objective-reward IPPO is the validated Territory selection. Neither PR2 variant produced a success in seed 0. Objective PR2 is in fact all-zero across both seeds. Shaped PR2 gives (0.000, 0.000) and (0.000, 0.004). Both PR2 profiles are rejected without long validation.

Jobs 6685--6688 and 6689--6692 were two zero-training submission attempts for this same matrix. The shared runner resolves notebook paths relative to ~/Projects: the first batch incorrectly used a repository-relative path and the second an absolute path, so both exited with code 66 before repository setup. The valid batch uses the canonical project-qualified path pltlf/notebooks/... and reaches repository setup and training normally.

CX3 was inspected as overflow capacity but was not used for this pass. Its checkout was current yet contained untracked live Marimo snapshots, so the clean-checkout requirement for scheduler runners was not satisfied. The DoC CPU cluster had sufficient immediate capacity.

Results¶

Final retained-suite outcome¶

Every retained task below was validated over the three contiguous training seeds 0--2, 1,000,000 environment steps per seed, and 1,000 frozen-policy objective episodes per seed. p is the best fixed agent's mean across all three seeds; the interval is the report's 95% marginal mean interval. No threshold was lowered from 0.5.

All eight retained environments therefore satisfy the requested fixed-agent point-estimate gate. The narrow Coin result is not claimed as interval- certified improvement over 0.5; its point estimate and every individual seed meet the stated criterion. Classic Pursuit's role-swapping minima are retained rather than selecting favourable seeds.

Four original curated tasks were removed under the explicit removal allowance:

The agent-owned, cross-inhibiting Opposed Cycles replacement candidate also failed: its best fixed mean was 0.024, so it did not replace either Chemistry task. Removed notebooks and Python APIs remain directly runnable and no saved artifact was deleted.

Matrix Stag Hunt completed its two-seed screen. IPPO_CER produced p_hat = 1.0 for both agents on both seeds. PR2_IPPO_CER produced 0.0 on seed zero and 1.0 on seed one, so its per-agent aggregate is 0.5 but is not stable. NASHQ_CER also produced 1.0 for both agents on both seeds.

Its fixed 500-episode uniform-random diagnostic (environment seeds 0--499, action RNG seed 20260810, unchanged 20-round trace) gives 0.106 for each agent. Sustained Stag is therefore not a chance-level objective despite the shared-satisfaction exception recorded in assumptions.md.

The stable Stag IPPO_CER profile was promoted immediately to a three-seed, 1,000,000-step validation with 1,000 frozen-policy episodes per seed. Job 6635 runs on amd48 and writes to exports/o/stag-v/.

Validation job 6635 completed cleanly in 03:44:40. Both Stag agents remain at p_hat=1.0 in every one of the three 1,000,000-step seeds, with 1,000 frozen- policy episodes per seed. The per-agent marginal Hoeffding interval is [0.9571, 1.0]; the simultaneous Bonferroni-Hoeffding interval is [0.9532, 1.0]. Training took 13,380.57 seconds in aggregate and frozen satisfaction evaluation took 72.30 seconds. Stag is therefore fully validated, not merely screened.

The initial Pursuit seed-zero snapshot below showed at least one provisional profile above the gate for every task. Promotions use the later two-seed aggregates, not this snapshot alone:

A fixed uniform-random diagnostic closes the chance-baseline gap for these tasks. Each task uses environment seeds 0--499, action RNG seed 20260810, its unchanged 100-step trace, and 500 episodes. Per-agent probabilities are:

The final selected-agent means for the first three tasks are 0.651, 0.943, and 0.671, respectively, so all exceed both 0.5 and their same-role random baseline. Classic-3's +0.033 margin over its random guard is modest and is reported as such; its validation evidence is strengthened by the fact that every seed has some opposed role at p >= 0.835, not by pretending the guard is uniformly dominant. Capture-3's first validation profile improves on its random intruders but remains below 0.5 and therefore still fails.

Capture-3 is the clearest algorithm split: its IPPO profile credits both intruders at 0.938, while the corresponding PR2 profile fails the screen.

Classic-2 IPPO_CER completed with a best-agent mean of 0.792; both seeds pass individually (0.716 and 0.868). It was promoted to three seeds, 1,000,000 steps, and 1,000 frozen-policy episodes in job 6644, writing to exports/o/c2-v/. Validation completes at (0.993, 0.476), (0.185, 0.807), and (0.776, 0.132). The winning role swaps in seed 1, but the final fixed- agent means are (0.651, 0.472), so player 0 clears the gate. Its reported mean marginal Hoeffding interval is [0.6084, 0.6823] and simultaneous mean interval is [0.6045, 0.6849]; player 1's are [0.4287, 0.5146] and [0.4249, 0.5185]. Job 6644 completed cleanly in 05:38:35; aggregate training and satisfaction-evaluation times are 17,558.55 and 2,721.16 seconds. Classic-2 is fully validated.

Capture-2 IPPO_CER completed at a best per-agent mean of only 0.509. The successful role swaps across seeds: player 0 is 0.780 then 0.238, while player 1 is 0.220 then 0.762. This clears the aggregate point-estimate gate but is not promoted while the PR2 screen remains available as a potentially more stable choice.

Capture-2 PR2_IPPO_CER subsequently completed with player 1 at 0.992 and 0.860 across the two seeds (0.926 mean). That stable intruder result was promoted to a three-seed, 1,000,000-step validation with 1,000 evaluation episodes in job 6645, writing to exports/o/cap2-v/. Validation completes at (0.000, 1.000), (0.147, 0.853), and (0.770, 0.975). Fixed-agent means are (0.306, 0.943) and player 1 never falls below 0.853. Its reported mean marginal Hoeffding interval is [0.8997, 0.9653] and simultaneous mean interval is [0.8959, 0.9666]; player 0's corresponding intervals are [0.2770, 0.3486] and [0.2745, 0.3525]. Aggregate training and satisfaction-evaluation times are 17,874.16 and 6,180.83 seconds. Job 6645 completed cleanly in 06:41:18. Capture-2 is fully validated.

Both three-player screens also completed. Classic-3 PR2_IPPO_CER gives the guard a 0.998 mean, versus a best-agent 0.798 for IPPO_CER; PR2 was promoted in validation job 6646 (exports/o/cl3-v/). Capture-3 reverses that ordering: IPPO_CER gives each intruder a 0.792 mean, while PR2's best agent is only 0.369; IPPO was promoted in job 6647 (exports/o/ca3-v/). Both jobs use three seeds, 1,000,000 steps, and 1,000 frozen-policy episodes per seed. Classic-3 validation seeds 0 and 1 are (0.999, 0.000, 0.000) and (0.983, 0.098, 0.098). Capture-3 validation seeds 0 and 1 are (0.288, 0.712, 0.712) and (0.289, 0.697, 0.697).

Classic-3 job 6646 completes cleanly in 07:59:26; its third seed switches the successful role to the intruders at (0.030, 0.835, 0.835). Across all three predeclared seeds the fixed-agent means are (0.671, 0.311, 0.311), so the guard still clears the gate. Its reported mean marginal Hoeffding interval is [0.6320, 0.6910] and simultaneous mean interval is [0.6280, 0.6930]. Aggregate training and satisfaction-evaluation times are 27,894.64 and 835.12 seconds. Classic-3 is fully validated; the role switch is retained rather than selecting only the two guard-dominant seeds.

Capture-3 job 6647 then completed cleanly in 07:04:44, but its predeclared third seed collapsed to (0.001, 0.003, 0.003). The final fixed-agent means are consequently (0.193, 0.471, 0.471), so this profile fails the p >= 0.5 gate despite its first two seeds. The intruders' reported mean marginal Hoeffding interval is [0.4410, 0.5136] and their simultaneous mean interval is [0.4370, 0.5196]; neither establishes success. Aggregate training and satisfaction-evaluation times are 25,377.48 and 57.41 seconds.

Three robustness controls keep the same three training seeds, 1,000,000 steps, 1,000 frozen-policy episodes, unchanged temporal satisfaction evaluator, and IPPO_CER algorithm:

The first tests whether the constant 3e-4 update destabilises a useful late policy; the second isolates update size without changing the entropy bonus or rollout geometry. Both retain objective-mode training. The third changes only the training reward: the guard receives capture credit and each intruder receives collision-free terminal credit from the existing role-local helper. It neither shares success across opposing roles nor changes the frozen-policy formulas. Capture-3 remains unvalidated until a predeclared control clears the gate.

The first control seeds are encouraging but non-final. Learning-rate annealing gives (0.012, 0.988, 0.988), and fixed lr=1e-4 gives (0.216, 0.777, 0.777), while the handcrafted role reward gives (0.068, 0.930, 0.930). Every control's intruder result exceeds the 0.334 random baseline. Every remaining seed is retained because the original constant-rate profile failed only on its third predeclared seed.

Seed 1 separates the two learning-rate controls. Annealing collapses to (0, 0.001, 0.001), while fixed lr=1e-4 remains stable at (0.154, 0.806, 0.806). Both third seeds continue: seed 2 is mandatory because it was exactly where the original constant-rate validation collapsed. No two-seed snapshot is promoted as final evidence.

The aligned role-reward control is stable in seed 1 as well, at (0.100, 0.898, 0.898) after seed 0's (0.068, 0.930, 0.930). Its third seed has started. This is the strongest two-seed point estimate, but it remains provisional under the same seed-2 rule.

The fixed lr=1e-4 control completes seed 2 at (0.016, 0.981, 0.981) and finishes cleanly in 06:34:58. Across all three 1,000,000-step seeds its fixed-agent means are (0.129, 0.855, 0.855). Each intruder's reported mean marginal interval is [0.8117, 0.8896] and simultaneous interval is [0.8057, 0.8936]; both decisively exceed the 0.334 same-role random baseline. Training took 7,953.26, 7,783.01, and 7,859.70 seconds, while the corresponding 1,000-episode evaluations took only 17.92, 18.01, and 12.74 seconds. Capture-3 is therefore validated. The annealed and aligned-reward third seeds continue only to determine whether either is a preferable final default.

The annealed control then completes seed 2 at (0.059, 0.936, 0.936) and finishes cleanly in 06:43:14. Its fixed intruder mean is 0.642, with marginal interval [0.6127, 0.6743] and simultaneous interval [0.6087, 0.6783]; it formally passes. However, its seed-1 intruder result is only 0.001, versus a 0.777 minimum for fixed lr=1e-4. Annealing is therefore rejected as the final default on stability and aggregate performance, not because its point mean fails the gate.

The aligned role-reward control is stronger still. Seed 2 gives (0.178, 0.822, 0.822) and job 6764 completes cleanly in 06:36:50. Across all three seeds its fixed-agent means are (0.115, 0.883, 0.883), with intruder minimum 0.822, marginal interval [0.8404, 0.9263], and simultaneous interval [0.8344, 0.9323]. Training takes 8,089.89, 7,893.69, and 7,728.67 seconds; evaluations take 14.76, 14.67, and 12.66 seconds. This profile beats fixed lr=1e-4 in both mean (0.883 versus 0.855) and worst seed (0.822 versus 0.777). Formula-aligned handcrafted-reward IPPO is therefore the final Capture-3 selection; frozen evaluation continues to use the unchanged opposed guard/intruder formulas.

Capture_3_mo.py now defaults to reward_mode="handcrafted" so an IPPO-CER run reproduces the selected reward profile. The experiment layer continues to construct a separate objective-mode environment for frozen satisfaction and Nash checks, so this default does not change either agent's formula.

For Coins, a separate fixed-policy diagnostic sampled uniform random actions for 500 episodes using environment seeds 0--499, action RNG seed 20260810, the same 256-step horizon, and regrow_rate=0.005. It produced p_hat=0.484 for player 0 and 0.498 for player 1. Objective-IPPO job 6612 completed both seeds: seed 0 gave (0.498, 0.522) and seed 1 gave (0.524, 0.472), for means (0.511, 0.497). The apparent winner swaps with the seed and remains at the random baseline, so this variant is rejected rather than promoted. Slurm marks the job FAILED only because the old runner attempted to write an auxiliary Marimo snapshot with an overlong filename after the authoritative report and checkpoints were written; the full two-seed report is valid. Objective PR2 and the aligned handcrafted-reward pair were retained at this stage. Objective-PR2 job 6613 also completed: seed 0 gave (0.004, 0.378) and seed 1 gave (0.322, 0.000), for means (0.163, 0.189). It is both below the gate and strongly seed- dependent, so it is rejected. Its Slurm FAILED state has the same post-report auxiliary-filename cause as job 6612. The later one-shot, repeated, zero- entropy, and annealed global-observation controls all failed as documented above. Only the local-observation IPPO profile cleared its random baseline and gate, and it is now in long validation job 6768.

For each completed pair, record per-seed and mean p_hat, confidence intervals, wall time, and the highest agent probability. A pass only advances to long validation if the objective is also legitimate under the assumptions in assumptions.md.