Skip to content

NashConv Auto-Resume

Temporal-game experiment configs support an opt-in NashConv retry pass:

  • nashconv_auto_resume: enables the retry when calculate_conv is also true.
  • nashconv_auto_resume_threshold: convergence threshold; a run retries when nashconv > threshold.
  • nashconv_auto_resume_step_fraction: maximum extra training budget as a fraction of the original resolved TOTAL_TIMESTEPS.

The library default disables auto-resume; its default threshold is 0.05 and its default budget fraction is 0.25. Runnable notebooks explicitly enable the pass with those threshold and fraction values.

The implementation lives in src/experiments/temporal_game_experiment.py, so Pursuit, Coins, Chemistry, Harvest, Territory, Gift Refinements, and matrix experiment wrappers inherit the same behavior. Only runs that actually receive NashConv evaluation are eligible. In quick mode this means only run 0 can retry; in full mode each evaluated run is checked independently.

When a run exceeds the threshold, the runner computes the largest whole number of trainer updates whose step count is no more than floor(original_total_timesteps * step_fraction). If that is less than one update, the retry is skipped and logged. Otherwise, the trainer is called once with initial_train_states from the first pass, so policy/value state truly continues, while replay buffers and environment instances are fresh for the retry pass.

Resume trainer configs include:

  • TOTAL_TIMESTEPS: the extra retry steps only.
  • SCHEDULE_UPDATE_OFFSET: the original update count.
  • SCHEDULE_TOTAL_UPDATES: original plus retry update count.

The offset/total fields let epsilon and learning-rate schedules continue on the extended training timeline instead of restarting at update 0. The final resumed checkpoint overwrites the original run checkpoint with the post-resume train state and records AUTO_RESUME_EXTRA_TIMESTEPS in the checkpoint config.

The experiment result stores only the final NashConv payload in nash_runs. The before/after trail is available in auto_resume_events, with run seed, initial/final NashConv, threshold, configured fraction, actual extra steps and updates, and final convergence status. Timings use the existing nashconv stage for the initial check plus *_auto_resume stages for retry training, checkpoint save, and post-resume NashConv.