Skip to content

Verification

Verification checks what a trained policy profile actually achieved. The main tools are final satisfaction evaluation, approximate NashConv, and AlphaRank.

The callable estimators and report writers are documented in the Verification API.

Final Satisfaction

Final satisfaction estimates each agent's probability of satisfying its temporal objective under the learned joint policy.

The estimator:

  1. Runs fresh evaluation episodes.
  2. Records a success bit per agent and episode.
  3. Computes p_hat = successes / episodes.
  4. Reports a Hoeffding confidence interval.

Reports are saved as:

satisfaction.npy
satisfaction_report.json

Important fields include:

  • eval_episodes
  • delta
  • confidence
  • success_threshold
  • per_agent

Use final satisfaction to confirm whether a learned profile actually solves the temporal objective.

Approximate NashConv

Approximate NashConv is the closest model-free analogue of unilateral deviation incentive used in this repo.

For each evaluated agent:

  1. Freeze all other agents at the learned policy.
  2. Train a best-response or exploiter policy for the selected agent.
  3. Estimate baseline objective success under the original profile.
  4. Estimate best-response objective success under the deviating profile.
  5. Measure the positive improvement.

The aggregate NashConv score summarizes how much agents can improve by unilateral deviation. Lower is better, but interpretation depends on the baseline satisfaction.

Reports are saved as:

nash.npy
nash_report.json

The JSON report includes aggregate summary statistics and per-agent baseline, best-response, and delta values.

AlphaRank

AlphaRank is an optional ranking analysis over learned policies. In this repo it is used as a cross-play style comparison tool after the strategy set has been produced.

Reports are saved as:

alpharank.npy
alpharank_report.json

AlphaRank is off by default in most notebooks because it can add evaluation cost.

Interpreting Results

Read metrics together:

  • High satisfaction and low NashConv is the strongest positive signal.
  • Low satisfaction and low NashConv can mean the best-response search also failed.
  • High satisfaction and high NashConv means the objective can be reached, but the profile may be exploitable.
  • AlphaRank helps compare strategy sets, but it does not replace per-agent objective satisfaction.

For sparse temporal goals, increase final satisfaction episodes or inspect label traces before drawing conclusions.

Success Thresholds

The experiment spec includes success_threshold, defaulting to 0.5. Rollout verifiers use this threshold when reducing objective reward histories to success outcomes. Most current temporal wrappers produce terminal 0.0 or 1.0 objective rewards, so the default threshold is natural.