Verification¶

Verification checks what a trained policy profile actually achieved. The main tools are final satisfaction evaluation, approximate NashConv, and AlphaRank.

The callable estimators and report writers are documented in the Verification API.

Final Satisfaction¶

Final satisfaction estimates each agent's probability of satisfying its temporal objective under the learned joint policy.

The estimator:

1. Runs fresh evaluation episodes.
2. Records a success bit per agent and episode.
3. Computes p_hat = successes / episodes.
4. Reports both a marginal Hoeffding interval and a Bonferroni-Hoeffding interval with simultaneous 1 - delta coverage across all agents.

Reports are saved as:

satisfaction.npy
satisfaction_report.json

Important fields include:

• eval_episodes
• delta
• confidence
• joint_confidence
• success_threshold
• per_agent

Use final satisfaction to confirm whether a learned profile actually solves the temporal objective.

Approximate NashConv¶

Approximate NashConv is the closest model-free analogue of unilateral deviation incentive used in this repo.

For each evaluated agent:

1. Freeze all other agents at the learned policy.
2. Train a best-response or exploiter policy for the selected agent.
3. Estimate baseline objective success under the original profile.
4. Estimate best-response objective success under the deviating profile.
5. Measure the positive improvement.

The aggregate NashConv score summarizes how much agents can improve by unilateral deviation. Lower is better, but interpretation depends on the baseline satisfaction.

calculate_conv_quick=True evaluates only the first trained run. Full mode evaluates every run. In both modes the best response is a finite-budget learned candidate, not an exact optimizer.

Reports are saved as:

nash.npy
nash_report.json

The JSON report includes aggregate summary statistics and per-agent baseline, best-response, and delta values. delta_joint_ci simultaneously covers the rollout-estimated deviation improvement for every agent, and nashconv_joint_ci propagates those bounds through the aggregate score. The overall failure probability is configured independently through nashconv_delta, which defaults to 0.05.

These joint bounds cover Monte Carlo error for the learned candidate best responses. They do not certify an epsilon-Nash equilibrium against all possible deviations because best-response training itself remains approximate.

When nashconv_auto_resume=True, a run above nashconv_auto_resume_threshold can receive one continuation-training pass, capped by nashconv_auto_resume_step_fraction of the original training budget, followed by another NashConv evaluation.

AlphaRank¶

AlphaRank is an optional ranking analysis over learned policies. Each completed run contributes one strategy per player, and the verifier evaluates the full cross-play payoff tensor before ranking joint profiles. With R runs and N agents, each agent payoff table has R**N cells, so the evaluation cost grows quickly.

Reports are saved as:

alpharank.npy
alpharank_report.json

AlphaRank requires OpenSpiel's Python AlphaRank implementation. The dependency is imported lazily: ordinary training works without it, while an enabled AlphaRank stage fails before cross-play rollout construction if it is missing. The stage is off by default in most notebooks because it can add substantial evaluation cost.

Interpreting Results¶

Read metrics together:

• High satisfaction and low NashConv is the strongest positive signal.
• Low satisfaction and low NashConv can mean the best-response search also failed.
• High satisfaction and high NashConv means the objective can be reached, but the profile may be exploitable.
• AlphaRank helps compare strategy sets, but it does not replace per-agent objective satisfaction.

For sparse temporal goals, increase final satisfaction episodes or inspect label traces before drawing conclusions.

Success Thresholds¶

The experiment spec includes success_threshold, defaulting to 0.5. Rollout verifiers use this threshold when reducing objective reward histories to success outcomes. Most current temporal wrappers produce terminal 0.0 or 1.0 objective rewards, so the default threshold is natural.