Skip to content

Visualisation Package

The LTLf plotting code now lives in src/visualisation as the canonical import location. In-repo notebooks should import:

  • from src.visualisation import plot_ewma_success_all_agents
  • from src.visualisation import plot_suite_satisfaction_comparisons
  • from src.visualisation.ltlf_success_utils import satisfaction_utility_by_run

notebooks/visualisation has been removed. Older notebook code must be updated to import from src.visualisation directly.

What Changed

The migration intentionally changes presentation, not semantics:

  • EWMA success computation, run alignment, confidence bands, and utility aggregation are unchanged.
  • The plots now use the rl-engine paper-style visual language: light panel background, dashed grid, lighter spines, and figure-level legend styling.
  • Plot exports use a 5.0 x 3.0 inch Matplotlib canvas with moderate paper fonts. This keeps labels legible after LaTeX scaling while giving the plotted data more of the available area.
  • Single-panel plot helpers accept paper_panel=True for source images that will be shrunk into dense paper grids. That preset uses a compact 3.2 x 2.1 inch canvas with larger relative axis labels, tick labels, offset text, grid lines, and spines. Learning curves use a 2.0-point default stroke and a 2.1-point paper-panel stroke. It does not change default filenames; callers should save explicit alternates such as *_paper.png.
  • Suite notebooks and the generic regraph path use EWMA alpha=0.002, giving slightly smoother comparison curves than the previous 0.0025 setting.
  • The legend is placed at figure level above the axis, with layout space reserved after legend measurement so wrapped legends do not overlap axis titles.
  • PDF and PostScript exports embed TrueType fonts. Axes rasterize artists below z-order 1.9, keeping low-layer backgrounds compact while plotted learning curves remain vector graphics.
  • Translucent shared-legend handles are composited against the figure background before export so their apparent colors remain stable in vector viewers without preserving alpha.
  • When a plot helper is given save_path, the base filename is saved without an inline legend and a second *_legend.png file is saved with the measured figure-level legend. The returned/displayed figure keeps the legend so notebooks remain readable.
  • Suite comparison plots and regraph per-algorithm exports do not set default titles; the notebook artifacts rely on filenames, legends, and paper captions for the best-variant, all-variant, and per-algorithm distinctions.

Suite-Level Best-Variant Plot

plot_best_ewma_success_by_algorithm(...) is the shared lower-level comparison plotter for best-variant suite panels.

  • By default it averages the EWMA P(satisfy) curves across agents within each run first.
  • Passing reduce_agents="max" instead plots the per-episode best agent satisfaction within each run.
  • Passing agent="player_0" or another agent id plots only that agent.
  • It then compares the base and _CER variants inside each algorithm family.
  • The winner is chosen by the mean final plotted EWMA value across runs, using the same agent reducer as the displayed curve.
  • Ties stay with the non-CER variant, so CER only appears when it genuinely outperforms the base version.
  • Base and CER variants share a learner-family colour. Base variants use solid lines and CER variants use dashed lines; algorithm curves have no markers.

Display labels are humanised for suite comparisons:

  • IPPO_CER -> IPPO-CER
  • PR2_IPPO_CER -> PR2-IPPO-CER
  • NASHQ -> Nash-Q Learning
  • NASHQ_CER -> Nash-Q Learning (CER)

Suite-Level All-Variant Plot

plot_all_ewma_success_by_algorithm(...) is the companion comparison plotter that shows every selected base and CER variant together.

  • It averages EWMA satisfaction across agents within each run, matching the best-variant plot's aggregation semantics, and accepts the same reduce_agents and agent options.
  • Base and CER variants are both shown when present.
  • Base and CER variants share a learner-family colour. CER variants use dashed lines and base variants use solid lines; no algorithm marker shapes are used.

Current experiment notebooks save per-algorithm plots plus suite-root comparison artifacts, each with matching *_legend.png variants:

  • objective_satisfaction_by_agent.png and objective_satisfaction_by_agent_legend.png in each algorithm directory.
  • avg_agent_best_algos.png and avg_agent_all_algos.png in the suite root.
  • best_agent_best_algos.png and best_agent_all_algos.png in the suite root.
  • summary/agent_1_best_algos.png, summary/agent_1_all_algos.png, etc. for sorted agents, using 1-indexed filenames.

When the campaign launcher runs one isolated job per algorithm--environment pair, the notebook process contains only that job's algorithm. Its suite-root and summary/ comparison files are therefore partial views, and later jobs overwrite the same filenames. In particular, an all_algos filename written by an isolated job does not imply that all sibling algorithm directories were loaded. After the full campaign finishes, regenerate the canonical comparison files with notebooks/generic/regraph_mo.py (or the corresponding notebooks.utils.regraph.plot_suite entry point), which discovers and loads all completed sibling runs.npy files together. Per-algorithm reports and arrays are unaffected by this overwrite behavior.

Every suite comparison also has a *_no_hc.png sibling, plus its *_no_hc_legend.png export. These variants omit the Best Handcrafted endpoint marker while keeping the same objective-trained curves, algorithm selection, agent reduction, axes, and styling. The public best/all plotters expose this behavior through include_handcrafted=False. Handcrafted runs are not drawn as individual curves or included in best-family selection. The files remain beside their corresponding comparison rather than moving into a separate no_hc/ tree, so existing suite-root and summary/ discovery keeps working and the filename alone identifies the variant.

Every full-scale comparison also has a *_tightened.png sibling (and matching legend export). Tightened views choose a probability range from the displayed mean curves, always retain 0.5 as a reference, round outward to 0.05, and stay within [0,1]. The unsuffixed export retains the full [0,1] y-axis and remains available whenever the complete confidence-band extent is required.

plot_suite_satisfaction_comparisons(...) is the notebook-facing helper that writes the suite-root and summary/ comparison artifacts. It also removes only the previous best_average_objective_satisfaction*.png and all_average_objective_satisfaction*.png files during regeneration so stale renamed artifacts do not survive in an export directory. When passed paper_panel=True, it forwards the paper-panel preset to every generated satisfaction plot while preserving the same artifact names.

The paper's learning-curve PNGs use this preset, but neither the training notebooks (just_vis=True) nor regraph.plot_suite enable it, so their outputs do not match the paper. To regenerate paper figures, copy the suite without checkpoints/, load it with load_suite_algorithm_runs, and call plot_suite_satisfaction_comparisons(runs, suite.directory, alpha=0.002, x_axis="episode", align="truncate", band="ci95", show_seed_traces=False, paper_panel=True) with MPLBACKEND=Agg. Checked on 2026-09-28: this reproduced the paper's Coins PNGs byte for byte, and on 2026-09-30 the Pursuit Classic 2 PNGs. scripts/regenerate_paper_figures.py automates this: it mirrors each suite's runs.npy and satisfaction_report.json as symlinks under --output and plots each suite in its own subprocess (Pursuit histories exceed 3 GB); run it as a cluster job with at least 16 GB of memory.

The paper's main learning-curve figure uses each suite's best_agent_best_algos_tightened.png (best agent per episode, stronger of the base and -CER variant per learner family; switched from best_agent_all_algos on 2026-10-04), and its per-agent appendix figures use summary/agent_<k>_{best,all}_algos_tightened.png. Paper directories differ from suite paths (gridworlds/coins/own_coin_first/2 maps to figures/experiments/coins/own_coin_first_2, and similarly for Stag Hunt, Harvest and Territory; Pursuit maps to pursuit/<task>).

Since 2026-10-02 the script also passes display_smoothing=0.03 (--display-smoothing, PAPER_DISPLAY_SMOOTHING), so the paper panels no longer match the plain recipe above. The comparison and per-agent plotters accept display_smoothing (default 0, off): a centred moving average (centered_moving_average in ltlf_success_common.py) over that fraction of the panel's x-span, applied to the plotted mean, band and seed traces. Reasons and choices:

  • The EWMA's effective window (1/alpha, ~500 episodes) spans most of the 200-1000-episode Coins, Harvest and Territory ranges but under 1% of the 50k-150k-episode Pursuit ranges, where the curves looked jagged and dashed strokes broke up. A fraction of the span gives every panel the same on-screen smoothness; on the short panels it is a few episodes and changes nothing visible.
  • The window is shared by all curves in a panel (computed from the longest grid), so a short run is not smoothed less than a long one.
  • It is centred and shrinks symmetrically near the ends: no lag, and the last plotted value stays the EWMA endpoint the handcrafted marker is compared against. The last ~1.5% of a curve is therefore less smoothed.
  • Variant selection (select_best_algorithm_variants) and handcrafted markers still use the raw EWMA values; only the drawn lines change.
  • Alternative: a smaller alpha would smooth more but lags further and would flatten the short Coins/Territory curves into their initial values.

objective_satisfaction_by_agent.png is per algorithm, not a comparison across algorithms: it draws one EWMA satisfaction curve per agent using that algorithm's recorded run histories. In matrix Stag Hunt the temporal objective is shared symmetrically by both players, so player_0 and player_1 can have identical success histories and their per-agent curves may overlap exactly. Pursuit objectives are role-specific, so guard and intruder curves usually separate.

Current persisted arrays and reports avoid repeating the agent count in the filename because the suite directory already encodes it:

  • runs.npy
  • nash.npy
  • nash_report.json
  • optional alpharank.npy
  • optional alpharank_report.json

notebooks/generic/regraph_mo.py uses notebooks/utils/regraph.py to discover available suites by scanning exports/**/runs.npy, reload their algorithm histories, and regenerate the same plot artifacts.

The suite dropdown intentionally starts with no selected value. This keeps the notebook from regenerating plot artifacts as soon as it opens; plotting and the satisfaction summary cells are stopped with mo.stop() until the user selects a suite.

Axis Formatting

plot_ewma_success_all_agents() formats the x-axis by mode:

  • x_axis="episode" keeps plain episode numbering with no scientific notation.
  • x_axis="t_end" applies a scientific step-style formatter for larger timestep values.
  • Both modes cap the major x-axis tick count so labels do not collide after the generated PNGs are scaled down in the paper.

That split is deliberate: the rl-engine reference style uses step formatting, but applying it to episode counts made the probability plots harder to read.