Skip to content

Test Suites and Runtime

The 2026-08-08 profiling baseline covered the then-current 266-test tree: 256 routine tests and 10 slow JAX-based NashConv, satisfaction, and AlphaRank integration checks. The full run completed in 6m56s on the development workstation; a parallel verification matching the CI layout completed the then-current fast shard in 3m46s and the slow shard in 2m10s.

The current tree collects 295 tests: 285 routine tests and the same 10-test slow tier. Coins, native-object-observation, notebook, and launcher coverage was added after the timing baseline, so the historical durations should be treated as planning estimates rather than current benchmarks. CI can still execute the two shards in parallel without dropping coverage.

Use the repository runner during development:

# Select non-slow tests from tracked and untracked working-tree changes.
uv run python scripts/run_tests.py changed

# Broad non-slow regression suite.
uv run python scripts/run_tests.py fast

# One component, excluding its slow tests by default.
uv run python scripts/run_tests.py verification

# Include that component's expensive integrations.
uv run python scripts/run_tests.py verification --include-slow

# Expensive tier only, or every test.
uv run python scripts/run_tests.py slow
uv run python scripts/run_tests.py full

changed compares the working tree to HEAD. Pass --base origin/main to include committed branch changes, or repeat --changed-file PATH to inspect an explicit set. Arguments after -- are forwarded to pytest.

Change-to-suite map

Every tests/test_*.py module has one primary marker in tests/suite_manifest.py. Collection fails if a new test module has not been classified. Cross-cutting implementation paths can select several components.

Changed code Routine suite Include slow when
src/environments/ env environment behavior changes affect training integrations
pursuit environment code env, rl, verification usually; its integration coverage lives in learner and verification tests
matrix environment code env, experiments experiment construction or training changes
src/monitor/ env, monitor monitor state or wrapper behavior changes
src/monitor/counterfactual.py env, monitor, rl replay enumeration or wrapper contracts change
src/rl/ rl, verification trainer construction, updates, replay loops, or evaluator integration changes
src/verify/ experiments, verification rollout evaluation, NashConv, AlphaRank, confidence, or driver integration changes
src/experiments/ experiments orchestration or artifact persistence changes
pursuit and temporal-game experiment drivers experiments, verification verification stages or real trainer calls change
environment-specific experiment adapters env, experiments wrapper construction or dispatch changes
src/visualisation/ visualisation plot exports or presentation behavior changes
notebooks/ experiments notebook configuration or launcher behavior changes
notebooks/utils/regraph.py experiments, visualisation regraph discovery or export behavior changes
scripts/run_tests.py exact runner test never
a tests/test_*.py file that exact file the edited test has the slow marker
dependency, pytest, or unknown implementation configuration fast use full if integrations may change
documentation only no pytest selection run uv run zensical build --strict

The component names are env, monitor, rl, verification, experiments, visualisation, and tooling. They describe primary ownership rather than dependency isolation. Use fast for broad or uncertain changes.

Runtime policy

The slow marker is reserved for tests measured at roughly ten seconds or longer. Slow tests receive a 240-second timeout; normal tests retain the 60-second project default. Re-profile before changing the manifest:

uv run python -m pytest -q --durations=50

JAX's persistent compilation cache is stored under .cache/jax-test-compilation, which is already ignored. Cache keys include the compiled program and configuration, so implementation changes create new entries naturally.