# Reproducible benchmark guide

This document describes the benchmark runner shipped with TP 1.0. It measures
the current retained/reactive runtime; it is not a historical TP 0.3 stack
benchmark and it does not impose a release threshold.

## Run

Use the Makefile entry point:

~~~sh
make benchmark
~~~

The equivalent batch command is:

~~~sh
emacs -Q --batch -L . -l tp-benchmark.el -f tp-benchmark-run
~~~

The runner uses fixed seeds 1, 7, 42, 747555 and generated seed 8675309. Each
scenario performs a correctness assertion before timing the operation. Record
the Emacs version, machine, seed, and full output when comparing runs.

## Scenarios

For every seed, the runner executes:

| Scenario | Fixtures |
| --- | --- |
| large-text | strings of 100,000 and 1,000,000 characters; set and presence-aware search |
| fragmented | 1,000, 10,000, and 50,000 alternating property intervals |
| retained-keyed-reconcile | retained content with 10, 100, and 1,000 stable keyed entries |
| signal-sparse-update | one target binding beside 1, 100, and 10,000 unrelated bindings |
| transaction-batch | 1, 100, and 10,000 writes to one retained surface |
| equal-write-noop | the same write counts, all equal to the committed value |

The retained scenarios verify stable object reconciliation, correct published
text, dependency-local recomputation, one publication for a batch, and no
revision change for equal writes. A failed assertion aborts the run instead of
producing misleading timing evidence.

## Output

Each row is a whitespace-separated key/value record. The stable fields are:

scenario, status, fixture, seed, requested, actual, operations, objects,
subscribers, invalidated, recomputed, skipped, text-operations,
property-operations, touched, revision, published, elapsed, gcs, and note.

The output is deliberately machine-readable enough for local comparison, but
it is not a compatibility format. Interpret it together with the scenario
source in tp-benchmark.el.

## Interpretation

These measurements are advisory. Runtime, garbage collection, Emacs build,
machine load, and buffer implementation details affect absolute timings.
Compare like-for-like runs, inspect correctness failures first, and use the
reports/counters to explain a regression:

- fragmented measures expose interval-run scaling;
- retained-keyed-reconcile measures keyed object reuse and publication work;
- signal-sparse-update checks that unrelated bindings are not recomputed;
- transaction-batch measures deduplicated recomputation and one surface commit;
- equal-write-noop checks that equal values do not publish a new revision.

For the contracts behind these scenarios, read [API semantics](API-SEMANTICS.md),
[architecture](ARCHITECTURE.md), and the [public API reference](API-REFERENCE.md).