docs: distinguish M0a performance evidence lanes

This commit is contained in:
Kinneyzhang 2026-08-31 12:14:14 +08:00
parent bf22251d40
commit 51a00b3ee5
4 changed files with 101 additions and 4 deletions

View File

@ -186,8 +186,11 @@ triplet.
- Pressure fixture: `etaf-research-shelf-fixture-size` defaults to 256 and
`etaf-research-shelf-page-size` defaults to 12; existing local rows are
preserved and missing fixture rows are topped up with fresh IDs.
- Performance: one Data mutation -> one Runtime generation/publication; normal
warm actions remain under the accepted 105ms p50 target.
- Performance: one Data mutation -> one Runtime generation/publication. The
current active latency lane uses 5 unmeasured warmups and 30 measured samples
at fixed 1413×62 geometry; every scenario must satisfy p95 and max at or below
50ms. The accepted 105ms p50 wording is historical design context, not the
current evaluator gate; `DOC-PERF-001` keeps its formal disposition in M0b1.
- Compatibility: any same-basename `.etaf`/`.el` example is discoverable;
`.ecss` is optional, and the bundled Research Shelf remains only one
consumer.
@ -195,6 +198,18 @@ triplet.
refresh tests, SQLite temp-file integration tests, mount/remount tests,
repeated selection/mutation tests, and clean GUI screenshots.
## Performance evidence lanes
The latency lane, trace lane, and GUI lane share one
repository/environment/scenario/fixture/build identity but remain separate
measurements. The latency lane owns the uninstrumented 5/30 p95/max 50ms gate.
The trace lane owns a representative instrumented invocation with cost class,
work counters, turns, allocation, and GC. The GUI lane owns the real Emacs
action sequence, screenshots, recording, and temporal-review verdict. Batch
verifier duration is not GUI first paint; action-start through forced redisplay
completion remains an explicitly separate future first-paint gate and an M0a
observed gap. No one lane may be used to claim another lane passed.
## Open questions
- [ ] User may rename the product after reviewing the first HTML/ETAF render;

View File

@ -108,12 +108,24 @@ ETAF 私有 registry 变量Component setup 中创建的 Controller 自动归
- 用 inert reader、source tab/session、未保存 source refresh 和 SQLite 临时文件
测试 mount/remount、筛选、分页、重复选行、mutation、错误状态和 cleanupGUI
用干净 fullscreen 截图验证真实布局。
- 目标:一次 Data mutation 对应一次 Runtime generation/publicationwarm 交互
维持已接受的 105ms p50 预算。
- 目标:一次 Data mutation 对应一次 Runtime generation/publication。当前有效的
延迟 lane 在固定 1413×62 几何下执行 5 次未计时 warmup 与 30 个计时 sample
每个场景的 p95 和 max 均须不超过 50ms。已接受的 105ms p50 是历史设计上下文,
不是当前 evaluator gate其正式处置仍由 M0b1 的 `DOC-PERF-001` 负责。
- 压测入口:`etaf-research-shelf-fixture-size` 默认 256
`etaf-research-shelf-page-size` 默认 12已有本地记录保留不足部分使用新 ID
补齐 fixture。
## 性能 evidence lanes
延迟 lane、追踪 lane 与 GUI lane 共享同一组仓库、环境、场景、fixture 和 build identity
但保持为三种独立测量。延迟 lane 负责无 instrumentation 的 5/30、p95/max 50ms
门禁;追踪 lane 负责一次代表性 instrumented invocation 的 cost class、work counters、
turns、allocation 与 GCGUI lane 负责真实 Emacs action sequence、截图、录像与
temporal-review verdict。Batch verifier duration 不是 GUI first paint从 action-start
到 forced redisplay completion 的 first-paint 计时仍是独立 future gate也是 M0a
明确记录的 observed gap。任何一条 lane 都不能替另一条 lane 宣称通过。
## 待 review
- [ ] 用户可调整产品命名或 palettepair 边界、组合/复用原则和公开交互合同不变。

View File

@ -110,6 +110,26 @@ For repeated absolute-latency runs, use `make perf-prepare` once after source
changes, let compilation activity settle, then run `make perf-evaluator` without
rebuilding the dependency graph.
## Performance evidence lanes
Performance evidence has three separate lanes correlated by one
repository/environment/scenario/fixture/build identity:
- The latency lane runs the fixed 1413×62 batch evaluator with 5 unmeasured
warmups and 30 measured samples. The current active gate requires both p95
and max to be at or below 50ms for every scenario.
- The trace lane runs a separate instrumented representative invocation. Its
cost classes, work counters, turns, allocation, and GC data are not inserted
into the timed latency distribution.
- The GUI lane captures the real Emacs interaction sequence, screenshots, and
temporal-review verdict. Batch verifier duration is not GUI first paint;
action-start to forced-redisplay-complete first-paint timing remains a
separately identified future gate and an observed M0a gap.
Evidence from one lane supports only that lane's conclusion. A
performance-complete or user-visible-non-regression claim requires the
applicable gates from all three lanes.
The bundled Research Shelf example installs a deterministic 256-record SQLite
fixture with 12 records per page. Bind `etaf-research-shelf-fixture-size` and
`etaf-research-shelf-page-size` for smaller tests or larger pressure runs. Its

View File

@ -0,0 +1,50 @@
;;; etaf-playground-docs-tests.el --- Documentation contracts -*- lexical-binding: t; -*-
;;; Code:
(require 'ert)
(defconst etaf-playground-docs-tests--root
(file-name-directory
(directory-file-name
(file-name-directory (or load-file-name buffer-file-name))))
"Absolute etaf-playground repository root.")
(defun etaf-playground-docs-tests--contents (name)
"Return the literal contents of repository file NAME."
(with-temp-buffer
(insert-file-contents
(expand-file-name name etaf-playground-docs-tests--root))
(buffer-string)))
(defun etaf-playground-docs-tests--normalized-contents (name)
"Return NAME with runs of whitespace collapsed for prose assertions."
(replace-regexp-in-string
"[[:space:]]+" " " (etaf-playground-docs-tests--contents name)))
(ert-deftest etaf-performance-docs-distinguish-evidence-lanes ()
"Latency, trace, and GUI evidence remain explicitly separate lanes."
(dolist (file '("README.md" "DESIGN.md" "DESIGN.zh-CN.md"))
(let ((text (etaf-playground-docs-tests--normalized-contents file)))
(should (string-match-p "latency lane\\|延迟 lane" text))
(should (string-match-p "trace lane\\|追踪 lane" text))
(should (string-match-p "GUI lane" text))
(should (string-match-p
"repository/environment/scenario/fixture/build identity\\|仓库、环境、场景、fixture 和 build identity"
text))
(should (string-match-p
"not GUI first paint\\|不是 GUI first paint"
text)))))
(ert-deftest etaf-performance-docs-separate-current-gate-from-historical-budget ()
"The active 50ms gate must not be confused with the historical 105ms text."
(dolist (file '("DESIGN.md" "DESIGN.zh-CN.md"))
(let ((text (etaf-playground-docs-tests--normalized-contents file)))
(should (string-match-p "5.*warmup.*30.*sample\\|5.*warmup.*30.*sample" text))
(should (string-match-p "p95.*max.*50ms\\|p95.*max.*50ms" text))
(should (string-match-p "105ms p50" text))
(should (string-match-p "historical\\|历史" text))
(should (string-match-p "M0b1" text)))))
(provide 'etaf-playground-docs-tests)
;;; etaf-playground-docs-tests.el ends here