docs: distinguish M0a performance evidence lanes
This commit is contained in:
parent
bf22251d40
commit
51a00b3ee5
19
DESIGN.md
19
DESIGN.md
@ -186,8 +186,11 @@ triplet.
|
||||
- Pressure fixture: `etaf-research-shelf-fixture-size` defaults to 256 and
|
||||
`etaf-research-shelf-page-size` defaults to 12; existing local rows are
|
||||
preserved and missing fixture rows are topped up with fresh IDs.
|
||||
- Performance: one Data mutation -> one Runtime generation/publication; normal
|
||||
warm actions remain under the accepted 105ms p50 target.
|
||||
- Performance: one Data mutation -> one Runtime generation/publication. The
|
||||
current active latency lane uses 5 unmeasured warmups and 30 measured samples
|
||||
at fixed 1413×62 geometry; every scenario must satisfy p95 and max at or below
|
||||
50ms. The accepted 105ms p50 wording is historical design context, not the
|
||||
current evaluator gate; `DOC-PERF-001` keeps its formal disposition in M0b1.
|
||||
- Compatibility: any same-basename `.etaf`/`.el` example is discoverable;
|
||||
`.ecss` is optional, and the bundled Research Shelf remains only one
|
||||
consumer.
|
||||
@ -195,6 +198,18 @@ triplet.
|
||||
refresh tests, SQLite temp-file integration tests, mount/remount tests,
|
||||
repeated selection/mutation tests, and clean GUI screenshots.
|
||||
|
||||
## Performance evidence lanes
|
||||
|
||||
The latency lane, trace lane, and GUI lane share one
|
||||
repository/environment/scenario/fixture/build identity but remain separate
|
||||
measurements. The latency lane owns the uninstrumented 5/30 p95/max 50ms gate.
|
||||
The trace lane owns a representative instrumented invocation with cost class,
|
||||
work counters, turns, allocation, and GC. The GUI lane owns the real Emacs
|
||||
action sequence, screenshots, recording, and temporal-review verdict. Batch
|
||||
verifier duration is not GUI first paint; action-start through forced redisplay
|
||||
completion remains an explicitly separate future first-paint gate and an M0a
|
||||
observed gap. No one lane may be used to claim another lane passed.
|
||||
|
||||
## Open questions
|
||||
|
||||
- [ ] User may rename the product after reviewing the first HTML/ETAF render;
|
||||
|
||||
@ -108,12 +108,24 @@ ETAF 私有 registry 变量;Component setup 中创建的 Controller 自动归
|
||||
- 用 inert reader、source tab/session、未保存 source refresh 和 SQLite 临时文件
|
||||
测试 mount/remount、筛选、分页、重复选行、mutation、错误状态和 cleanup;GUI
|
||||
用干净 fullscreen 截图验证真实布局。
|
||||
- 目标:一次 Data mutation 对应一次 Runtime generation/publication;warm 交互
|
||||
维持已接受的 105ms p50 预算。
|
||||
- 目标:一次 Data mutation 对应一次 Runtime generation/publication。当前有效的
|
||||
延迟 lane 在固定 1413×62 几何下执行 5 次未计时 warmup 与 30 个计时 sample,
|
||||
每个场景的 p95 和 max 均须不超过 50ms。已接受的 105ms p50 是历史设计上下文,
|
||||
不是当前 evaluator gate;其正式处置仍由 M0b1 的 `DOC-PERF-001` 负责。
|
||||
- 压测入口:`etaf-research-shelf-fixture-size` 默认 256,
|
||||
`etaf-research-shelf-page-size` 默认 12;已有本地记录保留,不足部分使用新 ID
|
||||
补齐 fixture。
|
||||
|
||||
## 性能 evidence lanes
|
||||
|
||||
延迟 lane、追踪 lane 与 GUI lane 共享同一组仓库、环境、场景、fixture 和 build identity,
|
||||
但保持为三种独立测量。延迟 lane 负责无 instrumentation 的 5/30、p95/max 50ms
|
||||
门禁;追踪 lane 负责一次代表性 instrumented invocation 的 cost class、work counters、
|
||||
turns、allocation 与 GC;GUI lane 负责真实 Emacs action sequence、截图、录像与
|
||||
temporal-review verdict。Batch verifier duration 不是 GUI first paint;从 action-start
|
||||
到 forced redisplay completion 的 first-paint 计时仍是独立 future gate,也是 M0a
|
||||
明确记录的 observed gap。任何一条 lane 都不能替另一条 lane 宣称通过。
|
||||
|
||||
## 待 review
|
||||
|
||||
- [ ] 用户可调整产品命名或 palette;pair 边界、组合/复用原则和公开交互合同不变。
|
||||
|
||||
20
README.md
20
README.md
@ -110,6 +110,26 @@ For repeated absolute-latency runs, use `make perf-prepare` once after source
|
||||
changes, let compilation activity settle, then run `make perf-evaluator` without
|
||||
rebuilding the dependency graph.
|
||||
|
||||
## Performance evidence lanes
|
||||
|
||||
Performance evidence has three separate lanes correlated by one
|
||||
repository/environment/scenario/fixture/build identity:
|
||||
|
||||
- The latency lane runs the fixed 1413×62 batch evaluator with 5 unmeasured
|
||||
warmups and 30 measured samples. The current active gate requires both p95
|
||||
and max to be at or below 50ms for every scenario.
|
||||
- The trace lane runs a separate instrumented representative invocation. Its
|
||||
cost classes, work counters, turns, allocation, and GC data are not inserted
|
||||
into the timed latency distribution.
|
||||
- The GUI lane captures the real Emacs interaction sequence, screenshots, and
|
||||
temporal-review verdict. Batch verifier duration is not GUI first paint;
|
||||
action-start to forced-redisplay-complete first-paint timing remains a
|
||||
separately identified future gate and an observed M0a gap.
|
||||
|
||||
Evidence from one lane supports only that lane's conclusion. A
|
||||
performance-complete or user-visible-non-regression claim requires the
|
||||
applicable gates from all three lanes.
|
||||
|
||||
The bundled Research Shelf example installs a deterministic 256-record SQLite
|
||||
fixture with 12 records per page. Bind `etaf-research-shelf-fixture-size` and
|
||||
`etaf-research-shelf-page-size` for smaller tests or larger pressure runs. Its
|
||||
|
||||
50
tests/etaf-playground-docs-tests.el
Normal file
50
tests/etaf-playground-docs-tests.el
Normal file
@ -0,0 +1,50 @@
|
||||
;;; etaf-playground-docs-tests.el --- Documentation contracts -*- lexical-binding: t; -*-
|
||||
|
||||
;;; Code:
|
||||
|
||||
(require 'ert)
|
||||
|
||||
(defconst etaf-playground-docs-tests--root
|
||||
(file-name-directory
|
||||
(directory-file-name
|
||||
(file-name-directory (or load-file-name buffer-file-name))))
|
||||
"Absolute etaf-playground repository root.")
|
||||
|
||||
(defun etaf-playground-docs-tests--contents (name)
|
||||
"Return the literal contents of repository file NAME."
|
||||
(with-temp-buffer
|
||||
(insert-file-contents
|
||||
(expand-file-name name etaf-playground-docs-tests--root))
|
||||
(buffer-string)))
|
||||
|
||||
(defun etaf-playground-docs-tests--normalized-contents (name)
|
||||
"Return NAME with runs of whitespace collapsed for prose assertions."
|
||||
(replace-regexp-in-string
|
||||
"[[:space:]]+" " " (etaf-playground-docs-tests--contents name)))
|
||||
|
||||
(ert-deftest etaf-performance-docs-distinguish-evidence-lanes ()
|
||||
"Latency, trace, and GUI evidence remain explicitly separate lanes."
|
||||
(dolist (file '("README.md" "DESIGN.md" "DESIGN.zh-CN.md"))
|
||||
(let ((text (etaf-playground-docs-tests--normalized-contents file)))
|
||||
(should (string-match-p "latency lane\\|延迟 lane" text))
|
||||
(should (string-match-p "trace lane\\|追踪 lane" text))
|
||||
(should (string-match-p "GUI lane" text))
|
||||
(should (string-match-p
|
||||
"repository/environment/scenario/fixture/build identity\\|仓库、环境、场景、fixture 和 build identity"
|
||||
text))
|
||||
(should (string-match-p
|
||||
"not GUI first paint\\|不是 GUI first paint"
|
||||
text)))))
|
||||
|
||||
(ert-deftest etaf-performance-docs-separate-current-gate-from-historical-budget ()
|
||||
"The active 50ms gate must not be confused with the historical 105ms text."
|
||||
(dolist (file '("DESIGN.md" "DESIGN.zh-CN.md"))
|
||||
(let ((text (etaf-playground-docs-tests--normalized-contents file)))
|
||||
(should (string-match-p "5.*warmup.*30.*sample\\|5.*warmup.*30.*sample" text))
|
||||
(should (string-match-p "p95.*max.*50ms\\|p95.*max.*50ms" text))
|
||||
(should (string-match-p "105ms p50" text))
|
||||
(should (string-match-p "historical\\|历史" text))
|
||||
(should (string-match-p "M0b1" text)))))
|
||||
|
||||
(provide 'etaf-playground-docs-tests)
|
||||
;;; etaf-playground-docs-tests.el ends here
|
||||
Loading…
Reference in New Issue
Block a user