diff --git a/DESIGN.md b/DESIGN.md index 4ad964d..4f34064 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -186,8 +186,11 @@ triplet. - Pressure fixture: `etaf-research-shelf-fixture-size` defaults to 256 and `etaf-research-shelf-page-size` defaults to 12; existing local rows are preserved and missing fixture rows are topped up with fresh IDs. -- Performance: one Data mutation -> one Runtime generation/publication; normal - warm actions remain under the accepted 105ms p50 target. +- Performance: one Data mutation -> one Runtime generation/publication. The + current active latency lane uses 5 unmeasured warmups and 30 measured samples + at fixed 1413×62 geometry; every scenario must satisfy p95 and max at or below + 50ms. The accepted 105ms p50 wording is historical design context, not the + current evaluator gate; `DOC-PERF-001` keeps its formal disposition in M0b1. - Compatibility: any same-basename `.etaf`/`.el` example is discoverable; `.ecss` is optional, and the bundled Research Shelf remains only one consumer. @@ -195,6 +198,18 @@ triplet. refresh tests, SQLite temp-file integration tests, mount/remount tests, repeated selection/mutation tests, and clean GUI screenshots. +## Performance evidence lanes + +The latency lane, trace lane, and GUI lane share one +repository/environment/scenario/fixture/build identity but remain separate +measurements. The latency lane owns the uninstrumented 5/30 p95/max 50ms gate. +The trace lane owns a representative instrumented invocation with cost class, +work counters, turns, allocation, and GC. The GUI lane owns the real Emacs +action sequence, screenshots, recording, and temporal-review verdict. Batch +verifier duration is not GUI first paint; action-start through forced redisplay +completion remains an explicitly separate future first-paint gate and an M0a +observed gap. No one lane may be used to claim another lane passed. + ## Open questions - [ ] User may rename the product after reviewing the first HTML/ETAF render; diff --git a/DESIGN.zh-CN.md b/DESIGN.zh-CN.md index 3db17cd..d7095a0 100644 --- a/DESIGN.zh-CN.md +++ b/DESIGN.zh-CN.md @@ -108,12 +108,24 @@ ETAF 私有 registry 变量;Component setup 中创建的 Controller 自动归 - 用 inert reader、source tab/session、未保存 source refresh 和 SQLite 临时文件 测试 mount/remount、筛选、分页、重复选行、mutation、错误状态和 cleanup;GUI 用干净 fullscreen 截图验证真实布局。 -- 目标:一次 Data mutation 对应一次 Runtime generation/publication;warm 交互 - 维持已接受的 105ms p50 预算。 +- 目标:一次 Data mutation 对应一次 Runtime generation/publication。当前有效的 + 延迟 lane 在固定 1413×62 几何下执行 5 次未计时 warmup 与 30 个计时 sample, + 每个场景的 p95 和 max 均须不超过 50ms。已接受的 105ms p50 是历史设计上下文, + 不是当前 evaluator gate;其正式处置仍由 M0b1 的 `DOC-PERF-001` 负责。 - 压测入口:`etaf-research-shelf-fixture-size` 默认 256, `etaf-research-shelf-page-size` 默认 12;已有本地记录保留,不足部分使用新 ID 补齐 fixture。 +## 性能 evidence lanes + +延迟 lane、追踪 lane 与 GUI lane 共享同一组仓库、环境、场景、fixture 和 build identity, +但保持为三种独立测量。延迟 lane 负责无 instrumentation 的 5/30、p95/max 50ms +门禁;追踪 lane 负责一次代表性 instrumented invocation 的 cost class、work counters、 +turns、allocation 与 GC;GUI lane 负责真实 Emacs action sequence、截图、录像与 +temporal-review verdict。Batch verifier duration 不是 GUI first paint;从 action-start +到 forced redisplay completion 的 first-paint 计时仍是独立 future gate,也是 M0a +明确记录的 observed gap。任何一条 lane 都不能替另一条 lane 宣称通过。 + ## 待 review - [ ] 用户可调整产品命名或 palette;pair 边界、组合/复用原则和公开交互合同不变。 diff --git a/README.md b/README.md index 25ecd37..cd4fb23 100644 --- a/README.md +++ b/README.md @@ -110,6 +110,26 @@ For repeated absolute-latency runs, use `make perf-prepare` once after source changes, let compilation activity settle, then run `make perf-evaluator` without rebuilding the dependency graph. +## Performance evidence lanes + +Performance evidence has three separate lanes correlated by one +repository/environment/scenario/fixture/build identity: + +- The latency lane runs the fixed 1413×62 batch evaluator with 5 unmeasured + warmups and 30 measured samples. The current active gate requires both p95 + and max to be at or below 50ms for every scenario. +- The trace lane runs a separate instrumented representative invocation. Its + cost classes, work counters, turns, allocation, and GC data are not inserted + into the timed latency distribution. +- The GUI lane captures the real Emacs interaction sequence, screenshots, and + temporal-review verdict. Batch verifier duration is not GUI first paint; + action-start to forced-redisplay-complete first-paint timing remains a + separately identified future gate and an observed M0a gap. + +Evidence from one lane supports only that lane's conclusion. A +performance-complete or user-visible-non-regression claim requires the +applicable gates from all three lanes. + The bundled Research Shelf example installs a deterministic 256-record SQLite fixture with 12 records per page. Bind `etaf-research-shelf-fixture-size` and `etaf-research-shelf-page-size` for smaller tests or larger pressure runs. Its diff --git a/tests/etaf-playground-docs-tests.el b/tests/etaf-playground-docs-tests.el new file mode 100644 index 0000000..5a0c963 --- /dev/null +++ b/tests/etaf-playground-docs-tests.el @@ -0,0 +1,50 @@ +;;; etaf-playground-docs-tests.el --- Documentation contracts -*- lexical-binding: t; -*- + +;;; Code: + +(require 'ert) + +(defconst etaf-playground-docs-tests--root + (file-name-directory + (directory-file-name + (file-name-directory (or load-file-name buffer-file-name)))) + "Absolute etaf-playground repository root.") + +(defun etaf-playground-docs-tests--contents (name) + "Return the literal contents of repository file NAME." + (with-temp-buffer + (insert-file-contents + (expand-file-name name etaf-playground-docs-tests--root)) + (buffer-string))) + +(defun etaf-playground-docs-tests--normalized-contents (name) + "Return NAME with runs of whitespace collapsed for prose assertions." + (replace-regexp-in-string + "[[:space:]]+" " " (etaf-playground-docs-tests--contents name))) + +(ert-deftest etaf-performance-docs-distinguish-evidence-lanes () + "Latency, trace, and GUI evidence remain explicitly separate lanes." + (dolist (file '("README.md" "DESIGN.md" "DESIGN.zh-CN.md")) + (let ((text (etaf-playground-docs-tests--normalized-contents file))) + (should (string-match-p "latency lane\\|延迟 lane" text)) + (should (string-match-p "trace lane\\|追踪 lane" text)) + (should (string-match-p "GUI lane" text)) + (should (string-match-p + "repository/environment/scenario/fixture/build identity\\|仓库、环境、场景、fixture 和 build identity" + text)) + (should (string-match-p + "not GUI first paint\\|不是 GUI first paint" + text))))) + +(ert-deftest etaf-performance-docs-separate-current-gate-from-historical-budget () + "The active 50ms gate must not be confused with the historical 105ms text." + (dolist (file '("DESIGN.md" "DESIGN.zh-CN.md")) + (let ((text (etaf-playground-docs-tests--normalized-contents file))) + (should (string-match-p "5.*warmup.*30.*sample\\|5.*warmup.*30.*sample" text)) + (should (string-match-p "p95.*max.*50ms\\|p95.*max.*50ms" text)) + (should (string-match-p "105ms p50" text)) + (should (string-match-p "historical\\|历史" text)) + (should (string-match-p "M0b1" text))))) + +(provide 'etaf-playground-docs-tests) +;;; etaf-playground-docs-tests.el ends here