perf: attribute source live append costs

.phrase/phases/phase-kp-overhaul-20260726/change_post_audit_hardening_20260820.md

@@ -52,6 +52,20 @@
   claimed as complete evidence.
 - **Commit:** `60b299b` and `postmortem` follow-up commit.
 
+## 2026-08-20 — Attribute incremental append cost and preserve raw evidence
+
+- **Modify** `tests/ekp-live-commit-evaluator.el` to time
+  `ekp-layout-plan-append` and `ekp--dp-cache-append` separately in each raw
+  sample and report them beside total/publication timings.
+- **Modify** `tests/run-live-commit-evaluator.sh` to write each run into a
+  unique raw directory and atomically replace the report after comparison.
+- **Evidence:** source-fresh width-80/two-row/GC-excluded candidate p95 is
+  17.161 ms C append / 1.187 ms C append-DP, and 73.293 ms Elisp append /
+  57.063 ms Elisp append-DP; raw JSONL is nonempty and parity/zero-work/GC/
+  conflict/non-regression remain true. No production optimization is claimed
+  yet because the measured DP/append owners require an exact redesign.
+- **Commit:** Pending measurement commit.
+
 ## Verification
 
 - Source-first normal ERT: 295/295.

.phrase/phases/phase-kp-overhaul-20260726/task_post_audit_hardening_20260820.md

@@ -55,3 +55,7 @@
   - `issue028` is part of this task: each baseline/candidate evaluator round
     must explicitly load source files from its own code root, never rely on
     local `.elc` precedence.
+  - The evaluator now records `append_ms` and `append_dp_ms` and preserves
+    nonempty raw JSONL per run. Current source-fresh p95 is 17.161/1.187 ms
+    for C append/append-DP and 73.293/57.063 ms for Elisp; choose a
+    production optimization only after this attribution remains stable.

postmortem/20260820-source-fresh-live-commit-gate.md

@@ -20,17 +20,20 @@ cannot satisfy the source gate.
 
 The corrected width-80, two-row, GC-excluded matrix preserves layout parity,
 zero-work ordinary keys, valid GC exclusion, conflict freedom, and
-non-regression. Its source candidate p95/p99 are 23.294 ms for C and 78.593
+non-regression. Its source candidate total p95/p99 are 23.217/23.217 ms for
-ms for Elisp, so the evaluator correctly remains red. The public source
+C and 79.724/79.724 ms for Elisp, so the evaluator correctly remains red.
-benchmark independently records C append/hard-boundary p99 near 19.9/30.7 ms
+The newly measured append owners account for C at 17.161 ms append and
-and Elisp near 49.3/50.8 ms; byte-compiled production remains below 16 ms.
+1.187 ms append-DP p95, and Elisp at 73.293 ms append and 57.063 ms
+append-DP p95. The public source benchmark independently records C
+append/hard-boundary p99 near 19.9/30.7 ms and Elisp near 49.3/50.8 ms;
+byte-compiled production remains below 16 ms.
 
-Profiling attributes the Elisp structural cost primarily to the incremental
+The evaluator now attributes the Elisp structural cost primarily to the
-1D DP pass and the C cost to live-plan construction/publication around the
+incremental 1D DP pass and the C cost to live-plan construction around the
-native call. No single cache-key or validation call owns enough time to make
+native call. No cache-key or validation call owns enough time to make a safe
-a safe one-line optimization. A broad DP or projection rewrite would need a
+one-line optimization. A broad DP or projection rewrite would need a new
-new red/green parity matrix and architecture decision; it is not smuggled into
+red/green parity matrix and architecture decision; it is not smuggled into
-this correctness hardening change.
+this measurement change.
 
 ## Consequences
 
@@ -40,6 +43,9 @@ this correctness hardening change.
 - The next performance task must optimize a measured structural owner while
   preserving exact C/Elisp parity, source-clean projection, and zero-work
   point motion.
+- Every run now keeps nonempty baseline/candidate JSONL under a unique raw
+  directory and replaces the report only after comparison, so interruption
+  cannot erase the last evidence.
 
 ## Rollback
 

tests/ekp-live-commit-evaluator.el

@@ -49,6 +49,8 @@
 (defvar ekp-live-commit-evaluator--plan-ms 0.0)
 (defvar ekp-live-commit-evaluator--para-ms 0.0)
 (defvar ekp-live-commit-evaluator--dp-ms 0.0)
+(defvar ekp-live-commit-evaluator--append-ms 0.0)
+(defvar ekp-live-commit-evaluator--append-dp-ms 0.0)
 (defvar ekp-live-commit-evaluator--module-ms 0.0)
 (defvar ekp-live-commit-evaluator--install-ms 0.0)
 (defvar ekp-live-commit-evaluator--clear-ms 0.0)
@@ -95,6 +97,8 @@
         ekp-live-commit-evaluator--plan-ms 0.0
         ekp-live-commit-evaluator--para-ms 0.0
         ekp-live-commit-evaluator--dp-ms 0.0
+        ekp-live-commit-evaluator--append-ms 0.0
+        ekp-live-commit-evaluator--append-dp-ms 0.0
         ekp-live-commit-evaluator--module-ms 0.0
         ekp-live-commit-evaluator--install-ms 0.0
         ekp-live-commit-evaluator--clear-ms 0.0
@@ -147,13 +151,17 @@
       crossed)))
 
 (defun ekp-live-commit-evaluator--append-wrapper (function)
-  "Return a FUNCTION wrapper that records append-plan hits."
+  "Return a FUNCTION wrapper that times and records append-plan hits."
   (lambda (&rest arguments)
     (cl-incf ekp-live-commit-evaluator--append-calls)
-    (let ((plan (apply function arguments)))
+    (let ((started (float-time)))
-      (when plan
+      (prog1
-        (cl-incf ekp-live-commit-evaluator--append-hits))
+          (let ((plan (apply function arguments)))
-      plan)))
+            (when plan
+              (cl-incf ekp-live-commit-evaluator--append-hits))
+            plan)
+        (cl-incf ekp-live-commit-evaluator--append-ms
+                 (* 1000.0 (- (float-time) started)))))))
 
 (defun ekp-live-commit-evaluator--instrument (function)
   "Call FUNCTION with live-commit layer instrumentation installed."
@@ -162,6 +170,7 @@
         (append-plan (symbol-function 'ekp-layout-plan-append))
         (para (symbol-function 'ekp--get-para))
         (dp (symbol-function 'ekp--dp-cache-para))
+        (append-dp (symbol-function 'ekp--dp-cache-append))
         (module (symbol-function 'ekp-c-break-with-arrays))
         (install (symbol-function 'ekp-buffer--install-live-prefix))
         (suffix (symbol-function 'ekp-buffer--clear-live-suffix))
@@ -187,6 +196,9 @@
          ((symbol-function 'ekp--dp-cache-para)
           (ekp-live-commit-evaluator--timed-wrapper
            'ekp-live-commit-evaluator--dp-ms nil dp))
+         ((symbol-function 'ekp--dp-cache-append)
+          (ekp-live-commit-evaluator--timed-wrapper
+           'ekp-live-commit-evaluator--append-dp-ms nil append-dp))
          ((symbol-function 'ekp-c-break-with-arrays)
           (ekp-live-commit-evaluator--timed-wrapper
            'ekp-live-commit-evaluator--module-ms
@@ -243,6 +255,8 @@
          (plan_ms . ,ekp-live-commit-evaluator--plan-ms)
          (para_ms . ,ekp-live-commit-evaluator--para-ms)
          (dp_ms . ,ekp-live-commit-evaluator--dp-ms)
+         (append_ms . ,ekp-live-commit-evaluator--append-ms)
+         (append_dp_ms . ,ekp-live-commit-evaluator--append-dp-ms)
          (module_ms . ,ekp-live-commit-evaluator--module-ms)
          (publication_ms
           . ,(+ ekp-live-commit-evaluator--install-ms
@@ -471,6 +485,10 @@
                                   baseline 'para_ms))
                    (dp . ,(ekp-live-commit-evaluator--statistics
                            baseline 'dp_ms))
+                   (append . ,(ekp-live-commit-evaluator--statistics
+                               baseline 'append_ms))
+                   (append_dp . ,(ekp-live-commit-evaluator--statistics
+                                  baseline 'append_dp_ms))
                    (module . ,(ekp-live-commit-evaluator--statistics
                                baseline 'module_ms))
                    (publication
@@ -487,6 +505,10 @@
                                    candidate 'para_ms))
                     (dp . ,(ekp-live-commit-evaluator--statistics
                             candidate 'dp_ms))
+                    (append . ,(ekp-live-commit-evaluator--statistics
+                                candidate 'append_ms))
+                    (append_dp . ,(ekp-live-commit-evaluator--statistics
+                                   candidate 'append_dp_ms))
                     (module . ,(ekp-live-commit-evaluator--statistics
                                 candidate 'module_ms))
                     (publication

tests/run-live-commit-evaluator.sh

@@ -27,9 +27,11 @@ test -f "$BASELINE_ROOT/dictionaries/hyph_en_US.dic"
 
 make -C "$ROOT/ekp_c" clean all PROFILE=portable
 mkdir -p "$RAW"
-BASELINE_JSONL="$RAW/baseline.jsonl"
+RUN_RAW=$(mktemp -d "$RAW/run.XXXXXX")
-CANDIDATE_JSONL="$RAW/candidate.jsonl"
+BASELINE_JSONL="$RUN_RAW/baseline.jsonl"
+CANDIDATE_JSONL="$RUN_RAW/candidate.jsonl"
 REPORT="$GOAL/latest-report.json"
+REPORT_TMP="$REPORT.tmp.$$"
 : >"$BASELINE_JSONL"
 : >"$CANDIDATE_JSONL"
 
@@ -59,12 +61,23 @@ while test "$round" -le "$ROUNDS"; do
   round=$((round + 1))
 done
 
-EKP_LIVE_COMMIT_MODE=compare \
+if EKP_LIVE_COMMIT_MODE=compare \
-EKP_LIVE_COMMIT_BASELINE_JSONL="$BASELINE_JSONL" \
+   EKP_LIVE_COMMIT_BASELINE_JSONL="$BASELINE_JSONL" \
-EKP_LIVE_COMMIT_CANDIDATE_JSONL="$CANDIDATE_JSONL" \
+   EKP_LIVE_COMMIT_CANDIDATE_JSONL="$CANDIDATE_JSONL" \
-EKP_LIVE_COMMIT_REPORT="$REPORT" \
+   EKP_LIVE_COMMIT_REPORT="$REPORT_TMP" \
-  "$EMACS_BIN" -Q --batch -L "$ROOT" -L "$ROOT/tests" \
+   "$EMACS_BIN" -Q --batch -L "$ROOT" -L "$ROOT/tests" \
-    -l "$ROOT/tests/ekp-live-commit-evaluator.el"
+     -l "$ROOT/tests/ekp-live-commit-evaluator.el"; then
+    compare_status=0
+  else
+    compare_status=$?
+  fi
+
+if test -f "$REPORT_TMP"; then
+  mv "$REPORT_TMP" "$REPORT"
+fi
+printf '%s\n' "live-commit-evaluator: raw samples stored in $RUN_RAW"
+
+test "$compare_status" -eq 0 || exit "$compare_status"
 
 if test "${EKP_LIVE_COMMIT_SKIP_AUDIT:-0}" = 1; then
   printf '%s\n' "live-commit-evaluator: performance and parity gates pass"