geekinney/ekp
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

fix: restore TeX final-pass reachability

Browse Source
This commit is contained in:
Kinneyzhang 2026-08-02 15:30:12 +08:00
parent 665164911f
commit e823d89a4a
31 changed files with 7192 additions and 454 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
.phrase/docs/CHANGE.md
View File

@ -2,6 +2,10 @@
## phase-kp-overhaul-20260726
- 2026-08-02 complete fixed final-pass emergency stretch and TeX-style
active-path preservation (`task036`; `issue021` closed after user visual
confirmation):
`.phrase/phases/phase-kp-overhaul-20260726/change_final_pass_emergency_stretch_20260802.md`
- 2026-07-28 repository system audit:
`.phrase/phases/phase-kp-overhaul-20260726/change_repository_audit_20260728.md`
- 2026-07-28 complete DP cache signature (`task002`):
@ -82,6 +86,18 @@
projection reuse (`task030` implementation verified; locked
source-instrumented 16 ms gate and `issue018` remain open):
`.phrase/phases/phase-kp-overhaul-20260726/change_narrow_live_append_latency_20260730.md`
- 2026-08-01 implement and developer-verify the rigid-inline-atom
emergency-break correction (`task033` complete; `issue021` awaits user
confirmation):
`.phrase/phases/phase-kp-overhaul-20260726/change_overlong_no_break_atom_20260801.md`
- 2026-08-01 implement configurable break policies, record G002-G008
implementation/GUI/performance/repository/cleanup/review-blocker evidence,
and close developer work for `task034` (`issue021` still awaits user visual
confirmation; final independent code review APPROVE and architecture CLEAR):
`.phrase/phases/phase-kp-overhaul-20260726/change_configurable_break_policies_20260801.md`
- 2026-08-02 record the failed line-width emergency-stretch attempt
(`task035` closed as falsified; superseded by completed `task036` above):
`.phrase/phases/phase-kp-overhaul-20260726/change_final_pass_emergency_stretch_20260802.md`
## phase-doc-improvement-20260125
- See `.phrase/phases/phase-doc-improvement-20260125/change_log.md`

1
.phrase/docs/ISSUES.md
View File

@ -20,3 +20,4 @@
- issue018 [ ] [Unique live appends miss the frame budget at very narrow widths.](../phases/phase-kp-overhaul-20260726/issue_narrow_live_append_latency_20260729.md)
- issue019 [ ] [Per-edit frontier replanning destabilizes live projection and cannot restore reversible edits exactly.](../phases/phase-kp-overhaul-20260726/issue_stable_live_edit_transaction_20260729.md)
- issue020 [ ] [C-backed resize reflow still takes roughly 60–70 ms.](../phases/phase-kp-overhaul-20260726/issue_c_resize_latency_20260729.md)
- issue021 [x] [Final-pass emergency layout can isolate CJK source lines at narrow widths.](../phases/phase-kp-overhaul-20260726/issue_overlong_no_break_atom_20260801.md)

175
.phrase/phases/phase-kp-overhaul-20260726/change_configurable_break_policies_20260801.md Normal file
View File

@ -0,0 +1,175 @@
# Change: Configurable Break Policies 2026-08-01
Superseded note: the historical hard-atom adjacency rule recorded below was
removed by `task036`. Explicit atoms forbid only interior breaks; current
final-pass reachability semantics are recorded in
`change_final_pass_emergency_stretch_20260802.md`.
## 2026-08-01 — Plan task034 and record the red baseline
- **Add** `task034` for configurable break policies and the remaining
orphan-glyph quality failure in the 280px showcase paragraph.
- **Modify** the text-property layout spec to make inline code wrapping,
token policies, kinsoku profiles, overlong-token behavior, buffer measure,
and region break-policy precedence explicit.
- **Modify** the text-property layout plan with M13, preserving the approved
architecture: resolve policy intervals before tokenization, compile to the
existing hyphen/break vectors before DP, and keep the C boundary at 15
arguments and 15 batch fields.
- **Red baseline:** the latest screenshot shows the previous atom cascade is
gone, but the crafted inline-code paragraph still permits pathological
single-CJK source lines around the atom, including `行`, `内`, and `永`.
The implementation oracle must reject any such single-CJK source line when
a legal non-emergency alternative exists.
- **Verification contract:** focused RED/GREEN ERT for core, buffer, command,
cache, and diagnostics; C/Elisp parity; 300-case property fuzz;
warning-as-error Elisp compilation; C builds/tests; checkdoc/package/static
release gates; and reviewed fullscreen GUI evidence for inline wrapping,
no orphan glyph, explicit no-break, block verbatim, measure modes, and
overlong-token modes.
- **Behavior/Risk:** Planning records only. Runtime behavior is unchanged.
This entry opened `task034`; later entries record its implementation and
closure.
## 2026-08-01 — Record task034 implementation and G004 GUI evidence
- **Modify** `task034`, M13, and the text-property layout spec with verified
implementation facts from G002/G003/G004. At this point `task034` was still
open pending later gates; G006 closure is recorded below.
- **Implementation evidence recorded:** core policy compilation preserves the
existing C boundary; buffer policy ownership separates block skip faces from
inline faces; public local/profile/region controls and diagnostics exist;
showcase verification distinguishes automatic inline code from explicit
`ekp-no-break` and verbatim block code.
- **GUI evidence recorded:** G004 passed GUI verifier ERT 7/7 and the clean
single-window evidence run at `/tmp/ekp-g004-evidence.Tp77dW` reports 12/12
checkpoints with no failed assertions, 25.75s/206-frame recording, no black
segments, automatic inline wrapping 3→2→3 lines across 280→340→280, every
internal inline split as source whitespace, exact source, zero overlays, C
active, and a settled no-hyphen→normal policy transition with stale
nil-plan spans at zero.
- **Verification status:** G005 repository-wide gates and G006 cleanup are
recorded below.
- **Behavior/Risk:** Documentation synchronization only in this entry. Do not
close `task034` or any user-confirmation issue from this G004-only evidence
at this point in the sequence.
## 2026-08-01 — Record G005 final performance and repository gate evidence
- **Modify** `task034` and the policy-boundary postmortem with the confirmed
G005 root causes and final gate evidence. At this point `task034` was still
open pending G006 cleanup; G006 closure is recorded below.
- **Root causes recorded:** paragraph identity included unconditional policy
measure, causing paragraph rebuilds; volatile property clearing fragmented
equal property runs; policy analysis ran before paragraph-cache hits; and
repeated same-paragraph/same-width requests duplicated semantic plan
assembly.
- **Fix evidence recorded:** policy measure was removed from the semantic
signature; adjacent equal filtered intervals are canonicalized; policy full
analysis now uses a bounded two-tier cache keyed by clean source, canonical
layout and raw face policy intervals, policy signature, and width context,
with a measure tier only for actual width-sensitive no-break/overflow
candidates; and semantic plans use a bounded per-paragraph plan cache. G007
later strengthened the consumer boundary from shallow copies to
consumer-owned copies of plan-owned mutable payloads; see the G008 entry
below.
- **Performance evidence recorded:** formal four-interleaved evaluator reports
layout parity true. Core baseline p50/p95 was 38.6679/51.6782ms and
candidate p50/p95 was 21.7102/33.0040ms, for 43.8549%/36.1354% gains.
Resize baseline p50/p95 was 43.3831/55.6250ms and candidate p50/p95 was
22.0919/32.8202ms, for 49.0773%/40.9973% gains. Both candidate p95 values
are under 50ms and both gain sets exceed 20%.
- **Live evaluator recorded:** the source-instrumented locked goal remains the
known `validation_failed` debt; parity, zero-work, GC, conflict, and
all-width-nonregression checks are true. Current C p95/p99 is
26.449/26.740ms, a 76.65%/77.23% improvement. Current Elisp p95/p99 is
49.940/52.017ms, a 91.64%/91.35% improvement. This is consistent with
historical open `issue018` and is not a regression.
- **Production public-path evidence recorded:** three byte-compiled runs
passed with zero GC. C append p99 was 1.361-1.368ms and hard p99 was
1.876-1.891ms. Elisp append p99 was 1.692-1.775ms and hard p99 was
2.100-2.230ms.
- **Verification recorded:** default ERT passed 262/262; seeded permuted ERT
seed 20260728 passed 255/255; isolated per-test process suite exited 0;
the subsequently added alias guard passed targeted verification; property
fuzz passed 300/300; warning-as-error byte compilation passed; pinned
package-lint at `35996f478d81e51dae4fa30d051f741895d07399` exited 0 with
only an external obsolete warning from the local names dependency; checkdoc
was empty; release, 49-entry dictionary manifest, pinned dictionary update,
shell syntax, CI YAML, and diff-check passed; portable, native, debug, and
sanitize C builds were warning-clean; module 1.6/4-thread smoke passed; and
focused C verification passed 19/19 including the 15-argument/15-field
boundary.
- **Verification status:** G005 gates are complete. G006 cleanup is recorded
below.
## 2026-08-01 — Close task034 with G006 cleanup evidence
- **Modify** `task034`, M13, the global change index, and the policy-boundary
postmortem with G006 cleanup evidence.
- **Cleanup evidence recorded:** G006 edited only `ekp.el` and
`ekp-buffer.el`, removing redundant policy/cache code without changing the
locked behavior.
- **Verification recorded:** targeted cleanup suites passed 6/6 and 3/3; full
core ERT passed 125/125; full buffer ERT passed 120/120; warning-as-error
byte compilation passed; diff-check passed; and the C build gate passed.
- **Behavior/Risk:** `task034` is closed for developer implementation,
repository gates, performance gates, and cleanup gates. `issue021` remains
open pending user visual confirmation. Independent final code/architecture
review has not yet been claimed.
## 2026-08-01 — Resolve G006/G007 review blockers with G008 ownership fixes
- **Modify** `task034`, M13, developer documentation, changelog, and
postmortems with the G007/G008 ownership and backend-contract corrections.
- **Review blockers resolved:** G006 cleanup's independent review found that
shallow semantic-plan copies left nested line/gap/glue payloads mutable
through cache hits, and that malformed non-nil C results could still be
treated too softly. G007 changed the semantic-plan cache boundary to return
consumer-owned copies of every plan-owned mutable payload: source string,
context, boxes, offsets, lines, glues, gaps, and signatures. `para` remains
intentionally shared because paragraph-cache ownership and append identity
rely on that object. G007 final review then found one remaining alias:
`copy-tree` did not copy strings inside the returned context, so mutating a
dynamic policy suffix from the returned plan could poison the cached context.
G008 resolves that final blocker by using the recursive context copier for
cons/vector/string payloads both when the context snapshot/cache key is
created and when a plan is returned.
- **Backend contract recorded:** only a nil whole C result or nil per-item
breaks may fall back to Elisp. Any malformed non-nil single or batch output
signals `ekp-backend-contract-error`. The C entry remains 15 arguments and
the batch payload remains 15 fields.
- **Hard-atom boundary recorded:** the ordinary boundary immediately before a
following explicit `ekp-no-break` atom is forbidden before DP. This direct
pre-DP adjacency rule is separate from the final-pass emergency-stretch
model and is locked by a focused regression.
- **Verification recorded:** formal resize parity is true; core baseline
p50/p95 is 36.595/48.357ms and candidate p50/p95 is 23.266/36.363ms, for
36.42%/24.80% gains. Resize baseline p50/p95 is 41.443/53.541ms and
candidate p50/p95 is 23.761/36.889ms, for 42.67%/31.10% gains. ERT passes
268/268; property fuzz passes 300/300; release gates pass; byte-compiled
current public path records zero GC with C append/hard p99
1.440-1.464/2.022-2.050ms and Elisp append/hard p99
1.648-1.687/2.136-2.335ms, all below 16ms.
- **Behavior/Risk:** G007 resolved the first recorded G006 review blockers,
and G008 resolves the final string-leaf context blocker, but this change
record does not claim final independent APPROVE/CLEAR. `issue021` remains
open pending user visual confirmation, and historical `issue018` remains
open.
## 2026-08-02 — Record final independent review clearance
- **Review remediation:** the first final code-review pass found warning-only
defects in changed test fixtures and a stale C README setter example. The
fixtures now byte-compile with warnings as errors, and the README documents
the optional eighth `EMERGENCY-STRETCH` argument.
- **Verification:** the remediation-focused ERT passes 5/5, full ERT passes
288/288, every changed test file compiles warning-clean, and diff-check
passes.
- **Independent verdicts:** the code-reviewer re-review returns `APPROVE` with
zero findings; the subsequent architect review returns `CLEAR` and confirms
core K-P ownership, content-independent final-pass reachability, Elisp/C
parity, stable ABI, cache ownership, and complete nonempty GUI oracles.
- **Behavior/Risk:** `task034` and its independent developer gate are
complete. `issue021` closed after the user's 2026-08-02 visual
confirmation, and historical `issue018` remains open.

91
.phrase/phases/phase-kp-overhaul-20260726/change_final_pass_emergency_stretch_20260802.md Normal file
View File

@ -0,0 +1,91 @@
# Change: Final-Pass Emergency Stretch 2026-08-02
## 2026-08-02 — Close issue021 after user visual confirmation
- **Modify** the global issue index and issue detail to close `issue021`.
- **User confirmation:** the user accepted the visible result with “可以了，
提交吧” and requested the commit.
- **Behavior/Risk:** no runtime change; this closes the required user-visible
acceptance gate after the developer, dynamic GUI, and independent review
gates had already passed.
## 2026-08-02 — Complete task036 with TeX active-path preservation
- **Modify** `ekp.el` and `ekp_c/ekp_kp.c`: keep strict K-P unchanged; use
fixed `ekp-emergency-stretch-pixel` for ordinary final-pass underfull
candidates; and, only when an overfull candidate would otherwise extinguish
the final active path to a breakpoint, install the best provisional path
with tight fitness and zero incremental demerits.
- **Delete** the wrong-layer rule that forbade an otherwise legal boundary
immediately before an explicit hard atom. Hard atoms forbid only their
interior breaks; an overwide atom remains intact but need not stand alone.
- **Modify** the renderer to distribute the chosen line's actual rest using
its actual TeX glue-set proportions. Elisp 1D, looseness/parshape, and C
behavior match; the public 15-field/15-argument C contract is unchanged.
- **Strengthen tests** so isolated-CJK oracles require a nonempty plan that
exactly covers the source, preventing an empty result from false-greening.
- **Review remediation:** make the changed test fixtures warning-clean under
`byte-compile-error-on-warn` and document the optional eighth
`EMERGENCY-STRETCH` argument in the C README API example.
- **Verification:** focused root regressions 8/8, emergency selector 10/10,
core/buffer/GUI oracle 8/8, full ERT 288/288, seeded and isolated suites
279/279, fuzz 300/300, warning-clean portable/native/debug/sanitize C
builds, warning-as-error production and changed-test Elisp compilation,
checkdoc/package/static/release gates, formal performance gates, and
reviewed 42.78-second dynamic GUI evidence at
`/tmp/ekp-g009-evidence-retry.UOpPNp` with `VERDICT=PASS`. Final independent
code review returns `APPROVE` and architecture review returns `CLEAR`.
- **Behavior/Risk:** no CJK-orphan, unit, atom-adjacency, or screenshot
heuristic exists. `task036` is complete; `issue021` closed after the
user's 2026-08-02 visual confirmation.
The entries below are historical steps. Their hard/atomic fixed-cost wording
was superseded by the completed task036 semantics above.
## 2026-08-02 — Open task036 after task035 full-regression failure
- **Modify** `task035` to record the line-width-sized emergency stretch as a
falsified attempt, not accepted current behavior.
- **Add** `task036` for TeX-style fixed-dimension final-pass emergency
stretch. The implementation must expose `ekp-emergency-stretch-pixel`
where nil auto-resolves to roughly three display-font `M` widths and a
non-negative integer fixes the pixel value.
- **Preserve** the constraints: strict pass unchanged, 15-field C paragraph
ABI unchanged, renderer semantics synchronized, fixed artificial emergency
transition only for truly overfull first permitted hard/atomic runs, and no
CJK-orphan/unit/screenshot-specific penalty.
- **Verification planned:** five new RED cases for fixed dimension behavior,
nil auto, integer override, C parity without ABI growth, and renderer
width/glue consistency; the three regressions broken by task035; focused
G009 GUI/core/public-buffer/C parity; final fullscreen dynamic GUI evidence;
full ERT/fuzz/build/static/release gates.
- **Behavior/Risk:** Documentation records the next atom of work only.
Runtime behavior is not claimed fixed by this entry.
## 2026-08-02 — Close task035 as falsified
- **Modify** core K-P documentation to state the attempted final-pass model:
strict pass unchanged; final pass gives ordinary underfull candidates a
line-width-sized finite background emergency stretch; those candidates
still use adjustment ratio, badness, fitness, and demerits.
- **Modify** user-facing hard-atom documentation to avoid claiming a
hard-coded fullest-prefix fallback. A hard atom may occupy one overflow
line, while the ordinary prefix remains a normal K-P decision with finite
emergency stretch.
- **Modify** `issue021`, `task035`, the spec, plan, changelog, developer
docs, C README, and global change/issue indexes to record the corrected
owner and current acceptance boundary.
- **Add** `postmortem/20260802-final-pass-emergency-stretch.md` to explain
why the ordinary-underfull fixed fallback was the wrong layer, why the
TeX-style emergency-stretch model is used, and why the earlier 84px oracle
was invalid.
- **Verification:** Focused core single-CJK regressions pass 2/2. The
implementation lane reports focused G009 core/C/hard-atom/buffer/GUI-oracle
coverage 18/18, warning-as-error byte compilation pass, and C build pass.
Full regression then falsified the line-width stretch detail and forced the
`task036` fixed-dimension follow-up. This documentation pass ran stale-text
search and `git diff --check`.
- **Behavior/Risk:** `task035` is not current truth. No public configuration,
C ABI, or renderer ownership change is accepted by this record. `issue021`
remains open until task036 and the final user-visible visual acceptance gate
pass.

44
.phrase/phases/phase-kp-overhaul-20260726/change_overlong_no_break_atom_20260801.md Normal file
View File

@ -0,0 +1,44 @@
# Change: Rigid Inline Atom Break Cascade 2026-08-01
Superseded note: this file records the earlier task033 repair. The current
ordinary-underfull final-pass model is recorded in
`change_final_pass_emergency_stretch_20260802.md`; an overwide atom remains
intact but is not guaranteed a standalone line, and atom adjacency has no
special break rule.
## 2026-08-01 — Implement and developer-verify task033
- **Modify** `ekp.el` so the emergency pass preserves its established first
transition but adds a fullest-underfull prefix fallback only when a
forbidden break run jumps directly to an overfull permitted candidate.
- **Modify** `ekp_c/ekp_kp.c` with the same state transition and keep the
looseness/parshape Elisp path in parity.
- **Add** core, public-buffer, and GUI regressions covering diverse CJK
prefixes, exact source boundaries, atom integrity, Elisp/C parity, and
repeated 480→280→340→280 reflow.
- **Modify** user/developer documentation and add
`postmortem/20260801-forbidden-run-emergency-boundary.md` to record why the
broader fullest-prefix replacement was rejected.
- **Verification:** RED reproduced boundary 1 instead of 11. GREEN passes
normal, random-order, and isolated 201-test runs; 300 C/Elisp fuzz cases;
warning-as-error Elisp compilation; package-lint; checkdoc; release and
dictionary gates; warning-clean debug/sanitize/portable C builds; and
reviewed dynamic GUI evidence at
`/private/tmp/ekp-atom-gui-final-Beg8hb` with verdict PASS.
- **Behavior/Risk:** Strict K-P, ordinary emergency layouts, buffer
projection, source text, and public APIs are unchanged. The additional
relaxation is limited to emergency scans that cross forbidden break
positions before an overfull candidate.
## 2026-08-01 — Plan issue021 and task033
- **Add** `issue021` with the 280px showcase reproduction and deterministic
width-40 batch analogue.
- **Add** `task033` and M12 acceptance gates for the core emergency-break
correction, Elisp/C parity, and real GUI verification.
- **Modify** the text-property layout spec to require a full ordinary prefix
line and one intact rigid atom line rather than one-glyph emergency
fragmentation.
- **Behavior/Risk:** Planning records only. Runtime behavior is unchanged.
The change will affect only the second pass used after strict K-P has
already proved the paragraph unreachable.

69
.phrase/phases/phase-kp-overhaul-20260726/issue_overlong_no_break_atom_20260801.md Normal file
View File

@ -0,0 +1,69 @@
# Issue: Final-Pass CJK Isolation 2026-08-01
## issue021 [x] Final-pass emergency layout can isolate CJK source lines at narrow widths
- **Status:** Resolved. Developer verification and independent review passed;
the user accepted the visible result and requested the commit.
- **Summary:** At narrow measures, the final K-P pass can choose one-glyph CJK
source lines in the mixed showcase paragraph. The visible symptom was first
found around an explicit hard atom, then reproduced after automatic inline
policy was split from hard atoms.
- **Environment:** Graphical Emacs 30.2 on macOS, narrow showcase widths,
justify alignment, C module active for 1D layout. The same core problem is
visible through the public buffer path, so this is not a renderer-only
issue.
- **Repro:** Run `M-x ekp-showcase`, set a narrow width such as 168px or
280px, and inspect the mixed Chinese/Latin inline paragraph. The failure
appears when a CJK glyph becomes a complete source line even though the
paragraph has legal non-emergency alternatives.
- **Expected vs Actual:** Ordinary underfull candidates in the final pass
should remain normal K-P candidates: adjustment ratio, badness, fitness,
and demerits decide the global path. Separately, the final pass must not
lose all reachability when a legal candidate is overfull. The previous
model confused that reachability safeguard with a hard/atomic fixed-cost
scoring shortcut, and an empty-plan oracle could hide the failure.
- **Investigation:** The strict pass is not the bug. The buffer projector
reproduces the semantic plan exactly and owns no line-choice compensation.
The root is in core DP final-pass modeling: ordinary underfull candidates
must receive finite emergency stretch and stay inside normal K-P scoring.
TeX's `artificial_demerits` instead preserves the last active path in the
final pass independent of content class; it is not owned by CJK, units,
hard atoms, or their adjacency.
- **Required Outcome:** Eliminate unintended isolated CJK source lines at the
checked narrow showcase widths without adding CJK-orphan, number-unit, or
screenshot-specific penalties. Preserve source text, zero overlays,
strict-pass behavior, Elisp/C parity, and the public C ABI.
- **Fix Plan:** Keep strict K-P unchanged. In the final pass, add a
TeX-style fixed emergency-stretch dimension to ordinary underfull
candidates and score them through normal badness, fitness, and demerits.
Expose the dimension as `ekp-emergency-stretch-pixel`: nil auto-resolves to
roughly three display-font `M` widths, while a non-negative integer fixes
the pixel value. If an overfull candidate would otherwise remove the final
active path to a breakpoint and no non-overfull candidate survives there,
install the best provisional path with tight fitness and zero incremental
demerits. The Elisp 1D path, looseness/parshape path, renderer semantics,
and C engine share this rule without changing the 15-field C paragraph ABI.
- **Verification:**
- Focused core single-CJK regressions pass 2/2.
- The implementation lane's focused G009 suite covering core, C parity,
hard atoms, public buffer projection, and GUI oracle passes 18/18.
- Warning-as-error Elisp byte compilation and the C build pass.
- Full regression falsified task035's line-width-sized emergency stretch:
the stretch must be a fixed dimension, not derived from each candidate
measure.
- Task036's fixed-dimension RED cases pass and the three regressions broken
by task035 remain green. Direct regression proves artificial lines add
zero demerits and preserve a complete nonempty source-covering plan.
- Focused root regressions pass 8/8, emergency selection passes 10/10,
core/buffer/GUI oracle passes 8/8, full ERT passes 288/288, seeded and
isolated suites pass 279/279, fuzz passes 300/300, and all build/static/
release/performance gates pass.
- Reviewed 42.78-second fullscreen single-window evidence at
`/tmp/ekp-g009-evidence-retry.UOpPNp` covers 480→168→280→168 and
no-hyphen→normal policy change, reports no isolated CJK source lines,
exact source, zero overlays/stale spans, active C, and `VERDICT=PASS`.
- **User Confirmation:** On 2026-08-02, the user replied “可以了，提交吧”,
accepting the visible result and requesting submission.
- **Resolved At:** 2026-08-02.
- **Resolved By:** `task036`; final-pass active-path preservation in core K-P.
- **Commit:** This commit (`fix: restore TeX final-pass reachability`).

128
.phrase/phases/phase-kp-overhaul-20260726/plan_text_property_layout_engine_20260729.md
View File

@ -38,6 +38,12 @@ hyphens. No buffer path may use overlays.
character-modified tick, point, mark, or external modification hooks.
- Turning the mode off removes only EKP-owned projection properties and
restores every pre-existing property exactly.
- Final-pass emergency layout remains owned by core K-P. Ordinary underfull
candidates use finite emergency stretch and normal badness/demerits; the
final active path is preserved with TeX-style zero-increment artificial
demerits only when an overfull candidate would otherwise extinguish it.
The buffer renderer and policy compiler must not add CJK-orphan, unit,
hard-atom-adjacency, or screenshot-specific compensation.
## Architecture
@ -260,6 +266,128 @@ rows are not the durable planning unit.
complete resize improved by 43.51%/41.03% to 15.900/27.487 ms. Exact
frozen-C/Elisp parity and all automated, static, and GUI gates pass.
### M12 — Rigid inline atom emergency breaking
- Status: `task033` implemented and developer-verified as the earlier
rigid-atom repair. `task036` supersedes its ordinary-underfull emergency
model; `issue021` was closed after user-visible confirmation.
- Reproduce the showcase's narrow CJK prefix plus `ekp-no-break` atom through
the public string and semantic-plan paths before changing the algorithm.
- Historical task033 design: keep the strict K-P pass and the established
first-candidate emergency transition unchanged, then add a narrow
forbidden-run prefix fallback. The current task036 model replaces that
fallback with fixed final-pass emergency stretch plus content-independent
active-path preservation.
- Preserve atom integrity, source text, Elisp/C parity, looseness/parshape
semantics, append correctness, and the buffer projection contract.
- Gate: focused RED/GREEN core and buffer regressions, complete ERT in normal
and random order, isolated tests, C/Elisp fuzz, warning-as-error builds,
static/release checks, and clean static plus dynamic 280px GUI evidence.
- Result: the failing boundary moved from 1 to the full prefix boundary 11 in
Elisp, C, and public buffer paths. Normal/random/isolated ERT pass 201/201,
fuzz passes 300/300, static and build gates pass, and reviewed
480→280→340→280 GUI evidence returns PASS.
### M13 — Configurable break policies and orphan-glyph closure
- Status: `task034` implementation, focused automated evidence, and G004 GUI
evidence are recorded. G005 repository-wide gates and G006 cleanup are
complete. `task034` is closed for developer work, `issue021` is closed
after user visual confirmation, and independent final code review returned
APPROVE with architecture status CLEAR.
- Implement the locked A2/B2/C2/D1/E1/F1/G2/H1/I1/J1/K1/M1 contract:
inline code defaults to no-hyphen rather than no-break, known inline faces
are recognized through mode profiles, region policy outranks explicit
local values which outrank mode profiles and globals, block faces remain
verbatim, explicit no-break never downgrades, automatic no-break downgrades
to no-hyphen when overwide, inline and block faces use separate paths,
manual properties stay session-only, hyphenation defaults to auto,
URL/path/identifier default to no-hyphen, compact number-unit defaults to
no-break, kinsoku defaults to common, overlong tokens default to emergency,
and buffer measure defaults to the narrowest live window.
- Keep the chosen architecture: resolve private structural policy intervals
before tokenization, compile them into existing hyphen positions and
forbidden-break vectors before DP, preserve the original source in the
semantic plan, and keep the C boundary at the current 15 arguments and
15-field batch payload unless architecture review is reopened.
- Split block and inline ownership in `ekp-buffer`: `ekp-buffer-skip-faces`
remains paragraph-level verbatim, `ekp-buffer-inline-faces` annotates only
exact inline intervals, and `ekp-buffer-mode-policy-alist` is consulted
without auto-copying profile values into buffer locals.
- Add public region controls for `ekp-break-policy`: normal, enable
hyphenation, disable hyphenation, and clear. Existing no-break/verbatim
commands remain the only hard-atom and paragraph-bypass controls.
- Fix the remaining visible quality bug from the user's latest screenshot:
the 280px showcase path must not isolate any pathological single-CJK source
line around inline code, including `行`, `内`, or `永`, when a legal
non-emergency alternative exists.
- Gate: follow the RALPLAN test specification R1-R8 plus GUI verification.
Required evidence includes focused RED/GREEN ERT, full normal/permuted/
isolated ERT, 300-case property fuzz, C/Elisp parity, warning-as-error
Elisp compilation, C builds/tests, checkdoc/package/static/release gates,
and reviewed fullscreen single-window screenshots for inline wrapping,
no orphan glyph, explicit no-break, block verbatim, measure modes, and
overlong-token modes.
- Current evidence: G002 records core policy compilation, C parity, 300-case
fuzz, byte compilation, checkdoc, C build, and focused policy/cache gates.
G003 records buffer/profile/local/region controls, diagnostics, generation
reflow, source/editor invariants, focused core/buffer suites, byte
compilation, and checkdoc. G004 records GUI verifier ERT 7/7 and a clean
single-window run at `/tmp/ekp-g004-evidence.Tp77dW` with 12/12 checkpoints,
no failed assertions, 25.75s/206-frame recording, no black segments,
automatic inline wrapping 3→2→3 lines across 280→340→280, source-space
internal inline breaks, explicit hard atom, block verbatim, C active, and
no stale policy projection after the no-hyphen→normal transition.
- Final evidence: G005 recorded repository-wide default/seeded/isolated ERT,
property fuzz, warning-clean Elisp/C builds, checkdoc/package/release/
dictionary/static gates, performance checks, and full diff review. G006
cleanup touched only `ekp.el` and `ekp-buffer.el`; targeted cleanup suites
passed 6/6 and 3/3, full core ERT passed 125/125, full buffer ERT passed
120/120, and byte compilation, diff-check, and C build gates passed. G006
was then review-blocked on nested semantic-plan cache aliasing and malformed
non-nil C fallback semantics. G007 resolved those first blockers: cached
semantic plans return consumer-owned copies of plan-owned mutable payloads
while intentionally sharing `para`; malformed non-nil C single/batch output
signals `ekp-backend-contract-error`, with nil-result Elisp fallback and
the 15-argument/15-field ABI unchanged. G007 final review found one
remaining string-leaf context alias; G008 resolves it by recursively
copying cons/vector/string context payloads both for snapshot/cache-key
creation and returned plans. Fresh G008 gates record formal resize parity
true, ERT 268/268, fuzz 300/300, release pass, core p50/p95
23.266/36.363ms, resize p50/p95 23.761/36.889ms, and byte-compiled current
public path p99 below 16ms with zero GC. After review remediation made all
changed tests warning-clean and synchronized the eighth C setter parameter,
final independent code review returned APPROVE and architecture review
returned CLEAR.
### M14 — Final-pass emergency stretch
- Status: `task035` was falsified; `task036` is implemented and its developer
gates pass. `issue021` is closed after user-visible confirmation.
- Correct the core K-P owner, not the renderer or policy compiler. Strict
pass behavior remains unchanged.
- In the final pass, ordinary underfull candidates receive finite background
emergency stretch and still compute adjustment ratio, badness, fitness, and
demerits. Separately, when an overfull candidate would otherwise eliminate
the final active path to a breakpoint and no non-overfull candidate survives
there, install the best provisional path with tight fitness and zero
incremental demerits, matching TeX's `artificial_demerits` purpose.
- Do not add CJK-orphan, unit, or screenshot-specific penalties. Unit suffix
configuration remains only a token-classification input, not a layout
scoring rule.
- Gate: focused core/public-buffer/C parity tests and GUI-oracle checks must
reject any isolated CJK source line in the showcase paragraph at checked
widths. Final fullscreen visual review remains required before closing
`issue021`.
- Result: the Elisp 1D, looseness/parshape, and C paths share the same rule;
explicit atom interiors remain unbreakable but adjacent legal boundaries
remain legal; the public 15-field/15-argument C contract is unchanged.
Focused regressions pass 8/8, the emergency selector passes 10/10, the
core/buffer/GUI oracle passes 8/8, full ERT passes 288/288, seeded and
isolated core suites pass 279/279, fuzz passes 300/300, all build/static/
release gates pass, performance gates pass, and reviewed dynamic GUI
evidence at `/tmp/ekp-g009-evidence-retry.UOpPNp` returns PASS.
## Stop Gates
- Stop the affected paragraph instead of stealing a foreign replacing

52
.phrase/phases/phase-kp-overhaul-20260726/spec_text_property_layout_engine_20260729.md
View File

@ -141,6 +141,42 @@ overlay and inserts no layout character.
state.
- Major-mode changes and mode disable leave the logical text and prior
integrations intact.
- A rigid inline atom that jumps a candidate line from underfull to overfull
stays intact without forcing the preceding prose through one-box emergency
lines. In the final pass, ordinary underfull candidates receive finite
emergency stretch and remain normal K-P cost candidates. An atom wider than
the measure stays intact but may share an overflow line with preceding
ordinary content when TeX-style artificial demerits must preserve the last
active path. Atom adjacency has no special break or scoring rule.
- Inline code is not rigid by default. Known Org/Markdown inline faces use
the inline policy path, not the paragraph verbatim path. The default policy
permits legal wrapping at existing boundaries, suppresses discretionary
dictionary hyphenation, and preserves source whitespace literally. Known
code-block faces remain paragraph-level verbatim.
- Manual `ekp-no-break` is the only explicit hard-atom owner and is never
downgraded. Automatic no-break spans, such as compact number-unit tokens,
downgrade to no-hyphen when wider than the effective measure.
- The previous narrow showcase orphan-glyph failure is now a permanent
acceptance invariant: the inline-code paragraph must not isolate CJK glyphs
such as `行`, `内`, or `永` as one-character source lines around automatic
inline code when a non-emergency legal alternative exists. Dynamic evidence
must distinguish automatic inline code from explicit hard atoms and block
verbatim spans.
- The GUI oracle rejects any isolated CJK source line in the showcase
paragraph at the checked widths. This is a core K-P invariant, not a
unit-suffix, screenshot-specific, or renderer compensation rule.
- Effective policy precedence is deterministic: region properties, then
explicit buffer/file/dir-local values, then major-mode profiles, then
global defaults. Manual text properties are session-local; persistence
comes from mode syntax/profiles and file/dir locals.
- URL, path, and identifier tokens default to no-hyphen. Compact number-unit
tokens default to no-break. Hyphenation defaults to `auto`, with `on` and
`off` overrides. Kinsoku defaults to `common`, with `zh`, `ja`, `off`, and
custom additions available. Ordinary overlong tokens default to the current
emergency output, with `overflow` and `natural` alternatives.
- The buffer measure defaults to the narrowest live window. A positive
integer fixed measure and `(max . PIXELS)` cap are configurable and must be
reported by diagnostics when they create overflow risk.
- Reprojection preserves point, the mark marker, and `mark-active`
independently. An inactive historical mark must never become a visible
selection merely because width or layout options changed.
@ -190,3 +226,19 @@ overlay and inserts no layout character.
9. Default, permuted, and isolated ERT; C/Elisp fuzz; warning-as-error
Elisp/C builds; checkdoc; package/static/release gates; full diff review;
independent code review; and independent architecture review pass.
10. Elisp, C, semantic-plan, string-renderer, and real GUI paths keep an
`ekp-no-break` atom intact while proving that its preceding CJK prefix is
not fragmented into one-glyph lines at narrow measures.
11. Inline and token break policies are configurable at global, mode profile,
explicit local, and region scopes. The accepted defaults are: inline code
`no-hyphen`; block code verbatim; URL/path/identifier `no-hyphen`;
compact number-unit `no-break`; hyphenation `auto`; kinsoku `common`;
overlong token `emergency`; buffer measure `narrowest-window`.
12. Region `ekp-break-policy` supports `normal`, `hyphenate`, and
`no-hyphen`, and never creates a second hard-atom representation.
Overlapping explicit `ekp-no-break` wins over every new policy.
13. The showcase paragraph containing inline code, CJK prose, NBSP-backed
numbers, and units must have no pathological single-CJK source line in
the inspected 280px GUI path while retaining exact source text, zero
overlays, block-code verbatim display, internal source-space inline
breaks, and C/Elisp plan parity.

202
.phrase/phases/phase-kp-overhaul-20260726/task_repository_audit_20260728.md
View File

@ -722,3 +722,205 @@
checkdoc, release checks, and reviewed dynamic GUI evidence pass.
- Closure: developer verification is complete. `issue020` remains open
only for the required user-visible confirmation.
- task033 [x] Prevent rigid inline atoms from fragmenting preceding CJK.
- Source: `issue021`, the user's 2026-08-01 showcase screenshot, and
`plan_text_property_layout_engine_20260729.md` M12.
- Red tests: reproduce the deterministic batch underfull-to-overfull jump
and the real 280px showcase path; require the fullest permitted prefix
line, one intact atom line, and matching Elisp/C plans.
- Work: correct the emergency transition owner in core DP and its C/loose
equivalents without changing strict K-P results or buffer projection.
- Validation: focused RED/GREEN core and buffer ERT, normal/random/
isolated full ERT, C/Elisp fuzz, warning-as-error compilation, checkdoc,
release/static gates, and inspected static/dynamic GUI evidence.
- Impact: core line breaking, C parity, regression tests, user/developer
documentation, issue/change records, and no new public API.
- Result: retain the original emergency transition and add a fallback only
for a forbidden-run underfull-to-overfull jump. This was later superseded
by the accepted `task036` final-pass model; strict K-P and buffer
projection remain unchanged.
- Verification: the RED boundary was 1 instead of 11. Normal,
seeded-random, and isolated ERT pass 201/201; property fuzz passes
300/300; Elisp/static/release/dictionary and three C profile builds pass.
Reviewed 62.7-second GUI evidence at
`/private/tmp/ekp-atom-gui-final-Beg8hb` passes all nine checkpoints with
an intact atom, one-line full prefix, exact source, zero overlays, active
C engine, and no black or transient vertical frame.
- Closure: developer verification is complete. `issue021` remains open
only for the required user-visible confirmation.
- task034 [x] Make break policy configurable and eliminate inline-code
induced orphan glyph lines.
- Source: the approved RALPLAN PRD
`.omx/plans/prd-ekp-configurable-break-policies.md`, test spec
`.omx/plans/test-spec-ekp-configurable-break-policies.md`, and the
user's 2026-08-01 follow-up screenshot showing the showcase no longer
collapses into the previous one-glyph cascade but still permits
pathological single-CJK source lines around the inline atom, including
`行`, `内`, and `永`.
- Work: implement the locked A2/B2/C2/D1/E1/F1/G2/H1/I1/J1/K1/M1 policy
contract. Inline code defaults to legal wrapping with no dictionary
hyphenation; block code remains verbatim; explicit `ekp-no-break` stays
rigid forever; automatic no-break downgrades to no-hyphen when wider than
the effective measure; URL/path/identifier default to no-hyphen; compact
number-unit tokens default to no-break; kinsoku defaults to `common`;
ordinary overlong tokens default to current emergency output; and buffer
measure defaults to the narrowest live window with fixed and max-cap
alternatives.
- Public configuration: add global, major-mode profile, buffer/file/dir
local, and region controls for inline code, hyphenation, token policies,
kinsoku profiles, overlong-token behavior, and buffer measure. Region
`ekp-break-policy` provides `normal`, `hyphenate`, and `no-hyphen`; the
existing `ekp-no-break` remains the only manual hard-atom owner.
- Red tests: prove inline faces no longer cause paragraph skipping; prove
region > explicit local > mode profile > global precedence; prove token
classification/downgrade behavior; prove `common`/`zh`/`ja`/`off`/custom
kinsoku; prove `emergency`/`overflow`/`natural` overlong modes; prove
`narrowest-window`, fixed, and `(max . PIXELS)` measure resolution; and
reproduce the current showcase orphan-glyph screenshot as a failing
public path before the fix, with an oracle that rejects any pathological
single-CJK source line in the crafted paragraph when a legal
non-emergency alternative exists.
- Verification: focused RED/GREEN ERT for core, buffer, command, cache, and
diagnostics; C/Elisp parity without changing the 15-argument C entry or
15-field batch payload; 300-case property fuzz; warning-as-error Elisp
compilation; C builds/tests; checkdoc/package/static/release gates; and
clean fullscreen single-window GUI evidence for inline wrapping,
no-orphan showcase text, explicit no-break, block verbatim, measure
modes, and overlong-token modes.
- Impact: `ekp.el`, `ekp-buffer.el`, `ekp_c/ekp_kp.c` only if parity logic
requires a mirrored DP adjustment, tests, bilingual user/developer
documentation, spec/plan/tech reference, issue/change records, changelog,
and a postmortem explaining the policy boundary.
- Implementation evidence recorded: G002 completed core policy compilation
without changing the 15-argument/15-field C boundary; G003 completed
buffer ownership, consult-only mode profiles, local/region controls,
diagnostics, and generation reflow; G004 completed the showcase split
between automatic inline code, explicit no-break, and verbatim block;
G005 performance work removed unconditional policy measure from paragraph
semantic identity, canonicalized adjacent equal filtered property
intervals, moved repeated policy full analysis behind a bounded two-tier
cache, and added a bounded per-paragraph semantic plan cache; G006
cleanup touched only `ekp.el` and `ekp-buffer.el`, removing redundant
policy/cache code while preserving behavior; G007 resolved the first G006
review blockers by returning consumer-owned copies of all plan-owned
mutable payloads from semantic-plan cache hits, keeping `para`
intentionally shared for append identity, and by making malformed non-nil
C single/batch results signal `ekp-backend-contract-error` while
preserving nil-result Elisp fallback and the 15-argument/15-field C
boundary; G008 resolved the final string-leaf context alias by using the
recursive context copier for cons/vector/string payloads both when the
context snapshot/cache key is created and when a plan is returned.
- GUI evidence recorded: `/tmp/ekp-g004-evidence.Tp77dW` has 12/12
checkpoints, a 25.75s/206-frame recording, no black segments, no failed
assertions, automatic inline wrapping 3→2→3 lines across 280→340→280,
every internal inline split as source whitespace, C active, exact source,
zero overlays, and policy no-hyphen→normal settling with generation_delta
1, live nil-plan span 1, stale nil-plan span 0, and current projection.
- Verified so far: G002 focused core policy ERT 22/22, no-break 5/5, cache
15/15, C parity 5/5, 300-case fuzz, warning-clean C build, byte
compilation, checkdoc, and diff-check; G003 core ERT 118/118, buffer ERT
119/119, watcher invariants 2/2, byte compilation, checkdoc, and
diff-check; G004 GUI verifier ERT 7/7 and clean GUI evidence; G005
formal four-interleaved evaluator layout parity true, core baseline
p50/p95 38.6679/51.6782ms versus candidate 21.7102/33.0040ms for
43.8549%/36.1354% gains, resize baseline p50/p95 43.3831/55.6250ms
versus candidate 22.0919/32.8202ms for 49.0773%/40.9973% gains, both
candidate p95 values under 50ms and both gain sets at least 20%; live
evaluator source-instrumented locked goal remains the known
`validation_failed` debt, with parity, zero-work, GC, conflict, and
all-width-nonregression true, current C p95/p99 26.449/26.740ms
(76.65%/77.23% improvement), current Elisp p95/p99 49.940/52.017ms
(91.64%/91.35% improvement), consistent with historical open `issue018`
and not a regression; byte-compiled production public path passed three
runs with zero GC, C append p99 1.361-1.368ms, C hard p99
1.876-1.891ms, Elisp append p99 1.692-1.775ms, and Elisp hard p99
2.100-2.230ms; default ERT 262/262, seeded permuted ERT seed 20260728
255/255, isolated per-test process suite exit 0, targeted alias guard
passed, property fuzz 300/300, warning-as-error byte compilation,
package-lint pinned at `35996f478d81e51dae4fa30d051f741895d07399` exit
0 with only an external obsolete warning from the local names dependency,
empty checkdoc, release, 49-entry dictionary manifest, pinned dictionary
update, shell syntax, CI YAML, diff-check, portable/native/debug/sanitize
warning-clean C builds, module 1.6/4-thread smoke, and focused C 19/19
including the 15-argument/15-field boundary passed; G006 targeted
cleanup verification passed 6/6 and 3/3, full core ERT 125/125 and
buffer ERT 120/120 passed, and byte compilation, diff-check, and C build
gates passed; G008 formal resize parity true, core baseline/candidate
p50/p95 36.595/48.357ms versus 23.266/36.363ms for 36.42%/24.80%
gains, resize baseline/candidate p50/p95 41.443/53.541ms versus
23.761/36.889ms for 42.67%/31.10% gains, ERT 268/268, fuzz 300/300,
release gates pass, and byte-compiled current public path records zero
GC with C append/hard p99 1.440-1.464/2.022-2.050ms and Elisp
append/hard p99 1.648-1.687/2.136-2.335ms, all below 16ms.
- Closure: task034 developer implementation, repository gates, performance
gates, cleanup gates, G007 first-blocker fixes, and the G008 final
string-leaf context fix are complete.
`issue021` is closed after user visual confirmation, historical `issue018`
remains open, and final independent review returned APPROVE with
architecture status CLEAR.
- task035 [x] Falsify line-width-scaled final-pass emergency stretch.
- Source: `issue021`, the user's 2026-08-02 narrow-showcase screenshot,
and the G009 upstream K-P review.
- Red tests: reproduce the mixed showcase paragraph at narrow widths and
reject any isolated CJK source line through the core, public buffer path,
and GUI oracle.
- Work attempted: keep the strict pass unchanged; in the final pass add a
line-width-sized finite background emergency stretch to ordinary
underfull candidates and score them through adjustment ratio, badness,
fitness, and demerits. Keep the fixed artificial emergency transition
only for truly overfull first permitted hard/atomic runs. Do not add
CJK-orphan, unit, or screenshot-specific penalties.
- Validation: focused core single-CJK ERT passes 2/2; focused core, C,
hard-atom, buffer, and GUI-oracle G009 coverage passes 18/18 in the
implementation lane; warning-as-error byte compilation and the C build
pass. Full regression then falsified the line-width stretch detail: it
made emergency stretch scale with the candidate measure instead of a TeX
fixed dimension and regressed existing emergency behavior.
- Impact: `ekp.el`, `ekp_c/ekp_kp.c`, tests, bilingual documentation,
phase records, changelog, and postmortem. Public configuration and the C
ABI are unchanged.
- Closure: task035 is closed as a disproved implementation attempt, not as
the accepted current behavior. Continue with `task036`.
- task036 [x] Implement fixed-dimension final-pass emergency stretch and
preserve the final active path.
- Source: task035 full-regression failure, `issue021`, and the G009
original K-P correction.
- Work: replace the candidate-width/line-width emergency stretch detail
with a TeX-style fixed dimension exposed as
`ekp-emergency-stretch-pixel`. A nil value auto-resolves to roughly three
display-font `M` widths; a non-negative integer fixes the stretch in
pixels. Keep the strict pass unchanged and the 15-field C paragraph ABI
stable. In the final pass, preserve the best provisional overfull path
with tight fitness and zero incremental demerits only when no normal
active path survives to that breakpoint. Do not add CJK-orphan, unit,
atom-adjacency, or screenshot-specific penalties.
- Red tests: add five focused RED cases covering fixed dimension versus
measure scaling, nil auto resolution, non-negative integer override,
Elisp/C parity without C ABI growth, and renderer width/glue consistency.
- Regression gate: the original three regressions that task035 broke must
stay green alongside the new RED cases.
- Full gate: focused G009 core/public-buffer/C parity and GUI-oracle
coverage, final fullscreen dynamic GUI evidence, default/permuted/
isolated ERT, 300-case fuzz, warning-as-error byte compilation, C builds,
checkdoc/package/static/release gates, and diff-check.
- Result: production Elisp 1D, looseness/parshape, and C implementations
match; renderer glue distribution uses the chosen line's actual rest;
explicit hard atoms forbid only interior breaks and may share an overflow
line with preceding ordinary content. The test oracle now requires a
nonempty source-covering plan so an empty result cannot false-green.
- Validation: focused root regressions pass 8/8, emergency selection passes
10/10, core/buffer/GUI oracle passes 8/8, full ERT passes 288/288, seeded
and isolated suites pass 279/279, fuzz passes 300/300, portable/native/
debug/sanitize C builds and warning-as-error production plus changed-test
byte compilation pass, checkdoc/package/static/release gates pass, formal
core/resize performance gates pass, and reviewed 42.78-second
single-window dynamic GUI evidence at
`/tmp/ekp-g009-evidence-retry.UOpPNp` returns PASS.
Final independent code review returns APPROVE and architecture review
returns CLEAR.
- Closure: developer implementation and full quality gate complete;
`issue021` closed after the user's 2026-08-02 visual confirmation.

18
CHANGELOG.md
View File

@ -8,6 +8,18 @@ project aims to follow [Semantic Versioning](https://semver.org/).
### Fixed
- Malformed non-nil C backend results no longer fall back to Elisp. Nil whole
results and nil per-item breaks remain soft failures; malformed single or
batch output now signals `ekp-backend-contract-error`, so a broken backend
cannot silently produce a different layout.
- An overwide `ekp-no-break` inline atom no longer collapses its preceding
CJK text into one-glyph emergency lines. The final pass now gives ordinary
underfull candidates finite background emergency stretch and lets normal
badness/fitness/demerits choose the layout. If an overfull candidate would
otherwise extinguish every active path, TeX-style artificial demerits
preserve the best provisional path with zero incremental cost. The rule is
content-independent and identical in the Elisp 1D, looseness/parshape, and
C engines.
- C-backed core layout and complete resize/reflow now reuse prepared
paragraph, DP, and gap geometry and avoid duplicate projection
publication. The four-round same-machine gate records p95 values of
@ -148,6 +160,12 @@ project aims to follow [Semantic Versioning](https://semver.org/).
### Internal
- Cached semantic layout plans now return consumer-owned copies of all
plan-owned mutable payloads: strings, context, boxes, offsets, line/glue/
gap records, and signatures. Layout context snapshots and returned contexts
recursively copy conses, vectors, and strings, so mutable policy inputs
cannot alias the cache key. The paragraph object remains intentionally
shared for stable append identity.
- DP reuse identity, edge-space exclusion, and the lossless marker
vocabulary now each have one directly tested rule owner. The consolidation
removes formula/property-list drift without adding modules or hot-loop

93
DEVELOPER.md
View File

@ -121,10 +121,12 @@ badness = min(10000, 100·|adjustment/flexibility|³)
```
Fitness classes (tight/decent/loose/very-loose) follow the TeX ratio
thresholds. Special cases: single-box lines use flexibility 1 and
fitness decent; the last line pays `(line-penalty + short-badness)²`
where `short-badness = last-line-short-penalty × (1 − fill)` when the
fill ratio is below `ekp-last-line-min-ratio`.
thresholds. Special cases: single-box lines use flexibility 1 in the
strict pass; the final pass uses the same finite emergency stretch as
ordinary underfull lines. The last line pays
`(line-penalty + short-badness)²` where
`short-badness = last-line-short-penalty × (1 − fill)` when the fill ratio
is below `ekp-last-line-min-ratio`.
Deviations from the 1981 paper, by design: penalties are always added
as `+p²` (no negative/flagged penalties), there is no `q`/looseness in
@ -135,12 +137,22 @@ the main pass (see §6), and adjacent-fitness is a flat constant.
Some inputs admit no valid layout: an unbreakable box wider than the
line, or a rigid (all-`nws`) region that cannot stretch to the target.
A strict pass runs first; if the paragraph end is unreachable, a second
pass additionally allows **emergency breaks** — single-box lines with
demerits `(line-penalty + 10000)² + rest²`, at least as bad as any
regular line. This guarantees, by induction over positions, that every
input produces output (regression: narrow CJK used to return an empty
string), while the common case pays nothing and keeps pure K-P
optimality. Both engines implement the identical strategy.
pass adds a finite background emergency stretch to ordinary underfull
candidates and still scores them through the same adjustment ratio,
badness, fitness, and demerits used by the strict pass. That keeps
underfull final-pass choices inside the global K-P DP instead of forcing
them through a separate fixed-cost path.
Separately, the final pass implements TeX's `artificial_demerits` safeguard.
When an overfull candidate would otherwise remove the last active path to a
breakpoint and no non-overfull candidate survives there, the best provisional
path is installed with tight fitness and zero incremental demerits. This is a
reachability rule, not a hard-atom scoring shortcut: it never competes while a
normal active path survives, and it does not inspect CJK, units, or token
classes. Thus every input still produces a complete plan (regression: narrow
CJK used to return an empty result), while ordinary underfull lines compete
by normal K-P cost. The Elisp 1D, looseness/parshape, and C engines implement
the identical strategy.
## 5. Rendering
@ -162,6 +174,16 @@ mechanism. `ekp-render-layout-string` consumes it for the public string API;
the buffer integration can consume the same decisions without re-running or
reinterpreting the KP algorithm.
Cached semantic plans are immutable to the cache owner. A cache hit returns
a consumer-owned copy of every plan-owned mutable payload: source string,
context tree, box vector and box strings, source offsets, line records,
line glues, gap records, and line signatures. `ekp-layout-plan-para` is the
intentional exception: paragraph-cache ownership predates the semantic-plan
cache, and append planning depends on stable paragraph identity.
Layout context snapshots and returned plan contexts use the same recursive
copier for conses, vectors, and strings, so mutable policy inputs cannot alias
the cache key or a later consumer plan.
The two consumers deliberately have different representation rights.
#### String renderer
@ -281,9 +303,33 @@ the one authoritative width.
own box; `ekp-para-breaks-allowed` forbids gaps per kinsoku (full-
and halfwidth), `ekp-no-break' spans and NBSP-family joiners.
Forbidden gaps carry no glue. The DP skips them as candidates while
the line keeps extending; the emergency fallback treats a run with
no permitted inner break as atomic. C receives the sparse
`forbidden-positions` vector.
the line keeps extending. In the final pass, legal underfull candidates
receive finite emergency stretch and are scored by normal
badness/demerits. If an overfull candidate would extinguish the last active
final-pass path, TeX-style artificial demerits preserve that path with zero
incremental cost. C receives the sparse `forbidden-positions` vector.
Explicit hard atoms forbid only breaks inside their interval; an otherwise
legal boundary immediately before or after an atom remains legal. Atom
adjacency therefore receives no special scoring or break prohibition.
- **Configurable policy compilation**: buffer and core policy variables are
resolved before tokenization into private structural intervals. Region
`ekp-break-policy` wins first, explicit buffer/file/dir locals win over
mode profiles, and profiles win over global defaults. Token policies are
the only merged category map; scalar and face-list options replace the
lower scope. The core may use private properties such as
`ekp--face-break-policy`, `ekp--no-hyphen`, and
`ekp--literal-spacing` on an analysis copy, but cache keys, plan strings,
boxes, rendered strings, and `ekp--last-para` must contain only public
source properties. Automatic face no-break is measured over the
contiguous private face-policy span and downgrades to no-hyphen when
overwide; explicit `ekp-no-break` never downgrades.
- **Literal inline spaces**: face-derived `no-hyphen` preserves source
spaces as literal boxes inside a line, forbids a break that would move a
literal source-space box to line start, and permits the complementary
break after a source-space box. When such whitespace is the selected
visual break, source ownership belongs to the break gap metadata, not to a
trailing visible line box. This rule is policy-derived and must not depend
on whether the source also carries a public `face` property.
- **Alignment** (`ekp-alignment`): non-justify modes zero the glue
stretch/shrink arrays and class params; the DP widens `max_w` by an
extra per-line stretch R (`ekp-c-set-penalties` arg 7), so badness =
@ -306,7 +352,9 @@ the one authoritative width.
C module 1.6: `ekp-c-break-with-arrays' takes 15 args
(…, forbidden-positions, tail-protrudes, hyphen-protrude,
first-line-width); batch vectors have 15 elements;
`ekp-c-set-penalties' takes 4–7.
`ekp-c-set-penalties' takes 4–8. Policy compilation feeds the existing
hyphen-position and forbidden-break vectors; it must not add a sixteenth C
argument or batch field without a new architecture decision.
Performance after the feature wave (byte-compiled + C, Apple
Silicon, batch): justify zh w=200 ≈ 54 ms, range zh ≈ 117 ms —
@ -349,19 +397,22 @@ the source of truth for all font-dependent data.
pool is created lazily on the first multi-paragraph batch and sized
to the machine's cores; a full queue blocks the submitter rather
than dropping the task.
- `ekp-c-set-penalties` (4–7 args): called by `ekp--c-sync-params`
- `ekp-c-set-penalties` (4–8 args): called by `ekp--c-sync-params`
before *every* C entry, so `ekp-line-penalty` & friends always take
effect (regression: they were never synced before).
- `ekp-c-module-load` refuses modules older than
`ekp-c-module-required-version` and falls back to Elisp, preventing
arity mismatches after upgrades.
An unavailable module, an allocation/no-result nil, or an incompatible
module version falls back to the Elisp engine. Invalid direct API input
signals `ekp-c-invalid-input`, and any signal from an enabled backend
propagates through the public formatter; the dispatcher does not catch and
hide it. The module never silently produces a different layout on partial
failure. The two engines are verified byte-identical by
An unavailable module, an incompatible module version, a nil whole C result,
or a nil per-item/break result falls back to the Elisp engine. Invalid
direct API input signals `ekp-c-invalid-input`. Any non-nil malformed
backend result signals `ekp-backend-contract-error`: malformed cons shape,
non-list breaks, non-integer/out-of-range/non-increasing/partial breaks,
nonnumeric cost, or malformed batch result shape. Any signal from an
enabled backend propagates through the public formatter; the dispatcher does
not catch and hide it. The module never silently produces a different
layout on partial failure. The two engines are verified byte-identical by
`ekp-test-c-parity-simple` / `ekp-test-c-parity-files` and the 300-case
property fuzz.

67
DEVELOPER_ZH.md
View File

@ -109,7 +109,8 @@ badness = min(10000, 100·|adjustment/flexibility|³)
```
松紧等级(tight/decent/loose/very-loose)沿用 TeX 的比例阈值。特殊情
况:单盒行 flexibility 固定为 1、fitness 为 decent;末行代价为
况:严格遍的单盒行 flexibility 固定为 1;最终遍使用与普通欠宽行相同
的有限 emergency stretch。末行代价为
`(line-penalty + 短行badness)²`,填充率低于 `ekp-last-line-min-ratio`
时 `短行badness = last-line-short-penalty × (1 − 填充率)`。
@ -121,10 +122,17 @@ penalty/flagged 断点),主流程无 `q`/looseness(见 §6),相邻松紧惩
某些输入不存在合法排版:比行宽更宽的不可断盒子,或无法伸展到目标宽
的刚性(全 `nws`)区段。先跑严格遍;若段尾不可达,第二遍额外允许
**紧急断行**——demerits 为 `(line-penalty + 10000)² + rest²` 的单盒行,
不低于任何常规行的代价。由位置归纳可证:任何输入必有输出(回归:窄
栏 CJK 曾整段返回空串),常规输入不付任何代价、保持纯 K-P 最优。两个
引擎实现完全相同的策略。
为普通欠宽候选加入有限 background emergency stretch,并继续走同一套
adjustment ratio、badness、fitness 和 demerits。这样最终遍的欠宽选择
仍由全局 K-P DP 比较,不会被塞进另一条固定代价路径。
另外,最终遍实现 TeX 的 `artificial_demerits` 可达性保护。如果某个
超宽候选将使一个断点的最后活动路径消失,且该断点没有任何未超宽候选
存活,就以 tight fitness 和零增量 demerits 安装最佳暂存路径。这不是
hard atom 专用的评分捷径:只要正常活动路径还在,它就不参与竞争,也不
检查中文、单位或 token 类别。因此任何输入仍会得到覆盖完整源文本的
plan(回归:窄栏 CJK 曾整段返回空结果),而普通欠宽行仍按正常 K-P 代价
竞争。Elisp 1D、looseness/parshape 与 C 引擎实现相同语义。
## 5. 渲染
@ -143,6 +151,15 @@ penalty/flagged 断点),主流程无 `q`/looseness(见 §6),相邻松紧惩
`ekp-render-layout-string` 用它服务现有字符串 API;buffer 集成可复用
完全相同的决策,无需重新运行或重新解释 KP 算法。
缓存的 semantic plan 对缓存 owner 不可变。缓存命中会返回 consumer
自有副本,覆盖所有 plan 自有的可变 payload:源字符串、context 树、盒
向量与盒字符串、源 offset、line 记录、line glue、gap 记录和 line
signature。`ekp-layout-plan-para` 是有意例外:段落缓存所有权早于
semantic-plan cache,append planning 依赖稳定的段落身份。
layout context snapshot 与返回 plan context 使用同一个递归 copier,
覆盖 cons、vector 与 string,因此可变 policy 输入不会别名到 cache key
或后续 consumer plan。
两个消费方拥有不同且明确的表示权限。
#### 字符串渲染器
@ -242,8 +259,27 @@ idle formatter。point 移动本身严格零副作用;之后若在别处发生
- **断行许可**:每个 CJK 字符(含标点)独立成盒;
`ekp-para-breaks-allowed` 按禁则(全角与半角)、`ekp-no-break`
区间及 NBSP 族连接符禁止相应间隙,被禁间隙不携带 glue。DP 跳过
被禁候选但继续延伸行;紧急兜底把"内部无许可断点的连跑段"视为
原子。C 侧接收稀疏 `forbidden-positions` 向量。
被禁候选但继续延伸行。最终遍中的合法欠宽候选获得有限 emergency
stretch 并按正常 badness/demerits 评分;若超宽候选将消灭最后活动
路径,TeX 风格 artificial demerits 以零增量代价保留该路径。C 侧接收
稀疏 `forbidden-positions` 向量。显式 hard atom 只禁止其区间内部
断行;atom 前后本来合法的边界仍然合法。atom 邻接不获得特殊评分,
也不会额外禁止断点。
- **可配置 policy 编译**:buffer 与 core policy 变量在 tokenization
之前解析为私有结构区间。区域 `ekp-break-policy` 优先,显式
buffer/file/dir local 值优先于 mode profile,profile 优先于全局默认。
只有 token policy 是按类别合并的 map;标量与 face 列表都直接替换
低优先级值。core 可以在 analysis copy 上使用
`ekp--face-break-policy`、`ekp--no-hyphen`、`ekp--literal-spacing`
等私有属性,但 cache key、plan string、box、rendered string 与
`ekp--last-para` 只能保留公开源属性。自动 face no-break 按连续的
私有 face-policy 区间整体测宽,超宽时降级为 no-hyphen;显式
`ekp-no-break` 永不降级。
- **行内字面空格**:face 派生的 `no-hyphen` 在行内保留源空格为字面
box,禁止会把源空格 box 移到行首的断点,并允许互补的"空格后到内容"
断点。当这些空白成为选中的视觉断点时,源归属属于 break gap metadata,
而不是尾随可见行 box。该规则来自 policy,不能依赖源上是否另有公开
`face` 属性。
- **对齐**(`ekp-alignment`):非两端对齐把 glue 伸缩数组与类参数
置零,DP 给 `max_w` 加每行额外伸展 R(`ekp-c-set-penalties` 第 7
参),badness = 100·(欠宽/R)³;渲染层按模式分派剩余(尾部/对半/
@ -261,7 +297,8 @@ idle formatter。point 移动本身严格零副作用;之后若在别处发生
C 模块 1.6:`ekp-c-break-with-arrays` 15 参(…、forbidden-positions、
tail-protrudes、hyphen-protrude、first-line-width);batch 向量 15 元;
`ekp-c-set-penalties` 4–7 参。
`ekp-c-set-penalties` 4–8 参。policy 编译只进入现有断词位置与禁断
向量;没有新的架构决策时,不得增加第 16 个 C 参数或 batch 字段。
特性完成后的性能(字节编译 + C,Apple Silicon,batch):justify zh
w=200 ≈ 54 ms、range zh ≈ 117 ms——justify 与特性前持平,range 因盒
@ -296,16 +333,20 @@ C 模块(`ekp_c/`,版本 1.6)只执行阶段 ④。所有字体相关数据以 E
——每段一个任务(这是正确的并行粒度;DP 本身天然串行)。线程池在
首次多段落 batch 时惰性创建,按机器核心数定大小;队列满时提交方
阻塞等待而非丢弃任务。
- `ekp-c-set-penalties`(4–7 参数):`ekp--c-sync-params` 在**每次**
- `ekp-c-set-penalties`(4–8 参数):`ekp--c-sync-params` 在**每次**
进入 C 之前调用,保证 `ekp-line-penalty` 等变量始终生效(回归:此
前从未同步)。
- `ekp-c-module-load` 拒绝低于 `ekp-c-module-required-version` 的模块
并回落到 Elisp,避免升级后的参数数量不匹配。
模块不可用、分配/无结果返回 nil 或 ABI 版本不兼容时回落到 Elisp。
直接 API 的非法输入 signal `ekp-c-invalid-input`;已启用后端发出的
任何 signal 都会穿过公共 formatter,dispatcher 不捕获或隐藏。模块
不会在部分失败时静默产出不同的排版。两引擎输出逐字节一致,由
模块不可用、ABI 版本不兼容、整批 C 结果为 nil 或单项/断点结果为
nil 时回落到 Elisp。直接 API 的非法输入 signal
`ekp-c-invalid-input`。任何非 nil 的畸形后端结果 signal
`ekp-backend-contract-error`:结果 cons 形状错误、breaks 不是 list、
break 非整数/越界/非递增/未覆盖段尾、cost 非数字,或 batch 结果形状
错误。已启用后端发出的任何 signal 都会穿过公共 formatter,dispatcher
不捕获或隐藏。模块不会在部分失败时静默产出不同的排版。两引擎输出
逐字节一致,由
`ekp-test-c-parity-simple` / `ekp-test-c-parity-files` 及 300 例性质
fuzz 验证。

659
ekp-buffer.el
View File

@ -78,19 +78,110 @@ command and is not limited by this value."
"Seconds of work allowed in one lazy reflow tick."
:type 'number)
(defconst ekp-buffer-org-skip-faces
'(org-block org-block-begin-line org-block-end-line org-code
org-verbatim org-table org-meta-line)
(defconst ekp-buffer-org-block-faces
'(org-block org-block-begin-line org-block-end-line
org-table org-meta-line)
"Org block faces whose paragraphs stay verbatim.")
(defconst ekp-buffer-org-inline-faces
'(org-code org-verbatim)
"Org inline faces that receive automatic inline break policy.")
(defconst ekp-buffer-org-skip-faces ekp-buffer-org-block-faces
"Reasonable `ekp-buffer-skip-faces' preset for Org buffers.")
(defconst ekp-buffer-markdown-skip-faces
'(markdown-code-face markdown-inline-code-face markdown-pre-face
markdown-table-face)
(defconst ekp-buffer-markdown-block-faces
'(markdown-code-face markdown-pre-face markdown-table-face)
"Markdown block faces whose paragraphs stay verbatim.")
(defconst ekp-buffer-markdown-inline-faces
'(markdown-inline-code-face)
"Markdown inline faces that receive automatic inline break policy.")
(defconst ekp-buffer-markdown-skip-faces ekp-buffer-markdown-block-faces
"Reasonable `ekp-buffer-skip-faces' preset for Markdown buffers.")
(defun ekp-buffer--safe-symbol-list-p (value)
"Return non-nil when VALUE is a closed list of symbols."
(and (proper-list-p value) (seq-every-p #'symbolp value)))
(defun ekp-buffer--safe-measure-p (value)
"Return non-nil when VALUE is a documented measure mode."
(or (eq value 'narrowest-window)
(and (integerp value) (> value 0))
(and (consp value) (eq (car value) 'max)
(integerp (cdr value)) (> (cdr value) 0))))
(defun ekp-buffer--safe-mode-policy-entry-p (entry)
"Return non-nil when ENTRY is a closed mode profile entry."
(and (consp entry)
(symbolp (car entry))
(proper-list-p (cdr entry))
(seq-every-p
(lambda (setting)
(and (consp setting)
(pcase (car setting)
('ekp-inline-code-policy
(ekp--safe-break-policy-value-p (cdr setting)))
('ekp-hyphenation
(ekp--safe-hyphenation-value-p (cdr setting)))
('ekp-overlong-token-policy
(memq (cdr setting) '(emergency overflow natural)))
('ekp-emergency-stretch-pixel
(ekp--safe-emergency-stretch-pixel-p (cdr setting)))
('ekp-kinsoku-profile
(memq (cdr setting) '(common zh ja off custom)))
('ekp-cjk-no-line-start-extra
(stringp (cdr setting)))
('ekp-cjk-no-line-end-extra
(stringp (cdr setting)))
('ekp-number-unit-suffixes
(and (proper-list-p (cdr setting))
(seq-every-p #'stringp (cdr setting))))
('ekp-token-break-policies
(ekp--safe-token-break-policies-p (cdr setting)))
('ekp-buffer-skip-faces
(ekp-buffer--safe-symbol-list-p (cdr setting)))
('ekp-buffer-inline-faces
(ekp-buffer--safe-symbol-list-p (cdr setting)))
('ekp-buffer-measure
(ekp-buffer--safe-measure-p (cdr setting)))
(_ nil))))
(cdr entry))))
(defun ekp-buffer--safe-mode-policy-alist-p (value)
"Return non-nil when VALUE is a safe mode policy alist."
(and (proper-list-p value)
(seq-every-p #'ekp-buffer--safe-mode-policy-entry-p value)))
(defcustom ekp-buffer-skip-faces nil
"Faces whose paragraphs stay verbatim."
:type '(repeat face))
:type '(repeat face)
:safe #'ekp-buffer--safe-symbol-list-p)
(defcustom ekp-buffer-inline-faces nil
"Faces whose exact spans use `ekp-inline-code-policy'."
:type '(repeat face)
:safe #'ekp-buffer--safe-symbol-list-p)
(defcustom ekp-buffer-mode-policy-alist
`((org-mode . ((ekp-buffer-skip-faces . ,ekp-buffer-org-block-faces)
(ekp-buffer-inline-faces . ,ekp-buffer-org-inline-faces)))
(markdown-mode
. ((ekp-buffer-skip-faces . ,ekp-buffer-markdown-block-faces)
(ekp-buffer-inline-faces . ,ekp-buffer-markdown-inline-faces))))
"Mode profiles consulted by automatic and manual buffer layout."
:type '(alist :key-type symbol :value-type sexp)
:safe #'ekp-buffer--safe-mode-policy-alist-p)
(defcustom ekp-buffer-measure 'narrowest-window
"Measure used for automatic and default manual buffer layout.
`narrowest-window' uses the narrowest live window displaying the
buffer. A positive integer is a fixed pixel width. `(max . N)'
caps the narrowest live window at N pixels."
:type '(choice (const narrowest-window) (integer :tag "Fixed pixels")
(cons (const max) (integer :tag "Maximum pixels")))
:safe #'ekp-buffer--safe-measure-p)
(defvar-local ekp-buffer-skip-predicate nil
"Function called with a paragraph string that should stay verbatim.")
@ -110,6 +201,9 @@ command and is not limited by this value."
(defvar-local ekp-buffer--auto-width nil)
(defvar-local ekp-buffer--resize-timer nil)
(defvar-local ekp-buffer--composition-timer nil)
(defvar-local ekp-buffer--policy-reflow-timer nil)
(defvar-local ekp-buffer--policy-reflow-context nil)
(defvar-local ekp-buffer--policy-context-snapshot nil)
(defvar-local ekp-buffer--pending nil)
(defvar-local ekp-buffer--chunk-timer nil)
(defvar-local ekp-buffer--generation 0)
@ -155,15 +249,127 @@ command and is not limited by this value."
;;; Setup
(defconst ekp-buffer--policy-variables
'(ekp-inline-code-policy ekp-hyphenation ekp-token-break-policies
ekp-number-unit-suffixes
ekp-overlong-token-policy ekp-kinsoku-profile
ekp-emergency-stretch-pixel
ekp-cjk-no-line-start-extra ekp-cjk-no-line-end-extra
ekp-buffer-skip-faces ekp-buffer-inline-faces
ekp-buffer-mode-policy-alist ekp-buffer-measure)
"Variables that affect buffer layout policy or measure.")
(defun ekp-buffer--matching-mode-profile ()
"Return the first matching mode policy profile for the current buffer."
(seq-some
(lambda (entry)
(and (derived-mode-p (car entry))
(cdr entry)))
ekp-buffer-mode-policy-alist))
(defun ekp-buffer--profile-value (profile variable)
"Return VARIABLE's value from PROFILE, or nil if absent."
(assq variable profile))
(defun ekp-buffer--effective-scalar (profile variable)
"Return VARIABLE for PROFILE after local, mode-profile, then global precedence."
(if (local-variable-p variable)
(symbol-value variable)
(let ((current (symbol-value variable))
(default (default-value variable))
(profile-value (ekp-buffer--profile-value profile variable)))
(cond
((not (equal current default)) current)
(profile-value (cdr profile-value))
(t default)))))
(defun ekp-buffer--merge-token-policies (profile)
"Return effective token policies after merging global, PROFILE, and local."
(let ((merged (copy-tree (default-value 'ekp-token-break-policies)))
(current ekp-token-break-policies)
(default (default-value 'ekp-token-break-policies)))
(dolist (entry (cdr (ekp-buffer--profile-value
profile 'ekp-token-break-policies)))
(setf (alist-get (car entry) merged) (cdr entry)))
(when (or (local-variable-p 'ekp-token-break-policies)
(not (equal current default)))
(dolist (entry current)
(setf (alist-get (car entry) merged) (cdr entry))))
merged))
(defun ekp-buffer--policy-context (&optional width)
"Return the effective policy context for the current buffer at WIDTH."
(let* ((profile (ekp-buffer--matching-mode-profile))
(emergency-stretch
(ekp-buffer--effective-scalar
profile 'ekp-emergency-stretch-pixel))
(resolved-emergency-stretch
(let ((ekp-emergency-stretch-pixel emergency-stretch))
(ekp--resolved-emergency-stretch-pixel))))
(ekp--copy-layout-context-value
(list :inline-code-policy
(ekp-buffer--effective-scalar profile 'ekp-inline-code-policy)
:hyphenation
(ekp-buffer--effective-scalar profile 'ekp-hyphenation)
:token-break-policies (ekp-buffer--merge-token-policies profile)
:number-unit-suffixes
(ekp-buffer--effective-scalar profile 'ekp-number-unit-suffixes)
:overlong-token-policy
(ekp-buffer--effective-scalar profile 'ekp-overlong-token-policy)
:emergency-stretch-pixel resolved-emergency-stretch
:kinsoku-profile
(ekp-buffer--effective-scalar profile 'ekp-kinsoku-profile)
:cjk-no-line-start-extra
(ekp-buffer--effective-scalar
profile 'ekp-cjk-no-line-start-extra)
:cjk-no-line-end-extra
(ekp-buffer--effective-scalar profile 'ekp-cjk-no-line-end-extra)
:block-faces
(ekp-buffer--effective-scalar profile 'ekp-buffer-skip-faces)
:inline-faces
(ekp-buffer--effective-scalar profile 'ekp-buffer-inline-faces)
:measure
(ekp-buffer--effective-scalar profile 'ekp-buffer-measure)
:width width))))
(defmacro ekp-buffer--with-policy-context (context &rest body)
"Run BODY with CONTEXT bound as EKP's core policy environment."
(declare (indent 1))
`(let ((ekp-inline-code-policy
(plist-get ,context :inline-code-policy))
(ekp-hyphenation (plist-get ,context :hyphenation))
(ekp-token-break-policies
(plist-get ,context :token-break-policies))
(ekp-number-unit-suffixes
(plist-get ,context :number-unit-suffixes))
(ekp-overlong-token-policy
(plist-get ,context :overlong-token-policy))
(ekp-emergency-stretch-pixel
(plist-get ,context :emergency-stretch-pixel))
(ekp-kinsoku-profile (plist-get ,context :kinsoku-profile))
(ekp-cjk-no-line-start-extra
(plist-get ,context :cjk-no-line-start-extra))
(ekp-cjk-no-line-end-extra
(plist-get ,context :cjk-no-line-end-extra)))
,@body))
(defun ekp-buffer--face-member-p (face faces)
"Return non-nil when FACE intersects FACES."
(if (listp face)
(seq-intersection face faces)
(memq face faces)))
;;;###autoload
(defun ekp-org-setup ()
"Protect common Org structural faces in the current buffer."
(setq-local ekp-buffer-skip-faces ekp-buffer-org-skip-faces))
(setq-local ekp-buffer-skip-faces ekp-buffer-org-block-faces)
(setq-local ekp-buffer-inline-faces ekp-buffer-org-inline-faces))
;;;###autoload
(defun ekp-markdown-setup ()
"Protect common Markdown code faces in the current buffer."
(setq-local ekp-buffer-skip-faces ekp-buffer-markdown-skip-faces))
(setq-local ekp-buffer-skip-faces ekp-buffer-markdown-block-faces)
(setq-local ekp-buffer-inline-faces ekp-buffer-markdown-inline-faces))
;;; Width
@ -208,40 +414,66 @@ command and is not limited by this value."
(point))))
(defun ekp-buffer--effective-width (&optional buffer)
"Return the authoritative width for BUFFER.
The narrowest live window wins because text properties are buffer-wide."
(let ((windows (get-buffer-window-list
(or buffer (current-buffer)) nil t)))
(if windows
(apply #'min (mapcar #'ekp-buffer--window-pixel windows))
(ekp-buffer--window-pixel))))
"Return BUFFER's effective `ekp-buffer-measure' width.
The default uses the narrowest live window; fixed widths and `(max . N)'
come from the active buffer policy."
(with-current-buffer (or buffer (current-buffer))
(let* ((windows (get-buffer-window-list (current-buffer) nil t))
(narrowest (if windows
(apply #'min
(mapcar #'ekp-buffer--window-pixel windows))
(ekp-buffer--window-pixel)))
(measure (plist-get (ekp-buffer--policy-context) :measure)))
(cond
((eq measure 'narrowest-window) narrowest)
((and (integerp measure) (> measure 0)) measure)
((and (consp measure) (eq (car measure) 'max)
(integerp (cdr measure)) (> (cdr measure) 0))
(min narrowest (cdr measure)))
(t (error "Invalid ekp-buffer-measure: %S" measure))))))
(defun ekp-buffer--measure-report (&optional buffer)
"Return measure diagnostics for BUFFER."
(with-current-buffer (or buffer (current-buffer))
(let* ((windows (get-buffer-window-list (current-buffer) nil t))
(narrowest (if windows
(apply #'min
(mapcar #'ekp-buffer--window-pixel windows))
(ekp-buffer--window-pixel)))
(context (ekp-buffer--policy-context))
(requested (plist-get context :measure))
(effective (ekp-buffer--effective-width)))
(list :requested requested
:narrowest narrowest
:effective effective
:overflow-risk (and (integerp effective)
(> effective narrowest))))))
;;; Paragraphs and conflicts
(defun ekp-buffer--face-hit-p (string)
"Return non-nil when STRING carries a configured skip face."
(defun ekp-buffer--face-hit-p (string faces)
"Return non-nil when STRING carries one of FACES."
(let ((pos 0) (length (length string)) hit)
(while (and (< pos length) (not hit))
(let ((face (get-text-property pos 'face string)))
(setq hit (if (listp face)
(seq-intersection face ekp-buffer-skip-faces)
(memq face ekp-buffer-skip-faces)))
(setq hit (ekp-buffer--face-member-p face faces))
(setq pos (or (next-single-property-change
pos 'face string length)
length))))
hit))
(defun ekp-buffer--skip-paragraph-p (paragraph)
"Return non-nil when PARAGRAPH must stay verbatim."
(or (string-blank-p paragraph)
(text-property-not-all 0 (length paragraph)
'ekp-verbatim nil paragraph)
(text-property-not-all 0 (length paragraph) 'field nil paragraph)
(text-property-not-all 0 (length paragraph) 'read-only nil paragraph)
(and ekp-buffer-skip-faces
(ekp-buffer--face-hit-p paragraph))
(and ekp-buffer-skip-predicate
(funcall ekp-buffer-skip-predicate paragraph))))
(defun ekp-buffer--skip-paragraph-p (paragraph &optional context)
"Return non-nil when PARAGRAPH must stay verbatim under CONTEXT."
(let* ((context (or context (ekp-buffer--policy-context)))
(faces (plist-get context :block-faces)))
(or (string-blank-p paragraph)
(text-property-not-all 0 (length paragraph)
'ekp-verbatim nil paragraph)
(text-property-not-all 0 (length paragraph) 'field nil paragraph)
(text-property-not-all 0 (length paragraph) 'read-only nil paragraph)
(and faces (ekp-buffer--face-hit-p paragraph faces))
(and ekp-buffer-skip-predicate
(funcall ekp-buffer-skip-predicate paragraph)))))
(defun ekp-buffer--foreign-property-at-p (position property)
"Return non-nil when PROPERTY at POSITION is not owned by EKP."
@ -608,17 +840,53 @@ EFFECTIVE-END includes the source boundary owned by the line."
(goto-char end)))))
(nreverse ranges)))
(defun ekp-buffer--annotate-inline-faces (text context)
"Annotate inline face spans on TEXT using CONTEXT."
(let ((faces (plist-get context :inline-faces))
(policy (plist-get context :inline-code-policy))
(pos 0)
(length (length text))
annotated)
(when (and faces (not (eq policy 'normal)))
(while (< pos length)
(let* ((face (get-text-property pos 'face text))
(next (or (next-single-property-change pos 'face text length)
length)))
(when (ekp-buffer--face-member-p face faces)
(unless annotated
(setq annotated (copy-sequence text)))
(put-text-property
pos next 'ekp--face-break-policy policy annotated))
(setq pos next))))
(or annotated text)))
(defun ekp-buffer--planning-text (text context)
"Return TEXT annotated with transient inline policy from CONTEXT."
(ekp-buffer--annotate-inline-faces text context))
(defun ekp-buffer--layout-plan (text pixel context)
"Return layout plan for TEXT at PIXEL under CONTEXT."
(let ((planning-text (ekp-buffer--planning-text text context)))
(ekp-buffer--with-policy-context context
(let ((plan (ekp-layout-plan planning-text pixel)))
(setf (ekp-layout-plan-string plan) text)
plan))))
(defun ekp-buffer--layout-paragraph (beg end pixel)
"Project one hard paragraph from BEG to END at PIXEL."
(if (and ekp-buffer--automatic-pass
(> (- end beg) ekp-auto-justify-paragraph-limit))
(ekp-buffer--record-conflict
beg end "paragraph exceeds the automatic paragraph limit")
(let ((paragraph (buffer-substring beg end)))
(unless (ekp-buffer--skip-paragraph-p paragraph)
(let ((plan (ekp-layout-plan paragraph pixel)))
(when (ekp-buffer--projectable-p plan beg beg end)
(ekp-buffer--install-plan beg end pixel plan)))))))
(let* ((paragraph (buffer-substring beg end))
(context (ekp-buffer--policy-context pixel)))
(unless (ekp-buffer--skip-paragraph-p paragraph context)
(let ((plan (ekp-buffer--layout-plan paragraph pixel context)))
(if (eq (ekp-layout-plan-state plan) 'natural)
(ekp-buffer--record-conflict
beg end "overlong-token-natural")
(when (ekp-buffer--projectable-p plan beg beg end)
(ekp-buffer--install-plan beg end pixel plan))))))))
(defun ekp-buffer--dwim-bounds ()
"Return active region bounds or the hard paragraph at point."
@ -636,13 +904,16 @@ EFFECTIVE-END includes the source boundary owned by the line."
(list start finish
(and current-prefix-arg
(prefix-numeric-value current-prefix-arg))))))
(setq pixel (or pixel (ekp-buffer--window-pixel)))
(setq pixel (or pixel (ekp-buffer--effective-width)))
(let ((start (min beg end))
(finish (max beg end)))
(setq beg start
end finish))
(when (and font-lock-mode
(or ekp-buffer-skip-faces ekp-buffer-skip-predicate))
(let ((context (ekp-buffer--policy-context pixel)))
(or (plist-get context :block-faces)
(plist-get context :inline-faces)
ekp-buffer-skip-predicate)))
(font-lock-ensure beg end))
(let ((point-before (point))
(mark-before (and (mark t) (copy-marker (mark t))))
@ -759,20 +1030,76 @@ EFFECTIVE-END includes the source boundary owned by the line."
;;; Protection commands
(defvar ekp-break-policy nil
"Text property controlling regional break policy.
`normal' clears automatic token or face restrictions, `hyphenate'
enables discretionary hyphenation, and `no-hyphen' suppresses it.
This property is not a hard no-break switch; rigid atoms remain
owned by the separate `ekp-no-break' text property.")
(defun ekp-buffer--interactive-protection-args ()
"Return region arguments for an interactive protection command."
(barf-if-buffer-read-only)
(list (region-beginning) (region-end) t))
(defun ekp-buffer--after-region-policy-mutation (beg end)
"Refresh projections between BEG and END after a region policy mutation."
(if ekp-auto-justify-mode
(ekp-buffer--reflow-for-policy-change (current-buffer) t)
(ekp-buffer--clear-projection (min beg end) (max beg end))
(unless ekp-buffer--spans
(ekp-buffer--remove-integrations))))
(defun ekp-buffer--set-protection (beg end property enabled label announce)
"Set PROPERTY to ENABLED from BEG to END and optionally ANNOUNCE LABEL."
(if enabled
(add-text-properties beg end (list property t))
(remove-text-properties beg end (list property nil)))
(let ((ekp-buffer--inhibit t))
(if enabled
(add-text-properties beg end (list property t))
(remove-text-properties beg end (list property nil))))
(ekp-buffer--after-region-policy-mutation beg end)
(when announce
(message "EKP: %s on %d characters; current buffer session only"
label (- end beg))))
(defun ekp-buffer--set-break-policy (beg end policy label announce)
"Set `ekp-break-policy' POLICY from BEG to END."
(let ((ekp-buffer--inhibit t))
(if policy
(put-text-property beg end 'ekp-break-policy policy)
(remove-text-properties beg end '(ekp-break-policy nil))))
(ekp-buffer--after-region-policy-mutation beg end)
(when announce
(message "EKP: %s on %d characters; current buffer session only"
label (- end beg))))
;;;###autoload
(defun ekp-normal-break-region (beg end &optional announce)
"Mark BEG through END as ordinary break policy; ANNOUNCE reports it."
(interactive (ekp-buffer--interactive-protection-args))
(ekp-buffer--set-break-policy
beg end 'normal "Marked normal break policy" announce))
;;;###autoload
(defun ekp-enable-hyphenation-region (beg end &optional announce)
"Enable discretionary hyphenation in BEG through END; ANNOUNCE reports it."
(interactive (ekp-buffer--interactive-protection-args))
(ekp-buffer--set-break-policy
beg end 'hyphenate "Enabled hyphenation" announce))
;;;###autoload
(defun ekp-disable-hyphenation-region (beg end &optional announce)
"Disable discretionary hyphenation in BEG through END; ANNOUNCE reports it."
(interactive (ekp-buffer--interactive-protection-args))
(ekp-buffer--set-break-policy
beg end 'no-hyphen "Disabled hyphenation" announce))
;;;###autoload
(defun ekp-clear-break-policy-region (beg end &optional announce)
"Clear `ekp-break-policy' from BEG through END; ANNOUNCE reports it."
(interactive (ekp-buffer--interactive-protection-args))
(ekp-buffer--set-break-policy
beg end nil "Cleared break policy" announce))
;;;###autoload
(defun ekp-no-break-region (beg end &optional announce)
"Mark BEG through END as an unbreakable session-local atom.
@ -990,10 +1317,12 @@ LINE-END lets insertion at the hard-line end belong to its last span."
(defun ekp-buffer--live-marker-snapshot (base)
"Return live marker offsets relative to BASE."
(mapcar
(delq nil
(mapcar
(lambda (marker)
(cons marker (- (marker-position marker) base)))
(ekp-buffer--live-state-owned-markers)))
(when-let* ((position (marker-position marker)))
(cons marker (- position base))))
(ekp-buffer--live-state-owned-markers))))
(defun ekp-buffer--start-live-edit (beg end bounds)
"Start a stable edit transaction for BEG through END in BOUNDS."
@ -1098,21 +1427,26 @@ LINE-END lets insertion at the hard-line end belong to its last span."
(buffer-substring dirty-beg dirty-end)
(substring source finish))))
(defun ekp-buffer--live-context (width)
"Return non-text layout context for a live plan at WIDTH."
(list (ekp--dp-key width)
(copy-tree (ekp--width-context))
(mapcar (lambda (attribute)
(face-attribute 'default attribute nil t))
'(:family :height :width :weight :slant))
ekp-latin-lang
ekp-alignment
ekp-ragged-stretch-pixel
(and ekp-protrusion (copy-tree ekp-protrusion-ratios))
(copy-tree ekp-parshape)
ekp-first-line-indent
ekp-cjk-no-line-start-extra
(ekp--spacing-signature)))
(defun ekp-buffer--live-context (width &optional context)
"Return non-text layout context for a live plan at WIDTH.
Optional CONTEXT supplies a precomputed policy context."
(let ((context (or context (ekp-buffer--policy-context width))))
(ekp-buffer--with-policy-context context
(ekp--copy-layout-context-value
(list (ekp--dp-key width)
(ekp--width-context)
(mapcar (lambda (attribute)
(face-attribute 'default attribute nil t))
'(:family :height :width :weight :slant))
ekp-latin-lang
ekp-alignment
ekp-ragged-stretch-pixel
(and ekp-protrusion ekp-protrusion-ratios)
ekp-parshape
ekp-first-line-indent
context
(ekp--policy-signature)
(ekp--spacing-signature))))))
(defun ekp-buffer--live-key-equal-p (left right)
"Return non-nil when live cache keys LEFT and RIGHT are equivalent."
@ -1151,16 +1485,26 @@ LINE-END lets insertion at the hard-line end belong to its last span."
ekp-buffer--live-state))))
(when (and old-key old-plan
(equal (cadr key) (cadr old-key)))
(ekp-layout-plan-append old-plan text width))))
(let* ((context (ekp-buffer--policy-context width))
(planning-text (ekp-buffer--planning-text text context))
(plan (ekp-buffer--with-policy-context context
(ekp-layout-plan-append
old-plan planning-text width))))
(when plan
(setf (ekp-layout-plan-string plan) text))
plan))))
(defun ekp-buffer--live-plan-entry (text width)
"Return the cache entry for TEXT at WIDTH."
(let ((key (list text (ekp-buffer--live-context width))))
(let* ((context (ekp-buffer--policy-context width))
(key (list (copy-sequence text)
(ekp-buffer--live-context width context))))
(cons key
(or (ekp-buffer--live-cache-get key)
(ekp-buffer--live-cache-put
key (or (ekp-buffer--live-append-plan key text width)
(ekp-layout-plan text width)))))))
(ekp-buffer--layout-plan
text width context)))))))
(defun ekp-buffer--single-line-live-p (text width)
"Return non-nil if TEXT is conservatively known to fit WIDTH."
@ -1401,34 +1745,40 @@ When COMPLETE is non-nil, project every semantic line."
(beg end width text boundary complete)
"Publish TEXT's stable prefix from BEG to END at WIDTH.
BOUNDARY identifies the native row left natural unless COMPLETE is non-nil."
(cond
((or (>= beg end) (ekp-buffer--skip-paragraph-p text))
(ekp-buffer--commit-natural-live-state beg end width text))
((> (- end beg) ekp-auto-justify-paragraph-limit)
(ekp-buffer--record-conflict
beg end "paragraph exceeds the automatic paragraph limit")
(ekp-buffer--commit-natural-live-state beg end width text))
((and (not complete)
(ekp-buffer--single-line-live-p text width))
(ekp-buffer--commit-natural-live-state beg end width text))
((ekp-buffer--unsupported-live-text-p text)
(ekp-buffer--record-conflict
beg end "unsupported whitespace shrink")
(ekp-buffer--commit-natural-live-state beg end width text))
((ekp-buffer--foreign-property beg end)
(ekp-buffer--record-conflict beg end "foreign property in live line")
(ekp-buffer--commit-natural-live-state beg end width text))
(t
(let* ((entry (ekp-buffer--live-plan-entry text width))
(key (car entry))
(plan (cdr entry)))
(if (ekp-buffer--unsupported-gap plan beg)
(progn
(ekp-buffer--record-conflict
beg end "unsupported live whitespace shrink")
(ekp-buffer--commit-natural-live-state beg end width text))
(ekp-buffer--project-live-plan
key plan beg end width boundary complete))))))
(let ((context (ekp-buffer--policy-context width)))
(cond
((or (>= beg end) (ekp-buffer--skip-paragraph-p text context))
(ekp-buffer--commit-natural-live-state beg end width text))
((> (- end beg) ekp-auto-justify-paragraph-limit)
(ekp-buffer--record-conflict
beg end "paragraph exceeds the automatic paragraph limit")
(ekp-buffer--commit-natural-live-state beg end width text))
((and (not complete)
(ekp-buffer--single-line-live-p text width))
(ekp-buffer--commit-natural-live-state beg end width text))
((ekp-buffer--unsupported-live-text-p text)
(ekp-buffer--record-conflict
beg end "unsupported whitespace shrink")
(ekp-buffer--commit-natural-live-state beg end width text))
((ekp-buffer--foreign-property beg end)
(ekp-buffer--record-conflict beg end "foreign property in live line")
(ekp-buffer--commit-natural-live-state beg end width text))
(t
(let* ((entry (ekp-buffer--live-plan-entry text width))
(key (car entry))
(plan (cdr entry)))
(cond
((eq (ekp-layout-plan-state plan) 'natural)
(ekp-buffer--record-conflict beg end "overlong-token-natural")
(ekp-buffer--commit-natural-live-state beg end width text))
((ekp-buffer--unsupported-gap plan beg)
(progn
(ekp-buffer--record-conflict
beg end "unsupported live whitespace shrink")
(ekp-buffer--commit-natural-live-state beg end width text)))
(t
(ekp-buffer--project-live-plan
key plan beg end width boundary complete))))))))
(defun ekp-buffer--publish-live-prefix (&optional boundary complete)
"Publish one stable live plan through BOUNDARY.
@ -1844,6 +2194,8 @@ ACTIVE-BEG through ACTIVE-END remains owned by the refreshed live state."
(point-min) (point-max) width active-beg active-end)
(ekp-buffer--lazy-reflow
width active-beg active-end)))))
(setq ekp-buffer--policy-context-snapshot
(copy-tree (ekp-buffer--policy-context)))
(ekp-buffer--refresh-live-prefix boundary complete))))))
(defun ekp-buffer--run-scheduled-reflow (buffer)
@ -1868,6 +2220,83 @@ ACTIVE-BEG through ACTIVE-END remains owned by the refreshed live state."
#'ekp-buffer--run-scheduled-reflow
(current-buffer)))))))
(defun ekp-buffer--reflow-for-policy-change (buffer &optional force)
"Bump BUFFER generation and reflow once after a policy change.
When FORCE is non-nil, reflow even if the global policy context is
unchanged; region text properties are paragraph policy too."
(when (buffer-live-p buffer)
(with-current-buffer buffer
(when ekp-auto-justify-mode
(when (timerp ekp-buffer--policy-reflow-timer)
(cancel-timer ekp-buffer--policy-reflow-timer))
(let ((before-context ekp-buffer--policy-reflow-context)
(after-context (ekp-buffer--policy-context)))
(setq-local ekp-buffer--policy-reflow-timer nil)
(setq-local ekp-buffer--policy-reflow-context nil)
(if (and (not force) (equal before-context after-context))
(setq-local ekp-buffer--policy-context-snapshot
(copy-tree after-context))
(let ((point-before (point))
(mark-before (and (mark t) (copy-marker (mark t))))
(mark-active-before mark-active)
(modified-before (buffer-modified-p))
(undo-before buffer-undo-list))
(unwind-protect
(progn
(cl-incf ekp-buffer--generation)
(setq ekp-buffer--live-plan-cache nil)
(ekp-buffer--reflow
buffer (ekp-buffer--effective-width buffer))
(goto-char (min point-before (point-max)))
(when mark-before
(set-marker (mark-marker)
(marker-position mark-before)))
(setq mark-active mark-active-before)
(set-buffer-modified-p modified-before)
(setq buffer-undo-list undo-before))
(when mark-before (set-marker mark-before nil))))))))))
(defun ekp-buffer--schedule-policy-reflow (buffer before-context)
"Schedule one post-set policy reflow for BUFFER from BEFORE-CONTEXT."
(when (buffer-live-p buffer)
(with-current-buffer buffer
(when ekp-auto-justify-mode
(unless (timerp ekp-buffer--policy-reflow-timer)
(setq-local ekp-buffer--policy-reflow-context
(copy-tree before-context)))
(when (timerp ekp-buffer--policy-reflow-timer)
(cancel-timer ekp-buffer--policy-reflow-timer))
(setq-local
ekp-buffer--policy-reflow-timer
(run-with-timer
0 nil #'ekp-buffer--reflow-for-policy-change buffer))))))
(defun ekp-buffer--watched-raw-noop-p (symbol new-value where)
"Return non-nil when setting SYMBOL to NEW-VALUE in WHERE is a raw no-op."
(if (bufferp where)
nil
(equal (default-value symbol) new-value)))
(defun ekp-buffer--policy-variable-changed (symbol new-value operation where)
"Schedule reflow after SYMBOL is set to NEW-VALUE by OPERATION in WHERE."
(when (memq operation '(set makunbound))
(unless (ekp-buffer--watched-raw-noop-p symbol new-value where)
(if (bufferp where)
(with-current-buffer where
(ekp-buffer--schedule-policy-reflow
where (or ekp-buffer--policy-context-snapshot
(ekp-buffer--policy-context))))
(dolist (buffer (buffer-list))
(with-current-buffer buffer
(ekp-buffer--schedule-policy-reflow
buffer (or ekp-buffer--policy-context-snapshot
(ekp-buffer--policy-context)))))))))
(dolist (variable ekp-buffer--policy-variables)
(unless (memq #'ekp-buffer--policy-variable-changed
(get-variable-watchers variable))
(add-variable-watcher variable #'ekp-buffer--policy-variable-changed)))
(defun ekp-buffer--on-resize (window-or-frame)
"Schedule reflow for the buffer shown by WINDOW-OR-FRAME."
(let ((window (cond
@ -1947,9 +2376,9 @@ ACTIVE-BEG through ACTIVE-END remains owned by the refreshed live state."
(ekp-buffer--release-live-state)
(pcase-let ((`(,beg . ,end)
(ekp-buffer--para-bounds (cons (point) (point)))))
(ekp-justify-region
beg end (or ekp-buffer--auto-width
(ekp-buffer--window-pixel)))))
(ekp-justify-region
beg end (or ekp-buffer--auto-width
(ekp-buffer--effective-width)))))
(defvar-keymap ekp-auto-justify-mode-map
:doc "Keymap for `ekp-auto-justify-mode'."
@ -1963,6 +2392,11 @@ ACTIVE-BEG through ACTIVE-END remains owned by the refreshed live state."
["Justify Buffer" ekp-justify-buffer t]
["Unjustify Buffer" ekp-unjustify-buffer t]
"--"
["Mark Region Normal Break" ekp-normal-break-region (use-region-p)]
["Enable Region Hyphenation" ekp-enable-hyphenation-region (use-region-p)]
["Disable Region Hyphenation" ekp-disable-hyphenation-region (use-region-p)]
["Clear Region Break Policy" ekp-clear-break-policy-region (use-region-p)]
"--"
["Mark Region No-Break" ekp-no-break-region (use-region-p)]
["Clear No-Break Region" ekp-allow-break-region (use-region-p)]
["Mark Region Verbatim" ekp-verbatim-region (use-region-p)]
@ -1993,10 +2427,6 @@ ACTIVE-BEG through ACTIVE-END remains owned by the refreshed live state."
(let (enabled)
(unwind-protect
(progn
(unless (or ekp-buffer-skip-faces ekp-buffer-skip-predicate)
(cond
((derived-mode-p 'org-mode) (ekp-org-setup))
((derived-mode-p 'markdown-mode) (ekp-markdown-setup))))
(ekp-buffer--enable-native-wrap)
(setq ekp-buffer--auto-width (ekp-buffer--effective-width))
(ekp-buffer--reflow (current-buffer) ekp-buffer--auto-width)
@ -2010,7 +2440,8 @@ ACTIVE-BEG through ACTIVE-END remains owned by the refreshed live state."
;;;###autoload
(define-minor-mode ekp-auto-justify-mode
"Maintain a non-mutating KP display projection.
Completed paragraphs use the authoritative narrowest-window width.
Completed paragraphs use the effective `ekp-buffer-measure': narrowest
window by default, a fixed width, or a `(max . N)' cap.
Manual no-break and verbatim properties last only for the current buffer
session."
:lighter " EKP"
@ -2025,11 +2456,16 @@ session."
(cancel-timer ekp-buffer--resize-timer))
(when (timerp ekp-buffer--composition-timer)
(cancel-timer ekp-buffer--composition-timer))
(when (timerp ekp-buffer--policy-reflow-timer)
(cancel-timer ekp-buffer--policy-reflow-timer))
(ekp-buffer--cancel-pending)
(ekp-buffer--release-live-edit)
(ekp-buffer--release-live-state)
(setq ekp-buffer--resize-timer nil
ekp-buffer--composition-timer nil
ekp-buffer--policy-reflow-timer nil
ekp-buffer--policy-reflow-context nil
ekp-buffer--policy-context-snapshot nil
ekp-buffer--auto-width nil
ekp-buffer--live-plan-cache nil)
(ekp-buffer--clear-all)
@ -2042,12 +2478,27 @@ session."
(defun ekp-diagnose ()
"Report the authoritative width and any skipped projection conflicts."
(interactive)
(let ((width (ekp-buffer--effective-width))
(conflicts (length ekp-buffer--conflicts)))
(let* ((measure (ekp-buffer--measure-report))
(width (plist-get measure :effective))
(conflicts (length ekp-buffer--conflicts))
(policy (ekp-buffer--policy-context width)))
(message
"EKP: authoritative width %dpx (narrowest window), %d conflict%s"
width conflicts (if (= conflicts 1) "" "s"))
(list :width width :conflicts ekp-buffer--conflicts)))
(concat "EKP: requested %S, narrowest %d, effective %d, "
"overflow risk %s, %d conflict%s; "
"inline-code %S, hyphenation %S, kinsoku %S, overlong %S")
(plist-get measure :requested)
(plist-get measure :narrowest)
width
(if (plist-get measure :overflow-risk) "yes" "no")
conflicts (if (= conflicts 1) "" "s")
(plist-get policy :inline-code-policy)
(plist-get policy :hyphenation)
(plist-get policy :kinsoku-profile)
(plist-get policy :overlong-token-policy))
(append measure
(list :width width
:policy policy
:conflicts ekp-buffer--conflicts))))
(provide 'ekp-buffer)

5
ekp-utils.el
View File

@ -202,7 +202,10 @@ Rules:
(setq boxes (cons spaces boxes)))
;; Inside a no-break span: spacing is literal, glue would
;; stretch — preserve the run as a rigid space box.
((text-property-not-all 0 (length spaces) 'ekp-no-break nil spaces)
((or (text-property-not-all
0 (length spaces) 'ekp-no-break nil spaces)
(text-property-not-all
0 (length spaces) 'ekp--literal-spacing nil spaces))
(setq boxes (cons spaces boxes)))
;; Latin-Latin with multiple spaces: preserve all but last
((> (length spaces) 1)

1358
ekp.el
View File

File diff suppressed because it is too large Load Diff

16
ekp_c/README.md
View File

@ -67,7 +67,8 @@ metacharacters in the checkout path are not interpreted.
;; Synced automatically by ekp.el before every call:
(ekp-c-set-penalties LINE HYPHEN FITNESS LAST-RATIO
&optional CONSEC-HYPHEN LAST-SHORT EXTRA-STRETCH)
&optional CONSEC-HYPHEN LAST-SHORT EXTRA-STRETCH
EMERGENCY-STRETCH)
;; Single paragraph (15 args):
(ekp-c-break-with-arrays IDEAL-PREFIX MIN-PREFIX MAX-PREFIX
@ -93,9 +94,16 @@ when no indent is active).
The DP uses the same two-pass strategy as the Elisp engine: a strict
Knuth-Plass pass, then — only when the paragraph end is unreachable —
a second pass permitting emergency single-box breaks, so overlong
unbreakable tokens can never make the result empty. Badness saturates
at 10000 exactly like the Elisp side.
a final pass that adds finite background emergency stretch to ordinary
underfull candidates and still scores them through the same
badness/fitness/demerits path. Separately, if an overfull candidate would
extinguish the final active path to a breakpoint and no non-overfull
candidate survives there, TeX-style artificial demerits install the best
provisional path with tight fitness and zero incremental cost. This is a
content-independent reachability safeguard, not a hard-atom scoring rule.
Thus an overlong unbreakable token cannot make the result empty, while
ordinary underfull choices remain global K-P decisions. Badness saturates at
10000 exactly like the Elisp side.
Failure behavior: the full schema is checked before extraction. Malformed
direct API arguments signal `ekp-c-invalid-input`; allocation failure or an

18
ekp_c/ekp.c
View File

@ -183,6 +183,7 @@ typedef struct {
int32_t consecutive;
double last_short;
int32_t extra_stretch;
int32_t emergency_stretch;
} penalty_config_t;
static const char *parse_penalties(emacs_env *env, ptrdiff_t nargs,
@ -192,7 +193,8 @@ static const char *parse_penalties(emacs_env *env, ptrdiff_t nargs,
!i32_value_p(env, args[2]) || !finite_number_p(env, args[3]) ||
(nargs > 4 && !i32_value_p(env, args[4])) ||
(nargs > 5 && !finite_number_p(env, args[5])) ||
(nargs > 6 && !i32_value_p(env, args[6])))
(nargs > 6 && !i32_value_p(env, args[6])) ||
(nargs > 7 && !i32_value_p(env, args[7])))
return "EKP C penalties require finite signed 32-bit numbers";
out->line = clamp32(env->extract_integer(env, args[0]));
@ -209,9 +211,12 @@ static const char *parse_penalties(emacs_env *env, ptrdiff_t nargs,
: (double)env->extract_integer(env, args[5]);
out->extra_stretch = nargs > 6
? clamp32(env->extract_integer(env, args[6])) : 0;
out->emergency_stretch = nargs > 7
? clamp32(env->extract_integer(env, args[7])) : 0;
if (out->last_ratio < 0.0 || out->last_ratio > 1.0 ||
out->last_short < 0.0 || out->extra_stretch < 0)
out->last_short < 0.0 || out->extra_stretch < 0 ||
out->emergency_stretch < 0)
return "EKP C ratios and stretch values are outside valid ranges";
return NULL;
}
@ -239,6 +244,7 @@ static emacs_value Fekp_c_set_penalties(emacs_env *env, ptrdiff_t nargs,
ekp_global->consec_hyphen_penalty = config.consecutive;
ekp_global->last_line_short_penalty = config.last_short;
ekp_global->extra_stretch = config.extra_stretch;
ekp_global->emergency_stretch = config.emergency_stretch;
return env->intern(env, "t");
}
@ -758,7 +764,7 @@ int emacs_module_init(struct emacs_runtime *runtime)
defun(env, "ekp-c-cleanup", 0, 0, Fekp_c_cleanup,
"Cleanup EKP C module resources.");
defun(env, "ekp-c-set-penalties", 4, 7, Fekp_c_set_penalties,
defun(env, "ekp-c-set-penalties", 4, 8, Fekp_c_set_penalties,
"Set Knuth-Plass algorithm penalties.\n\n\
LINE-PENALTY: base penalty per line break (default 10)\n\
HYPHEN-PENALTY: penalty for hyphenated breaks (default 50)\n\
@ -766,9 +772,11 @@ FITNESS-PENALTY: penalty for adjacent line tightness mismatch (default 100)\n\
LAST-LINE-RATIO: minimum fill ratio for last line (default 0.5)\n\
CONSEC-HYPHEN-PENALTY: multiplier for consecutive hyphen runs (default 100)\n\
LAST-LINE-SHORT-PENALTY: multiplier for short last lines (default 50.0)\n\
EXTRA-STRETCH: per-line non-justify flexibility in pixels (default 0)\n\n\
EXTRA-STRETCH: per-line non-justify flexibility in pixels (default 0)\n\
EMERGENCY-STRETCH: fixed final-pass emergency stretch in pixels (default 0)\n\n\
(fn LINE-PENALTY HYPHEN-PENALTY FITNESS-PENALTY LAST-LINE-RATIO \
&optional CONSEC-HYPHEN-PENALTY LAST-LINE-SHORT-PENALTY EXTRA-STRETCH)");
&optional CONSEC-HYPHEN-PENALTY LAST-LINE-SHORT-PENALTY EXTRA-STRETCH \
EMERGENCY-STRETCH)");
defun(env, "ekp-c-break-with-arrays", 15, 15, Fekp_c_break_with_arrays,
"Break lines using Elisp's pre-computed prefix arrays (preferred API).\n\n\

153
ekp_c/ekp_kp.c
View File

@ -153,9 +153,11 @@ typedef struct {
/* Per-line flexibility for non-justify alignment (0 = justify):
* widens max_w, so flexibility = max_w - ideal includes it. */
int32_t extra_stretch;
/* Fixed final-pass emergency stretch, matching TeX's emergency pass. */
int32_t emergency_stretch;
/* Two-pass strategy: strict K-P first; emergency single-box
* breaks only in the second pass (when no valid layout exists). */
/* Two-pass strategy: strict K-P first; emergency transitions only
* in the second pass (when no valid layout exists). */
bool allow_emergency;
} dp_input_t;
@ -209,31 +211,46 @@ static inline bool dp_is_forbidden(const dp_input_t *in, size_t pos)
* Processes position i, trying all end positions k.
* Updates output arrays when better solutions found.
*/
/*
* Emergency break: record a single-box over/underfull line so that the
* DP can never dead-end (every reachable i can always record i+1).
* Demerits are at least as bad as the worst regular line, so these are
* only chosen when nothing better exists. Mirrors
* ekp--dp-relax-emergency in ekp.el.
*/
static inline void dp_relax_emergency(
const dp_input_t *in, size_t i, size_t k,
double prev_dem, int prev_hyph, int prev_lines,
int64_t rest, bool end_hyphen,
typedef struct {
bool present;
double previous_demerits;
size_t start;
int previous_hyphens;
int previous_lines;
int64_t rest;
bool end_hyphen;
} dp_artificial_candidate_t;
static inline void dp_remember_artificial(
dp_artificial_candidate_t *candidate, size_t start,
double previous_demerits, int previous_hyphens, int previous_lines,
int64_t rest, bool end_hyphen)
{
if (candidate->present &&
candidate->previous_demerits <= previous_demerits)
return;
candidate->present = true;
candidate->previous_demerits = previous_demerits;
candidate->start = start;
candidate->previous_hyphens = previous_hyphens;
candidate->previous_lines = previous_lines;
candidate->rest = rest;
candidate->end_hyphen = end_hyphen;
}
static inline void dp_install_artificial(
const dp_artificial_candidate_t *candidate, size_t k,
double *demerits, int32_t *backptrs, int64_t *rest_pixels,
uint8_t *fitness, int32_t *hyphen_counts, int32_t *line_counts)
{
double base = in->line_penalty + EKP_BADNESS_INF;
double dem = prev_dem + base * base + (double)rest * rest;
if (dem < demerits[k]) {
demerits[k] = dem;
backptrs[k] = i;
rest_pixels[k] = rest;
fitness[k] = FITNESS_VERY_LOOSE;
hyphen_counts[k] = end_hyphen ? prev_hyph + 1 : 0;
line_counts[k] = prev_lines + 1;
}
demerits[k] = candidate->previous_demerits;
backptrs[k] = (int32_t)candidate->start;
rest_pixels[k] = candidate->rest;
fitness[k] = FITNESS_TIGHT;
hyphen_counts[k] = candidate->end_hyphen
? candidate->previous_hyphens + 1 : 0;
line_counts[k] = candidate->previous_lines + 1;
}
static void dp_process_position(
@ -250,7 +267,9 @@ static void dp_process_position(
int64_t *rest_pixels,
uint8_t *fitness,
int32_t *hyphen_counts,
int32_t *line_counts)
int32_t *line_counts,
uint8_t *surviving_candidates,
dp_artificial_candidate_t *artificial_candidates)
{
size_t n = in->n;
/* Line 0 (i == 0) may have a different width: first-line indent */
@ -265,20 +284,16 @@ static void dp_process_position(
int64_t lead_space = in->lead_spaces ? in->lead_spaces[i] : 0;
/* Try extending to each position k > i */
bool saw_allowed = false;
for (size_t k = i + 1; k <= n; k++) {
bool is_last = (k == n);
/* Break forbidden here (kinsoku, no-break span): not a
* candidate; keep extending the line. */
if (!is_last && dp_is_forbidden(in, k))
if (!is_last && dp_is_forbidden(in, k)) {
continue;
}
bool is_single_box = (k == i + 1);
/* No permitted break strictly inside [i, k): the run is atomic
* and eligible for emergency handling, like a single box. */
bool atomic_run = !saw_allowed;
saw_allowed = true;
bool end_hyphen = dp_is_hyphen(in, k - 1);
int64_t hyph_w = end_hyphen ? in->hyphen_width : 0;
@ -303,29 +318,28 @@ static void dp_process_position(
int64_t max_w = (int64_t)in->max_prefix[k] - in->max_prefix[i] -
(lead_ideal + lead_stretch) - space_w + hyph_w +
in->extra_stretch;
int64_t emergency_stretch = in->allow_emergency
? in->emergency_stretch : 0;
int64_t effective_max_w = max_w + emergency_stretch;
/* Too long? (last line is never shrunk below its ideal) */
if (min_w > lw || (is_last && ideal > lw)) {
if (atomic_run && in->allow_emergency)
dp_relax_emergency(in, i, k, prev_dem, prev_hyph, prev_lines,
lw - ideal, end_hyphen,
demerits, backptrs, rest_pixels,
fitness, hyphen_counts, line_counts);
if (in->allow_emergency)
dp_remember_artificial(&artificial_candidates[k], i,
prev_dem, prev_hyph, prev_lines,
lw - ideal, end_hyphen);
break; /* No point trying longer lines */
}
if (in->allow_emergency)
surviving_candidates[k] = 1;
/* Valid break? */
bool valid = (min_w <= lw && max_w >= lw) ||
bool valid = (min_w <= lw && effective_max_w >= lw) ||
(is_last && ideal <= lw);
if (!valid) {
/* Rigid underfull atomic run: emergency-record so the
* position after it stays reachable (2nd pass only). */
if (atomic_run && in->allow_emergency)
dp_relax_emergency(in, i, k, prev_dem, prev_hyph, prev_lines,
lw - ideal, end_hyphen,
demerits, backptrs, rest_pixels,
fitness, hyphen_counts, line_counts);
/* An underfull active path survives to later breakpoints. */
continue;
}
@ -333,6 +347,8 @@ static void dp_process_position(
int64_t adjustment = lw - ideal;
int64_t flexibility = (adjustment > 0) ?
(max_w - ideal) : (ideal - min_w);
if (adjustment > 0)
flexibility += emergency_stretch;
double badness;
uint8_t fit;
@ -341,8 +357,12 @@ static void dp_process_position(
/* Single-box line: use fixed flexibility=1, fitness=decent.
* This must come BEFORE is_last check to match Elisp behavior. */
if (is_single_box) {
badness = compute_badness(adjustment, 1);
fit = FITNESS_DECENT;
int64_t flexibility =
(in->allow_emergency && adjustment > 0) ? emergency_stretch : 1;
badness = compute_badness(adjustment, flexibility);
fit = (in->allow_emergency && adjustment > 0)
? compute_fitness(adjustment, flexibility)
: FITNESS_DECENT;
int penalty = end_hyphen ? in->hyphen_penalty : 0;
dem = prev_dem + compute_demerits(badness, penalty,
@ -405,9 +425,8 @@ void ekp_result_destroy(ekp_result_t *r)
*
* All font-dependent calculations happen in Elisp. C module is pure algorithm.
*
* Note: This function now uses dp_process_position() for the core DP logic,
* sharing the same algorithm with process_dp_range(). Any bug fix only needs
* to be made once in dp_process_position().
* dp_process_position() owns the per-breakpoint transition logic used by
* both strict and final passes, including final-pass active-path preservation.
*/
ekp_result_t *ekp_break_with_prefixes(
@ -443,10 +462,15 @@ ekp_result_t *ekp_break_with_prefixes(
uint8_t *fitness = malloc((n + 1) * sizeof(uint8_t));
int32_t *hyph_counts = malloc((n + 1) * sizeof(int32_t));
int32_t *line_counts = malloc((n + 1) * sizeof(int32_t));
uint8_t *surviving_candidates = calloc(n + 1, sizeof(uint8_t));
dp_artificial_candidate_t *artificial_candidates =
calloc(n + 1, sizeof(dp_artificial_candidate_t));
if (!demerits || !backptrs || !rest_pixels || !fitness || !hyph_counts || !line_counts) {
if (!demerits || !backptrs || !rest_pixels || !fitness || !hyph_counts ||
!line_counts || !surviving_candidates || !artificial_candidates) {
free(demerits); free(backptrs); free(rest_pixels);
free(fitness); free(hyph_counts); free(line_counts);
free(surviving_candidates); free(artificial_candidates);
return NULL;
}
@ -469,6 +493,7 @@ ekp_result_t *ekp_break_with_prefixes(
int chp = ekp_global ? ekp_global->consec_hyphen_penalty : 100;
double llsp = ekp_global ? ekp_global->last_line_short_penalty : 50.0;
int32_t xstretch = ekp_global ? ekp_global->extra_stretch : 0;
int32_t estretch = ekp_global ? ekp_global->emergency_stretch : 0;
/* Create unified input structure */
dp_input_t in = {
@ -497,11 +522,12 @@ ekp_result_t *ekp_break_with_prefixes(
.consec_hyphen_penalty = chp,
.last_line_short_penalty = llsp,
.extra_stretch = xstretch,
.emergency_stretch = estretch,
.allow_emergency = false
};
/* Two passes: strict Knuth-Plass first; if the paragraph end is
* unreachable, rerun permitting emergency single-box breaks.
* unreachable, rerun permitting emergency transitions.
* Mirrors ekp--dp-cache-elisp. */
for (int pass = 0; pass < 2; pass++) {
in.allow_emergency = (pass == 1);
@ -514,10 +540,19 @@ ekp_result_t *ekp_break_with_prefixes(
hyph_counts[i] = 0;
line_counts[i] = 0;
}
memset(surviving_candidates, 0, (n + 1) * sizeof(uint8_t));
memset(artificial_candidates, 0,
(n + 1) * sizeof(dp_artificial_candidate_t));
demerits[0] = 0.0;
/* DP: for each valid start, try all ends */
for (size_t i = 0; i < n; i++) {
if (in.allow_emergency && demerits[i] >= EKP_INFINITY &&
!surviving_candidates[i] && artificial_candidates[i].present)
dp_install_artificial(&artificial_candidates[i], i,
demerits, backptrs, rest_pixels,
fitness, hyph_counts, line_counts);
if (demerits[i] >= EKP_INFINITY)
continue;
@ -531,9 +566,17 @@ ekp_result_t *ekp_break_with_prefixes(
rest_pixels,
fitness,
hyph_counts,
line_counts);
line_counts,
surviving_candidates,
artificial_candidates);
}
if (in.allow_emergency && demerits[n] >= EKP_INFINITY &&
!surviving_candidates[n] && artificial_candidates[n].present)
dp_install_artificial(&artificial_candidates[n], n,
demerits, backptrs, rest_pixels,
fitness, hyph_counts, line_counts);
if (demerits[n] < EKP_INFINITY)
break;
}
@ -542,6 +585,7 @@ ekp_result_t *ekp_break_with_prefixes(
if (demerits[n] >= EKP_INFINITY) {
free(demerits); free(backptrs); free(rest_pixels);
free(fitness); free(hyph_counts); free(line_counts);
free(surviving_candidates); free(artificial_candidates);
return NULL;
}
@ -550,6 +594,7 @@ ekp_result_t *ekp_break_with_prefixes(
if (!result) {
free(demerits); free(backptrs); free(rest_pixels);
free(fitness); free(hyph_counts); free(line_counts);
free(surviving_candidates); free(artificial_candidates);
return NULL;
}
@ -568,6 +613,7 @@ ekp_result_t *ekp_break_with_prefixes(
ekp_result_destroy(result);
free(demerits); free(backptrs); free(rest_pixels);
free(fitness); free(hyph_counts); free(line_counts);
free(surviving_candidates); free(artificial_candidates);
return NULL;
}
@ -584,6 +630,7 @@ ekp_result_t *ekp_break_with_prefixes(
free(demerits); free(backptrs); free(rest_pixels);
free(fitness); free(hyph_counts); free(line_counts);
free(surviving_candidates); free(artificial_candidates);
return result;
}
@ -709,6 +756,8 @@ int ekp_init(void)
ekp_global->last_line_ratio = 0.5;
ekp_global->consec_hyphen_penalty = 100;
ekp_global->last_line_short_penalty = 50.0;
ekp_global->extra_stretch = 0;
ekp_global->emergency_stretch = 0;
/* The thread pool is created lazily by the first batch call:
* plain single-paragraph use never starts worker threads. */

1
ekp_c/ekp_module.h
View File

@ -82,6 +82,7 @@ typedef struct {
double last_line_short_penalty; /* multiplier for short last lines */
int32_t extra_stretch; /* per-line flexibility for non-justify
* alignment (0 = justify) */
int32_t emergency_stretch; /* fixed final-pass emergency stretch */
} ekp_state_t;
/* Global state instance */

80
postmortem/20260801-forbidden-run-emergency-boundary.md Normal file
View File

@ -0,0 +1,80 @@
# Forbidden-Run Emergency Boundary
Superseded note: this record explains the earlier rigid-atom repair. Neither
the fullest-prefix fallback nor the later atom-adjacency prohibition is
current. The accepted final-pass model is recorded in
`postmortem/20260802-final-pass-active-path-preservation.md`.
## Context
At 280px, the showcase paragraph before an overwide `ekp-no-break` atom
collapsed into one-glyph CJK lines. The buffer projector reproduced the
semantic plan exactly, so the visible failure was owned by core line breaking,
not redisplay, overlays, resize timing, or projection.
## What the trace showed
The strict pass correctly found no valid candidate where widths jumped from
an underfull CJK prefix to an overfull rigid atom. The emergency pass retained
only the first permitted candidate from each state. That rule guarantees
reachability for arbitrary hard input, but here it made the atom boundary
reachable only through repeated one-glyph emergency transitions.
Replacing that rule globally with the latest underfull candidate was also
wrong. It fixed the screenshot but changed emergency layouts for ordinary
Latin text: discretionary hyphens disappeared and existing DP-parameter
sensitivity changed. Three established regressions failed independently,
while the unmodified baseline passed them.
## Decision
Keep the original first-candidate emergency transition. In the emergency pass
only, remember the latest permitted underfull candidate. Use that additional
candidate only when one or more forbidden break positions are crossed and the
next permitted candidate is already overfull.
This assigns the exception to the event that creates it: a rigid or otherwise
forbidden run skipping the valid width range. The Elisp 1D path, looseness/
parshape path, and C engine implement the same state transition.
## Rejected alternatives
- Compensate in `ekp-buffer.el`: projection would diverge from the semantic
plan and the public string renderer would remain wrong.
- Break the protected atom: this violates the `ekp-no-break` contract.
- Replace every first-candidate emergency with the fullest underfull one: this
changes unrelated emergency layout, hyphenation, and cost sensitivity.
- Add resize debounce or cache invalidation: the failure is deterministic in
a fresh batch plan and is unrelated to time or stale state.
## Consequences
Rigid atoms remain intact, their ordinary prefix uses the fullest permitted
line, and source-clean projection is unchanged. Ordinary emergency behavior,
strict Knuth-Plass results, discretionary hyphenation, and DP parameter
sensitivity retain their previous semantics. The only additional DP state is
three fallback values plus a flag local to one candidate scan.
Automatic inline no-hyphen and explicit hard atoms are different contracts.
The later policy work keeps automatic inline code breakable at source spaces.
For explicit `ekp-no-break`, the ordinary boundary immediately before the
following hard atom is forbidden before DP; that adjacency rule prevents a
manual hard atom from stranding the preceding ordinary box. It is separate
from this postmortem's emergency forbidden-run fallback, which applies only
after a forbidden run skips the valid width range.
## Verification and rollback
The focused regression failed before the fix and passes in Elisp, C, and the
public buffer path. Normal, seeded-random, and per-test isolated 201-test
suites pass, as do 300 C/Elisp fuzz cases, warning-as-error Elisp compilation,
pinned package-lint, checkdoc, release/dictionary gates, and warning-clean
debug, ASan/UBSan, and portable C builds.
Reviewed 62.7-second evidence at
`/private/tmp/ekp-atom-gui-final-Beg8hb` covers nine checkpoints across
480→280→340→280 and returns PASS: the prefix stays on one screen line, the
atom remains intact, source is exact, overlays remain zero, the C engine is
active, and no black or transient vertical frame appears. Rollback is a
direct revert of the forbidden-run fallback and its tests; no API, schema,
cache, or stored-data migration is involved.

162
postmortem/20260801-policy-compilation-boundary.md Normal file
View File

@ -0,0 +1,162 @@
# Policy Compilation Boundary
Superseded note: the historical hard-atom adjacency prohibition below was
removed by `task036`. Explicit atoms forbid only interior breaks; final-pass
reachability is owned by core K-P as recorded in
`20260802-final-pass-active-path-preservation.md`.
## Context
Configurable break policy could have been implemented by teaching every
planning caller about inline code, tokens, mode profiles, and local
overrides. That would have spread policy precedence across core DP,
buffer projection, C marshalling, showcase code, and diagnostics.
The accepted task034 contract required inline code to default to legal
wrapping without dictionary hyphenation, explicit `ekp-no-break` to remain
the only manual hard atom, token categories to have independent defaults,
and the existing 15-argument / 15-field C boundary to stay stable unless
architecture review was reopened.
## Decision
Compile policy before dynamic programming. The buffer layer resolves public
configuration and annotates exact structural intervals; the core turns those
intervals into the existing hyphenation, no-hyphen, and forbidden-break
vectors before DP. The semantic plan still preserves the original source
string and source properties for renderers and diagnostics.
This keeps policy as input classification, not a second DP algorithm or a
post-projection correction.
## Rejected alternatives
- Make inline code a hard atom by default. This caused the visible orphan
glyph failure and made ordinary code-like prose too rigid.
- Add a broader C schema for policy objects. The current vectors already
express the required break permissions, and expanding the ABI would add
release risk without a new semantic capability.
- Handle inline faces in the buffer projector after planning. Projection
would disagree with the semantic plan and C parity evidence.
- Copy mode profiles into buffer locals automatically. That makes consulted
defaults look like user overrides and hides precedence bugs.
## Consequences
Policy precedence is explicit: region properties, then explicit local values,
then mode profiles, then global defaults. Block faces stay paragraph-level
verbatim; inline faces affect only their exact spans. Manual `ekp-no-break`
remains session-local and strictly stronger than automatic policies.
The C boundary remains stable, but every new policy that affects break
legality must be reflected in the compiled paragraph inputs before DP. GUI
verification must keep separate examples for automatic inline wrapping,
manual hard atoms, and verbatim blocks so later regressions cannot pass by
collapsing all three concepts into one rendering path.
## Performance root causes
The first performance boundary was paragraph identity. The semantic signature
included policy measure unconditionally, so width-only policy context rebuilt
paragraphs even when no break rule actually depended on measure. The fix
removed measure from paragraph semantic identity and confined measure-sensitive
work to the policy analysis tier.
The second boundary was property-run identity. Clearing volatile properties
could split otherwise equal filtered property runs, making equivalent source
look different to the cache. Canonicalizing adjacent equal filtered intervals
restores the intended stable identity.
The third boundary was policy analysis order. Full policy analysis ran before
paragraph-cache hits, so repeated layout requests paid the analysis cost even
when the paragraph itself was reusable. The implementation now uses a bounded
two-tier full-analysis cache keyed by clean source, canonical layout and raw
face policy intervals, policy signature, and width context. The measure tier
is used only for actual width-sensitive no-break or overflow candidates.
The fourth boundary was semantic plan assembly. Repeated same-paragraph and
same-width requests duplicated plan construction. A bounded per-paragraph plan
cache stores the reusable plan and returns consumer-owned copies of every
plan-owned mutable payload so later renderer or buffer mutations cannot poison
the cached plan. The paragraph pointer remains intentionally shared because
the older paragraph cache owns that lifetime and append planning uses stable
paragraph identity.
G006 cleanup exposed two ownership boundaries during independent review.
First, shallow semantic-plan copies were not enough: nested line/gap/glue
payloads could still alias the cached plan. Second, malformed non-nil C
results were too soft if they reached Elisp fallback. G007 resolved those
first blockers by deep-copying plan-owned mutable payloads at the consumer
boundary and by making `ekp-backend-contract-error` the contract for malformed
non-nil single or batch backend results. Nil whole results and nil per-item
break results remain the only C soft-failure path that may fall back to Elisp.
G007 final review then found one remaining string-leaf alias: `copy-tree`
does not copy strings inside the layout context, so a returned context could
mutate a dynamic policy suffix and poison the cached context. G008 resolves
that final blocker with the same recursive context copier at both context
snapshot/cache-key creation and returned-plan copying. The copier owns conses,
vectors, and strings; `para` remains the only deliberate shared exception.
The explicit hard-atom adjacency rule is also documented here because it is
part of policy compilation, not the emergency fallback. Before DP runs, the
ordinary boundary immediately before a following explicit `ekp-no-break` atom
is forbidden. The later G009 line-breaking correction is narrower and belongs
to core K-P: ordinary underfull final-pass candidates receive finite
emergency stretch and remain normal badness/demerits candidates, while fixed
emergency transitions are reserved for truly overfull hard/atomic runs.
## Verification and remaining gate
Recorded evidence so far: G002 focused core policy and C parity gates pass;
G003 buffer policy ownership and public controls pass; G004 GUI verifier ERT
7/7 and clean single-window evidence at `/tmp/ekp-g004-evidence.Tp77dW`
passes 12/12 checkpoints with automatic inline wrapping, source-space
internal breaks, explicit hard atom, block verbatim, exact source, zero
overlays, C active, and no stale policy projection. G005 records layout
parity true in the formal four-interleaved evaluator; core baseline p50/p95
38.6679/51.6782ms versus candidate 21.7102/33.0040ms for
43.8549%/36.1354% gains; resize baseline p50/p95 43.3831/55.6250ms versus
candidate 22.0919/32.8202ms for 49.0773%/40.9973% gains; both candidate p95
values under 50ms and both gain sets at least 20%.
The live evaluator records the source-instrumented locked goal as the known
`validation_failed` debt, with parity, zero-work, GC, conflict, and
all-width-nonregression checks true. Current C p95/p99 is 26.449/26.740ms,
a 76.65%/77.23% improvement. Current Elisp p95/p99 is 49.940/52.017ms, a
91.64%/91.35% improvement. This is consistent with historical open
`issue018` and is not a regression. The byte-compiled production public path
passed three runs with zero GC: C append p99 1.361-1.368ms, C hard p99
1.876-1.891ms, Elisp append p99 1.692-1.775ms, and Elisp hard p99
2.100-2.230ms.
G005 repository gates are complete: default ERT 262/262; seeded permuted ERT
seed 20260728 255/255; isolated per-test process suite exit 0; targeted
alias guard passed; property fuzz 300/300; warning-as-error byte
compilation; pinned package-lint at
`35996f478d81e51dae4fa30d051f741895d07399` exit 0 with only an external
obsolete warning from the local names dependency; empty checkdoc; release,
49-entry dictionary manifest, pinned dictionary update, shell syntax, CI
YAML, and diff-check passed; portable/native/debug/sanitize C builds were
warning-clean; module 1.6/4-thread smoke passed; and focused C passed 19/19
including the 15-argument/15-field boundary.
G006 cleanup edited only `ekp.el` and `ekp-buffer.el`, removing redundant
policy/cache code without changing the locked behavior. Its first independent
review found the shallow-copy and C-boundary blockers recorded above, so G006
is review-blocked historically rather than final-clean.
G007 resolved the first review blockers, then G008 resolved the final
string-leaf context blocker. Fresh G008 gates: formal resize parity true;
core baseline p50/p95 36.595/48.357ms versus candidate 23.266/36.363ms for
36.42%/24.80% gains; resize baseline p50/p95 41.443/53.541ms versus
candidate 23.761/36.889ms for 42.67%/31.10% gains; ERT 268/268; property
fuzz 300/300; release gate pass; and the byte-compiled current public path
reports zero GC with C append/hard p99 1.440-1.464/2.022-2.050ms and Elisp
append/hard p99 1.648-1.687/2.136-2.335ms, all below 16ms. After the final
warning-clean test-fixture and C README remediation, independent code review
returned APPROVE and the subsequent architecture review returned CLEAR.
This record explains the implementation boundary. `task034` is closed for
developer work and independent review. `issue021` closed after the user's
2026-08-02 visual confirmation.

92
postmortem/20260802-final-pass-active-path-preservation.md Normal file
View File

@ -0,0 +1,92 @@
# Final-Pass Active-Path Preservation
## Context
The narrow showcase could still isolate a CJK glyph after ordinary underfull
final-pass lines were moved onto fixed emergency stretch and normal K-P
badness. The strict pass, renderer, policy compiler, units, and source
projection were not choosing those lines. The failure remained in the core
final-pass transition model shared by the Elisp and C implementations.
Two compensations had accumulated around the symptom: a high fixed-cost
transition described as hard/atomic-specific, and a rule forbidding the
otherwise legal boundary immediately before a manual hard atom. Both encoded
content and adjacency into a reachability problem.
## Root Cause
Fixed emergency stretch solves one problem: it gives ordinary underfull
candidates finite flexibility so adjustment ratio, badness, fitness, and
demerits can compare them globally. It does not solve the separate case where
an overfull candidate would remove the last active final-pass path.
The implementation treated that second case as a special hard-atom line with
a large invented cost. That is not the role of TeX's
`artificial_demerits`. In the final pass TeX preserves reachability when the
active list would otherwise lose the remaining path; the artificial break
adds no demerits. It is a control-flow safeguard, not another line-quality
score and not a token-class rule.
The upstream reference is the pinned TeX Live `tex.web` at commit
`1a25c04b49317750330b4cf95994ea0d08f9d5ec`:
https://raw.githubusercontent.com/TeX-Live/texlive-source/1a25c04b49317750330b4cf95994ea0d08f9d5ec/texk/web2c/tex.web
## Decision
Keep the strict pass unchanged.
In the final pass:
1. Ordinary underfull candidates receive the resolved fixed
`ekp-emergency-stretch-pixel` and use the normal adjustment-ratio,
badness, fitness, and demerits pipeline.
2. Remember the best provisional overfull candidate for each breakpoint.
3. If no normal state exists at that breakpoint and no non-overfull active
candidate survives to it, install the provisional path with tight fitness
and zero incremental demerits.
The artificial path therefore cannot beat a surviving normal path. It exists
only at the path-extinction boundary. The Elisp 1D, Elisp
looseness/parshape, and C paths implement the same state transition.
Delete the atom-adjacency prohibition. An explicit `ekp-no-break` interval
forbids only its interior breaks; otherwise legal boundaries immediately
before and after it remain legal. An overwide atom stays intact but may share
an overflow line with preceding ordinary content.
## Test-Oracle Failure
The isolated-CJK oracle originally checked only the lines returned by the
planner. An empty plan contained no isolated line, so the assertion could
pass while the core had failed to cover any source. The oracle now first
requires a nonempty, contiguous plan whose ranges cover the entire source,
then checks for isolated CJK lines. A direct regression also proves that an
artificial line adds zero demerits to the preceding path.
## Rejected Alternatives
- A CJK orphan penalty would encode a visible sample instead of the K-P
invariant.
- Unit suffix or hard-atom scoring would make content classification affect a
content-independent final-pass safeguard.
- Forbidding the boundary before an atom would delete a legitimate K-P choice
and compensate in policy compilation for a core DP bug.
- Renderer reshaping would make display diverge from the semantic plan.
- A large artificial cost would turn reachability into a competing heuristic
and differ from TeX's zero-increment behavior.
## Consequences
The public configuration remains the fixed emergency-stretch dimension; no
new orphan, unit, or atom-adjacency option exists. The public C contract stays
at 15 paragraph fields and 15 entry arguments. Renderer glue distribution
uses the selected line's actual rest and actual glue-set proportions.
Focused regressions pass 8/8, emergency selection passes 10/10, the
core/buffer/GUI oracle passes 8/8, full ERT passes 288/288, seeded and isolated
suites pass 279/279, fuzz passes 300/300, and all Elisp/C build, static,
release, and performance gates pass. Reviewed 42.78-second dynamic GUI
evidence at `/tmp/ekp-g009-evidence-retry.UOpPNp` covers
480→168→280→168 plus a no-hyphen→normal policy transition and returns
`VERDICT=PASS` with one window, exact source, zero overlays/stale spans,
active C, and no isolated CJK source line.

65
postmortem/20260802-final-pass-emergency-stretch.md Normal file
View File

@ -0,0 +1,65 @@
# Final-Pass Emergency Stretch
Superseded note: the fixed-dimension emergency-stretch decision remains
current, but the historical hard/atomic fixed-cost exception below does not.
The accepted TeX active-path semantics are recorded in
`20260802-final-pass-active-path-preservation.md`.
## Context
The narrow showcase still produced isolated CJK source lines after the
policy work split automatic inline code from explicit hard atoms. That made
the earlier forbidden-run explanation too narrow: the visible failure was not
owned by unit suffix policy, buffer projection, or a screenshot-specific CJK
penalty. It was the final K-P pass assigning ordinary underfull candidates to
a fixed-cost emergency path outside the normal badness and demerits model.
## Decision
Keep the strict pass unchanged. When strict K-P cannot reach the paragraph
end, rerun with a finite background emergency stretch available to ordinary
underfull candidates. Those candidates still compute adjustment ratio,
badness, fitness, and demerits, so the global DP chooses between them by the
same model as ordinary K-P.
Task035 tried making that stretch scale with the line/candidate width. Full
regression falsified that detail. Task036 must use a TeX-style fixed
dimension instead: `ekp-emergency-stretch-pixel` is nil for an automatic
value around three display-font `M` widths, or a non-negative integer for a
fixed pixel value.
Keep the artificial fixed-cost emergency transition only for a truly overfull
first permitted hard or atomic run. That case has no ordinary candidate to
score. The Elisp 1D path, Elisp looseness/parshape path, and C engine use the
same rule.
## Why the old fallback was wrong
The earlier repair treated the symptom as a forbidden-run boundary problem
and remembered a fuller prefix. That explained one rigid-atom screenshot, but
it left the deeper model split in place: some ordinary underfull lines were
still not compared by normal badness, fitness, and demerits. At narrow widths
that flattened costs enough for one-glyph lines to remain competitive.
The accepted direction matches TeX's emergency-pass shape more closely:
emergency stretch is background flexibility for badness calculation, not a
separate shortcut that bypasses the line-quality model. It must be a fixed
dimension for the paragraph context, not a value derived from each line
measure.
## Test lesson
The first 84px/168px CJK oracle was invalid because it assumed two CJK glyphs
always fit when the width is twice one glyph. It ignored glue, kinsoku, and
attached punctuation. The replacement tests use the real mixed showcase text
and reject isolated CJK source lines through the core, public buffer path, and
GUI oracle.
## Consequences
There is no CJK-orphan penalty, no number-unit special case, and no renderer
compensation. Policy compilation still decides which gaps are legal before
DP. Core K-P owns the final line choices, the renderer must agree with the
chosen fixed stretch dimension, and both engines must stay in parity whenever
emergency stretch behavior changes. The 15-field C paragraph ABI remains a
constraint for task036.

60
readme.md
View File

@ -26,7 +26,7 @@ typesetting, entirely inside Emacs.
- **Text properties preserved** — faces, colors and other properties
survive justification; inserted hyphens inherit the face of the word
they break.
- **Robust on hard input** — unbreakable overlong tokens (URLs, long
- **Robust on hard input** — unprotected overlong tokens (URLs, long
words at narrow widths) degrade to emergency breaks instead of losing
text; every input produces output.
- **Optional C module** — a dynamic module runs the DP in C with a
@ -173,15 +173,17 @@ blocking input in an unbounded Knuth-Plass pass; `M-x ekp-diagnose` reports
the reason. Run `M-x ekp-refill-paragraph` when you explicitly want the
unbounded full-quality pass for that paragraph.
Mode presets for verbatim protection — one call each:
Mode presets for explicit local protection — one call each:
```elisp
(add-hook 'org-mode-hook #'ekp-org-setup)
(add-hook 'markdown-mode-hook #'ekp-markdown-setup)
```
`ekp-auto-justify-mode` also applies the matching preset automatically
in Org and Markdown buffers when you have not configured your own.
`ekp-auto-justify-mode` consults `ekp-buffer-mode-policy-alist`
automatically in Org and Markdown buffers. It does not copy profile
values into buffer-local variables unless you explicitly call the setup
functions above.
### Protecting code and other verbatim text
@ -190,10 +192,22 @@ in Org and Markdown buffers when you have not configured your own.
`ekp-buffer-skip-faces` (e.g. `org-block`, `markdown-code-face`), or
matched by the buffer-local function `ekp-buffer-skip-predicate`
pass through completely untouched.
- Inline level: spans carrying `ekp-no-break`
- Automatic inline level: faces listed in `ekp-buffer-inline-faces`
(Org `org-code`/`org-verbatim`, Markdown inline code by default through
mode profiles) use `ekp-inline-code-policy`. The default `no-hyphen`
keeps source spaces literal and suppresses dictionary hyphenation, but it
may still wrap at legal source boundaries. Automatic `no-break` spans
downgrade to `no-hyphen` when wider than the effective measure.
- Explicit hard atom level: spans carrying `ekp-no-break`
(`M-x ekp-no-break-region`) become rigid atoms — never broken,
never hyphenated, spacing kept literal — ideal for inline code,
product names, or numbers with units.
product names, or numbers with units. An atom wider than the measure stays
intact but is not guaranteed a line of its own: the final pass may place it
on one overflow line with preceding ordinary content. Ordinary underfull
candidates use fixed emergency stretch and normal K-P costs. If an
overfull candidate would otherwise extinguish every active final-pass path,
the core preserves the last path with TeX-style artificial demerits. There
is no CJK-orphan, unit, or screenshot-specific rule.
Manual properties are deliberately **current-buffer-session only**:
plain-text saving and reopening do not persist them. Use
@ -235,6 +249,40 @@ not renderable (text cannot start before the line origin).
## Configuration
### Break policy and measure
The break-policy defaults are intended to make code readable without making
every code-looking span a hard atom:
| Option | Default | Scope | Effect |
|:-------|:--------|:------|:-------|
| `ekp-inline-code-policy` | `no-hyphen` | global, profile, local | `normal`, `no-hyphen`, or automatic fitting `no-break` for inline-face spans |
| `ekp-hyphenation` | `auto` | global, profile, local, region via command | `auto`/`on` use dictionaries when available; `off` suppresses discretionary hyphens |
| `ekp-token-break-policies` | URL/path/identifier `no-hyphen`, number-unit `no-break` | global, profile, local | per-token automatic policies; local/profile values merge by token category |
| `ekp-number-unit-suffixes` | common CSS, time, data, frequency, and metric units | global, profile, local | suffixes recognized by the compact number-unit classifier |
| `ekp-kinsoku-profile` | `common` | global, profile, local | `common`, `zh`, `ja`, `off`, or `custom` CJK line-start/end prohibitions |
| `ekp-cjk-no-line-start-extra` / `ekp-cjk-no-line-end-extra` | `""` | global, profile, local | additions used by the `custom` profile |
| `ekp-overlong-token-policy` | `emergency` | global, profile, local | `emergency`, `overflow`, or `natural` for ordinary overlong Latin-like tokens |
| `ekp-buffer-measure` | `narrowest-window` | global, profile, local | `narrowest-window`, fixed pixel integer, or `(max . PIXELS)` |
| `ekp-buffer-skip-faces` | profile-dependent | global, profile, local | paragraph-level verbatim faces |
| `ekp-buffer-inline-faces` | profile-dependent | global, profile, local | exact inline spans using `ekp-inline-code-policy` |
| `ekp-buffer-mode-policy-alist` | Org and Markdown profiles | global/local safe value | mode profiles consulted by automatic and manual buffer layout |
Effective precedence is deterministic: explicit region text properties
first, then explicit buffer/file/dir-local values, then the first matching
major-mode profile, then global defaults. `ekp-break-policy` region values
are `normal`, `hyphenate`, and `no-hyphen`; they override automatic token
or inline policies for the exact region but never create a hard atom.
`ekp-no-break` remains the only explicit hard-atom property and wins over
every automatic policy.
All listed variables have closed safe-local predicates where file/dir local
configuration is supported. `ekp-diagnose` reports the requested measure,
the narrowest live window, the effective measure, overflow risk, conflict
count, and the active inline/hyphenation/kinsoku/overlong policy summary.
The EKP menu exposes diagnose plus region commands for normal break,
hyphenation on, hyphenation off, clear break policy, no-break, and verbatim.
### Hyphenation language
```elisp

52
readme_zh.md
View File

@ -22,7 +22,7 @@ Emacs-kp 在 Emacs 内部完整实现了 Knuth-Plass 最优断行算法,支持
搜索、语法、保存及普通 Elisp 文本 API 看到的都是源字符。
- **文本属性保留** — face、颜色等属性完整保留;断词插入的连字符继承所
在单词的样式。
- **困难输入不丢内容** — 超长不可断 token(URL、窄栏长词)退化为紧急
- **困难输入不丢内容** — 未受保护的超长 token(URL、窄栏长词)退化为紧急
断行而不是吞掉文本;任何输入都有输出。
- **可选 C 模块** — 动态模块用 C 执行 DP,线程池并行处理多个段落(见
性能数据)。
@ -145,7 +145,7 @@ Elisp。已启用模块若 signal,则作为后端契约错误直接呈现。若
Knuth-Plass 计算阻塞输入;`M-x ekp-diagnose` 会报告这一原因。确实需要
对该段执行无上限完整质量排版时,显式运行 `M-x ekp-refill-paragraph`。
各 mode 的 verbatim 保护预设——各一行:
各 mode 的显式本地保护预设——各一行:
```elisp
(add-hook 'org-mode-hook #'ekp-org-setup)
@ -153,7 +153,9 @@ Knuth-Plass 计算阻塞输入;`M-x ekp-diagnose` 会报告这一原因。确实
```
在 Org 与 Markdown buffer 里,若你没有自定义配置,
`ekp-auto-justify-mode` 会自动套用对应预设。
`ekp-auto-justify-mode` 会自动查询 `ekp-buffer-mode-policy-alist`。
它不会把 profile 值复制成 buffer-local 变量;只有显式调用上面的
setup 函数时才会写入本地 face 列表。
### 保护代码块与 verbatim 文本
@ -161,9 +163,18 @@ Knuth-Plass 计算阻塞输入;`M-x ekp-diagnose` 会报告这一原因。确实
face 在 `ekp-buffer-skip-faces` 列表中(如 `org-block`、
`markdown-code-face`)、或被 buffer-local 的
`ekp-buffer-skip-predicate` 判定的段落**原样跳过**,一个字节都不动。
- 行内级:带 `ekp-no-break` 属性的区间(`M-x ekp-no-break-region`)
- 自动行内级:face 在 `ekp-buffer-inline-faces` 中的精确区间
(Org 的 `org-code`/`org-verbatim`,Markdown 行内代码默认由 mode
profile 提供)使用 `ekp-inline-code-policy`。默认 `no-hyphen`
保留源空格字面宽度并禁止词典断词,但仍可在合法源边界换行。自动
`no-break` 区间若宽于有效栏宽,会降级为 `no-hyphen`。
- 显式硬原子级:带 `ekp-no-break` 属性的区间(`M-x ekp-no-break-region`)
成为刚性原子——不断行、不断词、空格保持字面宽度——适合行内代码、
产品名、数字加单位。
产品名、数字加单位。原子宽于栏宽时仍保持完整,但不保证独占一行:
最终遍可能把它与前面的普通内容放在同一条溢出行。普通欠宽候选使用
固定的 emergency stretch 并按正常 K-P 代价评分;若超宽候选会让最终遍
的活动路径全部消失,核心按 TeX 的 artificial demerits 语义保留最后
路径。这里没有中文孤字、单位或截图专用规则。
手动属性明确只在**当前 buffer 会话**有效:普通文本保存与重新打开不会
恢复它们。使用 `M-x ekp-allow-break-region` /
@ -197,6 +208,37 @@ mode face 或 buffer-local 的 `ekp-buffer-skip-predicate`(Org/Markdown
## 配置
### 断行策略与测量宽度
默认策略的目标是让代码可读,但不把所有"看起来像代码"的片段都变成
硬原子:
| 选项 | 默认值 | 作用域 | 含义 |
|:-----|:-------|:-------|:-----|
| `ekp-inline-code-policy` | `no-hyphen` | 全局、profile、本地 | 行内 face 的 `normal`、`no-hyphen` 或自动适配 `no-break` |
| `ekp-hyphenation` | `auto` | 全局、profile、本地、区域命令 | `auto`/`on` 在词典可用时断词;`off` 禁止词典断词 |
| `ekp-token-break-policies` | URL/path/identifier `no-hyphen`,number-unit `no-break` | 全局、profile、本地 | 按 token 类别合并的自动策略 |
| `ekp-number-unit-suffixes` | 常见 CSS、时间、数据、频率、度量单位 | 全局、profile、本地 | 紧凑数字单位识别的后缀 |
| `ekp-kinsoku-profile` | `common` | 全局、profile、本地 | `common`、`zh`、`ja`、`off` 或 `custom` 禁则 |
| `ekp-cjk-no-line-start-extra` / `ekp-cjk-no-line-end-extra` | `""` | 全局、profile、本地 | `custom` profile 使用的附加禁则字符 |
| `ekp-overlong-token-policy` | `emergency` | 全局、profile、本地 | 普通超宽 Latin-like token 的 `emergency`、`overflow` 或 `natural` |
| `ekp-buffer-measure` | `narrowest-window` | 全局、profile、本地 | `narrowest-window`、固定像素整数或 `(max . PIXELS)` |
| `ekp-buffer-skip-faces` | 由 profile 提供 | 全局、profile、本地 | 段落级 verbatim face |
| `ekp-buffer-inline-faces` | 由 profile 提供 | 全局、profile、本地 | 使用行内策略的精确 face 区间 |
| `ekp-buffer-mode-policy-alist` | Org 与 Markdown profile | 全局/安全本地值 | 自动与手动 buffer 排版查询的 mode profile |
有效优先级固定为:显式区域文本属性 > 显式 buffer/file/dir-local 值 >
第一个匹配的 major-mode profile > 全局默认值。区域 `ekp-break-policy`
可取 `normal`、`hyphenate`、`no-hyphen`;它只覆盖精确区间内的自动
token/行内策略,不会创建硬原子。`ekp-no-break` 仍是唯一显式硬原子
属性,并且胜过所有自动策略。
支持 file/dir local 的上述变量都有封闭的 safe-local 谓词。
`M-x ekp-diagnose` 会报告请求宽度、最窄活动窗口、有效宽度、溢出风险、
冲突数量,以及当前行内/断词/禁则/超宽策略摘要。EKP 菜单提供诊断与
区域命令:普通断行、开启区域断词、关闭区域断词、清除区域断行策略、
no-break 与 verbatim。
### 断词语言
```elisp

1122
tests/ekp-buffer-tests.el
View File

File diff suppressed because it is too large Load Diff

18
tests/ekp-c-tests.el
View File

@ -86,6 +86,24 @@
(should-error (ekp-c-set-penalties "10" 50 100 0.5)
:type 'ekp-c-invalid-input))
(ert-deftest ekp-c-test-penalties-accept-emergency-stretch-eighth-param ()
"The direct C setter accepts a non-negative emergency stretch parameter."
(skip-unless (ekp-c-tests--available))
(should (ekp-c-set-penalties 10 50 100 0.5 100 50 0 7)))
(ert-deftest ekp-c-test-penalties-reject-invalid-emergency-stretch ()
"Invalid emergency stretch inputs use the module input condition."
(skip-unless (ekp-c-tests--available))
(should-error (ekp-c-set-penalties 10 50 100 0.5 100 50 0 -1)
:type 'ekp-c-invalid-input)
(should-error (ekp-c-set-penalties 10 50 100 0.5 100 50 0 "wide")
:type 'ekp-c-invalid-input))
(ert-deftest ekp-c-test-penalties-keep-legacy-four-arg-call ()
"The direct C setter remains compatible with the legacy 4-arg call."
(skip-unless (ekp-c-tests--available))
(should (ekp-c-set-penalties 10 50 100 0.5)))
(ert-deftest ekp-c-test-module-build-uses-argv-in-directory ()
"Interactive builds must not interpolate a module path into a shell."
(let (process-arguments process-directory)

30
tests/ekp-fuzz.el
View File

@ -17,15 +17,31 @@
"bcdfghjklmnpqrstvwxz" "word!" "(paren)" "don't"
"test," "end." "«quoted»" "naïve" "Ｆｕｌｌ" "１２３"))
(defconst fuzz--puncts '("，" "。" "、" "《" "》" "「" "」" "！" "？"))
(defconst fuzz--policy-tokens
'("https://example.test/a_b" "src/core/file_name.el"
"processKeyword42" "3.14MB" "100px"))
(defun fuzz--maybe-policy-propertize (token)
"Return TOKEN with deterministic public policy annotations sometimes."
(pcase (fuzz--rand 8)
(0 (propertize token 'ekp-break-policy 'normal))
(1 (propertize token 'ekp-break-policy 'hyphenate))
(2 (propertize token 'ekp-break-policy 'no-hyphen))
(3 (propertize token 'ekp-no-break t))
(4 (propertize token 'face 'ekp-fuzz-inline-code))
(_ token)))
(defun fuzz--gen-string ()
"Random mixed paragraph of 5-60 tokens."
(let ((n (+ 5 (fuzz--rand 56))) (parts nil))
(dotimes (_ n)
(pcase (fuzz--rand 10)
(pcase (fuzz--rand 11)
;; latin word
((or 0 1 2 3) (push (nth (fuzz--rand (length fuzz--words)) fuzz--words) parts)
(push " " parts))
((or 0 1 2 3)
(push (fuzz--maybe-policy-propertize
(nth (fuzz--rand (length fuzz--words)) fuzz--words))
parts)
(push " " parts))
;; CJK run
((or 4 5 6 7) (let ((len (1+ (fuzz--rand 6)))
(start (fuzz--rand (- (length fuzz--cjk) 7))))
@ -33,7 +49,13 @@
;; CJK punct
(8 (push (nth (fuzz--rand (length fuzz--puncts)) fuzz--puncts) parts))
;; spaces / zwsp
(9 (push (if (= 0 (fuzz--rand 3)) "​" " ") parts))))
(9 (push (if (= 0 (fuzz--rand 3)) "​" " ") parts))
;; built-in policy token categories
(10 (push (fuzz--maybe-policy-propertize
(nth (fuzz--rand (length fuzz--policy-tokens))
fuzz--policy-tokens))
parts)
(push " " parts))))
(string-trim (apply #'concat (nreverse parts)))))
(defun fuzz--content (s)

1048
tests/ekp-gui-verify.el
View File

File diff suppressed because it is too large Load Diff

21
tests/ekp-showcase.el
View File

@ -31,15 +31,29 @@
(defvar-local ekp-showcase--last-ms nil)
(defvar-local ekp-showcase--parshape-on nil)
(defface ekp-showcase-inline-code
'((t :inherit fixed-pitch
:background "#3a3f4b"
:foreground "#98c379"))
"Face for automatic inline-code policy spans in the showcase."
:group 'ekp)
(defconst ekp-showcase--alignments
'(justify ragged-right center ragged-left))
(defun ekp-showcase--sample ()
"Build the demo text: prose, punctuation, atoms, NBSP, a code block."
(let ((zh1 "汉字排版的质感藏在细节里:开启标点悬挂之后,行尾的句号。逗号,和问号?都会把墨迹之外的空白悬出齐边,右边缘因此在视觉上更加平直。避头尾规则保证「引号」与《书名号》永远紧贴内容,连续闭合标点如此。」也绝不拆行。")
(mixed "中英混排是 Emacs 里最常见的场景:The quick brown fox jumps over the lazy dog, 而 internationalization 这样的长词在窄栏会按 Liang 模式断词,连字符同样参与悬挂。")
(atoms (concat "行内原子演示:代码片段 "
(propertize "(ekp-pixel-justify STR W)"
(mixed (concat "中英混排是 Emacs 里最常见的场景: The quick brown fox "
"jumps over the lazy dog, 而 internationalization "
"这样的长词在窄栏会按 Liang 模式断词。自动行内代码 "
(propertize
"(ekp-pixel-justify paragraph-text target-width 'justify nil)"
'face 'ekp-showcase-inline-code)
" 可以在合法空白边界附近换行,但不会插入"
" discretionary hyphen。"))
(atoms (concat "行内原子演示:手动硬原子 "
(propertize "M-x ekp-demo"
'ekp-no-break t
'face '(:inherit fixed-pitch
:background "#3a3f4b"
@ -230,6 +244,7 @@
"Interactive showcase for ekp typesetting features."
(setq-local truncate-lines t)
(setq-local cursor-type 'bar)
(setq-local ekp-buffer-inline-faces '(ekp-showcase-inline-code))
;; keys stay pinned in the header line; live state lives in the
;; mode line (both always visible)
(setq header-line-format (ekp-showcase--keys-line)))

1730
tests/ekp-tests.el
View File

File diff suppressed because it is too large Load Diff