Complete task002 through task016 across cache correctness, editor lifecycle, C boundaries, build and release governance, dictionary provenance, performance, interactive workflows, tests, documentation, and final cleanup.
Two root causes made justified text overrun the window's right edge
in real user sessions (every line ending in the "$" truncation
marker), while emacs -Q looked fine:
1. Windows WITHOUT fringes draw the truncation/continuation
indicator in the text area's LAST COLUMN, so the usable width is
one character less than window-body-width. ekp reserved only a
2px margin: with fringes disabled (a common minimal setup, and
all ttys) every line that hit the target width exactly had its
final glyph displaced by the "$". ekp-region--window-pixel now
reserves one frame-char-width when the window has no right
fringe. (The same lesson ebox-playground encodes in its
viewport-width reserve.)
2. Measurement was blind to the buffer's display context:
string-pixel-width works in a bare hidden buffer, ignoring
face-remapping-alist — which is where text-scale-mode, themes
and per-buffer font tweaks live. Under a remap, the DP laid
lines out with one font's metrics and the display rendered them
with another's: scale +3 made 5 of 7 sample lines overflow a
1330px window by up to 900px (GUI-measured). Measurement now
runs with the destination buffer's face-remapping-alist (the
29/30-compatible equivalent of Emacs 31's string-pixel-width
BUFFER argument), and the width/paragraph caches key on that
context so buffers at different scales never alias. text-scale
changes also trigger a re-flow in ekp-auto-justify-mode.
Ground truth, measured with window-text-pixel-size in GUI Emacs
across 7 display contexts (plain / text-scale ±| face remap /
no-fringes / no-fringes+scale / narrow+scale): the widest justified
line equals the target width exactly in every case, zero lines
overflow. Verbatim code blocks are exempt by design (they never
reflow, like any code line in a narrow window).
New tooling so this never regresses invisibly:
- M-x ekp-diagnose: renders a probe line in YOUR buffer and reports
target vs rendered width — run it in any session where justified
text looks wrong.
- tests/ekp-gui-verify.el: M-x ekp-gui-verify (single check in a
customized session) and ekp-gui-verify-matrix (the 7-case table,
for emacs -Q).
- 2 new batch ERT tests pin the context-keyed caches (94 total).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
showcase: the demo font was applied with face-remap-add-relative, but
string-pixel-width (how ekp measures) ignores buffer face remapping
while honoring an explicit :family — so measurement used the session
default font and rendering used the demo font. On a wide follow-
window that per-glyph discrepancy accumulated past the safety margin
and every justified line overflowed the right edge (invisible at the
narrow 480px default). Pin the font as an appended :family face text
property instead, so measurement and rendering agree; inline atom/
code faces keep precedence.
checkdoc: all four elisp files are now checkdoc-clean (docstrings,
argument mentions, sentence spacing) with no logic change; the three
validation messages read naturally again (dropped the capitalized
"Ekp:" prefix checkdoc's capital-letter rule had forced).
92 ERT green; fuzz 300/300 with C parity; byte-compile warning-free.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- a dirty area larger than the lazy threshold (big paste, revert)
is chunked into the background queue instead of freezing the
command loop with one synchronous re-justification
- queued chunks intersecting the visible span move to the queue
front on scroll (window-scroll-functions): scrolling into an
unprocessed area no longer waits for the whole queue
- background ticks work under a time budget
(ekp-auto-justify-tick-budget, default 5 ms): several small
chunks per tick, guaranteed progress of at least one, peek-then-
pop so an aborted justification never loses a chunk
- ticks also yield while an input-method composition is active
3 new ERT tests (84 total).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- new commands ekp-justify-buffer / ekp-unjustify-buffer
- ekp-justify-region / ekp-unjustify-region without an active region
now act on the hard paragraph at point (DWIM) instead of erroring
- ekp-auto-justify-mode remaps fill-paragraph to the new
ekp-refill-paragraph — plain M-q treated glue spaces and soft
breaks as content and destroyed the original whitespace
- ekp-org-setup / ekp-markdown-setup wire the existing skip-face
presets in one call; enabling the mode in an Org/Markdown buffer
applies them automatically unless the user configured their own.
The markdown setup also stops font-lock from managing the display
property: refontification was stripping the pixel glue and
wrecking the layout permanently.
- justify runs font-lock-ensure over the region first when
face-based verbatim detection is configured — paragraphs jit-lock
had never fontified (off-screen code blocks) were not skipped
- showcase/demo buffers disable undo recording (width sweeps
generated thousands of useless undo entries)
5 new ERT tests (81 total).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- isearch searches the logical text: entering isearch temporarily
un-justifies (spans remembered as markers), exiting restores the
layout. Justified CJK lines contain real injected spaces, which
made even same-line phrases unfindable.
- the kill ring receives the logical text via a buffer-local
filter-buffer-substring-function: yanking justified text elsewhere
transports words, not the source window's pixel layout.
- multi-window: the layout width is now the narrowest window showing
the buffer (was: whichever window resized last, leaving the other
window overflowing), and window-configuration-change-hook catches
the buffer becoming displayed, splits, and closed windows.
- paragraphs containing field or read-only text (comint prompts)
are never re-written; interactive justify/unjustify on read-only
buffers signals instead of editing through inhibit-read-only.
- edit re-flows wait for an active input-method composition (quail
preedit) to finish instead of re-writing the text under it.
4 new ERT tests (76 total).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Justification is a re-layout of the same logical text; it now behaves
that way toward the rest of the editor:
- saving a justified buffer writes the LOGICAL text: before-save-hook
unjustifies (spans remembered as markers), after-save-hook restores
the justified view; the deterministic pair stays off undo history.
Previously soft newlines and break hyphens were silently persisted
to disk and the original whitespace was unrecoverable after reopen.
- renderer marker properties (ekp-glue/soft-break/soft-hyphen/hidden/
justified) are registered in text-property-default-nonsticky: text
typed after a glue no longer inherits the marker and is no longer
deleted as a synthesized space by the next re-flow.
- pure re-layout preserves buffer-modified-p when it was nil: no more
lock files, auto-saves and "buffer modified" prompts from merely
enabling ekp-auto-justify-mode.
- changes applied by undo are not re-dirtied (no timer war against
the user's undo sequence).
- switching major modes tears justified state down cleanly
(change-major-mode-hook), widening first so narrowing never leaves
justified orphans.
5 new ERT tests (72 total).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- add COPYING (GPL-3.0-or-later) and license headers to all elisp
and C sources; real author/maintainer info replaces placeholders
- convert user-facing options to defcustom under new group `ekp'
(spacing internals managed by ekp-param-set stay defvars)
- autoload user commands: ekp-param-reset, ekp-clear-caches,
ekp-c-module-load, ekp-c-module-build
- ekp--load-dicts: a missing dictionaries/ directory now only
disables hyphenation instead of breaking (require 'ekp);
ekp--split-with-hyphen degrades gracefully (and resolves the
hyphenator once per call instead of once per word)
- package summary no longer redundantly says "for Emacs"
- Makefile: drop personal Windows EMACS_ROOT default; add guidance
- ekp_c/README.md: fix stale 1.1 version references (module is 1.4)
- DEVELOPER*.md: remove archive/ from file map (not in the repo)
- remove unreferenced 10 MB demo GIF
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Buffers past ekp-auto-justify-lazy-threshold (20k chars) no longer
re-justify wholesale on a width change: the visible span updates
synchronously and the rest follows in polite background chunks
(run-with-timer ticks that yield to pending input). Measured on a
300-paragraph, 132k-char article: perceived latency 332 ms → 15 ms,
background completion ~0.5 s.
Fixed in the process: adjacent chunk markers shared a boundary, and
the earlier chunk's delete+insert pushed the next chunk's start
marker back over the freshly inserted text — chunks grew linearly
(2.2k, 4.9k, 7.4k… chars) and background work went quadratic (14 s).
Start markers now use insertion-type t. Regression test asserts the
chunked result equals the one-shot result property-for-property.
Also: org/markdown skip-face presets (ekp-region-org-skip-faces /
ekp-region-markdown-skip-faces), MELPA header hygiene (Version + URL
on the main file, stale Package-Requires dropped from ekp-hyphen.el),
readme notes.
67 ERT green; fuzz 300/300.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Investigation: the apparent 7× bench regression was interpreted-mode
measurement (tests always deleted .elc); byte-compiled + C the real
P1/P2 cost was +70–90%. Worktree A/B across phase commits confirmed
the features themselves added ~+20% interpreted.
Optimizations (all verified by the 66-test suite + 300-case fuzz):
- per-char memo table for ekp--str-type (get-char-code-property was
the para-build hot spot)
- precomputed punctuation char lists (no per-call list rebuild)
- ekp--box-offsets memoized in the paragraph struct
- interned glue strings for the two dominant payload shapes
- rendered-output cache per (paragraph, width) inside dp-cache
(capped at 64 widths per paragraph), so resize sweeps that revisit
a width skip rendering entirely
- unjustify walks hop property boundaries instead of chars
Numbers (byte-compiled + C): justify zh w=200 53.8 ms — at parity
with the pre-feature record (57 ms) despite punctuation boxes and
break permissions; range zh 117 ms (+55%, inherent box-count cost).
Continuous reflow, 60-paragraph 26k-char article through the full
region layer: 187 ms → 73 ms per width change; incremental
single-paragraph re-justify after an edit: 62 ms → 17 ms.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Paragraphs stay verbatim (never justified, byte-identical through
justify/auto-mode/unjustify) when they carry the ekp-verbatim text
property, wear a face from ekp-region-skip-faces (point it at
org-block / markdown-code-face and code blocks are exempt), or match
the buffer-local ekp-region-skip-predicate — three layers of the same
general mechanism, no mode-specific code. Inline code inside prose
is the existing ekp-no-break atom (rigid, unbreakable, unhyphenated).
New commands: ekp-verbatim-region / ekp-clear-verbatim-region.
Core engine untouched: block protection lives entirely in the region
layer, so skipped text never even reaches the tokenizer.
Tests: 66 ERT green (property / face / predicate paths, exact
roundtrips around justified prose).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ekp-protrusion enables microtype-style character protrusion at the
right edge: a line ending in fullwidth punctuation (。、」…) hangs
the whitespace half of the glyph past the flush edge — visually
identical to CLREQ line-end punctuation compression — and latin
periods/commas/quotes and soft hyphens protrude by their configured
ratios (ekp-protrusion-ratios).
Mechanics: per-box tail protrusion, folded through trailing space
boxes into a per-gap array; each DP candidate widens its effective
target (lw = width + release), mirrored exactly in the renderer's
per-line effective width and in the C-result reconstruction, so DP,
rendering and both engines stay coherent. Left-edge protrusion is
documented as out of scope (Emacs cannot render before the line
origin). ekp-region reserves the protrusion width in auto layouts
(ekp-region-protrusion-reserve).
Off by default; all-zero arrays make it a no-op on the shared path.
C module 1.4: break-with-arrays 12→14 args (tail-protrudes array,
hyphen-protrude scalar), batch vectors 14 elements.
Tests: 59 ERT green (deterministic hang check, flush-when-off,
protrusion C parity); fuzz 300/300.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- text property ekp-no-break turns any span into a rigid typesetting
atom: no line break inside, no hyphenation, literal spacing (space
runs inside the span become preserved space boxes instead of
stretchable glue) — inline code, product names, numbers with units
- character-driven joins: NBSP, NARROW NBSP, FIGURE SPACE keep their
neighbors together (they are boxes whose adjacent gaps are
unbreakable and glue-free — the character supplies its spacing);
WORD JOINER / ZWNBSP attach invisibly like combining marks
- new commands: ekp-no-break-region / ekp-allow-break-region
- overlong atoms fall back to single emergency lines (atomic-run
handling from the break-permission layer); zero C changes needed
Tests: 52 ERT green incl. atomicity across widths, hyphenation
suppression, NBSP/WJ adjacency sweeps; fuzz 300/300 with C parity.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The renderer (shared by both engines) is now lossless: synthesized
glue carries the original text it replaced (ekp-glue), soft line
breaks carry the whitespace swallowed around the break
(ekp-soft-break), break hyphens are marked (ekp-soft-hyphen), and
paragraph-edge whitespace survives as zero-display ekp-hidden text.
Orphaned ekp--combine-glues-and-boxes / ekp--interleave removed.
New ekp-region.el:
- ekp-justify-region / ekp-unjustify-region: in-place justification
with exact structural restore (character- and property-exact),
robust to edits made while justified
- ekp-auto-justify-mode: keeps the buffer justified at the window
width; debounced re-flow on window resize, incremental
per-paragraph re-justification after edits (served by the
paragraph cache)
- fix found in live GUI testing: buffer-local members of
window-size-change-functions receive the WINDOW as argument and
may run with an unrelated buffer current; the handler now resolves
window and buffer explicitly (regression test included)
Tests: 47 ERT (36 core + 11 region) passing; 300-case fuzz 0
failures (C/elisp parity unchanged); byte-compile clean with
error-on-warn. Verified interactively in GUI Emacs: justified at
1403px maximized, auto-reflowed to 614px on frame resize.
docs: interactive-use section in readme.md / readme_zh.md
ci: byte-compile list includes ekp-region.el
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
