# Changelog All notable changes to emacs-kp are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/), and the project aims to follow [Semantic Versioning](https://semver.org/). ## [Unreleased] ### Fixed - Malformed non-nil C backend results no longer fall back to Elisp. Nil whole results and nil per-item breaks remain soft failures; malformed single or batch output now signals `ekp-backend-contract-error`, so a broken backend cannot silently produce a different layout. - An overwide `ekp-no-break` inline atom no longer collapses its preceding CJK text into one-glyph emergency lines. The final pass now gives ordinary underfull candidates finite background emergency stretch and lets normal badness/fitness/demerits choose the layout. If an overfull candidate would otherwise extinguish every active path, TeX-style artificial demerits preserve the best provisional path with zero incremental cost. The rule is content-independent and identical in the Elisp 1D, looseness/parshape, and C engines. - C-backed core layout and complete resize/reflow now reuse prepared paragraph, DP, and gap geometry and avoid duplicate projection publication. The four-round same-machine gate records p95 values of 27.687 ms and 27.487 ms respectively, with exact frozen-C/Elisp parity. - Reprojecting a buffer now preserves the mark marker and `mark-active` independently. Showcase width keys and resize reflow no longer turn an old inactive mark into a highlighted region. - The unfinished live line now keeps its source-edge whitespace natural. A leading/trailing space or tab appears in the same input turn, and whitespace exposed by backspace is no longer hidden by `display ""` until another glyph arrives. - Enabling or disabling a theme, or changing a frame's default font, now invalidates live plan history and reflows every active auto-justify buffer with the new glyph metrics. - Changing any Knuth-Plass cost parameter now selects a correctly keyed DP/render result immediately; cached paragraphs no longer require `ekp-clear-caches`. Structurally equal non-zero-looseness signatures now reuse their cached result instead of missing an `eql` cons key. - Changing `ekp-default-cws-stretch-pixel` in automatic spacing mode now invalidates both paragraph-cache lookup paths immediately. Unchanged spacing signatures still reuse the cached paragraph. - Buffer layout no longer rewrites the source character stream. Saving, region writes, direct Elisp reads, syntax, and search therefore operate on logical text without a temporary serialization copy. - Live editing now separates committed projection from a local dirty edit transaction. Same-row typing performs no whole-hard-line planning, middle-row edits preserve unaffected break anchors, and exact source reversal restores the saved projection `equal-including-properties`. - Point-only motion no longer changes live state or projection anywhere, including after leaving the active hard paragraph. Structural commits are owned by visual-row crossing, hard completion, the next real edit elsewhere, explicit refill, and width/font/layout-context changes. - Kill/copy integration now composes with and restores an existing `filter-buffer-substring-function`, including DELETE operations, and strips only EKP-owned projection properties from copied text. Removing the final layout span outside auto mode restores the exact prior filter. - Dictionary syntax no longer degrades silently: files containing libhyphen replacement/slash patterns fail with an explicit condition because the fixed-width DP cannot model their conditional rewrites. - Mutating layout-relevant text properties on an already cached string now invalidates the same-string paragraph fast path. It reuses the complete structural cache key instead of a partial parallel signature. ### Changed - The core now exposes semantic `ekp-layout-plan`, `ekp-layout-line`, and `ekp-layout-gap` records. The compatible string renderer and the buffer renderer consume the same break, glue, indentation, and discretionary- hyphen decisions. - Buffer/region layout is now a pure text-property projection on existing source characters: ASCII spaces combine `space-width` and absolute-pixel `min-width`; zero-source CJK/mixed glue uses `min-width` on a complete grapheme; indentation uses `line-prefix`; and visual breaks/hyphens use replacing display strings. EKP creates no overlay and inserts no layout character into a buffer. - Auto-justify now uses stable hard-line transactions. A dirty island stays native between commit events, so ordinary keys neither run DP nor rewrite unrelated properties. Native visual-row crossing atomically republishes every completed row from the unchanged whole-hard-line core plan; the new row remains natural. A buffer-local 16-entry LRU reuses recent text/context plans, and common-prefix signatures minimize writes at commits. IME and resize publication remain generation-checked; no edit-idle or cursor-motion formatter exists. - Auto-justify now owns its native soft-wrap precondition. It temporarily disables `truncate-lines` and narrow partial-window truncation, including Emacs's default sub-50-column behavior, then restores the prior values and buffer-local ownership on teardown. - One buffer uses the narrowest displayed window as its authoritative width. Foreign replacing display owners and unsupported non-ASCII whitespace shrink keep only the affected hard paragraph natural and are reported by `ekp-diagnose`. - Automatic work bounds a single hard paragraph with `ekp-auto-justify-paragraph-limit` (default 2,048 characters). Longer paragraphs remain naturally editable; `ekp-refill-paragraph` is the explicit unbounded quality pass. - **Breaking:** the editor integration module is now `ekp-buffer.el` and provides `ekp-buffer`; replace `(require 'ekp-region)` with `(require 'ekp-buffer)`. Module-owned configuration names now use the `ekp-buffer-` prefix. Public commands whose names describe an actual region or buffer operand are unchanged. - The bundled dictionary inventory is now 49 reproducible entries with a pinned LibreOffice commit, per-file SHA-256/source/license manifest, and a portable verifier/exporter. Sanskrit was removed because the pinned upstream data has no authoritative license statement for that exact hyphenation file. - Auto-justify mode now exposes its existing formatting, protection, and diagnostic commands in an EKP menu. Interactive no-break/verbatim changes report that their text properties last only for the current buffer session. ### Tests - ERT now covers source/tick/undo invariants, exact ASCII and CJK glue projection, display-only static breaks and hyphens, zero-projection underfilled input, whole-hard-line semantic-prefix selection, non-frozen earlier breaks, common-prefix differential writes, plan-cache reuse, point-motion projection stability, stable dirty transactions, exact reversible restoration, atomic visual-row commits, hard-paragraph completion, IME/stale generations, foreign ownership, narrowest-window policy, inactive/active mark preservation, immediate edge whitespace, consecutive spaces, newline/yank/real-undo paths, and the oversized-paragraph guard. GUI probes verify pixel widths. Reviewed temporal recordings verify width-key selection and immediate edge-whitespace states, plus stable middle-row edits, exact reversible restoration, atomic visual-row commits, zero-work point motion, public yank/undo, resize commits, and hard-paragraph completion without delayed snap or overlays. - ERT fixtures now restore every mutable EKP option they isolate. The parshape C-bypass regression drives the public formatter, and reusable permuted-order plus per-test fresh-process runners prevent alphabetical execution from hiding leaked state. - The GUI fit matrix now exits with status 1 when any row fails. A batch-safe forced-failure control locks the automation contract while the real seven-case matrix remains an explicitly loaded developer tool. - Release invariants now have one local gate. CI action inputs are immutable, Windows runs the Elisp baseline, and the `.phrase` decision source is versionable instead of ignored. ### Performance - Live structural commits now extend prepared paragraph data from the last complete-word boundary, resume pure-Elisp DP from a safe retained state, reuse unchanged layout lines, and reconstruct only the buffer's dirty source island. Three repeated byte-compiled public-command runs at 80 pixels measure append p99 at 1.158–1.326 ms for C and 1.429–1.438 ms for pure Elisp; same-row editing and point motion still perform zero planning. The stricter frozen source-instrumented matrix improves C p99 by 78.37% to 25.785 ms and Elisp p99 by 92.03% to 47.578 ms, but remains openly red against its locked 16 ms absolute stress target (`issue018`/`task030`). No stale reuse, debounce, skipped publication, or global GC workaround hides either result. - Tokenization now accumulates fragments and joins once per emitted box; dense hyphen insertion likewise joins original word slices once. On the 1,000–8,000-character adversarial benchmark, the 8,000-character cases improved from 3.133 s to 1.100 s and from 0.945 s to 0.013 s respectively. - Cached “no hyphen” results now use an explicit miss sentinel, so repeated lookups do not rerun the dictionary computation. ### Internal - Cached semantic layout plans now return consumer-owned copies of all plan-owned mutable payloads: strings, context, boxes, offsets, line/glue/ gap records, and signatures. Layout context snapshots and returned contexts recursively copy conses, vectors, and strings, so mutable policy inputs cannot alias the cache key. The paragraph object remains intentionally shared for stable append identity. - DP reuse identity, edge-space exclusion, and the lossless marker vocabulary now each have one directly tested rule owner. The consolidation removes formula/property-list drift without adding modules or hot-loop allocations. - Final cleanup removed nine dead pass-through accessors and replaced direct use of Emacs's private substring-filter dispatcher with the public `filter-buffer-substring` path while preserving DELETE lifecycle ownership. - The paragraph fast path no longer needs style-variable watchers: both the one-entry path and hash cache have one structural identity owner. ### C module (1.6) - Signed 32-bit module inputs now use one `extract_integer` validation instead of calling Lisp `integerp`, `>=`, and `<=` for every value. The frozen 80-pixel candidate module layer measures 0.697/0.701 ms p95/p99, down from 2.615/2.655 ms, without changing the ABI or DP output. - The 15-field single and batch APIs now preflight vector shape, lengths, scalar types, and signed 32-bit input range before extraction. Caller errors signal `ekp-c-invalid-input`; allocation/no-result still returns nil. Enabled-backend signals are no longer swallowed by the Elisp dispatcher. - Line metric, flexibility, and remaining-space intermediates are now 64-bit, eliminating signed overflow when valid int32 widths and protrusions combine. - The default C build is portable (`PROFILE=portable`). Native CPU/LTO, debug, and sanitizer flags are explicit profiles, and `ekp-c-module-build` now passes argv directly to `make` from the module directory instead of interpolating a shell command. ## [1.0.0] - 2026-07-27 First tagged release. Highlights of the work leading up to it: ### Editor integration - Saving an auto-justified buffer now writes the **logical** text to disk; soft line breaks, glue spaces and break hyphens are layout, not content, and no longer reach the file. - `isearch` searches the logical text, so CJK phrases and hyphenated words are found across the justified layout. - Copying/killing puts the logical text on the kill ring (via a buffer-local `filter-buffer-substring-function`). - Justification no longer flips `buffer-modified-p` on an unmodified buffer — no stray lock files or auto-saves — and re-flow no longer fights `undo`. - Text typed adjacent to synthesized glue/hyphens no longer inherits the marker properties (`rear-nonsticky`), so it is never mistaken for layout and deleted on the next re-flow. - Switching major mode or turning the minor mode off restores the logical text cleanly, widening first so narrowing leaves no justified orphans. - Paragraphs containing `field`/`read-only` text (comint prompts) are left verbatim. ### New commands and options - `ekp-justify-buffer` / `ekp-unjustify-buffer`. - `ekp-justify-region` / `ekp-unjustify-region` act on the paragraph at point when no region is active (DWIM). - `ekp-org-setup` / `ekp-markdown-setup` wire the verbatim-protection presets in one call; `ekp-auto-justify-mode` applies the matching one automatically in Org/Markdown buffers. - `fill-paragraph` is remapped to `ekp-refill-paragraph` while the mode is on. - User options are now `defcustom`s under the `ekp` / `ekp-region` groups. - `ekp-cjk-no-line-start-extra`: Japanese line-start prohibition for small kana, the prolonged sound mark and iteration marks (JIS X 4051). - `ekp-auto-justify-tick-budget`: time budget per background re-flow tick. ### Typography - First-line indent (`ekp-first-line-indent`) now runs on the fast 1D dynamic program and the C engine, instead of the heavy 2D path. - Dictionary `LEFTHYPHENMIN` / `RIGHTHYPHENMIN` are honored (English now keeps ≥2 before and ≥3 after a break, per its dictionary). ### Performance - Session-global box-width cache eliminates cross-paragraph duplicate measurement (≈40% of `string-pixel-width` calls on the Chinese sample); byte-compiled justify of that sample dropped ~40%. - Cache keys ignore volatile `fontified` bookkeeping, restoring the paragraph-cache hit rate in font-locked buffers. - Large-buffer re-flow: huge edits chunk in the background, scrolled regions get priority, background ticks honor a time budget. ### Fixed - The DP cache no longer returns a stale layout after `ekp-looseness` changes (results are keyed by looseness). - Changing a style variable (`ekp-alignment`, …) invalidates the same-string fast path. - Missing `dictionaries/` directory only disables hyphenation instead of breaking `(require 'ekp)`. ### C module (1.4 → 1.5) - **Breaking:** the unused experimental C tokenization path (`ekp-c-break-lines`, `ekp-c-hyphenate`, `ekp-c-load-hyphenator`, `ekp-c-set-spacing`, and the `ekp_paragraph.c` / `ekp_hyphen.c` sources) is removed. It carried a heap overflow reachable from Lisp. Rebuild with `make -C ekp_c clean all`. - `ekp-c-break-with-arrays` gains a 15th argument, `FIRST-LINE-WIDTH`, supporting first-line indent in C. - Thread pool sized to the machine's cores, created lazily; a full queue blocks instead of dropping tasks. - Allocation failures and bad arguments fail the call cleanly (Elisp fallback) instead of silently producing a different layout; extracted integers are clamped to int32. ### Packaging - Added `COPYING` (GPL-3.0-or-later) and license headers to all sources; real author/maintainer metadata. - Autoload cookies on all interactive commands. - Removed internal working files and an unused demo GIF from the repo. ## Historical Earlier work (the core KP overhaul and the P1/P2 typography wave — optimal line breaking, CJK kinsoku, hyphenation, pixel-exact justification, alignment modes, hanging punctuation, parshape, no-break atoms, verbatim protection, the C module) predates this changelog; see the git history.