ekp/readme.md

Emacs-KP: Knuth-Plass Line Breaking for Emacs

中文文档 | Developer Guide

Emacs-kp implements the Knuth-Plass optimal line breaking algorithm with full support for CJK (Chinese, Japanese, Korean) and Latin mixed text typesetting, entirely inside Emacs.

Features

• Optimal line breaking — the Knuth-Plass dynamic program finds the globally optimal set of breaks for a paragraph, not greedy first-fit.
• CJK support — every CJK character is a breakable box; kinsoku rules keep punctuation attached (，。 never start a line, 「《 never end one); dedicated inter-CJK and CJK↔Latin spacing.
• Hyphenation — Frank Liang's algorithm (the TeX algorithm) with 70+ Hunspell pattern dictionaries bundled.
• Pixel-accurate justification — every justified line renders at exactly the requested pixel width, using display (space :width ...) properties; works with variable-width fonts.
• Text properties preserved — faces, colors and other properties survive justification; inserted hyphens inherit the face of the word they break.
• Robust on hard input — unbreakable overlong tokens (URLs, long words at narrow widths) degrade to emergency breaks instead of losing text; every input produces output.
• Optional C module — a dynamic module runs the DP in C with a thread pool that processes paragraphs in parallel (see benchmarks).

Requirements

• Emacs 29.1+ (uses string-pixel-width and object-intervals)
• Optional, for the C module: a C11 compiler and pthreads

Installation

Clone the repository and add it to your load-path (the dictionaries/ directory must sit next to the .el files):

(add-to-list 'load-path "/path/to/emacs-kp")
(require 'ekp)
(require 'ekp-region)   ; buffer/region commands

Byte-compiling is strongly recommended — the Elisp engine is about 10× faster compiled.

Quick Start

(require 'ekp)

;; Justify a paragraph to 600 pixels
(insert (ekp-pixel-justify "Your paragraph text here..." 600))

;; Find the best width in a range; returns (justified-text . width)
(ekp-pixel-range-justify "Your text" 400 800)

Multiline strings are treated as one paragraph per line; blank lines are preserved.

C module (recommended for long texts)

cd ekp_c && make        # requires C11 compiler, produces ekp.dylib/.so/.dll

(ekp-c-module-load)     ; prints "ekp-c module loaded (version 1.4, N threads)"

Once loaded (and since ekp-use-c-module defaults to t), all justification calls automatically use the C engine. The Elisp and C engines produce identical output; Elisp is the always-available fallback. If the module on disk is older than the Elisp code expects, loading refuses with a message asking you to rebuild.

Interactive Use (buffer & region)

ekp-region.el turns the string API into buffer-level commands:

(require 'ekp-region)

• M-x ekp-justify-region — justify the region to the window text width (with a numeric prefix argument, to that many pixels). With no active region, it justifies the paragraph at point.
• M-x ekp-justify-buffer — justify the whole buffer.
• M-x ekp-unjustify-region / ekp-unjustify-buffer — restore the original text exactly, including collapsed whitespace runs. Justification is lossless: every synthesized space, soft line break, and soft hyphen carries the original text it replaced, so restoring is a structural transform that also works after you edited the justified text.
• M-x ekp-auto-justify-mode — keep the whole buffer justified to the window width. Re-flows (debounced by ekp-auto-justify-resize-delay) when the window width changes, and after edits re-justifies only the touched paragraphs (ekp-auto-justify-edit-delay), so unchanged paragraphs hit the paragraph cache. Turning the mode off restores the buffer exactly.

The buffer is treated as a live document, not just a canvas:

• Saving writes the logical text — soft line breaks, glue spaces and break hyphens never reach disk; the on-screen buffer stays justified.
• Searching (isearch) sees the logical text, so CJK phrases and hyphenated words are found across the layout.
• Copying puts the logical text on the kill ring, so pasted text carries words, not pixel spacing.
• Merely enabling the mode never marks the buffer modified (no stray lock files or auto-saves), and undo is not fought by the re-flow timer.

ekp-region-margin-pixel (default 2) is subtracted from the window width as a rounding safety margin.

Large buffers (over ekp-auto-justify-lazy-threshold characters, default 20 000) re-flow visible-first: the portion on screen updates synchronously and the rest follows in idle background chunks, with a per-tick time budget (ekp-auto-justify-tick-budget) and priority for whatever you scroll to.

Mode presets for verbatim protection — one call each:

(add-hook 'org-mode-hook      #'ekp-org-setup)
(add-hook 'markdown-mode-hook #'ekp-markdown-setup)

ekp-auto-justify-mode also applies the matching preset automatically in Org and Markdown buffers when you have not configured your own.

Protecting code and other verbatim text

• Block level: paragraphs carrying the ekp-verbatim text property (M-x ekp-verbatim-region), wearing a face listed in ekp-region-skip-faces (e.g. org-block, markdown-code-face), or matched by the buffer-local function ekp-region-skip-predicate pass through completely untouched.
• Inline level: spans carrying ekp-no-break (M-x ekp-no-break-region) become rigid atoms — never broken, never hyphenated, spacing kept literal — ideal for inline code, product names, or numbers with units.

Typography

• Alignment — ekp-alignment: justify (default), ragged-right, ragged-left, or center. Non-justify modes keep word spacing natural while Knuth-Plass still minimizes raggedness within ekp-ragged-stretch-pixel (≈2 em) per line.
• Hanging punctuation — set ekp-protrusion to t and line-final punctuation (。、」 as well as periods, commas and break hyphens) hangs past the flush edge by ekp-protrusion-ratios. The 0.5 default for fullwidth closers is visually equivalent to CLREQ line-end punctuation compression. ekp-auto-justify-mode reserves the protrusion width automatically.
• Paragraph shapes — ekp-first-line-indent (t = 2 em) for the CJK paragraph convention, or full TeX-style ekp-parshape with per-line (INDENT . WIDTH). First-line indent runs on the fast 1D path and the C engine; only full ekp-parshape and ekp-looseness fall back to the Elisp-only 2D dynamic program.
• Unbreakables — NO-BREAK SPACE, NARROW NBSP, FIGURE SPACE and WORD JOINER characters keep their neighbors together out of the box.
• Kinsoku covers full- and halfwidth punctuation: a line never starts with 。、」!? or a lone .,;:!?, never ends with 「( etc. Japanese line-start prohibition also covers small kana, the prolonged sound mark and iteration marks (っ ょ ー 々), configurable via ekp-cjk-no-line-start-extra.

Limitations worth knowing: mid-line CLREQ punctuation compression (e.g. 「字。下」 squeezed inside a line) cannot be rendered — Emacs cannot shrink a glyph's advance — which is why line-edge compression is delivered via protrusion instead; left-edge protrusion is likewise not renderable (text cannot start before the line origin).

Configuration

Hyphenation language

(setq ekp-latin-lang "de_DE")   ; default "en_US"

Any dictionaries/hyph_<lang>.dic works; short codes like "de" resolve to the first matching dictionary. Each dictionary's own LEFTHYPHENMIN / RIGHTHYPHENMIN are honored (English keeps ≥2 letters before and ≥3 after a break); pass explicit margins to ekp-hyphen-create to override.

Spacing parameters

Three glue classes control spacing (all values in pixels):

Group	Between
lws-*	two Latin words
mws-*	a Latin word and a CJK char
cws-*	two CJK characters

Each class has an ideal width, a maximum stretch and a maximum shrink:

(ekp-param-set lws-ideal lws-stretch lws-shrink
               mws-ideal mws-stretch mws-shrink
               cws-ideal cws-stretch cws-shrink)
;; e.g. (ekp-param-set 7 3 2  5 2 1  0 2 0)

• If you never call ekp-param-set, defaults are derived automatically from the font of each string.
• Explicit parameters persist until you call ekp-param-reset, which returns to automatic per-string defaults.

Algorithm parameters

Variable	Default	Meaning
ekp-line-penalty	10	Base cost per line; higher prefers fewer lines
ekp-hyphen-penalty	50	Cost of a hyphenated break (added as penalty²)
ekp-adjacent-fitness-penalty	100	Cost when adjacent lines differ in tightness by >1 class
ekp-consecutive-hyphen-penalty	100	Multiplier for runs of hyphenated lines (× count²)
ekp-last-line-min-ratio	0.5	Minimum fill ratio for the last line
ekp-last-line-short-penalty	50	Cost multiplier for a too-short last line
ekp-looseness	0	Target line count offset: +1 = one line more than optimal, −1 = one fewer

All parameters take effect with both engines: the Elisp side syncs them to the C module before every call. ekp-looseness is handled by a dedicated Elisp path (the C module is bypassed automatically while it is non-zero).

Caching

Tokenization, measurement, and DP results are cached per paragraph; box widths are additionally cached session-wide, so a glyph shared across paragraphs is measured only once.

• ekp-para-cache-limit (default 256): max cached paragraphs; the cache is flushed when the limit is reached.
• M-x ekp-clear-caches clears everything (use after changing fonts or themes that affect glyph widths).

Performance

Measured on the bundled sample texts (tests/ekp-bench.el), batch Emacs 30.2, Apple Silicon; see DEVELOPER.md for methodology:

Case (text-zh.txt ≈ 3.6 KB)	Elisp (byte-compiled)	C module
justify, width 200px	150 ms	41 ms
optimal-width search 340–380	529 ms	106 ms
DP only, width 400px	30 ms	2.5 ms

Byte-compile the package — the Elisp engine is ~10× faster compiled. Both engines produce identical output; the C module pays off most for optimal-width search and long multi-paragraph texts. (Absolute numbers vary with machine and power state; the ratios are the point.)

Known Limitations

• Measurement follows the current buffer's face remappings (text-scale-mode, themes, ekp-org-setup-style tweaks) and reserves the truncation-indicator column in windows without fringes, so justified lines fit the real display. If lines ever look truncated or short in an exotic setup, run M-x ekp-diagnose in that buffer — it reports whether measurement matches rendering (and M-x ekp-gui-verify runs a full fit check).
• One font is assumed per Latin/CJK script per paragraph when computing spacing defaults; mixed-font paragraphs work but spacing defaults come from the first font found.
• ekp-pixel-range-justify minimizes average demerits with a ternary search plus a local scan; cost is not perfectly unimodal in width, so the result is a very good, but not guaranteed global, optimum.
• In batch/tty Emacs, pixel widths degrade to character columns (the full pipeline still works; useful for testing).

Interactive Demo

emacs -Q -L /path/to/emacs-kp -l tests/ekp-showcase.el -f ekp-showcase

One buffer, live keys: -/+ change the pixel width (per-reflow time in the header line), d runs an animated width sweep with an fps report, a cycles alignment, p toggles hanging punctuation, i first-line indent, s a wedge parshape, c compares the C engine with pure Elisp, w follows the window width via ekp-auto-justify-mode. The sample text includes a protected code block, an inline no-break atom and NBSP-joined numbers.

Testing

tests/run-tests.sh /path/to/emacs     # 67 ERT tests, all batch-safe

Credits

• Core algorithm: "Breaking Paragraphs into Lines" by Donald E. Knuth and Michael F. Plass (1981)
• Hyphenation: Frank Liang's algorithm, adapted from Pyphen
• Dictionaries: hyphenation patterns from the LibreOffice dictionaries (GPL/LGPL/MPL; see each dictionaries/README_hyph_*.txt)