74a780ce95
The self-contained C path (ekp-c-break-lines, ekp-c-load-hyphenator, ekp-c-hyphenate, ekp-c-set-spacing, ekp_paragraph.c, ekp_hyphen.c, ~1500 lines) was never used by ekp.el, diverged semantically from the real pipeline (no kinsoku, no protrusion, no two-pass emergency), and contained an exploitable heap overflow reachable from Lisp: ekp_para_create sized its box array as box_count * 2, but a long word hyphenates into arbitrarily many syllable boxes, overflowing the calloc'd buffer. Deleting the path deletes the bug class. Hardening of the live path: - thread pool: sized from the machine's core count instead of a hardcoded 8; created lazily on the first multi-paragraph batch (single-paragraph users never start worker threads); a full queue now blocks the submitter until a worker makes room — tasks were silently dropped before, degrading the batch to the Elisp fallback exactly when parallelism mattered most. - unified failure gate: a partial allocation used to silently drop kinsoku, hyphenation or protrusion data and continue with a subtly different layout; any allocation failure or pending Lisp signal (non-local exit from a bad element type) now fails the whole call, and ekp.el falls back to the Elisp engine. The Elisp bridge wraps both C entry points in condition-case, and a whole-batch nil no longer crashes the per-paragraph loop. - integer safety: every extracted pixel value is clamped to int32 instead of silently wrapping. - EKP_INFINITY (the unreachable-state sentinel) is now a real infinity: extremely degenerate paragraphs could legitimately accumulate demerits past the old 1e10 constant, making C consider reachable states dead and diverge from the Elisp engine. BREAKING: the four experimental module functions are gone; rebuild with make -C ekp_c clean all (version gate unchanged at 1.5). 92 ERT green; fuzz 300/300 byte-identical across engines. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| ekp_kp.c | ||
| ekp_module.h | ||
| ekp_thread_pool.c | ||
| ekp.c | ||
| Makefile | ||
| README.md | ||
EKP C Dynamic Module
C implementation of the Knuth-Plass DP for emacs-kp (module version 1.5).
The division of labor: Elisp owns all font-dependent data (tokenization, pixel measurement, glue values, prefix sums); the C module runs only the O(n²) dynamic program. This keeps the two engines byte-identical in output while making the hot loop native.
Architecture
ekp_c/
├── ekp_module.h # Core data structures and API declarations
├── ekp.c # Emacs module entry point (emacs_module_init)
├── ekp_kp.c # Knuth-Plass DP + two-pass emergency strategy
├── ekp_thread_pool.c # Thread pool (parallelism across paragraphs)
└── Makefile
Parallelism model: the DP for one paragraph is sequential (each
position depends on all earlier ones), so the thread pool parallelizes
across paragraphs via ekp-c-break-batch — the correct granularity,
with zero synchronization in the inner loop. The pool is created
lazily on the first multi-paragraph batch, sized to the machine's
core count; a full queue blocks the submitter instead of dropping
tasks.
Building
cd ekp_c
make # → ekp.dylib (macOS) / ekp.so (Linux) / ekp.dll (Windows)
Requirements: C11 compiler, Emacs module headers, pthreads.
Windows builds need MinGW-w64 (for pthreads) and
make EMACS_ROOT=<path to your Emacs installation>.
make DEBUG=1 # Debug build with ASan/UBSan
make clean
make info
API (as used by ekp.el)
(ekp-c-init) ; init global state
(ekp-c-version) ; => "1.5" — checked by ekp-c-module-load
(ekp-c-thread-count) ; worker count (created lazily on first batch)
(ekp-c-cleanup)
;; Synced automatically by ekp.el before every call:
(ekp-c-set-penalties LINE HYPHEN FITNESS LAST-RATIO
&optional CONSEC-HYPHEN LAST-SHORT EXTRA-STRETCH)
;; Single paragraph (15 args):
(ekp-c-break-with-arrays IDEAL-PREFIX MIN-PREFIX MAX-PREFIX
GLUE-IDEALS GLUE-SHRINKS GLUE-STRETCHES
HYPHEN-POS HYPHEN-WIDTH LINE-WIDTH
LEAD-SPACES TRAIL-SPACES FORBIDDEN-POS
TAIL-PROTRUDES HYPHEN-PROTRUDE
FIRST-LINE-WIDTH)
;; => (BREAKS . TOTAL-COST)
;; Many paragraphs in parallel: vector of 15-element vectors
(ekp-c-break-batch PARAGRAPHS) ; => vector of (BREAKS . COST)
LEAD-SPACES / TRAIL-SPACES are the space-box run widths that the
Elisp renderer strips from line edges; the DP excludes them from line
metrics so both layers agree exactly (since 1.1). FORBIDDEN-POS
carries the kinsoku / no-break gap indices (since 1.2),
TAIL-PROTRUDES / HYPHEN-PROTRUDE the right-edge protrusion
allowances (since 1.4), and FIRST-LINE-WIDTH the width of line 0
for first-line indentation (since 1.5; pass the line width or ≤0
when no indent is active).
The DP uses the same two-pass strategy as the Elisp engine: a strict Knuth-Plass pass, then — only when the paragraph end is unreachable — a second pass permitting emergency single-box breaks, so overlong unbreakable tokens can never make the result empty. Badness saturates at 10000 exactly like the Elisp side.
Failure behavior: any allocation failure or bad argument makes the call return nil, and ekp.el falls back to the Elisp engine — the C module never silently degrades to a subtly different layout.
Performance
Measured with tests/ekp-bench.el (batch Emacs 30.2, Apple Silicon,
byte-compiled Elisp around the C calls, min of 3 cold-cache runs):
| Case | Elisp engine (compiled) | C engine |
|---|---|---|
| justify text-zh.txt w=200 | 150 ms | 41 ms |
| justify mixed text w=300 | 82 ms | 31 ms |
| range-justify zh 340–380 | 529 ms | 106 ms |
| range-justify mix 280–320 | 762 ms | 52 ms |
| DP only, text-zh w=400 | 30 ms | 2.5 ms |
The pure-DP speedup is ~12×; end-to-end gains are smaller because
tokenization, measurement and rendering stay in Elisp. The C engine
matters most for range-justify (many widths per text) and
multi-paragraph batches. Absolute numbers vary with the machine and
power state; regenerate them with the two commands in DEVELOPER.md §9.