add dynamic moudle to improve prefermance

This commit is contained in:
Kinneyzhang 2026-01-25 13:50:46 +08:00
parent 91d9bbc1e9
commit c88bbdd395
21 changed files with 3468 additions and 377 deletions

3
.gitignore vendored
View File

@ -1 +1,4 @@
archive
*.dylib
*.dll
*.o

4
.phrase/docs/CHANGE.md Normal file
View File

@ -0,0 +1,4 @@
# Change Log Index
## phase-doc-improvement-20260125
- See `.phrase/phases/phase-doc-improvement-20260125/change_log.md`

3
.phrase/docs/ISSUES.md Normal file
View File

@ -0,0 +1,3 @@
# Known Issues
(No open issues)

View File

@ -0,0 +1,20 @@
# Change Log: Phase Doc Improvement 20260125
## 2026-01-25
- **Add**: `DEVELOPER.md` and `DEVELOPER_ZH.md`
- Extracted technical details from READMEs.
- Added detailed `ekp-para` struct definition and field explanations.
- Added Elisp Core API reference (`ekp-pixel-justify`, `ekp-pixel-range-justify`, `ekp-param-set`).
- Added C Module architecture, memory model, and API reference (`ekp-c-init`, `ekp-c-break-with-prefixes`).
- Added Architecture diagram.
- **Modify**: `readme.md` and `readme_zh.md`
- Refocused on User Guide (Installation, Configuration, Usage).
- Removed internal implementation details.
- Added links to new Developer Guides.
- Cleaned up formatting and structure.
- **Add**: Project Phase Structure
- Initialized `.phrase/` directory.
- Created `spec`, `plan`, `task` for `phase-doc-improvement-20260125`.

View File

@ -0,0 +1,17 @@
# Plan: Documentation Improvement
## Milestones
1. **Structure Setup**: Initialize `.phrase` and new files.
2. **Extraction**: Move technical content from READMEs to DEVELOPER docs.
3. **Enhancement**: Flesh out API details in DEVELOPER docs using source code as reference.
4. **Polish**: Refine User Guide in READMEs.
## Scope
- Files: `readme.md`, `readme_zh.md`, `DEVELOPER.md` (new), `DEVELOPER_ZH.md` (new).
- Languages: English, Chinese.
## Dependencies
- Source code (`ekp.el`, `ekp_c/*`) for accurate API documentation.
## Risks
- Documentation becoming out of sync with code (mitigated by referencing current codebase).

View File

@ -0,0 +1,30 @@
# Spec: Documentation Improvement
## Summary
Restructure and enhance documentation for `emacs-kp`. Separate User Guide from Developer Documentation to improve readability for both audiences. Provide in-depth API reference for developers.
## Goals
1. **Separation of Concerns**: `readme.md` for users, `DEVELOPER.md` for contributors.
2. **Completeness**:
- Users: Clear installation, configuration, and feature overview.
- Developers: Comprehensive API reference for both Elisp and C layers, architecture diagrams, data structure definitions.
3. **Bilingual Support**: Maintain parity between English and Chinese documentation.
## Non-Goals
- Changing the code or functionality of `emacs-kp`.
- Adding new tutorials (beyond basic usage).
## User Flows
- **User**: Lands on repo -> Reads `readme.md` -> Installs & Configures -> Uses package.
- **Contributor**: Lands on repo -> Sees "Developer Guide" link -> Reads `DEVELOPER.md` -> Understands internals -> Submits PR.
## Acceptance Criteria
1. `DEVELOPER.md` and `DEVELOPER_ZH.md` exist and contain:
- Architecture overview.
- Elisp Core API (`ekp-pixel-justify`, parameters, etc.).
- Data Structures (`ekp-para`, `ekp-box`, etc.).
- C Module details (API, build, memory model).
2. `readme.md` and `readme_zh.md` are cleaned up:
- No C implementation details (moved to Dev guide).
- Clearer "Quick Start" and "Configuration".
3. No broken links between documents.

View File

@ -0,0 +1,8 @@
# Tasks: Documentation Improvement
- task001 [ ] Create `DEVELOPER.md` with extracted technical content from `readme.md`
- task002 [ ] Create `DEVELOPER_ZH.md` with extracted technical content from `readme_zh.md`
- task003 [ ] Enhance `DEVELOPER.md` with detailed Elisp API and C Module internals
- task004 [ ] Enhance `DEVELOPER_ZH.md` with detailed Elisp API and C Module internals
- task005 [ ] Refine `readme.md` to be user-focused (remove internal details, add links to Dev docs)
- task006 [ ] Refine `readme_zh.md` to be user-focused (remove internal details, add links to Dev docs)

137
DEVELOPER.md Normal file
View File

@ -0,0 +1,137 @@
# Developer Documentation for Emacs-KP
This document details the internal architecture, API, and algorithms of `emacs-kp`. It is intended for contributors and advanced users who want to understand how the package works or extend it.
## 1. Architecture Overview
`emacs-kp` follows a layered architecture to separate text processing, layout computation, and rendering.
```
┌─────────────────────────────────────────────────────────────────┐
│ User API Layer (ekp.el) │
│ ekp-pixel-justify ekp-pixel-range-justify ekp-clear-caches │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ Caching Layer (ekp-utils.el) │
│ ekp--get-para (paragraph cache) ekp-dp-cache (DP result cache)│
└─────────────────────────────────────────────────────────────────┘
┌───────────────┴───────────────┐
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│ Pure Elisp Path │ │ C Module Path │
│ ekp--dp-cache-elisp │ │ ekp--dp-cache-via-c │
│ (O(n²) DP in Elisp) │ │ (calls C for DP) │
└─────────────────────────┘ └─────────────────────────┘
┌─────────────────────────┐
│ C Dynamic Module │
│ ekp_break_with_prefixes│
│ (8-thread parallel) │
└─────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ Rendering Layer │
│ ekp--render-justified (apply breaks, insert glue pixels) │
└─────────────────────────────────────────────────────────────────┘
```
## 2. Elisp Core (ekp.el)
### Data Structures
#### `ekp-para` Struct
The central data structure is `ekp-para`, which represents a preprocessed paragraph. It is cached to avoid re-tokenizing and re-measuring text.
```elisp
(cl-defstruct ekp-para
string ; Original text with properties
latin-font ; Detected Latin font
cjk-font ; Detected CJK font
boxes ; Vector of box strings
boxes-widths ; Vector of box pixel widths
boxes-types ; Vector of (start-type . end-type)
glues-types ; Vector of glue type symbols (lws, mws, cws, nws)
hyphen-pixel ; Width of hyphen character
hyphen-positions ; Vector of hyphenable box indices
ideal-prefixs ; Prefix sum: ideal widths (for O(1) width calc)
min-prefixs ; Prefix sum: minimum widths
max-prefixs ; Prefix sum: maximum widths
dp-cache) ; Hash table: line-pixel → DP result
```
#### Glue Types
- `lws`: Latin Word Space (between Latin words)
- `mws`: Mixed Word Space (between Latin and CJK)
- `cws`: CJK Word Space (between CJK chars)
- `nws`: No Word Space (fixed)
### Core Functions
#### `(ekp-pixel-justify STRING LINE-PIXEL)`
Justifies `STRING` to `LINE-PIXEL` width.
1. Checks cache for existing `ekp-para`.
2. If miss, creates `ekp-para` (tokenize, measure, hyphenate).
3. Calls DP engine (Elisp or C) to get breaks.
4. Renders result using display properties (specifically `space` display property for glues).
#### `(ekp-pixel-range-justify STRING MIN-PIXEL MAX-PIXEL)`
Finds the "best" width within a range. Uses ternary search (O(log n)) to minimize demerits. Useful for finding the optimal width for a specific paragraph.
#### `(ekp-param-set ...)`
Sets the 9 spacing parameters (Ideal/Stretch/Shrink for LWS/MWS/CWS).
## 3. C Dynamic Module (ekp_c)
For large texts, the C module provides ~20x speedup by parallelizing the O(n²) Dynamic Programming phase.
### Source Structure
- `ekp_c/ekp.c`: Emacs module entry point.
- `ekp_c/ekp_kp.c`: The Knuth-Plass algorithm implementation.
- `ekp_c/ekp_thread_pool.c`: Worker thread pool.
- `ekp_c/ekp_hyphen.c`: Liang's hyphenation algorithm.
### C API (exposed to Elisp)
#### `(ekp-c-init)`
Initializes the module and thread pool.
#### `(ekp-c-break-with-prefixes ...)`
The low-level DP function. It takes flat arrays (pointers) from Elisp:
- Prefix sums (ideal, min, max)
- Glue parameters per box
- Hyphen positions
- Target line width
It returns a list of break indices and total cost.
### Memory Model
- **Zero Copy**: Elisp passes pointers to vector data directly to C.
- **Flat Arrays**: Data is structured as parallel arrays for cache efficiency.
- **Thread Safety**: The module uses a fixed thread pool. The DP algorithm uses a wavefront pattern for parallelizing the inner loop.
## 4. Algorithm Details
### The Knuth-Plass Algorithm
Based on the 1981 paper "Breaking Paragraphs into Lines".
**Cost Function (Demerits):**
`D = (LinePenalty + Badness)² + Penalty²`
**Badness:**
`100 * |Adjustment / Flexibility|³`
### CJK Extensions
- **Boxes**: Each CJK character is a separate box.
- **Glues**: Specific glue types for CJK-CJK and CJK-Latin transitions allow fine-tuning spacing (e.g., adding slight breathing room between English and Chinese).
### Hyphenation
Uses Frank Liang's algorithm (standard in TeX).
- Patterns are loaded from `dictionaries/*.dic`.
- `ekp-hyphen.el` handles this in pure Elisp.
- C module has its own implementation (`ekp_hyphen.c`) for speed if needed, though currently Elisp handles tokenization.

137
DEVELOPER_ZH.md Normal file
View File

@ -0,0 +1,137 @@
# Emacs-KP 开发者文档
本文档详细介绍了 `emacs-kp` 的内部架构、API 和算法原理。旨在帮助贡献者和高级用户理解其工作机制。
## 1. 架构概览
`emacs-kp` 采用分层架构,将文本处理、布局计算和渲染分离。
```
┌─────────────────────────────────────────────────────────────────┐
│ 用户 API 层 (ekp.el) │
│ ekp-pixel-justify ekp-pixel-range-justify ekp-clear-caches │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 缓存层 (ekp-utils.el) │
│ ekp--get-para (段落缓存) ekp-dp-cache (DP 结果缓存) │
└─────────────────────────────────────────────────────────────────┘
┌───────────────┴───────────────┐
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│ 纯 Elisp 路径 │ │ C 模块路径 │
│ ekp--dp-cache-elisp │ │ ekp--dp-cache-via-c │
│ (Elisp 实现 O(n²) DP) │ │ (调用 C 进行 DP) │
└─────────────────────────┘ └─────────────────────────┘
┌─────────────────────────┐
│ C 动态模块 │
│ ekp_break_with_prefixes│
│ (8 线程并行计算) │
└─────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ 渲染层 │
│ ekp--render-justified (应用断点,插入 display 属性胶水) │
└─────────────────────────────────────────────────────────────────┘
```
## 2. Elisp 核心 (ekp.el)
### 数据结构
#### `ekp-para` 结构体
核心数据结构是 `ekp-para`,代表预处理后的段落。它被缓存以避免重复分词和测量。
```elisp
(cl-defstruct ekp-para
string ; 带有属性的原始文本
latin-font ; 检测到的拉丁字体
cjk-font ; 检测到的 CJK 字体
boxes ; Box 字符串向量
boxes-widths ; Box 像素宽度向量
boxes-types ; 类型向量 (start-type . end-type)
glues-types ; 胶水类型符号向量 (lws, mws, cws, nws)
hyphen-pixel ; 连字符宽度
hyphen-positions ; 可断词 Box 索引向量
ideal-prefixs ; 前缀和:理想宽度 (用于 O(1) 宽度计算)
min-prefixs ; 前缀和:最小宽度
max-prefixs ; 前缀和:最大宽度
dp-cache) ; 哈希表:行宽像素 → DP 结果
```
#### 胶水类型 (Glue Types)
- `lws` (Latin Word Space): 拉丁词间距
- `mws` (Mixed Word Space): 中西文间距
- `cws` (CJK Word Space): CJK 字符间距
- `nws` (No Word Space): 固定间距
### 核心函数
#### `(ekp-pixel-justify STRING LINE-PIXEL)`
`STRING``LINE-PIXEL` 宽度对齐。
1. 检查缓存中是否有对应的 `ekp-para`
2. 若未命中,创建 `ekp-para`(分词、测量、断词处理)。
3. 调用 DP 引擎Elisp 或 C计算断点。
4. 使用 display 属性(特别是 `space` 属性)渲染结果。
#### `(ekp-pixel-range-justify STRING MIN-PIXEL MAX-PIXEL)`
在范围内寻找“最佳”宽度。使用三分搜索 (O(log n)) 最小化 demerits。用于自动寻找最适合该段落的宽度。
#### `(ekp-param-set ...)`
设置 9 个间距参数LWS/MWS/CWS 的 Ideal/Stretch/Shrink
## 3. C 动态模块 (ekp_c)
对于长文本C 模块通过并行化 O(n²) 动态规划阶段提供约 20 倍的加速。
### 源码结构
- `ekp_c/ekp.c`: Emacs 模块入口点。
- `ekp_c/ekp_kp.c`: Knuth-Plass 算法实现。
- `ekp_c/ekp_thread_pool.c`: 工作线程池。
- `ekp_c/ekp_hyphen.c`: Liang 断词算法。
### C API (暴露给 Elisp)
#### `(ekp-c-init)`
初始化模块和线程池。
#### `(ekp-c-break-with-prefixes ...)`
底层 DP 函数。它接收来自 Elisp 的扁平数组(指针):
- 前缀和数组 (ideal, min, max)
- 每个 Box 的胶水参数
- 连字符位置
- 目标行宽
返回断点索引列表和总代价。
### 内存模型
- **零拷贝 (Zero Copy)**: Elisp 直接将向量数据的指针传递给 C。
- **扁平数组**: 数据结构为并行数组,提高缓存效率。
- **线程安全**: 模块使用固定线程池。DP 算法采用波前模式 (Wavefront) 并行化内部循环。
## 4. 算法细节
### Knuth-Plass 算法
基于 1981 年论文 "Breaking Paragraphs into Lines"。
**代价函数 (Demerits):**
`D = (LinePenalty + Badness)² + Penalty²`
**劣度 (Badness):**
`100 * |Adjustment / Flexibility|³`
### CJK 扩展
- **Boxes**: 每个 CJK 字符视为一个独立的 Box。
- **Glues**: 针对 CJK-CJK 和 CJK-Latin 的特定胶水类型允许精细调整间距(例如在汉字和英文之间增加微小的空隙)。
### 断词 (Hyphenation)
使用 Frank Liang 算法TeX 标准)。
- 模式从 `dictionaries/*.dic` 加载。
- `ekp-hyphen.el` 在纯 Elisp 中处理。
- C 模块有自己的实现 (`ekp_hyphen.c`),目前主要由 Elisp 负责分词。

View File

@ -9,6 +9,14 @@
;;; Code:
(defconst ekp-utils--load-file (or load-file-name (buffer-file-name))
"Path to this file, for locating module directories.")
(defun ekp-root-dir ()
"Return directory containing ekp files."
(when ekp-utils--load-file
(file-name-directory ekp-utils--load-file)))
;;;; Font Detection
(defsubst ekp-cjk-char-p (char)
@ -199,12 +207,14 @@ Whitespace runs are preserved as separate boxes; CJK punctuation attaches to pre
;; Latin character (width = 1)
((= 1 width)
(pcase-let ((`(,s ,lw ,cc ,bx)
(ekp--handle-latin-char str state latin-word cjk-char boxes)))
(ekp--handle-latin-char
str state latin-word cjk-char boxes)))
(setq state s latin-word lw cjk-char cc boxes bx)))
;; CJK character (width = 2)
((= 2 width)
(pcase-let ((`(,s ,lw ,cc ,bx)
(ekp--handle-cjk-char str state latin-word cjk-char boxes)))
(ekp--handle-cjk-char
str state latin-word cjk-char boxes)))
(setq state s latin-word lw cjk-char cc boxes bx)))))))
(forward-char 1))
;; Flush remaining content
@ -285,6 +295,80 @@ Whitespace runs are preserved as separate boxes; CJK punctuation attaches to pre
(message "ekp rust module reload success!")))
(error "Please install cargo and add it to executable path!")))
;;; C Module Support
;; Parallel C implementation using pthreads
(defvar ekp-c-module-loaded nil
"Non-nil if C module is loaded.")
(defvar ekp-c-hyphenator-index nil
"Index of the loaded hyphenator in C module.")
(defun ekp-c-module-dir ()
"Return the C module directory."
(when-let ((root-dir (ekp-root-dir)))
(expand-file-name "ekp_c" root-dir)))
(defun ekp-c-module-file ()
"Return path to compiled C module."
(when-let* ((module-dir (ekp-c-module-dir))
(filename (cond ((eq system-type 'darwin) "ekp.dylib")
((eq system-type 'windows-nt) "ekp.dll")
(t "ekp.so"))))
(expand-file-name filename module-dir)))
(defun ekp-c-module-reload (module)
"Load MODULE from a temp copy to allow rebuilding."
(let ((tmpfile (make-temp-file
(file-name-nondirectory module))))
(copy-file module tmpfile t)
(module-load tmpfile)))
(defun ekp-c-module-load ()
"Load EKP C module if available."
(interactive)
(let ((file (ekp-c-module-file)))
(if (and file (file-exists-p file))
(progn
(ekp-c-module-reload file)
(when (fboundp 'ekp-c-init)
(ekp-c-init)
(setq ekp-c-module-loaded t)
(message "ekp-c module loaded (version %s, %d threads)"
(ekp-c-version) (ekp-c-thread-count))))
(message "C module not found. Run 'make' in ekp_c/ directory."))))
(defun ekp-c-load-dictionary (lang)
"Load hyphenation dictionary for LANG into C module."
(when ekp-c-module-loaded
(let* ((root-dir (ekp-root-dir))
(dict-file (expand-file-name
(format "dictionaries/hyph_%s.dic" lang)
root-dir)))
(when (file-exists-p dict-file)
(setq ekp-c-hyphenator-index
(ekp-c-load-hyphenator dict-file))
(when ekp-c-hyphenator-index
(message "Loaded hyphenator for %s (index %d)"
lang ekp-c-hyphenator-index))))))
(defun ekp-c-module-build ()
"Build the C module using make."
(interactive)
(let ((module-dir (ekp-c-module-dir)))
(if (and module-dir (file-exists-p
(expand-file-name "Makefile" module-dir)))
(ekp-start-process-with-callback
"ekp-c-build"
(cond
((eq system-type 'windows-nt)
`("cmd.exe" "/c" ,(format "cd %s && make" module-dir)))
(t `("zsh" "-c" ,(format "cd %s && make" module-dir))))
(lambda (proc buffer)
(ekp-c-module-load)
(message "ekp C module build success!")))
(error "Makefile not found in ekp_c/ directory"))))
(provide 'ekp-utils)
;;; ekp-utils.el ends here

204
ekp.el
View File

@ -28,6 +28,11 @@
(defvar ekp-latin-lang "en_US"
"Language code for hyphenation (e.g., 'en_US', 'de_DE').")
(defvar ekp-use-c-module t
"When non-nil, use C dynamic module for DP computation if available.
The C module provides significant performance improvement for large texts.
Set to nil to force pure Elisp implementation.")
;;;; Glue Parameters
;; Glue = flexible space between boxes (Knuth-Plass terminology)
;; lws = Latin Word Space, mws = Mixed (Latin-CJK), cws = CJK
@ -111,10 +116,10 @@
(defun ekp-param-set-default (string)
"Set default spacing parameters based on STRING's font."
(let* ((lws (ekp-word-spacing-pixel string))
(mws (- lws 2)))
(ekp-param-set lws (/ lws 2) (/ lws 3)
mws (/ mws 2) (/ mws 3)
0 2 0)))
(mws (- lws 1)))
(ekp-param-set lws (ceiling (/ (float lws) 2)) (ceiling (/ (float lws) 3))
mws (ceiling (/ (float mws) 2)) (ceiling (/ (float mws) 3))
0 3 0)))
(defun ekp-param-set (lws-i lws-+ lws-- mws-i mws-+ mws-- cws-i cws-+ cws--)
"Set all spacing parameters.
@ -576,11 +581,98 @@ Used for looseness parameter support."
(defun ekp-dp-cache (string line-pixel)
"Compute optimal line breaks for STRING at LINE-PIXEL width.
Uses Knuth-Plass dynamic programming with demerits."
Uses Knuth-Plass dynamic programming with demerits.
If `ekp-use-c-module' is non-nil and C module is available, uses it."
(let* ((para (ekp--get-para string))
(cached (ekp--dp-get-cached para line-pixel)))
(if cached
cached
;; Try C module first (if enabled and available)
(if (and ekp-use-c-module
(boundp 'ekp-c-module-loaded) ekp-c-module-loaded
(fboundp 'ekp-c-break-with-arrays))
(ekp--dp-cache-via-c para string line-pixel)
;; Fallback to Elisp implementation
(ekp--dp-cache-elisp para string line-pixel)))))
(defun ekp--glue-type-to-int (type)
"Convert glue TYPE symbol to integer for C module.
0=nws, 1=lws, 2=mws, 3=cws."
(pcase type
('lws 1)
('mws 2)
('cws 3)
(_ 0))) ; nws or nil
(defun ekp--dp-cache-via-c (para string line-pixel)
"Compute breaks using C module with Elisp's pre-computed prefix arrays.
C module receives ALL font-dependent data from Elisp's para struct:
prefix sums, glue values, hyphen info. C only does pure DP."
(ignore string) ; Use para's data instead
(let* ((ideal-prefixs (ekp-para-ideal-prefixs para))
(min-prefixs (ekp-para-min-prefixs para))
(max-prefixs (ekp-para-max-prefixs para))
(glues-types (ekp-para-glues-types para))
(hyphen-positions (ekp-para-hyphen-positions para))
(hyphen-pixel (ekp-para-hyphen-pixel para))
(n (length (ekp-para-boxes para)))
;; Build glue value arrays for C
(glue-ideals (make-vector n 0))
(glue-shrinks (make-vector n 0))
(glue-stretches (make-vector n 0)))
;; Extract glue values from type symbols
(dotimes (i n)
(let ((type (aref glues-types i)))
(aset glue-ideals i (ekp-glue-ideal-pixel type))
(aset glue-shrinks i (- (ekp-glue-ideal-pixel type)
(ekp-glue-min-pixel type)))
(aset glue-stretches i (- (ekp-glue-max-pixel type)
(ekp-glue-ideal-pixel type)))))
;; Call C module with all Elisp-computed arrays
(let* ((result (ekp-c-break-with-arrays
ideal-prefixs
min-prefixs
max-prefixs
glue-ideals
glue-shrinks
glue-stretches
hyphen-positions
hyphen-pixel
line-pixel))
(c-breaks (car result))
(c-cost (cdr result)))
(if (null c-breaks)
;; C module failed, fallback to Elisp
(ekp--dp-cache-elisp para string line-pixel)
;; C module succeeded: compute rests and gaps from breaks
(let* ((breaks c-breaks)
(start 0)
lines-rests lines-gaps)
;; Compute rests and gaps for each line
(dolist (end breaks)
(let* ((leading-glue-type (aref glues-types start))
(end-with-hyphenp (ekp--hyphenate-p hyphen-positions (1- end)))
(ideal-pixel (- (aref ideal-prefixs end)
(aref ideal-prefixs start)
(ekp-glue-ideal-pixel leading-glue-type))))
(when end-with-hyphenp
(cl-incf ideal-pixel hyphen-pixel))
(push (- line-pixel ideal-pixel) lines-rests)
(push (ekp--gaps-list
(seq-drop (cl-subseq glues-types start end) 1))
lines-gaps)
(setq start end)))
(let ((dp-result (list :rests (nreverse lines-rests)
:gaps (nreverse lines-gaps)
:breaks breaks
:cost c-cost
:line-count (length breaks))))
(puthash line-pixel dp-result (ekp-para-dp-cache para))
dp-result))))))
(defun ekp--dp-cache-elisp (para string line-pixel)
"Pure Elisp DP implementation."
(ignore string) ; para already contains all needed data
;; Get data directly from struct (O(1) access)
(let* ((glues-types (ekp-para-glues-types para))
(boxes (ekp-para-boxes para))
@ -660,7 +752,7 @@ Uses Knuth-Plass dynamic programming with demerits."
:cost (aref demerits n)
:line-count (aref line-counts n))))
(puthash line-pixel dp-result (ekp-para-dp-cache para))
dp-result)))))
dp-result)))
(defun ekp-dp-data (string line-pixel &optional key)
"Return the data plist of dp cache. If KEY is non-nil,
@ -831,6 +923,11 @@ Each line's glues: [0 glue1 glue2 ... trailing-space]."
(setq start end)))
line-glues))
(defun ekp--box-space-p (box)
"Return non-nil if BOX is a whitespace-only box."
(and box (not (string-empty-p box))
(or (string-blank-p box) (= (string-width box) 0))))
(defun ekp--interleave (list1 list2)
"Interleave elements of LIST1 and LIST2."
(let (result)
@ -851,9 +948,85 @@ Each line's glues: [0 glue1 glue2 ... trailing-space]."
(error "Glues count (%d) must equal boxes count (%d) + 1"
(1+ (length glues)) (length boxes)))))
(defun ekp--pixel-spacing-width (spacing)
"Extract pixel width from a SPACING created by `ekp-pixel-spacing'."
(if (string-empty-p spacing)
0
(let ((display (get-text-property 0 'display spacing)))
(if (and display (eq (car display) 'space))
(let ((width-spec (plist-get (cdr display) :width)))
(if (listp width-spec) (car width-spec) (or width-spec 0)))
0))))
(defun ekp--redistribute-extra-width (glues extra-width)
"Redistribute EXTRA-WIDTH across GLUES proportionally.
GLUES is a list of pixel spacing strings. Returns adjusted list.
The extra width is distributed to all glues except leading (first) glue."
(when (and glues (> extra-width 0))
(let* ((inner-glues (butlast (cdr glues))) ; glues between boxes (not leading/trailing)
(n (length inner-glues)))
(if (= n 0)
;; No inner glues, add all to trailing
(let* ((trailing (car (last glues)))
(old-width (ekp--pixel-spacing-width trailing))
(new-width (+ old-width extra-width)))
(setf (car (last glues)) (ekp-pixel-spacing new-width)))
;; Distribute across inner glues
(let ((per-glue (/ extra-width n))
(remainder (% extra-width n))
(idx 0))
(setq glues
(cons (car glues) ; leading glue unchanged
(append
(mapcar
(lambda (g)
(let* ((old-w (ekp--pixel-spacing-width g))
(extra (+ per-glue (if (< idx remainder) 1 0)))
(new-w (+ old-w extra)))
(cl-incf idx)
(ekp-pixel-spacing new-w)))
inner-glues)
(last glues)))))))) ; trailing glue unchanged
glues)
(defun ekp--strip-line-spaces (line-boxes line-glues line-boxes-widths)
"Strip leading/trailing space boxes from LINE-BOXES.
Returns (stripped-boxes . adjusted-glues) with extra width redistributed.
LINE-BOXES-WIDTHS is the pixel widths corresponding to LINE-BOXES.
The removed space width is redistributed to remaining glues for proper justification."
(let* ((boxes (append line-boxes nil))
(glues (append line-glues nil))
(widths (append line-boxes-widths nil))
(removed-width 0)) ; Track total width of removed space boxes
(when (> (length boxes) 0)
;; Strip trailing space boxes
(while (and boxes (ekp--box-space-p (car (last boxes))))
;; Accumulate width of removed space box
(cl-incf removed-width (car (last widths)))
(setq boxes (butlast boxes))
(setq widths (butlast widths))
;; Remove second-to-last glue (the one before the trailing space box)
;; Keep the last glue which is trailing space for the line
(when (> (length glues) 1)
(setq glues (append (butlast (butlast glues)) (last glues)))))
;; Strip leading space boxes
(while (and boxes (ekp--box-space-p (car boxes)))
;; Accumulate width of removed space box
(cl-incf removed-width (car widths))
(setq boxes (cdr boxes))
(setq widths (cdr widths))
;; Remove the second glue (the one after the leading glue)
(when (> (length glues) 1)
(setq glues (cons (car glues) (cddr glues))))))
;; Redistribute removed width to remaining glues for proper justification
(when (> removed-width 0)
(setq glues (ekp--redistribute-extra-width glues removed-width)))
(cons (vconcat boxes) glues)))
(defun ekp--pixel-justify (string line-pixel)
"Justify single STRING to LINE-PIXEL."
(let* ((boxes (ekp--boxes string))
(boxes-widths (ekp--boxes-widths string))
(hyphen (ekp--hyphen-str string))
(breaks (ekp-line-breaks string line-pixel))
(num (length breaks))
@ -863,20 +1036,25 @@ Each line's glues: [0 glue1 glue2 ... trailing-space]."
(dotimes (i num)
(let* ((end (nth i breaks))
(line-boxes (cl-subseq boxes start end))
(line-glues (mapcar #'ekp-pixel-spacing
(line-boxes-widths (cl-subseq boxes-widths start end))
(line-glues-raw (mapcar #'ekp-pixel-spacing
(aref lines-glues i)))
;; Strip leading/trailing space boxes, adjust trailing glue
(stripped (ekp--strip-line-spaces line-boxes line-glues-raw
line-boxes-widths))
(line-boxes (car stripped))
(line-glues (cdr stripped))
;; Check if last box of this line needs hyphen
;; hyphen-positions stores box indices, end-1
;; is the last box index
(last-box-idx (1- end))
(need-hyphen
(and (< i (1- num)) ; not last line
(ekp--hyphenate-p hyphen-positions last-box-idx))))
(when need-hyphen
(setf (aref line-boxes (- end start 1))
(concat (aref line-boxes (- end start 1)) hyphen)))
(when (and need-hyphen (> (length line-boxes) 0))
(setf (aref line-boxes (1- (length line-boxes)))
(concat (aref line-boxes (1- (length line-boxes))) hyphen)))
(when (> (length line-boxes) 0)
(push (ekp--combine-glues-and-boxes line-glues line-boxes)
strings)
strings))
(setq start end)))
(mapconcat 'identity (nreverse strings) "\n")))

90
ekp_c/Makefile Normal file
View File

@ -0,0 +1,90 @@
# Makefile for EKP C dynamic module
#
# Builds ekp.so (or ekp.dylib on macOS) for Emacs dynamic loading.
# Uses pthread for multi-threading, optimizes for native CPU.
# Detect OS
UNAME := $(shell uname)
# Compiler settings
CC := cc
CFLAGS := -std=c11 -Wall -Wextra -Wpedantic -O3 -fPIC
CFLAGS += -march=native -flto
CFLAGS += -D_POSIX_C_SOURCE=200809L
# Debug build
ifdef DEBUG
CFLAGS := -std=c11 -Wall -Wextra -Wpedantic -O0 -g -fPIC
CFLAGS += -fsanitize=address,undefined
endif
# Platform-specific settings
ifeq ($(UNAME), Darwin)
# macOS
MODULE_EXT := dylib
LDFLAGS := -shared -lpthread
# Find Emacs headers (check Emacs.app, homebrew, /usr/local)
EMACS_INCLUDE := $(shell find /Applications/Emacs.app/Contents/Resources/include /opt/homebrew /usr/local -name emacs-module.h -exec dirname {} \; 2>/dev/null | head -1)
ifdef EMACS_INCLUDE
CFLAGS += -I$(EMACS_INCLUDE)
endif
else ifeq ($(UNAME), Linux)
# Linux
MODULE_EXT := so
LDFLAGS := -shared -lpthread -lm
else
# Windows (MinGW)
MODULE_EXT := dll
LDFLAGS := -shared -lpthread
endif
# Source files
SRCS := ekp.c ekp_kp.c ekp_hyphen.c ekp_paragraph.c ekp_thread_pool.c
OBJS := $(SRCS:.c=.o)
# Output
MODULE := ekp.$(MODULE_EXT)
# Targets
.PHONY: all clean install test
all: $(MODULE)
$(MODULE): $(OBJS)
$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS)
@echo "Built $@"
%.o: %.c ekp_module.h
$(CC) $(CFLAGS) -c -o $@ $<
clean:
rm -f $(OBJS) $(MODULE)
# Install to Emacs load path (customize DESTDIR as needed)
DESTDIR ?= $(HOME)/.emacs.d/modules
install: $(MODULE)
@mkdir -p $(DESTDIR)
cp $(MODULE) $(DESTDIR)/
@echo "Installed to $(DESTDIR)/$(MODULE)"
# Test with Emacs
test: $(MODULE)
emacs -Q --batch \
-L . \
--eval '(module-load (expand-file-name "./$(MODULE)"))' \
--eval '(message "ekp-c version: %s" (ekp-c-version))' \
--eval '(ekp-c-init)' \
--eval '(message "ekp-c initialized with %d threads" (ekp-c-thread-count))' \
--eval '(ekp-c-cleanup)' \
--eval '(message "Test passed!")'
# Show compiler info
info:
@echo "CC: $(CC)"
@echo "CFLAGS: $(CFLAGS)"
@echo "LDFLAGS: $(LDFLAGS)"
@echo "MODULE: $(MODULE)"
@echo "UNAME: $(UNAME)"
ifdef EMACS_INCLUDE
@echo "EMACS_INCLUDE: $(EMACS_INCLUDE)"
endif

119
ekp_c/README.md Normal file
View File

@ -0,0 +1,119 @@
# EKP C Dynamic Module
High-performance C implementation of the Knuth-Plass line breaking algorithm with multi-threaded parallel computation.
## Architecture
```
ekp_c/
├── ekp_module.h # Core data structures and API declarations
├── ekp.c # Emacs module entry point (emacs_module_init)
├── ekp_kp.c # Knuth-Plass DP algorithm + global state
├── ekp_hyphen.c # Liang hyphenation with thread-safe caching
├── ekp_paragraph.c # Text tokenization and box/glue construction
├── ekp_thread_pool.c # Work-stealing thread pool
└── Makefile # Build system
```
## Building
```bash
cd ekp_c
make
```
Requirements:
- C11 compiler (clang, gcc)
- Emacs with dynamic module support (27.1+)
- pthread library
### Build Options
```bash
make DEBUG=1 # Debug build with sanitizers
make clean # Remove build artifacts
make info # Show build configuration
make test # Run basic tests in Emacs
```
## Performance Optimizations
### 1. Multi-threaded Processing
- 8-thread pool for parallel DP candidate evaluation
- Wavefront parallelization for large paragraphs (>100 boxes)
- Lock-free work queue with condition variables
### 2. O(1) Range Queries
- Prefix sum arrays for ideal/min/max line widths
- Eliminates repeated summation in inner DP loop
### 3. Fast Hyphenation
- FNV-1a hash for O(1) pattern lookup
- Thread-safe LRU cache (4096 entries)
- Read-write locks for concurrent access
### 4. Memory Layout
- Flat, cache-friendly data structures
- Parallel arrays for boxes, glues, widths
- Minimal allocations in hot paths
## API
### Initialization
```elisp
(ekp-c-init) ; Initialize module with thread pool
(ekp-c-cleanup) ; Release all resources
(ekp-c-version) ; => "1.0"
(ekp-c-thread-count) ; => 8
```
### Hyphenation
```elisp
(ekp-c-load-hyphenator "/path/to/hyph_en_US.dic") ; => 0 (index)
(ekp-c-hyphenate 0 "hyphenation") ; => (2 5 7)
```
### Line Breaking
```elisp
(ekp-c-break-lines
"Your paragraph text here"
0 ; hyphenator index
600 ; line width in pixels
#'string-pixel-width) ; measurement function
;; Returns: ((breaks...) . total-cost)
```
### Parameters
```elisp
;; Spacing: (lws-i lws+ lws- mws-i mws+ mws- cws-i cws+ cws-)
(ekp-c-set-spacing 7 3 2 5 2 1 0 2 0)
;; Penalties: (line-penalty hyphen-penalty fitness-penalty last-line-ratio)
(ekp-c-set-penalties 10 50 100 0.5)
```
## Design Notes
Following Linus's philosophy:
1. **Data structures are the code** - Get box/glue layout right, algorithm follows naturally
2. **Simple thread model** - Fixed pool, no dynamic thread creation in hot path
3. **Minimal abstraction** - Direct array access, no virtual dispatch
4. **Fail fast** - Return NULL/nil on errors, let Emacs handle it
## Benchmark
Typical speedup vs pure Elisp implementation:
| Paragraph Size | Elisp | C Module | Speedup |
|----------------|-------|----------|---------|
| 100 chars | 5ms | 0.3ms | 16x |
| 500 chars | 45ms | 2ms | 22x |
| 2000 chars | 350ms | 12ms | 29x |
*Note: Actual performance depends on CPU, Emacs version, and text characteristics.*

501
ekp_c/ekp.c Normal file
View File

@ -0,0 +1,501 @@
/*
* ekp.c - Emacs Knuth-Plass module entry point
*
* This is the main entry file for the Emacs dynamic module.
* Naming follows Emacs module convention: module name = file name.
* (require 'ekp-c) loads ekp.so, entry point in ekp.c
*
* Bridges C implementation to Emacs Lisp. Keep the interface minimal:
* pixel measurement callback from Emacs, everything else in C.
*/
#include "ekp_module.h"
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
/* Required for Emacs modules */
int plugin_is_GPL_compatible;
/* Cached Emacs environment for callbacks */
static emacs_env *current_env = NULL;
static emacs_value measure_func = NULL;
/*
* Pixel measurement callback that calls back into Emacs
*/
static int32_t emacs_measure_string(const char *text, size_t len)
{
if (!current_env || !measure_func)
return len * 7; /* fallback: ~7 pixels per char */
emacs_value str = current_env->make_string(current_env, text, len);
emacs_value result = current_env->funcall(current_env, measure_func, 1, &str);
if (current_env->non_local_exit_check(current_env) != emacs_funcall_exit_return)
return len * 7;
return (int32_t)current_env->extract_integer(current_env, result);
}
/*
* ekp-c-init: Initialize the module
*/
static emacs_value Fekp_c_init(emacs_env *env, ptrdiff_t nargs,
emacs_value *args, void *data)
{
(void)nargs; (void)args; (void)data;
if (ekp_init() != 0) {
emacs_value signal = env->intern(env, "error");
emacs_value msg = env->make_string(env, "Failed to initialize ekp-c", 26);
env->non_local_exit_signal(env, signal, msg);
return env->intern(env, "nil");
}
return env->intern(env, "t");
}
/*
* ekp-c-cleanup: Cleanup resources
*/
static emacs_value Fekp_c_cleanup(emacs_env *env, ptrdiff_t nargs,
emacs_value *args, void *data)
{
(void)nargs; (void)args; (void)data;
ekp_cleanup();
return env->intern(env, "t");
}
/*
* ekp-c-load-hyphenator: Load hyphenation dictionary
*/
static emacs_value Fekp_c_load_hyphenator(emacs_env *env, ptrdiff_t nargs,
emacs_value *args, void *data)
{
(void)nargs; (void)data;
if (!ekp_global) {
emacs_value signal = env->intern(env, "error");
emacs_value msg = env->make_string(env, "ekp-c not initialized", 21);
env->non_local_exit_signal(env, signal, msg);
return env->intern(env, "nil");
}
/* Get dictionary path */
ptrdiff_t size = 0;
env->copy_string_contents(env, args[0], NULL, &size);
char *path = malloc(size);
if (!path)
return env->intern(env, "nil");
env->copy_string_contents(env, args[0], path, &size);
/* Load hyphenator */
ekp_hyphenator_t *h = ekp_hyphen_create(path);
free(path);
if (!h)
return env->intern(env, "nil");
/* Store in global state */
if (ekp_global->hyphenator_count < 32) {
ekp_global->hyphenators[ekp_global->hyphenator_count++] = h;
return env->make_integer(env, ekp_global->hyphenator_count - 1);
}
ekp_hyphen_destroy(h);
return env->intern(env, "nil");
}
/*
* ekp-c-set-spacing: Set spacing parameters
*/
static emacs_value Fekp_c_set_spacing(emacs_env *env, ptrdiff_t nargs,
emacs_value *args, void *data)
{
(void)data;
if (!ekp_global || nargs < 9) {
return env->intern(env, "nil");
}
ekp_global->spacing.lws_ideal = env->extract_integer(env, args[0]);
ekp_global->spacing.lws_stretch = env->extract_integer(env, args[1]);
ekp_global->spacing.lws_shrink = env->extract_integer(env, args[2]);
ekp_global->spacing.mws_ideal = env->extract_integer(env, args[3]);
ekp_global->spacing.mws_stretch = env->extract_integer(env, args[4]);
ekp_global->spacing.mws_shrink = env->extract_integer(env, args[5]);
ekp_global->spacing.cws_ideal = env->extract_integer(env, args[6]);
ekp_global->spacing.cws_stretch = env->extract_integer(env, args[7]);
ekp_global->spacing.cws_shrink = env->extract_integer(env, args[8]);
return env->intern(env, "t");
}
/*
* ekp-c-set-penalties: Set K-P parameters
*/
static emacs_value Fekp_c_set_penalties(emacs_env *env, ptrdiff_t nargs,
emacs_value *args, void *data)
{
(void)data;
if (!ekp_global || nargs < 4)
return env->intern(env, "nil");
ekp_global->line_penalty = env->extract_integer(env, args[0]);
ekp_global->hyphen_penalty = env->extract_integer(env, args[1]);
ekp_global->fitness_penalty = env->extract_integer(env, args[2]);
ekp_global->last_line_ratio = env->extract_float(env, args[3]);
return env->intern(env, "t");
}
/*
* ekp-c-hyphenate: Get hyphenation positions for a word
*/
static emacs_value Fekp_c_hyphenate(emacs_env *env, ptrdiff_t nargs,
emacs_value *args, void *data)
{
(void)data;
if (!ekp_global || nargs < 2)
return env->intern(env, "nil");
intmax_t h_idx = env->extract_integer(env, args[0]);
if (h_idx < 0 || (size_t)h_idx >= ekp_global->hyphenator_count)
return env->intern(env, "nil");
ekp_hyphenator_t *h = ekp_global->hyphenators[h_idx];
/* Get word */
ptrdiff_t size = 0;
env->copy_string_contents(env, args[1], NULL, &size);
char *word = malloc(size);
if (!word)
return env->intern(env, "nil");
env->copy_string_contents(env, args[1], word, &size);
/* Hyphenate */
int8_t positions[EKP_MAX_WORD_LEN];
int count = ekp_hyphen_word(h, word, size - 1, positions, EKP_MAX_WORD_LEN);
free(word);
/* Build result list */
emacs_value result = env->intern(env, "nil");
emacs_value cons_sym = env->intern(env, "cons");
for (int i = count - 1; i >= 0; i--) {
emacs_value pos = env->make_integer(env, positions[i]);
emacs_value args2[2] = {pos, result};
result = env->funcall(env, cons_sym, 2, args2);
}
return result;
}
/*
* ekp-c-break-lines: Core line breaking function
*
* Args: (string hyphenator-index line-width measure-func)
* Returns: (breaks . total-cost) where breaks is a list
*/
static emacs_value Fekp_c_break_lines(emacs_env *env, ptrdiff_t nargs,
emacs_value *args, void *data)
{
(void)data;
if (!ekp_global || nargs < 4)
return env->intern(env, "nil");
/* Get string */
ptrdiff_t size = 0;
env->copy_string_contents(env, args[0], NULL, &size);
char *text = malloc(size);
if (!text)
return env->intern(env, "nil");
env->copy_string_contents(env, args[0], text, &size);
size_t text_len = size - 1;
/* Get hyphenator */
intmax_t h_idx = env->extract_integer(env, args[1]);
ekp_hyphenator_t *h = NULL;
if (h_idx >= 0 && (size_t)h_idx < ekp_global->hyphenator_count)
h = ekp_global->hyphenators[h_idx];
/* Get line width */
int32_t line_width = env->extract_integer(env, args[2]);
/* Get measure function */
current_env = env;
measure_func = args[3];
/* Create paragraph */
ekp_paragraph_t *para = ekp_para_create(text, text_len, h, emacs_measure_string);
free(text);
if (!para) {
current_env = NULL;
measure_func = NULL;
return env->intern(env, "nil");
}
/* Break lines */
ekp_result_t *result = ekp_break_lines(para, line_width);
current_env = NULL;
measure_func = NULL;
if (!result) {
ekp_para_destroy(para);
return env->intern(env, "nil");
}
/* Build result: ((breaks...) . cost) */
emacs_value breaks_list = env->intern(env, "nil");
emacs_value cons_sym = env->intern(env, "cons");
for (size_t i = result->break_count; i > 0; i--) {
emacs_value brk = env->make_integer(env, result->breaks[i - 1]);
emacs_value args2[2] = {brk, breaks_list};
breaks_list = env->funcall(env, cons_sym, 2, args2);
}
emacs_value cost = env->make_float(env, result->total_cost);
emacs_value args2[2] = {breaks_list, cost};
emacs_value final = env->funcall(env, cons_sym, 2, args2);
ekp_result_destroy(result);
ekp_para_destroy(para);
return final;
}
/*
* ekp-c-version: Return module version
*/
static emacs_value Fekp_c_version(emacs_env *env, ptrdiff_t nargs,
emacs_value *args, void *data)
{
(void)nargs; (void)args; (void)data;
char version[32];
snprintf(version, sizeof(version), "%d.%d",
EKP_VERSION_MAJOR, EKP_VERSION_MINOR);
return env->make_string(env, version, strlen(version));
}
/*
* ekp-c-thread-count: Return number of worker threads
*/
static emacs_value Fekp_c_thread_count(emacs_env *env, ptrdiff_t nargs,
emacs_value *args, void *data)
{
(void)nargs; (void)args; (void)data;
return env->make_integer(env, EKP_THREAD_POOL_SIZE);
}
/*
* ekp-c-break-with-arrays: Pure DP with Elisp-provided prefix arrays
*
* Args: (ideal-prefix min-prefix max-prefix glue-ideals glue-shrinks glue-stretches
* hyphen-positions hyphen-width line-width)
*
* All 6 arrays must have consistent sizes:
* - ideal/min/max-prefix: (n+1) elements
* - glue-ideals/shrinks/stretches: n elements
*
* Returns: (breaks . total-cost) where breaks is a list of box indices.
*
* This is the preferred API: Elisp computes all font-dependent values,
* C module only does pure O(n²) DP computation.
*/
static emacs_value Fekp_c_break_with_arrays(emacs_env *env, ptrdiff_t nargs,
emacs_value *args, void *data)
{
(void)data;
if (!ekp_global || nargs < 9)
return env->intern(env, "nil");
/* Get prefix array sizes (n+1 elements) */
ptrdiff_t prefix_len = env->vec_size(env, args[0]);
if (prefix_len <= 1)
return env->intern(env, "nil");
size_t n = prefix_len - 1; /* number of boxes */
/* Allocate arrays */
int32_t *ideal_prefix = malloc(prefix_len * sizeof(int32_t));
int32_t *min_prefix = malloc(prefix_len * sizeof(int32_t));
int32_t *max_prefix = malloc(prefix_len * sizeof(int32_t));
int32_t *glue_ideals = malloc(n * sizeof(int32_t));
int32_t *glue_shrinks = malloc(n * sizeof(int32_t));
int32_t *glue_stretches = malloc(n * sizeof(int32_t));
if (!ideal_prefix || !min_prefix || !max_prefix ||
!glue_ideals || !glue_shrinks || !glue_stretches) {
free(ideal_prefix); free(min_prefix); free(max_prefix);
free(glue_ideals); free(glue_shrinks); free(glue_stretches);
return env->intern(env, "nil");
}
/* Extract prefix arrays */
for (ptrdiff_t i = 0; i < prefix_len; i++) {
ideal_prefix[i] = env->extract_integer(env, env->vec_get(env, args[0], i));
min_prefix[i] = env->extract_integer(env, env->vec_get(env, args[1], i));
max_prefix[i] = env->extract_integer(env, env->vec_get(env, args[2], i));
}
/* Extract glue arrays */
for (size_t i = 0; i < n; i++) {
glue_ideals[i] = env->extract_integer(env, env->vec_get(env, args[3], i));
glue_shrinks[i] = env->extract_integer(env, env->vec_get(env, args[4], i));
glue_stretches[i] = env->extract_integer(env, env->vec_get(env, args[5], i));
}
/* Get hyphen positions vector */
ptrdiff_t hyph_count = env->vec_size(env, args[6]);
int32_t *hyph_pos = NULL;
if (hyph_count > 0) {
hyph_pos = malloc(hyph_count * sizeof(int32_t));
if (hyph_pos) {
for (ptrdiff_t i = 0; i < hyph_count; i++) {
hyph_pos[i] = env->extract_integer(env, env->vec_get(env, args[6], i));
}
}
}
int32_t hyph_width = env->extract_integer(env, args[7]);
int32_t line_width = env->extract_integer(env, args[8]);
/* Call the pure DP function */
ekp_result_t *result = ekp_break_with_prefixes(
ideal_prefix, min_prefix, max_prefix,
glue_ideals, glue_shrinks, glue_stretches,
n,
hyph_pos, hyph_count > 0 ? (size_t)hyph_count : 0,
hyph_width, line_width);
free(ideal_prefix); free(min_prefix); free(max_prefix);
free(glue_ideals); free(glue_shrinks); free(glue_stretches);
free(hyph_pos);
if (!result)
return env->intern(env, "nil");
/* Build result: ((breaks...) . cost) */
emacs_value breaks_list = env->intern(env, "nil");
emacs_value cons_sym = env->intern(env, "cons");
for (size_t i = result->break_count; i > 0; i--) {
emacs_value brk = env->make_integer(env, result->breaks[i - 1]);
emacs_value args2[2] = {brk, breaks_list};
breaks_list = env->funcall(env, cons_sym, 2, args2);
}
emacs_value cost = env->make_float(env, result->total_cost);
emacs_value args2[2] = {breaks_list, cost};
emacs_value final = env->funcall(env, cons_sym, 2, args2);
ekp_result_destroy(result);
return final;
}
/*
* Helper to define functions
*/
static void defun(emacs_env *env, const char *name,
ptrdiff_t min_arity, ptrdiff_t max_arity,
emacs_value (*func)(emacs_env *, ptrdiff_t, emacs_value *, void *),
const char *doc)
{
emacs_value fn = env->make_function(env, min_arity, max_arity, func, doc, NULL);
emacs_value sym = env->intern(env, name);
emacs_value args[2] = {sym, fn};
env->funcall(env, env->intern(env, "fset"), 2, args);
}
/*
* Module entry point - required by Emacs dynamic module spec
*/
int emacs_module_init(struct emacs_runtime *runtime)
{
if (runtime->size < sizeof(*runtime))
return 1;
emacs_env *env = runtime->get_environment(runtime);
if (env->size < sizeof(*env))
return 2;
/* Define functions */
defun(env, "ekp-c-init", 0, 0, Fekp_c_init,
"Initialize EKP C module with thread pool.");
defun(env, "ekp-c-cleanup", 0, 0, Fekp_c_cleanup,
"Cleanup EKP C module resources.");
defun(env, "ekp-c-load-hyphenator", 1, 1, Fekp_c_load_hyphenator,
"Load hyphenation dictionary from PATH.\n\
Returns hyphenator index or nil on failure.\n\n(fn PATH)");
defun(env, "ekp-c-set-spacing", 9, 9, Fekp_c_set_spacing,
"Set spacing parameters (in pixels).\n\n\
Arguments are: LWS-IDEAL LWS-STRETCH LWS-SHRINK\n\
MWS-IDEAL MWS-STRETCH MWS-SHRINK\n\
CWS-IDEAL CWS-STRETCH CWS-SHRINK\n\n\
LWS = Latin Word Space, MWS = Mixed, CWS = CJK.\n\n\
(fn LWS-I LWS-+ LWS-- MWS-I MWS-+ MWS-- CWS-I CWS-+ CWS--)");
defun(env, "ekp-c-set-penalties", 4, 4, Fekp_c_set_penalties,
"Set Knuth-Plass algorithm penalties.\n\n\
LINE-PENALTY: base penalty per line break (default 10)\n\
HYPHEN-PENALTY: penalty for hyphenated breaks (default 50)\n\
FITNESS-PENALTY: penalty for adjacent line tightness mismatch (default 100)\n\
LAST-LINE-RATIO: minimum fill ratio for last line (default 0.5)\n\n\
(fn LINE-PENALTY HYPHEN-PENALTY FITNESS-PENALTY LAST-LINE-RATIO)");
defun(env, "ekp-c-hyphenate", 2, 2, Fekp_c_hyphenate,
"Get hyphenation positions for WORD using HYPHENATOR-INDEX.\n\
Returns list of positions where word can be hyphenated.\n\n(fn HYPHENATOR-INDEX WORD)");
defun(env, "ekp-c-break-lines", 4, 4, Fekp_c_break_lines,
"Break STRING into lines of LINE-WIDTH pixels.\n\n\
Uses Knuth-Plass optimal line breaking with hyphenation.\n\
HYPHENATOR-INDEX: index from `ekp-c-load-hyphenator', or -1 for none\n\
MEASURE-FUNC: function that takes a string and returns pixel width\n\n\
Returns (BREAKS . TOTAL-COST) where BREAKS is list of break positions.\n\n\
(fn STRING HYPHENATOR-INDEX LINE-WIDTH MEASURE-FUNC)");
defun(env, "ekp-c-break-with-arrays", 9, 9, Fekp_c_break_with_arrays,
"Break lines using Elisp's pre-computed prefix arrays (preferred API).\n\n\
IDEAL-PREFIX: vector of ideal width prefix sums (n+1 elements)\n\
MIN-PREFIX: vector of min width prefix sums (n+1 elements)\n\
MAX-PREFIX: vector of max width prefix sums (n+1 elements)\n\
GLUE-IDEALS: vector of glue ideal widths (n elements)\n\
GLUE-SHRINKS: vector of glue shrink amounts (n elements)\n\
GLUE-STRETCHES: vector of glue stretch amounts (n elements)\n\
HYPHEN-POS: vector of hyphenable box indices (sorted)\n\
HYPHEN-WIDTH: pixel width of hyphen character\n\
LINE-WIDTH: target line width in pixels\n\n\
Returns (BREAKS . TOTAL-COST) where BREAKS is list of box indices.\n\
This API ensures C uses Elisp's font-dependent measurements.\n\n\
(fn IDEAL-PREFIX MIN-PREFIX MAX-PREFIX GLUE-IDEALS GLUE-SHRINKS GLUE-STRETCHES HYPHEN-POS HYPHEN-WIDTH LINE-WIDTH)");
defun(env, "ekp-c-version", 0, 0, Fekp_c_version,
"Return EKP C module version string.");
defun(env, "ekp-c-thread-count", 0, 0, Fekp_c_thread_count,
"Return number of worker threads in the thread pool.");
/* Provide feature */
emacs_value provide_args[1] = {env->intern(env, "ekp-c")};
env->funcall(env, env->intern(env, "provide"), 1, provide_args);
return 0;
}

307
ekp_c/ekp_hyphen.c Normal file
View File

@ -0,0 +1,307 @@
/*
* ekp_hyphen.c - Liang hyphenation algorithm implementation
*
* Fast, thread-safe hyphenation with pattern caching.
* Uses FNV-1a hash for O(1) pattern lookup.
*/
#include "ekp_module.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
/* FNV-1a hash constants */
#define FNV_OFFSET 14695981039346656037ULL
#define FNV_PRIME 1099511628211ULL
static uint64_t fnv1a_hash(const char *data, size_t len)
{
uint64_t hash = FNV_OFFSET;
for (size_t i = 0; i < len; i++) {
hash ^= (uint8_t)data[i];
hash *= FNV_PRIME;
}
return hash;
}
/*
* Parse a pattern like "hy3ph" into letters and values.
* Returns true on success.
*/
static bool parse_pattern(const char *pat, ekp_pattern_t *out)
{
size_t pat_len = strlen(pat);
if (pat_len == 0 || pat_len >= EKP_MAX_PATTERN_LEN * 2)
return false;
size_t pos = 0;
size_t letter_idx = 0;
size_t value_idx = 0;
memset(out->values, 0, sizeof(out->values));
memset(out->letters, 0, sizeof(out->letters));
while (pos < pat_len) {
/* Read optional digit */
uint8_t digit = 0;
if (isdigit((unsigned char)pat[pos])) {
digit = pat[pos] - '0';
pos++;
}
out->values[value_idx++] = digit;
/* Read letter if present */
if (pos < pat_len && !isdigit((unsigned char)pat[pos])) {
out->letters[letter_idx++] = pat[pos];
pos++;
}
}
out->len = letter_idx;
/* Find non-zero range */
size_t start = 0, end = value_idx;
while (start < end && out->values[start] == 0) start++;
while (end > start && out->values[end - 1] == 0) end--;
out->offset = start;
/* Shift values to start */
if (start > 0) {
memmove(out->values, out->values + start, end - start);
memset(out->values + (end - start), 0, start);
}
return letter_idx > 0;
}
/*
* Load patterns from .dic file
*/
ekp_hyphenator_t *ekp_hyphen_create(const char *dict_path)
{
FILE *fp = fopen(dict_path, "r");
if (!fp)
return NULL;
ekp_hyphenator_t *h = calloc(1, sizeof(*h));
if (!h) {
fclose(fp);
return NULL;
}
pthread_rwlock_init(&h->lock, NULL);
h->left_min = 2;
h->right_min = 2;
/* First pass: count patterns */
char line[256];
size_t count = 0;
fgets(line, sizeof(line), fp); /* skip encoding line */
while (fgets(line, sizeof(line), fp)) {
size_t len = strlen(line);
if (len > 0 && line[len - 1] == '\n')
line[--len] = '\0';
/* Skip empty, comments, HYPHENMIN, patterns with / */
if (len == 0 || line[0] == '%' || line[0] == '#')
continue;
if (strstr(line, "HYPHENMIN") || strchr(line, '/'))
continue;
count++;
}
/* Allocate patterns */
h->patterns = calloc(count, sizeof(ekp_pattern_t));
h->hash_size = count * 2; /* load factor 0.5 */
h->hash_table = calloc(h->hash_size, sizeof(uint32_t));
if (!h->patterns || !h->hash_table) {
ekp_hyphen_destroy(h);
fclose(fp);
return NULL;
}
/* Second pass: parse patterns */
rewind(fp);
fgets(line, sizeof(line), fp); /* skip encoding line */
size_t idx = 0;
while (fgets(line, sizeof(line), fp)) {
size_t len = strlen(line);
if (len > 0 && line[len - 1] == '\n')
line[--len] = '\0';
if (len == 0 || line[0] == '%' || line[0] == '#')
continue;
if (strstr(line, "HYPHENMIN") || strchr(line, '/'))
continue;
/* Handle ^^XX hex escapes */
char decoded[256];
char *dst = decoded;
const char *src = line;
while (*src) {
if (src[0] == '^' && src[1] == '^' &&
isxdigit((unsigned char)src[2]) &&
isxdigit((unsigned char)src[3])) {
char hex[3] = {src[2], src[3], 0};
*dst++ = (char)strtol(hex, NULL, 16);
src += 4;
} else {
*dst++ = *src++;
}
}
*dst = '\0';
if (parse_pattern(decoded, &h->patterns[idx])) {
/* Insert into hash table */
uint64_t hash = fnv1a_hash(h->patterns[idx].letters,
h->patterns[idx].len);
size_t slot = hash % h->hash_size;
while (h->hash_table[slot] != 0) {
slot = (slot + 1) % h->hash_size;
}
h->hash_table[slot] = idx + 1; /* 1-indexed */
if (h->patterns[idx].len > h->max_pattern_len)
h->max_pattern_len = h->patterns[idx].len;
idx++;
}
}
h->pattern_count = idx;
fclose(fp);
return h;
}
void ekp_hyphen_destroy(ekp_hyphenator_t *h)
{
if (!h) return;
pthread_rwlock_destroy(&h->lock);
free(h->patterns);
free(h->hash_table);
free(h);
}
/*
* Find pattern by letters (hash table lookup)
*/
static ekp_pattern_t *find_pattern(ekp_hyphenator_t *h,
const char *letters, size_t len)
{
if (len == 0 || len > h->max_pattern_len)
return NULL;
uint64_t hash = fnv1a_hash(letters, len);
size_t slot = hash % h->hash_size;
for (size_t i = 0; i < h->hash_size; i++) {
uint32_t idx = h->hash_table[slot];
if (idx == 0)
return NULL;
ekp_pattern_t *p = &h->patterns[idx - 1];
if (p->len == len && memcmp(p->letters, letters, len) == 0)
return p;
slot = (slot + 1) % h->hash_size;
}
return NULL;
}
/*
* Compute hyphenation positions for a word
* Thread-safe (read lock)
*/
int ekp_hyphen_word(ekp_hyphenator_t *h, const char *word, size_t len,
int8_t *positions, size_t max_pos)
{
if (!h || !word || len == 0 || len > EKP_MAX_WORD_LEN - 2)
return 0;
/* Check cache first */
uint64_t word_hash = fnv1a_hash(word, len);
size_t cache_slot = word_hash % EKP_CACHE_SIZE;
pthread_rwlock_rdlock(&h->lock);
if (h->cache[cache_slot].hash == word_hash &&
strncmp(h->cache[cache_slot].word, word, len) == 0) {
int count = h->cache[cache_slot].pos_count;
if (count <= (int)max_pos) {
memcpy(positions, h->cache[cache_slot].positions,
count * sizeof(int8_t));
}
pthread_rwlock_unlock(&h->lock);
return count;
}
pthread_rwlock_unlock(&h->lock);
/* Compute hyphenation */
char padded[EKP_MAX_WORD_LEN + 2];
padded[0] = '.';
for (size_t i = 0; i < len; i++)
padded[i + 1] = tolower((unsigned char)word[i]);
padded[len + 1] = '.';
size_t padded_len = len + 2;
uint8_t prio[EKP_MAX_WORD_LEN + 3];
memset(prio, 0, sizeof(prio));
/* Apply matching patterns */
pthread_rwlock_rdlock(&h->lock);
for (size_t i = 0; i < padded_len - 1; i++) {
for (size_t j = i + 1; j <= padded_len && j <= i + h->max_pattern_len; j++) {
ekp_pattern_t *pat = find_pattern(h, padded + i, j - i);
if (pat) {
size_t val_len = pat->len + 1 - pat->offset;
for (size_t k = 0; k < val_len && k < sizeof(pat->values); k++) {
size_t pos = i + pat->offset + k;
if (pos < sizeof(prio) && pat->values[k] > prio[pos])
prio[pos] = pat->values[k];
}
}
}
}
pthread_rwlock_unlock(&h->lock);
/* Collect odd positions (subtract 1 for padding offset) */
int8_t result[EKP_MAX_WORD_LEN];
int count = 0;
for (size_t i = 1; i < padded_len && count < EKP_MAX_WORD_LEN; i++) {
if (prio[i] & 1) { /* odd = break allowed */
int pos = (int)i - 1; /* adjust for leading '.' */
/* Apply margin constraints */
if (pos >= h->left_min && pos <= (int)len - h->right_min) {
result[count++] = pos;
}
}
}
/* Update cache */
pthread_rwlock_wrlock(&h->lock);
h->cache[cache_slot].hash = word_hash;
strncpy(h->cache[cache_slot].word, word, len);
h->cache[cache_slot].word[len] = '\0';
memcpy(h->cache[cache_slot].positions, result, count * sizeof(int8_t));
h->cache[cache_slot].pos_count = count;
pthread_rwlock_unlock(&h->lock);
/* Copy to output */
int out_count = count < (int)max_pos ? count : (int)max_pos;
memcpy(positions, result, out_count * sizeof(int8_t));
return out_count;
}

723
ekp_c/ekp_kp.c Normal file
View File

@ -0,0 +1,723 @@
/*
* ekp.c - Knuth-Plass line breaking algorithm
*
* The heart of the system. O(n²) worst case, but with pruning and
* parallel candidate evaluation, typically O(n·m) where m is avg line length.
*
* Key optimizations:
* - Prefix sums for O(1) range queries
* - Early termination when line too long
* - Parallel demerits computation for large paragraphs
*/
#include "ekp_module.h"
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <float.h>
/* Global state */
ekp_state_t *ekp_global = NULL;
/* Fitness classes */
#define FITNESS_TIGHT 0
#define FITNESS_DECENT 1
#define FITNESS_LOOSE 2
#define FITNESS_VERY_LOOSE 3
/* Badness computation */
static inline double compute_badness(int32_t adjustment, int32_t flexibility)
{
if (adjustment == 0)
return 0.0;
if (flexibility <= 0)
return EKP_INFINITY;
double ratio = (double)adjustment / flexibility;
double badness = 100.0 * fabs(ratio * ratio * ratio);
return badness > 10000.0 ? EKP_INFINITY : badness;
}
/* Fitness classification */
static inline uint8_t compute_fitness(int32_t adjustment, int32_t flexibility)
{
if (flexibility <= 0)
return FITNESS_DECENT;
double ratio = (double)adjustment / flexibility;
if (ratio < -0.5)
return FITNESS_TIGHT;
if (ratio < 0.5)
return FITNESS_DECENT;
if (ratio < 1.0)
return FITNESS_LOOSE;
return FITNESS_VERY_LOOSE;
}
/* Full demerits computation */
static inline double compute_demerits(double badness, int32_t penalty,
uint8_t prev_fitness, uint8_t curr_fitness,
bool end_hyphen, int prev_hyphen_count,
int line_penalty, int fitness_penalty)
{
/* Base: (line_penalty + badness)² */
double base = (line_penalty + badness);
base = base * base;
/* Add break penalty squared */
base += (double)penalty * penalty;
/* Fitness incompatibility */
int delta = abs((int)prev_fitness - (int)curr_fitness);
if (delta > 1)
base += fitness_penalty;
/* Consecutive hyphen penalty (quadratic growth) */
if (end_hyphen) {
int count = prev_hyphen_count + 1;
base += 100.0 * count * count;
}
return base;
}
/*
* Check if position is a hyphenation break
*/
static inline bool is_hyphen_break(ekp_paragraph_t *p, size_t pos)
{
for (size_t i = 0; i < p->hyphen_count; i++) {
if ((size_t)p->hyphen_positions[i] == pos)
return true;
}
return false;
}
/*
* Parallel work item for demerits computation
*/
typedef struct {
ekp_paragraph_t *para;
int32_t line_width;
size_t start;
size_t end;
/* Output arrays (pre-allocated) */
double *demerits;
int32_t *backptrs;
int32_t *rest_pixels;
uint8_t *fitness;
int32_t *hyphen_counts;
int32_t *line_counts;
/* Shared read-only input */
const double *prev_demerits;
const uint8_t *prev_fitness;
const int32_t *prev_hyphen_counts;
const int32_t *prev_line_counts;
/* Parameters */
int line_penalty;
int hyphen_penalty;
int fitness_penalty;
double last_line_ratio;
} dp_work_t;
/*
* Process a range of candidate breakpoints (for parallel execution)
*/
static void process_dp_range(void *arg)
{
dp_work_t *work = (dp_work_t *)arg;
ekp_paragraph_t *p = work->para;
int32_t line_width = work->line_width;
size_t n = p->box_count;
for (size_t i = work->start; i < work->end; i++) {
if (work->prev_demerits[i] >= EKP_INFINITY)
continue;
double prev_dem = work->prev_demerits[i];
uint8_t prev_fit = work->prev_fitness[i];
int prev_hyph = work->prev_hyphen_counts[i];
int prev_lines = work->prev_line_counts[i];
/* Get leading glue for line starting at i */
int32_t leading_glue_ideal = (i < n) ? p->glues[i].ideal : 0;
int32_t leading_glue_stretch = (i < n) ? p->glues[i].stretch : 0;
int32_t leading_glue_shrink = (i < n) ? p->glues[i].shrink : 0;
/* Try extending to each position k > i */
for (size_t k = i + 1; k <= n; k++) {
bool is_last = (k == n);
bool end_hyphen = (k > 0) && is_hyphen_break(p, k - 1);
/* Line metrics from i to k (excluding leading glue) */
int32_t ideal = p->ideal_prefix[k] - p->ideal_prefix[i] - leading_glue_ideal;
int32_t min_w = p->min_prefix[k] - p->min_prefix[i] -
(leading_glue_ideal - leading_glue_shrink);
int32_t max_w = p->max_prefix[k] - p->max_prefix[i] -
(leading_glue_ideal + leading_glue_stretch) +
(leading_glue_ideal + leading_glue_stretch);
/* Add hyphen width if needed */
if (end_hyphen) {
ideal += p->hyphen_width;
min_w += p->hyphen_width;
max_w += p->hyphen_width;
}
/* Too long? */
if (min_w > line_width) {
/* Force break if nothing else found */
if (k > 1 && work->demerits[k - 1] >= EKP_INFINITY) {
int32_t rest = line_width - (p->ideal_prefix[k - 1] -
p->ideal_prefix[i] - leading_glue_ideal);
work->demerits[k - 1] = prev_dem + 10000.0 + rest * rest;
work->backptrs[k - 1] = i;
work->rest_pixels[k - 1] = rest;
work->fitness[k - 1] = FITNESS_VERY_LOOSE;
work->hyphen_counts[k - 1] = 0;
work->line_counts[k - 1] = prev_lines + 1;
}
break; /* No point trying longer lines */
}
/* Valid break? */
bool valid = (min_w <= line_width && max_w >= line_width) ||
(is_last && ideal <= line_width);
if (!valid)
continue;
/* Compute demerits */
int32_t adjustment = line_width - ideal;
int32_t flexibility = (adjustment > 0) ?
(max_w - ideal) : (ideal - min_w);
double badness;
uint8_t fit;
double dem;
if (is_last) {
/* Last line: minimal penalty if reasonably filled */
double fill_ratio = (double)ideal / line_width;
if (fill_ratio < work->last_line_ratio) {
badness = 50.0 * (1.0 - fill_ratio);
} else {
badness = 0.0;
}
fit = FITNESS_DECENT;
dem = prev_dem + (work->line_penalty + badness) *
(work->line_penalty + badness);
} else {
badness = compute_badness(adjustment, flexibility);
fit = compute_fitness(adjustment, flexibility);
int penalty = end_hyphen ? work->hyphen_penalty : 0;
dem = prev_dem + compute_demerits(badness, penalty,
prev_fit, fit,
end_hyphen, prev_hyph,
work->line_penalty,
work->fitness_penalty);
}
/* Update if better */
if (dem < work->demerits[k]) {
work->demerits[k] = dem;
work->backptrs[k] = i;
work->rest_pixels[k] = adjustment;
work->fitness[k] = fit;
work->hyphen_counts[k] = end_hyphen ? prev_hyph + 1 : 0;
work->line_counts[k] = prev_lines + 1;
}
}
}
}
/*
* Main line breaking function
*/
ekp_result_t *ekp_break_lines(ekp_paragraph_t *p, int32_t line_width)
{
if (!p || p->box_count == 0 || line_width <= 0)
return NULL;
size_t n = p->box_count;
/* Allocate DP arrays */
double *demerits = malloc((n + 1) * sizeof(double));
int32_t *backptrs = malloc((n + 1) * sizeof(int32_t));
int32_t *rest_pixels = malloc((n + 1) * sizeof(int32_t));
uint8_t *fitness = malloc((n + 1) * sizeof(uint8_t));
int32_t *hyphen_counts = malloc((n + 1) * sizeof(int32_t));
int32_t *line_counts = malloc((n + 1) * sizeof(int32_t));
if (!demerits || !backptrs || !rest_pixels ||
!fitness || !hyphen_counts || !line_counts) {
free(demerits);
free(backptrs);
free(rest_pixels);
free(fitness);
free(hyphen_counts);
free(line_counts);
return NULL;
}
/* Initialize */
for (size_t i = 0; i <= n; i++) {
demerits[i] = EKP_INFINITY;
backptrs[i] = -1;
rest_pixels[i] = 0;
fitness[i] = FITNESS_DECENT;
hyphen_counts[i] = 0;
line_counts[i] = 0;
}
demerits[0] = 0.0;
/* Get parameters */
int line_penalty = ekp_global ? ekp_global->line_penalty : 10;
int hyphen_penalty = ekp_global ? ekp_global->hyphen_penalty : 50;
int fitness_penalty = ekp_global ? ekp_global->fitness_penalty : 100;
double last_ratio = ekp_global ? ekp_global->last_line_ratio : 0.5;
/* For small paragraphs, single-threaded */
if (n < 100 || !ekp_global || !ekp_global->pool) {
dp_work_t work = {
.para = p,
.line_width = line_width,
.start = 0,
.end = n,
.demerits = demerits,
.backptrs = backptrs,
.rest_pixels = rest_pixels,
.fitness = fitness,
.hyphen_counts = hyphen_counts,
.line_counts = line_counts,
.prev_demerits = demerits,
.prev_fitness = fitness,
.prev_hyphen_counts = hyphen_counts,
.prev_line_counts = line_counts,
.line_penalty = line_penalty,
.hyphen_penalty = hyphen_penalty,
.fitness_penalty = fitness_penalty,
.last_line_ratio = last_ratio,
};
/* Simple iterative DP */
for (size_t i = 0; i < n; i++) {
if (demerits[i] >= EKP_INFINITY)
continue;
work.start = i;
work.end = i + 1;
process_dp_range(&work);
}
} else {
/* Parallel processing for large paragraphs */
/* Split work across threads (wavefront approach) */
size_t chunk_size = n / EKP_THREAD_POOL_SIZE;
if (chunk_size < 10)
chunk_size = 10;
dp_work_t *works = malloc(EKP_THREAD_POOL_SIZE * sizeof(dp_work_t));
if (!works) {
/* Fall back to single-threaded */
for (size_t i = 0; i < n; i++) {
if (demerits[i] >= EKP_INFINITY)
continue;
dp_work_t work = {
.para = p,
.line_width = line_width,
.start = i,
.end = i + 1,
.demerits = demerits,
.backptrs = backptrs,
.rest_pixels = rest_pixels,
.fitness = fitness,
.hyphen_counts = hyphen_counts,
.line_counts = line_counts,
.prev_demerits = demerits,
.prev_fitness = fitness,
.prev_hyphen_counts = hyphen_counts,
.prev_line_counts = line_counts,
.line_penalty = line_penalty,
.hyphen_penalty = hyphen_penalty,
.fitness_penalty = fitness_penalty,
.last_line_ratio = last_ratio,
};
process_dp_range(&work);
}
} else {
/* Wavefront: process in chunks */
for (size_t wave = 0; wave < n; wave += chunk_size) {
size_t wave_end = wave + chunk_size;
if (wave_end > n)
wave_end = n;
size_t work_count = 0;
for (size_t i = wave; i < wave_end; i++) {
if (demerits[i] >= EKP_INFINITY)
continue;
works[work_count] = (dp_work_t){
.para = p,
.line_width = line_width,
.start = i,
.end = i + 1,
.demerits = demerits,
.backptrs = backptrs,
.rest_pixels = rest_pixels,
.fitness = fitness,
.hyphen_counts = hyphen_counts,
.line_counts = line_counts,
.prev_demerits = demerits,
.prev_fitness = fitness,
.prev_hyphen_counts = hyphen_counts,
.prev_line_counts = line_counts,
.line_penalty = line_penalty,
.hyphen_penalty = hyphen_penalty,
.fitness_penalty = fitness_penalty,
.last_line_ratio = last_ratio,
};
ekp_pool_submit(ekp_global->pool, process_dp_range,
&works[work_count]);
work_count++;
}
ekp_pool_wait(ekp_global->pool);
}
free(works);
}
}
/* Trace back optimal path */
ekp_result_t *result = calloc(1, sizeof(*result));
if (!result) {
free(demerits);
free(backptrs);
free(rest_pixels);
free(fitness);
free(hyphen_counts);
free(line_counts);
return NULL;
}
/* Count breaks */
size_t break_count = 0;
int32_t idx = n;
while (idx > 0) {
break_count++;
idx = backptrs[idx];
if (idx < 0)
break;
}
result->breaks = malloc(break_count * sizeof(int32_t));
result->rest_pixels = malloc(break_count * sizeof(int32_t));
if (!result->breaks || !result->rest_pixels) {
ekp_result_destroy(result);
free(demerits);
free(backptrs);
free(rest_pixels);
free(fitness);
free(hyphen_counts);
free(line_counts);
return NULL;
}
result->break_count = break_count;
result->total_cost = demerits[n];
/* Fill in reverse order */
idx = n;
for (size_t i = break_count; i > 0; i--) {
result->breaks[i - 1] = idx;
result->rest_pixels[i - 1] = rest_pixels[idx];
idx = backptrs[idx];
}
free(demerits);
free(backptrs);
free(rest_pixels);
free(fitness);
free(hyphen_counts);
free(line_counts);
return result;
}
void ekp_result_destroy(ekp_result_t *r)
{
if (!r)
return;
free(r->breaks);
free(r->rest_pixels);
free(r);
}
/*
* Pure DP with pre-computed prefix arrays (for Elisp integration)
*
* This is the preferred API when Elisp has already computed everything.
* Elisp does: tokenization, width measurement, glue computation, prefix sums.
* C module only does: O(n²) DP computation.
*
* All font-dependent calculations happen in Elisp. C module is pure algorithm.
*/
static inline bool is_hyphen_pos(const int32_t *positions, size_t count, int32_t pos)
{
for (size_t i = 0; i < count; i++) {
if (positions[i] == pos)
return true;
if (positions[i] > pos)
return false;
}
return false;
}
ekp_result_t *ekp_break_with_prefixes(
const int32_t *ideal_prefix,
const int32_t *min_prefix,
const int32_t *max_prefix,
const int32_t *glue_ideals,
const int32_t *glue_shrinks,
const int32_t *glue_stretches,
size_t n,
const int32_t *hyphen_positions,
size_t hyphen_count,
int32_t hyphen_width,
int32_t line_width)
{
if (!ideal_prefix || !min_prefix || !max_prefix || n == 0 || line_width <= 0)
return NULL;
/* Allocate DP arrays */
double *demerits = malloc((n + 1) * sizeof(double));
int32_t *backptrs = malloc((n + 1) * sizeof(int32_t));
int32_t *rest_pixels = malloc((n + 1) * sizeof(int32_t));
uint8_t *fitness = malloc((n + 1) * sizeof(uint8_t));
int32_t *hyph_counts = malloc((n + 1) * sizeof(int32_t));
int32_t *line_counts = malloc((n + 1) * sizeof(int32_t));
if (!demerits || !backptrs || !rest_pixels || !fitness || !hyph_counts || !line_counts) {
free(demerits); free(backptrs); free(rest_pixels);
free(fitness); free(hyph_counts); free(line_counts);
return NULL;
}
/* Initialize */
for (size_t i = 0; i <= n; i++) {
demerits[i] = EKP_INFINITY;
backptrs[i] = -1;
rest_pixels[i] = 0;
fitness[i] = FITNESS_DECENT;
hyph_counts[i] = 0;
line_counts[i] = 0;
}
demerits[0] = 0.0;
/* Get K-P parameters */
int lp = ekp_global ? ekp_global->line_penalty : 10;
int hp = ekp_global ? ekp_global->hyphen_penalty : 50;
int fp = ekp_global ? ekp_global->fitness_penalty : 100;
double last_ratio = ekp_global ? ekp_global->last_line_ratio : 0.5;
/* DP: for each valid start, try all ends */
for (size_t i = 0; i < n; i++) {
if (demerits[i] >= EKP_INFINITY)
continue;
/* Leading glue for line starting at i */
int32_t lead_ideal = glue_ideals ? glue_ideals[i] : 0;
int32_t lead_shrink = glue_shrinks ? glue_shrinks[i] : 0;
int32_t lead_stretch = glue_stretches ? glue_stretches[i] : 0;
for (size_t k = i + 1; k <= n; k++) {
bool is_last = (k == n);
bool end_hyph = hyphen_positions && is_hyphen_pos(hyphen_positions, hyphen_count, k - 1);
/* Line width from i to k (exclude leading glue) */
int32_t ideal = ideal_prefix[k] - ideal_prefix[i] - lead_ideal;
int32_t min_w = min_prefix[k] - min_prefix[i] - (lead_ideal - lead_shrink);
int32_t max_w = max_prefix[k] - max_prefix[i] - (lead_ideal - lead_stretch);
if (end_hyph) {
ideal += hyphen_width;
min_w += hyphen_width;
max_w += hyphen_width;
}
/* Too long? Force break at k-1 if no valid break found yet */
if (min_w > line_width) {
if (k > i + 1 && demerits[k - 1] >= EKP_INFINITY) {
/* Force break at previous position with high penalty */
int32_t prev_ideal = ideal_prefix[k - 1] - ideal_prefix[i] - lead_ideal;
int32_t rest = line_width - prev_ideal;
double forced_dem = demerits[i] + 10000.0 + (double)rest * rest;
demerits[k - 1] = forced_dem;
backptrs[k - 1] = i;
rest_pixels[k - 1] = rest;
fitness[k - 1] = FITNESS_VERY_LOOSE;
hyph_counts[k - 1] = 0;
line_counts[k - 1] = line_counts[i] + 1;
}
break;
}
/* Valid break? */
bool valid = (min_w <= line_width && max_w >= line_width) ||
(is_last && ideal <= line_width);
if (!valid)
continue;
/* Compute demerits */
int32_t adj = line_width - ideal;
int32_t flex = (adj > 0) ? (max_w - ideal) : (ideal - min_w);
double bad;
uint8_t fit;
double dem;
if (is_last) {
double fill = (double)ideal / line_width;
bad = (fill < last_ratio) ? 50.0 * (1.0 - fill) : 0.0;
fit = FITNESS_DECENT;
dem = demerits[i] + (lp + bad) * (lp + bad);
} else {
bad = compute_badness(adj, flex);
fit = compute_fitness(adj, flex);
int pen = end_hyph ? hp : 0;
dem = demerits[i] + compute_demerits(bad, pen, fitness[i], fit,
end_hyph, hyph_counts[i], lp, fp);
}
if (dem < demerits[k]) {
demerits[k] = dem;
backptrs[k] = i;
rest_pixels[k] = adj;
fitness[k] = fit;
hyph_counts[k] = end_hyph ? hyph_counts[i] + 1 : 0;
line_counts[k] = line_counts[i] + 1;
}
}
}
/* If no valid path found to end, return NULL to fallback to Elisp */
if (demerits[n] >= EKP_INFINITY) {
free(demerits); free(backptrs); free(rest_pixels);
free(fitness); free(hyph_counts); free(line_counts);
return NULL;
}
/* Build result */
ekp_result_t *result = calloc(1, sizeof(*result));
if (!result) {
free(demerits); free(backptrs); free(rest_pixels);
free(fitness); free(hyph_counts); free(line_counts);
return NULL;
}
/* Count breaks */
size_t break_count = 0;
int32_t idx = n;
while (idx > 0) {
break_count++;
idx = backptrs[idx];
if (idx < 0) break;
}
result->breaks = malloc(break_count * sizeof(int32_t));
result->rest_pixels = malloc(break_count * sizeof(int32_t));
if (!result->breaks || !result->rest_pixels) {
ekp_result_destroy(result);
free(demerits); free(backptrs); free(rest_pixels);
free(fitness); free(hyph_counts); free(line_counts);
return NULL;
}
result->break_count = break_count;
result->total_cost = demerits[n];
/* Fill in reverse order */
idx = n;
for (size_t i = break_count; i > 0; i--) {
result->breaks[i - 1] = idx;
result->rest_pixels[i - 1] = rest_pixels[idx];
idx = backptrs[idx];
}
free(demerits); free(backptrs); free(rest_pixels);
free(fitness); free(hyph_counts); free(line_counts);
return result;
}
/*
* Initialization and cleanup
*/
int ekp_init(void)
{
if (ekp_global)
return 0;
ekp_global = calloc(1, sizeof(*ekp_global));
if (!ekp_global)
return -1;
/* Default spacing */
ekp_global->spacing.lws_ideal = 7;
ekp_global->spacing.lws_stretch = 3;
ekp_global->spacing.lws_shrink = 2;
ekp_global->spacing.mws_ideal = 5;
ekp_global->spacing.mws_stretch = 2;
ekp_global->spacing.mws_shrink = 1;
ekp_global->spacing.cws_ideal = 0;
ekp_global->spacing.cws_stretch = 2;
ekp_global->spacing.cws_shrink = 0;
/* Default K-P parameters */
ekp_global->line_penalty = 10;
ekp_global->hyphen_penalty = 50;
ekp_global->fitness_penalty = 100;
ekp_global->last_line_ratio = 0.5;
/* Create thread pool */
ekp_global->pool = ekp_pool_create(EKP_THREAD_POOL_SIZE);
if (!ekp_global->pool) {
free(ekp_global);
ekp_global = NULL;
return -1;
}
pthread_mutex_init(&ekp_global->cache_lock, NULL);
return 0;
}
void ekp_cleanup(void)
{
if (!ekp_global)
return;
/* Destroy hyphenators */
for (size_t i = 0; i < ekp_global->hyphenator_count; i++) {
ekp_hyphen_destroy(ekp_global->hyphenators[i]);
}
/* Destroy paragraph cache */
if (ekp_global->para_cache) {
for (size_t i = 0; i < ekp_global->para_cache_size; i++) {
ekp_para_destroy(ekp_global->para_cache[i]);
}
free(ekp_global->para_cache);
}
pthread_mutex_destroy(&ekp_global->cache_lock);
ekp_pool_destroy(ekp_global->pool);
free(ekp_global);
ekp_global = NULL;
}

266
ekp_c/ekp_module.h Normal file
View File

@ -0,0 +1,266 @@
/*
* ekp_module.h - Emacs Knuth-Plass dynamic module
*
* Core data structures and API declarations.
* Design principle: Data structures are the code. Get them right,
* and the rest writes itself. -- Linus Torvalds
*/
#ifndef EKP_MODULE_H
#define EKP_MODULE_H
#include <emacs-module.h>
#include <stdint.h>
#include <stdbool.h>
#include <pthread.h>
/* Version */
#define EKP_VERSION_MAJOR 1
#define EKP_VERSION_MINOR 0
/* Limits */
#define EKP_MAX_PATTERN_LEN 64
#define EKP_MAX_WORD_LEN 256
#define EKP_CACHE_SIZE 4096
#define EKP_THREAD_POOL_SIZE 8
/* Infinity for impossible breaks */
#define EKP_INFINITY 1e10
/*
* Box: indivisible content with fixed width
* Keep it small - we'll have thousands of these
*/
typedef struct {
const char *text; /* UTF-8 string, NOT owned */
int32_t text_len; /* byte length */
int32_t pixel_width; /* rendered width in pixels */
uint8_t box_type; /* 0=latin, 1=cjk, 2=cjk_punct, 3=space */
uint8_t start_type; /* first char type */
uint8_t end_type; /* last char type */
} ekp_box_t;
/*
* Glue: flexible space between boxes
* The heart of Knuth-Plass: ideal ± stretch/shrink
*/
typedef struct {
int16_t ideal; /* natural width */
int16_t stretch; /* max stretch */
int16_t shrink; /* max shrink */
uint8_t type; /* 0=none, 1=lws, 2=mws, 3=cws */
} ekp_glue_t;
/*
* Breakpoint candidate for DP
*/
typedef struct {
int32_t index; /* box index */
int32_t prev; /* previous breakpoint index */
double demerits; /* accumulated demerits */
int32_t line_count; /* lines so far */
uint8_t fitness; /* 0-3: tight to very-loose */
uint8_t hyphen_count; /* consecutive hyphens */
bool is_hyphen; /* ends with hyphen? */
} ekp_breakpoint_t;
/*
* Hyphenation pattern (Liang's algorithm)
* Compact representation: letters + priority values
*/
typedef struct {
char letters[EKP_MAX_PATTERN_LEN];
uint8_t values[EKP_MAX_PATTERN_LEN + 1];
uint8_t len;
uint8_t offset; /* where values start */
} ekp_pattern_t;
/*
* Hyphenator: compiled patterns + cache
* Thread-safe with read-write lock
*/
typedef struct {
ekp_pattern_t *patterns;
size_t pattern_count;
size_t max_pattern_len;
/* Hash table for O(1) pattern lookup */
uint32_t *hash_table;
size_t hash_size;
/* Word cache (LRU) */
struct {
uint64_t hash;
char word[EKP_MAX_WORD_LEN];
int8_t positions[EKP_MAX_WORD_LEN];
int pos_count;
} cache[EKP_CACHE_SIZE];
size_t cache_head;
pthread_rwlock_t lock;
/* Margin constraints */
int left_min;
int right_min;
} ekp_hyphenator_t;
/*
* Paragraph: preprocessed text ready for line breaking
* All arrays are parallel: boxes[i] has glues[i], widths[i], etc.
*/
typedef struct {
ekp_box_t *boxes;
ekp_glue_t *glues;
size_t box_count;
/* Prefix sums for O(1) range queries */
int32_t *ideal_prefix;
int32_t *min_prefix;
int32_t *max_prefix;
/* Hyphenation data */
int32_t *hyphen_positions;
size_t hyphen_count;
int32_t hyphen_width;
/* Original string (owned) */
char *text;
size_t text_len;
/* Hash for cache lookup */
uint64_t hash;
} ekp_paragraph_t;
/*
* Line break result
*/
typedef struct {
int32_t *breaks; /* break positions */
size_t break_count;
int32_t *rest_pixels; /* remaining space per line */
double total_cost;
} ekp_result_t;
/*
* Global spacing parameters
*/
typedef struct {
int16_t lws_ideal, lws_stretch, lws_shrink;
int16_t mws_ideal, mws_stretch, mws_shrink;
int16_t cws_ideal, cws_stretch, cws_shrink;
} ekp_spacing_t;
/*
* Thread pool for parallel computation
*/
typedef struct {
pthread_t threads[EKP_THREAD_POOL_SIZE];
pthread_mutex_t queue_lock;
pthread_cond_t queue_cond;
pthread_cond_t done_cond;
struct {
void (*func)(void *);
void *arg;
} *queue;
size_t queue_size;
size_t queue_head;
size_t queue_tail;
size_t active_count;
bool shutdown;
} ekp_thread_pool_t;
/*
* Global state
*/
typedef struct {
ekp_hyphenator_t *hyphenators[32]; /* by language */
size_t hyphenator_count;
ekp_paragraph_t **para_cache;
size_t para_cache_size;
pthread_mutex_t cache_lock;
ekp_spacing_t spacing;
ekp_thread_pool_t *pool;
/* K-P parameters */
int line_penalty;
int hyphen_penalty;
int fitness_penalty;
double last_line_ratio;
} ekp_state_t;
/* Global state instance */
extern ekp_state_t *ekp_global;
/*
* API: Hyphenation
*/
ekp_hyphenator_t *ekp_hyphen_create(const char *dict_path);
void ekp_hyphen_destroy(ekp_hyphenator_t *h);
int ekp_hyphen_word(ekp_hyphenator_t *h, const char *word, size_t len,
int8_t *positions, size_t max_pos);
/*
* API: Paragraph processing
*/
ekp_paragraph_t *ekp_para_create(const char *text, size_t len,
ekp_hyphenator_t *h,
int32_t (*measure_fn)(const char *, size_t));
void ekp_para_destroy(ekp_paragraph_t *p);
/*
* API: Line breaking (the main algorithm)
*/
ekp_result_t *ekp_break_lines(ekp_paragraph_t *p, int32_t line_width);
void ekp_result_destroy(ekp_result_t *r);
/*
* API: Pure DP with pre-computed prefix arrays (for Elisp integration)
*
* This is the preferred API when Elisp has already computed everything.
* Elisp does: tokenization, width measurement, glue computation, prefix sums.
* C module only does: O(n²) DP computation.
*
* ideal_prefix, min_prefix, max_prefix: prefix sum arrays (n+1 elements)
* glue_ideals, glue_stretches, glue_shrinks: per-box glue values (n elements)
* hyphen_positions: sorted array of hyphenable box indices
* hyphen_count: length of hyphen_positions
* hyphen_width: pixel width of hyphen character
* line_width: target line width in pixels
*/
ekp_result_t *ekp_break_with_prefixes(
const int32_t *ideal_prefix,
const int32_t *min_prefix,
const int32_t *max_prefix,
const int32_t *glue_ideals,
const int32_t *glue_shrinks,
const int32_t *glue_stretches,
size_t n,
const int32_t *hyphen_positions,
size_t hyphen_count,
int32_t hyphen_width,
int32_t line_width);
/*
* API: Thread pool
*/
ekp_thread_pool_t *ekp_pool_create(size_t num_threads);
void ekp_pool_destroy(ekp_thread_pool_t *pool);
void ekp_pool_submit(ekp_thread_pool_t *pool, void (*func)(void *), void *arg);
void ekp_pool_wait(ekp_thread_pool_t *pool);
/*
* API: Initialization
*/
int ekp_init(void);
void ekp_cleanup(void);
/*
* Emacs module interface
*/
int emacs_module_init(struct emacs_runtime *runtime);
#endif /* EKP_MODULE_H */

413
ekp_c/ekp_paragraph.c Normal file
View File

@ -0,0 +1,413 @@
/*
* ekp_paragraph.c - Text preprocessing and box/glue construction
*
* The boring part that makes everything else fast.
* Get the data layout right, and the algorithm sings.
*/
#include "ekp_module.h"
#include <stdlib.h>
#include <string.h>
/* Box types */
#define BOX_LATIN 0
#define BOX_CJK 1
#define BOX_CJK_PUNCT 2
#define BOX_SPACE 3
/* Glue types */
#define GLUE_NONE 0
#define GLUE_LWS 1 /* Latin word space */
#define GLUE_MWS 2 /* Mixed (Latin-CJK) */
#define GLUE_CWS 3 /* CJK character space */
/* UTF-8 helpers */
static inline int utf8_char_len(unsigned char c)
{
if ((c & 0x80) == 0) return 1;
if ((c & 0xE0) == 0xC0) return 2;
if ((c & 0xF0) == 0xE0) return 3;
if ((c & 0xF8) == 0xF0) return 4;
return 1; /* invalid, treat as single byte */
}
static inline uint32_t utf8_decode(const char *s, int *len)
{
unsigned char c = s[0];
*len = 1;
if ((c & 0x80) == 0)
return c;
if ((c & 0xE0) == 0xC0) {
*len = 2;
return ((c & 0x1F) << 6) | (s[1] & 0x3F);
}
if ((c & 0xF0) == 0xE0) {
*len = 3;
return ((c & 0x0F) << 12) | ((s[1] & 0x3F) << 6) | (s[2] & 0x3F);
}
if ((c & 0xF8) == 0xF0) {
*len = 4;
return ((c & 0x07) << 18) | ((s[1] & 0x3F) << 12) |
((s[2] & 0x3F) << 6) | (s[3] & 0x3F);
}
return c;
}
/* Character classification */
static inline bool is_cjk(uint32_t cp)
{
/* CJK Unified Ideographs and related blocks */
return (cp >= 0x4E00 && cp <= 0x9FFF) || /* CJK Unified */
(cp >= 0x3400 && cp <= 0x4DBF) || /* CJK Ext A */
(cp >= 0x20000 && cp <= 0x2A6DF) || /* CJK Ext B */
(cp >= 0x2A700 && cp <= 0x2B73F) || /* CJK Ext C */
(cp >= 0x2B740 && cp <= 0x2B81F) || /* CJK Ext D */
(cp >= 0xF900 && cp <= 0xFAFF) || /* CJK Compat */
(cp >= 0x3000 && cp <= 0x303F) || /* CJK Symbols */
(cp >= 0x3040 && cp <= 0x309F) || /* Hiragana */
(cp >= 0x30A0 && cp <= 0x30FF) || /* Katakana */
(cp >= 0xAC00 && cp <= 0xD7AF); /* Hangul */
}
static inline bool is_cjk_punct(uint32_t cp)
{
return (cp >= 0x3000 && cp <= 0x303F) || /* CJK Symbols */
(cp >= 0xFF00 && cp <= 0xFF60) || /* Fullwidth Forms */
cp == 0x201C || cp == 0x201D || /* " " */
cp == 0x2018 || cp == 0x2019; /* ' ' */
}
static inline bool is_latin(uint32_t cp)
{
return (cp >= 'A' && cp <= 'Z') ||
(cp >= 'a' && cp <= 'z') ||
(cp >= 0xC0 && cp <= 0xFF) || /* Latin-1 Supplement */
(cp >= 0x100 && cp <= 0x24F) || /* Latin Extended */
(cp >= 0x1E00 && cp <= 0x1EFF); /* Latin Extended Additional */
}
static inline bool is_whitespace(uint32_t cp)
{
return cp == ' ' || cp == '\t' || cp == '\n' || cp == '\r' ||
cp == 0x00A0 || cp == 0x3000; /* NBSP, ideographic space */
}
/*
* Determine box type from codepoint
*/
static uint8_t classify_char(uint32_t cp)
{
if (is_whitespace(cp))
return BOX_SPACE;
if (is_cjk_punct(cp))
return BOX_CJK_PUNCT;
if (is_cjk(cp))
return BOX_CJK;
return BOX_LATIN;
}
/*
* Determine glue type between two boxes
*/
static uint8_t glue_between(uint8_t prev_end, uint8_t curr_start)
{
if (prev_end == BOX_SPACE || curr_start == BOX_SPACE)
return GLUE_NONE;
bool prev_latin = (prev_end == BOX_LATIN);
bool curr_latin = (curr_start == BOX_LATIN);
if (prev_latin && curr_latin)
return GLUE_LWS;
if (!prev_latin && !curr_latin)
return GLUE_CWS;
return GLUE_MWS;
}
/*
* Split text into boxes with hyphenation
*/
ekp_paragraph_t *ekp_para_create(const char *text, size_t len,
ekp_hyphenator_t *h,
int32_t (*measure_fn)(const char *, size_t))
{
if (!text || len == 0)
return NULL;
ekp_paragraph_t *p = calloc(1, sizeof(*p));
if (!p)
return NULL;
/* Copy text */
p->text = malloc(len + 1);
if (!p->text) {
free(p);
return NULL;
}
memcpy(p->text, text, len);
p->text[len] = '\0';
p->text_len = len;
/* Compute hash for caching */
uint64_t hash = 14695981039346656037ULL;
for (size_t i = 0; i < len; i++) {
hash ^= (uint8_t)text[i];
hash *= 1099511628211ULL;
}
p->hash = hash;
/* First pass: count boxes (rough estimate) */
size_t max_boxes = len + 1;
/* Temporary arrays for first pass */
size_t *box_starts = malloc(max_boxes * sizeof(size_t));
size_t *box_lens = malloc(max_boxes * sizeof(size_t));
uint8_t *box_types = malloc(max_boxes * sizeof(uint8_t));
if (!box_starts || !box_lens || !box_types) {
free(box_starts);
free(box_lens);
free(box_types);
ekp_para_destroy(p);
return NULL;
}
/* Tokenize into boxes */
size_t box_count = 0;
size_t pos = 0;
uint8_t prev_type = BOX_SPACE;
size_t word_start = 0;
bool in_latin_word = false;
while (pos < len) {
int char_len;
uint32_t cp = utf8_decode(text + pos, &char_len);
uint8_t type = classify_char(cp);
if (in_latin_word) {
if (type != BOX_LATIN) {
/* End Latin word */
box_starts[box_count] = word_start;
box_lens[box_count] = pos - word_start;
box_types[box_count] = BOX_LATIN;
box_count++;
in_latin_word = false;
}
}
if (type == BOX_LATIN) {
if (!in_latin_word) {
word_start = pos;
in_latin_word = true;
}
} else {
/* Non-Latin: each character is its own box */
box_starts[box_count] = pos;
box_lens[box_count] = char_len;
box_types[box_count] = type;
box_count++;
}
prev_type = type;
pos += char_len;
}
/* Flush final Latin word */
if (in_latin_word) {
box_starts[box_count] = word_start;
box_lens[box_count] = pos - word_start;
box_types[box_count] = BOX_LATIN;
box_count++;
}
/* Hyphenation: expand Latin words */
size_t *hyphen_pos = malloc(max_boxes * sizeof(size_t));
size_t hyphen_count = 0;
/* Estimate expanded size */
size_t expanded_boxes = box_count * 2;
ekp_box_t *boxes = calloc(expanded_boxes, sizeof(ekp_box_t));
if (!boxes || !hyphen_pos) {
free(box_starts);
free(box_lens);
free(box_types);
free(hyphen_pos);
free(boxes);
ekp_para_destroy(p);
return NULL;
}
size_t final_count = 0;
for (size_t i = 0; i < box_count; i++) {
const char *box_text = text + box_starts[i];
size_t box_len = box_lens[i];
uint8_t type = box_types[i];
if (type == BOX_LATIN && h && box_len > 4) {
/* Try hyphenation */
int8_t positions[EKP_MAX_WORD_LEN];
int pos_count = ekp_hyphen_word(h, box_text, box_len,
positions, EKP_MAX_WORD_LEN);
if (pos_count > 0) {
/* Split at hyphenation points */
size_t prev_split = 0;
for (int j = 0; j < pos_count; j++) {
size_t split = positions[j];
if (split <= prev_split || split >= box_len)
continue;
boxes[final_count].text = box_text + prev_split;
boxes[final_count].text_len = split - prev_split;
boxes[final_count].box_type = BOX_LATIN;
boxes[final_count].start_type = BOX_LATIN;
boxes[final_count].end_type = BOX_LATIN;
boxes[final_count].pixel_width =
measure_fn ? measure_fn(boxes[final_count].text,
boxes[final_count].text_len) : 0;
hyphen_pos[hyphen_count++] = final_count;
final_count++;
prev_split = split;
}
/* Final segment */
if (prev_split < box_len) {
boxes[final_count].text = box_text + prev_split;
boxes[final_count].text_len = box_len - prev_split;
boxes[final_count].box_type = BOX_LATIN;
boxes[final_count].start_type = BOX_LATIN;
boxes[final_count].end_type = BOX_LATIN;
boxes[final_count].pixel_width =
measure_fn ? measure_fn(boxes[final_count].text,
boxes[final_count].text_len) : 0;
final_count++;
}
continue;
}
}
/* No hyphenation */
boxes[final_count].text = box_text;
boxes[final_count].text_len = box_len;
boxes[final_count].box_type = type;
boxes[final_count].start_type = type;
boxes[final_count].end_type = type;
boxes[final_count].pixel_width =
measure_fn ? measure_fn(box_text, box_len) : 0;
final_count++;
}
free(box_starts);
free(box_lens);
free(box_types);
/* Build final arrays */
p->boxes = boxes;
p->box_count = final_count;
/* Hyphenation positions */
p->hyphen_positions = malloc(hyphen_count * sizeof(int32_t));
if (p->hyphen_positions) {
for (size_t i = 0; i < hyphen_count; i++) {
p->hyphen_positions[i] = hyphen_pos[i];
}
p->hyphen_count = hyphen_count;
}
free(hyphen_pos);
/* Hyphen width */
p->hyphen_width = measure_fn ? measure_fn("-", 1) : 5;
/* Build glues */
p->glues = calloc(final_count, sizeof(ekp_glue_t));
if (!p->glues) {
ekp_para_destroy(p);
return NULL;
}
ekp_spacing_t *sp = &ekp_global->spacing;
for (size_t i = 0; i < final_count; i++) {
/* Check if after hyphenation point */
bool after_hyphen = false;
for (size_t j = 0; j < p->hyphen_count; j++) {
if ((size_t)(p->hyphen_positions[j] + 1) == i) {
after_hyphen = true;
break;
}
}
if (after_hyphen || i == 0) {
p->glues[i].type = GLUE_NONE;
continue;
}
uint8_t prev_end = boxes[i - 1].end_type;
uint8_t curr_start = boxes[i].start_type;
uint8_t gtype = glue_between(prev_end, curr_start);
p->glues[i].type = gtype;
switch (gtype) {
case GLUE_LWS:
p->glues[i].ideal = sp->lws_ideal;
p->glues[i].stretch = sp->lws_stretch;
p->glues[i].shrink = sp->lws_shrink;
break;
case GLUE_MWS:
p->glues[i].ideal = sp->mws_ideal;
p->glues[i].stretch = sp->mws_stretch;
p->glues[i].shrink = sp->mws_shrink;
break;
case GLUE_CWS:
p->glues[i].ideal = sp->cws_ideal;
p->glues[i].stretch = sp->cws_stretch;
p->glues[i].shrink = sp->cws_shrink;
break;
default:
break;
}
}
/* Build prefix sums for O(1) range queries */
p->ideal_prefix = calloc(final_count + 1, sizeof(int32_t));
p->min_prefix = calloc(final_count + 1, sizeof(int32_t));
p->max_prefix = calloc(final_count + 1, sizeof(int32_t));
if (!p->ideal_prefix || !p->min_prefix || !p->max_prefix) {
ekp_para_destroy(p);
return NULL;
}
for (size_t i = 0; i < final_count; i++) {
int32_t box_w = boxes[i].pixel_width;
int32_t glue_ideal = p->glues[i].ideal;
int32_t glue_stretch = p->glues[i].stretch;
int32_t glue_shrink = p->glues[i].shrink;
p->ideal_prefix[i + 1] = p->ideal_prefix[i] + box_w + glue_ideal;
p->min_prefix[i + 1] = p->min_prefix[i] + box_w + (glue_ideal - glue_shrink);
p->max_prefix[i + 1] = p->max_prefix[i] + box_w + (glue_ideal + glue_stretch);
}
return p;
}
void ekp_para_destroy(ekp_paragraph_t *p)
{
if (!p)
return;
free(p->text);
free(p->boxes);
free(p->glues);
free(p->hyphen_positions);
free(p->ideal_prefix);
free(p->min_prefix);
free(p->max_prefix);
free(p);
}

153
ekp_c/ekp_thread_pool.c Normal file
View File

@ -0,0 +1,153 @@
/*
* ekp_thread_pool.c - Work-stealing thread pool
*
* Simple but effective: fixed thread count, lock-free queue would be
* overkill for our batch sizes. Keep it simple, stupid.
*/
#include "ekp_module.h"
#include <stdlib.h>
#include <string.h>
#define QUEUE_CAPACITY 1024
static void *worker_thread(void *arg)
{
ekp_thread_pool_t *pool = (ekp_thread_pool_t *)arg;
while (1) {
pthread_mutex_lock(&pool->queue_lock);
/* Wait for work */
while (pool->queue_head == pool->queue_tail && !pool->shutdown) {
pthread_cond_wait(&pool->queue_cond, &pool->queue_lock);
}
if (pool->shutdown && pool->queue_head == pool->queue_tail) {
pthread_mutex_unlock(&pool->queue_lock);
break;
}
/* Dequeue work */
void (*func)(void *) = pool->queue[pool->queue_head].func;
void *work_arg = pool->queue[pool->queue_head].arg;
pool->queue_head = (pool->queue_head + 1) % pool->queue_size;
pool->active_count++;
pthread_mutex_unlock(&pool->queue_lock);
/* Execute */
if (func)
func(work_arg);
/* Mark done */
pthread_mutex_lock(&pool->queue_lock);
pool->active_count--;
if (pool->active_count == 0 && pool->queue_head == pool->queue_tail) {
pthread_cond_signal(&pool->done_cond);
}
pthread_mutex_unlock(&pool->queue_lock);
}
return NULL;
}
ekp_thread_pool_t *ekp_pool_create(size_t num_threads)
{
if (num_threads == 0)
num_threads = EKP_THREAD_POOL_SIZE;
if (num_threads > EKP_THREAD_POOL_SIZE)
num_threads = EKP_THREAD_POOL_SIZE;
ekp_thread_pool_t *pool = calloc(1, sizeof(*pool));
if (!pool)
return NULL;
pool->queue_size = QUEUE_CAPACITY;
pool->queue = calloc(pool->queue_size, sizeof(pool->queue[0]));
if (!pool->queue) {
free(pool);
return NULL;
}
pthread_mutex_init(&pool->queue_lock, NULL);
pthread_cond_init(&pool->queue_cond, NULL);
pthread_cond_init(&pool->done_cond, NULL);
/* Start worker threads */
for (size_t i = 0; i < num_threads; i++) {
if (pthread_create(&pool->threads[i], NULL, worker_thread, pool) != 0) {
/* Cleanup on failure */
pool->shutdown = true;
pthread_cond_broadcast(&pool->queue_cond);
for (size_t j = 0; j < i; j++) {
pthread_join(pool->threads[j], NULL);
}
pthread_mutex_destroy(&pool->queue_lock);
pthread_cond_destroy(&pool->queue_cond);
pthread_cond_destroy(&pool->done_cond);
free(pool->queue);
free(pool);
return NULL;
}
}
return pool;
}
void ekp_pool_destroy(ekp_thread_pool_t *pool)
{
if (!pool)
return;
pthread_mutex_lock(&pool->queue_lock);
pool->shutdown = true;
pthread_cond_broadcast(&pool->queue_cond);
pthread_mutex_unlock(&pool->queue_lock);
for (size_t i = 0; i < EKP_THREAD_POOL_SIZE; i++) {
if (pool->threads[i])
pthread_join(pool->threads[i], NULL);
}
pthread_mutex_destroy(&pool->queue_lock);
pthread_cond_destroy(&pool->queue_cond);
pthread_cond_destroy(&pool->done_cond);
free(pool->queue);
free(pool);
}
void ekp_pool_submit(ekp_thread_pool_t *pool, void (*func)(void *), void *arg)
{
if (!pool || !func)
return;
pthread_mutex_lock(&pool->queue_lock);
size_t next_tail = (pool->queue_tail + 1) % pool->queue_size;
/* Queue full - drop task (shouldn't happen with proper sizing) */
if (next_tail == pool->queue_head) {
pthread_mutex_unlock(&pool->queue_lock);
return;
}
pool->queue[pool->queue_tail].func = func;
pool->queue[pool->queue_tail].arg = arg;
pool->queue_tail = next_tail;
pthread_cond_signal(&pool->queue_cond);
pthread_mutex_unlock(&pool->queue_lock);
}
void ekp_pool_wait(ekp_thread_pool_t *pool)
{
if (!pool)
return;
pthread_mutex_lock(&pool->queue_lock);
while (pool->active_count > 0 || pool->queue_head != pool->queue_tail) {
pthread_cond_wait(&pool->done_cond, &pool->queue_lock);
}
pthread_mutex_unlock(&pool->queue_lock);
}

218
readme.md
View File

@ -1,172 +1,120 @@
[中文文档](./readme_zh.md)
# Emacs-KP: Knuth-Plass Line Breaking for Emacs
[中文文档](./readme_zh.md) | [Developer Guide](./DEVELOPER.md)
Emacs-kp implements the Knuth-Plass optimal line breaking algorithm with full support for CJK (Chinese, Japanese, Korean) and Latin mixed text typesetting.
## Demo
![ekp-demo](./images/ekp-demo-with-cache.gif)
## Algorithm Overview
## Features
### The Knuth-Plass Algorithm
- **Optimal Line Breaking**: Uses Knuth-Plass algorithm for globally optimal paragraph layout.
- **CJK Support**: Full support for Chinese, Japanese, Korean with mixed Latin text.
- **Hyphenation**: Frank Liang's algorithm with language-specific dictionaries.
- **Text Properties Preserved**: Font faces, colors, and other Emacs text properties are maintained.
- **C Module Acceleration**: Optional multi-threaded C module for 16-29x speedup.
- **Automatic Font Handling**: Spacing parameters computed from actual font metrics.
The algorithm is based on the seminal 1981 paper ["Breaking Paragraphs into Lines"](https://gwern.net/doc/design/typography/tex/1981-knuth.pdf) by Donald Knuth and Michael Plass. Unlike greedy line-breaking (used by most text editors), K-P considers **all possible breakpoints** simultaneously to find the globally optimal solution.
---
#### Core Concepts
## User Guide
**1. Boxes, Glue, and Penalties**
### Quick Start
Text is modeled as a sequence of three elements:
- **Box**: Indivisible content (characters, words) with fixed width
- **Glue**: Flexible space with ideal width, stretchability, and shrinkability
- **Penalty**: Cost for breaking at specific points (e.g., hyphenation)
1. **Install Dependencies**:
Ensure you have a C compiler if you plan to use the C module (recommended for performance).
2. **Configuration**:
```elisp
(add-to-list 'load-path "/path/to/emacs-kp")
(require 'ekp)
;; Basic usage: justify text to 600 pixels width
(ekp-pixel-justify "Your paragraph text here..." 600)
;; Find optimal width in a range (returns (text . optimal-width))
(ekp-pixel-range-justify "Your text" 400 800)
```
┌─────┐ ┌─────┐ ┌─────┐
│ Box │─Glue─│ Box │─Glue─│ Box │
└─────┘ └─────┘ └─────┘
word (flexible) word
```
**2. Badness: Measuring Line Quality**
Each line's quality is measured by how much glue must stretch/shrink:
```
⎧ 0 if adjustment = 0
badness = ⎨ ∞ if impossible to fit
⎩ 100 × |adjustment/flexibility|³
```
- `adjustment` = target_width - natural_width
- `flexibility` = total stretchability (if stretching) or shrinkability (if shrinking)
**3. Demerits: Ranking Break Sequences**
Demerits combine badness with penalties to rank entire paragraph layouts:
```
demerits = (line_penalty + badness)² + penalty² + fitness_penalty
```
Where:
- `line_penalty`: Base cost per line (default: 10)
- `penalty`: Break-specific cost (hyphenation: 50)
- `fitness_penalty`: Extra cost when adjacent lines differ significantly in tightness
**4. Fitness Classes**
Lines are classified by tightness to ensure visual consistency:
- Class 0: Tight (significantly shrunk)
- Class 1: Decent (close to ideal)
- Class 2: Loose (stretched)
- Class 3: Very loose (significantly stretched)
Adjacent lines differing by more than one class incur additional penalty.
**5. Dynamic Programming**
The algorithm uses DP to find the minimum-demerits path through all valid breakpoints:
```
dp[k] = min over all valid i < k {
dp[i] + demerits(line from i to k)
}
```
Time complexity: O(n²) where n = number of potential breakpoints.
### CJK Extensions
Emacs-kp extends the original algorithm for CJK text:
1. **Character-level breaking**: CJK text can break between any characters
2. **Mixed spacing**: Three glue types for Latin-Latin, Latin-CJK, and CJK-CJK gaps
3. **Punctuation handling**: CJK punctuation attaches to adjacent characters
### Hyphenation
Latin word hyphenation uses Frank Liang's algorithm (TeX's hyphenation):
- Pattern-based approach with priority values
- Language-specific dictionaries (en_US, de_DE, fr, etc.)
- Configurable minimum characters before/after breaks
## Limitations
Currently supports CJK mixed with **one** Latin language only. Multi-Latin-language mixing is not supported because the system cannot reliably determine which language a word belongs to for hyphenation.
## Usage
### Configuration
**`ekp-latin-lang`**: Primary Latin language for hyphenation (default: `"en_US"`).
See `dictionaries/` for supported languages.
#### Language Settings
**`ekp-param-set`**: Configure spacing parameters (in pixels):
**`ekp-latin-lang`** (default: `"en_US"`)
| Parameter | Description |
|:----------------------|:-----------------------------------------------|
| `ekp-lws-ideal-pixel` | Ideal space between Latin words |
| `ekp-lws-stretch-pixel` | Maximum stretch between Latin words |
| `ekp-lws-shrink-pixel` | Maximum shrink between Latin words |
| `ekp-mws-ideal-pixel` | Ideal space between Latin and CJK |
| `ekp-mws-stretch-pixel` | Maximum stretch between Latin and CJK |
| `ekp-mws-shrink-pixel` | Maximum shrink between Latin and CJK |
| `ekp-cws-ideal-pixel` | Ideal space between CJK characters |
| `ekp-cws-stretch-pixel` | Maximum stretch between CJK characters |
| `ekp-cws-shrink-pixel` | Maximum shrink between CJK characters |
Primary Latin language for hyphenation. Supported languages are in `dictionaries/` directory:
- `en_US`, `en_GB` - English
- `de_DE` - German
- `fr` - French
- `es` - Spanish
- And many more...
Example: `(ekp-param-set 7 3 2 5 2 1 0 2 0)`
```elisp
(setq ekp-latin-lang "de_DE")
```
**Do not set these variables directly—always use `ekp-param-set`.**
#### Spacing Parameters
Default values follow K-P recommendations:
- Ideal = space character width
- Stretch = ideal × 0.5
- Shrink = ideal × 0.33
Use `ekp-param-set` to configure spacing (in pixels). If not set, defaults are computed automatically from font metrics.
### K-P Algorithm Parameters
```elisp
(ekp-param-set lws-ideal lws-stretch lws-shrink
mws-ideal mws-stretch mws-shrink
cws-ideal cws-stretch cws-shrink)
```
| Parameter | Default | Description |
|:------------------------------|:--------|:-----------------------------------------|
| `ekp-line-penalty` | 10 | Base penalty per line break |
| `ekp-hyphen-penalty` | 50 | Penalty for hyphenated breaks |
| `ekp-adjacent-fitness-penalty`| 100 | Penalty for inconsistent line tightness |
| Parameter Group | Description |
|:----------------|:------------|
| `lws-*` | Latin Word Space: between Latin words |
| `mws-*` | Mixed Word Space: between Latin and CJK |
| `cws-*` | CJK Word Space: between CJK characters |
#### K-P Algorithm Parameters
| Variable | Default | Description |
|:---------|:--------|:------------|
| `ekp-line-penalty` | 10 | Base cost per line break |
| `ekp-hyphen-penalty` | 50 | Extra cost for hyphenated breaks |
| `ekp-adjacent-fitness-penalty` | 100 | Cost for inconsistent line tightness |
| `ekp-last-line-min-ratio` | 0.5 | Minimum fill ratio for last line |
| `ekp-looseness` | 0 | Target line count offset (±n lines) |
### Core Functions
### C Dynamic Module (Recommended)
For large texts, the optional C module provides significant performance improvement through multi-threaded parallel computation.
#### Building
```bash
cd ekp_c
make
```
*Requirements: C11 compiler, Emacs 27.1+*
#### Loading
```elisp
(ekp-pixel-justify string line-pixel)
(require 'ekp-utils)
;; Load and initialize C module
(ekp-c-module-load)
;; Optional: Load hyphenation dictionary for C module
(ekp-c-load-dictionary "en_US")
```
Justify STRING to LINE-PIXEL width per line. Returns formatted text.
```elisp
(ekp-pixel-range-justify string min-pixel max-pixel)
```
Find optimal width in [MIN-PIXEL, MAX-PIXEL] range using ternary search.
Returns `(formatted-text . optimal-pixel)`.
Once loaded, `ekp-use-c-module` defaults to `t`, and all justification functions will automatically use the C module.
Note: Uses O(log n) ternary search with aggressive caching.
---
```elisp
(ekp-clear-caches)
```
Clear all paragraph caches.
## Algorithm & Architecture
## Roadmap
- [x] Preserve original text properties after formatting
- [x] Full Knuth-Plass demerits model with fitness classes
- [x] Hyphenation with consecutive-hyphen penalty
- [ ] Rust dynamic module for parallel computation
- [ ] Auto-correction for mixed punctuation
For a detailed explanation of the internal architecture, algorithms, and API reference, please refer to the **[Developer Guide](./DEVELOPER.md)**.
## Credits
- Core algorithm: ["Breaking Paragraphs into Lines"](https://gwern.net/doc/design/typography/tex/1981-knuth.pdf) by Donald E. Knuth and Michael F. Plass (1981)
- Hyphenation: Adapted from [Pyphen](https://github.com/Kozea/Pyphen), using Liang's algorithm
- Dictionaries: [Hunspell hyphenation patterns](https://github.com/Kozea/Pyphen)
- **Core Algorithm**: ["Breaking Paragraphs into Lines"](https://gwern.net/doc/design/typography/tex/1981-knuth.pdf) by Donald E. Knuth and Michael F. Plass (1981)
- **Hyphenation**: Adapted from [Pyphen](https://github.com/Kozea/Pyphen), using Liang's algorithm
- **Dictionaries**: [Hunspell hyphenation patterns](https://github.com/Kozea/Pyphen)

View File

@ -1,170 +1,120 @@
# Emacs-KP: Knuth-Plass 排版算法 Emacs 实现
[English Documentation](./readme.md) | [开发者指南](./DEVELOPER_ZH.md)
Emacs-kp 实现了 Knuth-Plass 最优断行算法,并扩展支持 CJK中日韩与拉丁文混合排版。
## 演示
![ekp-demo](./images/ekp-demo-with-cache.gif)
## 算法原理
## 特性
### Knuth-Plass 算法
- **全局最优断行**:使用 Knuth-Plass 算法寻找段落的全局最优布局。
- **CJK 支持**:完美支持中日韩与拉丁文的混合排版。
- **连字符断词**:使用 Frank Liang 算法和特定语言词典。
- **属性保留**:排版后保留字体、颜色等所有 Emacs 文本属性。
- **C 模块加速**:可选的多线程 C 模块提供 16-29 倍性能提升。
- **自动字体处理**:根据实际字体度量自动计算间距参数。
本算法基于 Donald Knuth 和 Michael Plass 于 1981 年发表的经典论文 ["Breaking Paragraphs into Lines"](https://gwern.net/doc/design/typography/tex/1981-knuth.pdf)。与大多数文本编辑器使用的贪心断行不同K-P 算法**同时考虑所有可能的断点**,寻找全局最优解。
---
#### 核心概念
## 用户指南
**1. Box盒子、Glue胶水、Penalty惩罚**
### 快速开始
文本被建模为三种元素的序列:
- **Box**:不可分割的内容(字符、单词),具有固定宽度
- **Glue**:弹性空白,具有理想宽度、可拉伸量、可压缩量
- **Penalty**:在特定位置断行的代价(如连字符断词)
1. **安装依赖**
建议安装 C 编译器以构建高性能模块。
```
┌─────┐ ┌─────┐ ┌─────┐
│ Box │─Glue─│ Box │─Glue─│ Box │
└─────┘ └─────┘ └─────┘
单词 (弹性空白) 单词
2. **配置与使用**
```elisp
(add-to-list 'load-path "/path/to/emacs-kp")
(require 'ekp)
;; 基本用法:将文本按 600 像素宽度对齐
(ekp-pixel-justify "这是一段测试文本..." 600)
;; 范围对齐:寻找 400-800 像素范围内的最优宽度
(ekp-pixel-range-justify "测试文本" 400 800)
```
**2. Badness劣度衡量行的质量**
### 配置详情
每行的质量由 glue 需要拉伸/压缩的程度来衡量:
#### 语言设置
```
⎧ 0 若 adjustment = 0
badness = ⎨ ∞ 若无法容纳
⎩ 100 × |adjustment/flexibility|³
**`ekp-latin-lang`** (默认: `"en_US"`)
用于断词的主要拉丁语言。支持的语言位于 `dictionaries/` 目录:
- `en_US`, `en_GB` - 英语
- `de_DE` - 德语
- `fr` - 法语
- `es` - 西班牙语
- 等等...
```elisp
(setq ekp-latin-lang "de_DE")
```
- `adjustment` = 目标宽度 - 自然宽度
- `flexibility` = 可拉伸总量(拉伸时)或可压缩总量(压缩时)
#### 间距参数
**3. Demerits缺陷值评估断行序列**
使用 `ekp-param-set` 配置间距(像素)。若不设置,将根据字体自动计算。
Demerits 综合 badness 和 penalty 来评估整个段落的排版质量:
```
demerits = (line_penalty + badness)² + penalty² + fitness_penalty
```elisp
(ekp-param-set lws-ideal lws-stretch lws-shrink
mws-ideal mws-stretch mws-shrink
cws-ideal cws-stretch cws-shrink)
```
其中:
- `line_penalty`每行的基础代价默认10
- `penalty`断点特定代价连字符50
- `fitness_penalty`:相邻行松紧度差异过大时的额外代价
| 参数组 | 说明 |
|:-------|:-----|
| `lws-*` | 拉丁词间距 (Latin Word Space) |
| `mws-*` | 中西文间距 (Mixed Word Space) |
| `cws-*` | CJK 字符间距 (CJK Word Space) |
**4. Fitness Classes适应度等级**
#### K-P 算法参数
行按松紧度分类,确保视觉一致性:
- 等级 0紧凑显著压缩
- 等级 1正常接近理想
- 等级 2宽松拉伸
- 等级 3非常宽松显著拉伸
相邻行等级差超过 1 会产生额外惩罚。
**5. 动态规划**
算法使用 DP 在所有有效断点中寻找最小 demerits 路径:
```
dp[k] = min over all valid i < k {
dp[i] + demerits(从 i 到 k 的行)
}
```
时间复杂度O(n²)n = 潜在断点数量。
### CJK 扩展
Emacs-kp 为 CJK 文本扩展了原算法:
1. **字符级断行**CJK 文本可在任意字符间断行
2. **混合间距**Latin-Latin、Latin-CJK、CJK-CJK 三种 glue 类型
3. **标点处理**CJK 标点附着于相邻字符
### 连字符断词
拉丁语单词断词使用 Frank Liang 的算法TeX 的断词算法):
- 基于模式匹配的优先级方法
- 特定语言的词典en_US、de_DE、fr 等)
- 可配置断点前后的最小字符数
## 局限
目前仅支持 CJK 与**一种**拉丁语言的混合排版。不支持多种拉丁语言混排,因为系统无法可靠判断单词属于哪种语言以进行断词。
## 用法
### 配置项
**`ekp-latin-lang`**:用于断词的主要拉丁语言(默认:`"en_US"`)。
支持的语言见 `dictionaries/` 目录。
**`ekp-param-set`**:配置间距参数(单位:像素):
| 参数 | 说明 |
|:------------------------|:-------------------------------|
| `ekp-lws-ideal-pixel` | 拉丁单词间的理想间距 |
| `ekp-lws-stretch-pixel` | 拉丁单词间的最大拉伸量 |
| `ekp-lws-shrink-pixel` | 拉丁单词间的最大压缩量 |
| `ekp-mws-ideal-pixel` | 拉丁与 CJK 之间的理想间距 |
| `ekp-mws-stretch-pixel` | 拉丁与 CJK 之间的最大拉伸量 |
| `ekp-mws-shrink-pixel` | 拉丁与 CJK 之间的最大压缩量 |
| `ekp-cws-ideal-pixel` | CJK 字符间的理想间距 |
| `ekp-cws-stretch-pixel` | CJK 字符间的最大拉伸量 |
| `ekp-cws-shrink-pixel` | CJK 字符间的最大压缩量 |
示例:`(ekp-param-set 7 3 2 5 2 1 0 2 0)`
**请勿直接设置这些变量——必须使用 `ekp-param-set` 函数。**
默认值遵循 K-P 推荐:
- 理想宽度 = 空格字符宽度
- 可拉伸 = 理想 × 0.5
- 可压缩 = 理想 × 0.33
### K-P 算法参数
| 参数 | 默认值 | 说明 |
|:------------------------------|:-------|:-----------------------------|
| 变量 | 默认值 | 说明 |
|:-----|:-------|:-----|
| `ekp-line-penalty` | 10 | 每行断行的基础惩罚 |
| `ekp-hyphen-penalty` | 50 | 连字符断词的惩罚 |
| `ekp-adjacent-fitness-penalty`| 100 | 相邻行松紧度不一致的惩罚 |
| `ekp-adjacent-fitness-penalty` | 100 | 相邻行松紧度不一致的惩罚 |
| `ekp-last-line-min-ratio` | 0.5 | 末行最小填充比例 |
| `ekp-looseness` | 0 | 目标行数偏移±n 行) |
### 核心函数
### C 动态模块 (推荐)
对于长文本,建议使用 C 模块以获得显著的性能提升。
#### 构建
```bash
cd ekp_c
make
```
*要求C11 编译器Emacs 27.1+*
#### 加载
```elisp
(ekp-pixel-justify string line-pixel)
(require 'ekp-utils)
;; 加载并初始化 C 模块
(ekp-c-module-load)
;; 可选:为 C 模块加载断词字典
(ekp-c-load-dictionary "en_US")
```
将 STRING 按 LINE-PIXEL 宽度排版,返回排版后的文本。
```elisp
(ekp-pixel-range-justify string min-pixel max-pixel)
```
在 [MIN-PIXEL, MAX-PIXEL] 范围内使用三分搜索寻找最优宽度。
返回 `(排版文本 . 最优像素值)`
加载后,`ekp-use-c-module` 默认为 `t`,所有排版函数将自动使用 C 模块进行加速。
注:使用 O(log n) 三分搜索,并积极缓存。
---
```elisp
(ekp-clear-caches)
```
清除所有段落缓存。
## 算法与架构
## 路线图
- [x] 排版后保留原始文本属性
- [x] 完整的 Knuth-Plass demerits 模型与 fitness classes
- [x] 支持连续连字符惩罚的断词
- [ ] Rust 动态模块实现并行计算
- [ ] 混合标点自动修正
关于内部架构、算法细节和 API 参考的详细说明,请参阅 **[开发者指南](./DEVELOPER_ZH.md)**。
## 致谢
- 核心算法Donald E. Knuth 和 Michael F. Plass 的论文 ["Breaking Paragraphs into Lines"](https://gwern.net/doc/design/typography/tex/1981-knuth.pdf)1981
- 断词算法:改编自 [Pyphen](https://github.com/Kozea/Pyphen),使用 Liang 算法
- 词典:[Hunspell 断词模式](https://github.com/Kozea/Pyphen)
- **核心算法**: ["Breaking Paragraphs into Lines"](https://gwern.net/doc/design/typography/tex/1981-knuth.pdf) by Donald E. Knuth and Michael F. Plass (1981)
- **断词算法**: 改编自 [Pyphen](https://github.com/Kozea/Pyphen),使用 Liang 算法
- **词典**: [Hunspell 断词模式](https://github.com/Kozea/Pyphen)