Commit Graph

100 Commits

Author SHA1 Message Date
Kinneyzhang
e823d89a4a fix: restore TeX final-pass reachability 2026-08-02 15:30:12 +08:00
Kinneyzhang
665164911f perf: make live structural commits incremental
Reuse prepared paragraph data, safe Elisp DP states, common layout lines, and dirty-island source reconstruction for exact append commits. Simplify C int32 validation and add the frozen task030 evaluator, regressions, and decision records.
2026-07-30 08:24:54 +08:00
Kinneyzhang
83352c4571 feat!: add source-clean live buffer layout
Replace the physical ekp-region renderer with ekp-buffer text-property projection backed by shared semantic layout plans and stable live-edit transactions.

Optimize the C-backed resize hot path, and add deterministic performance, ERT, GUI, fuzz, release, and documentation coverage for task017 through task032.

BREAKING CHANGE: require ekp-buffer instead of ekp-region and rename module-owned ekp-region-* settings to ekp-buffer-*.
2026-07-30 01:07:25 +08:00
Kinneyzhang
fd1210e557 fix: close repository audit findings
Complete task002 through task016 across cache correctness, editor lifecycle, C boundaries, build and release governance, dictionary provenance, performance, interactive workflows, tests, documentation, and final cleanup.
2026-07-28 22:43:10 +08:00
Kinneyzhang
29cef97ef8 fix: justified lines fit the real display, not an idealized one
Two root causes made justified text overrun the window's right edge
in real user sessions (every line ending in the "$" truncation
marker), while emacs -Q looked fine:

1. Windows WITHOUT fringes draw the truncation/continuation
   indicator in the text area's LAST COLUMN, so the usable width is
   one character less than window-body-width.  ekp reserved only a
   2px margin: with fringes disabled (a common minimal setup, and
   all ttys) every line that hit the target width exactly had its
   final glyph displaced by the "$".  ekp-region--window-pixel now
   reserves one frame-char-width when the window has no right
   fringe.  (The same lesson ebox-playground encodes in its
   viewport-width reserve.)

2. Measurement was blind to the buffer's display context:
   string-pixel-width works in a bare hidden buffer, ignoring
   face-remapping-alist — which is where text-scale-mode, themes
   and per-buffer font tweaks live.  Under a remap, the DP laid
   lines out with one font's metrics and the display rendered them
   with another's: scale +3 made 5 of 7 sample lines overflow a
   1330px window by up to 900px (GUI-measured).  Measurement now
   runs with the destination buffer's face-remapping-alist (the
   29/30-compatible equivalent of Emacs 31's string-pixel-width
   BUFFER argument), and the width/paragraph caches key on that
   context so buffers at different scales never alias.  text-scale
   changes also trigger a re-flow in ekp-auto-justify-mode.

Ground truth, measured with window-text-pixel-size in GUI Emacs
across 7 display contexts (plain / text-scale ±| face remap /
no-fringes / no-fringes+scale / narrow+scale): the widest justified
line equals the target width exactly in every case, zero lines
overflow.  Verbatim code blocks are exempt by design (they never
reflow, like any code line in a narrow window).

New tooling so this never regresses invisibly:
- M-x ekp-diagnose: renders a probe line in YOUR buffer and reports
  target vs rendered width — run it in any session where justified
  text looks wrong.
- tests/ekp-gui-verify.el: M-x ekp-gui-verify (single check in a
  customized session) and ekp-gui-verify-matrix (the 7-case table,
  for emacs -Q).
- 2 new batch ERT tests pin the context-keyed caches (94 total).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 03:19:27 +08:00
Kinneyzhang
448198bdb9 docs: cut the 1.0.0 changelog section
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 02:31:42 +08:00
Kinneyzhang
4dee06ec47 fix(showcase): follow-window right-edge truncation; checkdoc clean
showcase: the demo font was applied with face-remap-add-relative, but
string-pixel-width (how ekp measures) ignores buffer face remapping
while honoring an explicit :family — so measurement used the session
default font and rendering used the demo font.  On a wide follow-
window that per-glyph discrepancy accumulated past the safety margin
and every justified line overflowed the right edge (invisible at the
narrow 480px default).  Pin the font as an appended :family face text
property instead, so measurement and rendering agree; inline atom/
code faces keep precedence.

checkdoc: all four elisp files are now checkdoc-clean (docstrings,
argument mentions, sentence spacing) with no logic change; the three
validation messages read naturally again (dropped the capitalized
"Ekp:" prefix checkdoc's capital-letter rule had forced).

92 ERT green; fuzz 300/300 with C parity; byte-compile warning-free.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 02:28:29 +08:00
Kinneyzhang
81d32c7052 docs: record the paragraph-handle API as documented future work
Assessed the C marshal cost (dominates range-justify: ~12 ms/width
warm, mostly env extraction, not DP) and deliberately deferred the
make_user_ptr handle API: it is a breaking 2.0 ABI change introducing
C-side object lifetime, and the interactive paths already avoid the
repeat marshal via the dp-cache.  Documented in both DEVELOPER guides
as the clear next step.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 02:04:47 +08:00
Kinneyzhang
4ba97deec7 docs: sync all docs with the M1–M4 changes
- readme_zh: installation, editor-integration behavior, JIS kinsoku,
  HYPHENMIN, refreshed perf table (parity with English readme)
- DEVELOPER/DEVELOPER_ZH: C module 1.5 (15-arg API, first-line-width,
  lazy pool, clean failure fallback), first-line indent on the 1D/C
  path, file map includes ekp-region.el and the real tests/ layout
- correct dictionary attribution to LibreOffice (per dictionaries/
  update.sh and the bundled README_hyph_*.txt)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 02:01:44 +08:00
Kinneyzhang
03096facc5 ci: add lint, sanitizer, macOS and snapshot jobs; docs and changelog
- CI: package-lint + checkdoc job, ASan/UBSan fuzz job (make DEBUG=1),
  macOS dylib build + full suite, Emacs snapshot (advisory)
- readme: installation section, new commands and editor-integration
  behavior, JIS kinsoku option, HYPHENMIN note, refreshed perf table
- add CHANGELOG.md and CONTRIBUTING.md

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 01:56:28 +08:00
Kinneyzhang
74a780ce95 refactor!: remove the experimental C tokenization path; harden the module
The self-contained C path (ekp-c-break-lines, ekp-c-load-hyphenator,
ekp-c-hyphenate, ekp-c-set-spacing, ekp_paragraph.c, ekp_hyphen.c,
~1500 lines) was never used by ekp.el, diverged semantically from the
real pipeline (no kinsoku, no protrusion, no two-pass emergency), and
contained an exploitable heap overflow reachable from Lisp:
ekp_para_create sized its box array as box_count * 2, but a long word
hyphenates into arbitrarily many syllable boxes, overflowing the
calloc'd buffer.  Deleting the path deletes the bug class.

Hardening of the live path:

- thread pool: sized from the machine's core count instead of a
  hardcoded 8; created lazily on the first multi-paragraph batch
  (single-paragraph users never start worker threads); a full queue
  now blocks the submitter until a worker makes room — tasks were
  silently dropped before, degrading the batch to the Elisp fallback
  exactly when parallelism mattered most.
- unified failure gate: a partial allocation used to silently drop
  kinsoku, hyphenation or protrusion data and continue with a subtly
  different layout; any allocation failure or pending Lisp signal
  (non-local exit from a bad element type) now fails the whole call,
  and ekp.el falls back to the Elisp engine.  The Elisp bridge wraps
  both C entry points in condition-case, and a whole-batch nil no
  longer crashes the per-paragraph loop.
- integer safety: every extracted pixel value is clamped to int32
  instead of silently wrapping.
- EKP_INFINITY (the unreachable-state sentinel) is now a real
  infinity: extremely degenerate paragraphs could legitimately
  accumulate demerits past the old 1e10 constant, making C consider
  reachable states dead and diverge from the Elisp engine.

BREAKING: the four experimental module functions are gone; rebuild
with make -C ekp_c clean all (version gate unchanged at 1.5).

92 ERT green; fuzz 300/300 byte-identical across engines.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 01:49:25 +08:00
Kinneyzhang
3fc05c1405 feat: typography wave — dictionary HYPHENMIN, JIS kinsoku, fast indent
Three quality gaps closed, one of them unlocking the C engine for
the most common Chinese configuration:

- first-line indent now runs on the 1D DP and the C engine (module
  1.5: ekp-c-break-with-arrays grows a 15th arg FIRST-LINE-WIDTH).
  A line starts at box 0 exactly when i = 0, so no (position x
  line-count) state is needed; only true parshape and looseness
  still take the 2D Elisp path.  Indented Chinese sample, w=400:
  C engine 82 ms (this configuration previously bypassed C entirely
  and ran the heaviest Elisp DP).  The C-result reconstruction
  mirrors the per-line width, and an equivalence test pins
  indent == the equivalent parshape across engines.
- LEFTHYPHENMIN / RIGHTHYPHENMIN are parsed from dictionaries and
  applied (en_US declares 2/3; the hardcoded 2/2 allowed breaks
  like "gen-cy" that the dictionary forbids).  Explicit overrides
  still win; partial overrides keep the dictionary's other side.
- JIS X 4051 line-start prohibition for small kana, the prolonged
  sound mark and iteration marks (new defcustom
  ekp-cjk-no-line-start-extra, char-table backed): っょー々 can no
  longer start a line.  Pure break-permission change; spacing and
  the DP are untouched.

Also fixed in passing:
- ekp--split-with-hyphen extracted regexp groups AFTER resolving
  the hyphenator; dictionary compilation clobbers the match data
  (latent since the resolver was hoisted; exposed by the HYPHENMIN
  parser) — groups are captured first now.
- ekp--first-indent-pixel goes through the width cache; it runs
  once per rendered line and was re-measuring the reference glyph
  every time (C path was 397 ms before, 82 ms after).

4 new ERT tests (92 total); fuzz 300/300 against C 1.5.
C module must be rebuilt: make -C ekp_c clean all

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 01:36:52 +08:00
Kinneyzhang
bed359d439 perf: session-global width cache; fix two cache-aliasing bugs
Measurement was the largest cost outside the DP and entirely outside
the C module's reach.  Boxes repeat across paragraphs constantly
(CJK text re-uses a small alphabet of glyphs), yet each paragraph
measured its own copies: 43% of string-pixel-width calls on the
bundled Chinese sample were cross-paragraph duplicates.

- new session-global box-to-width cache behind all measurement sites
  (boxes, styled hyphens, protrusion tails); paragraph-local dedup
  stays as the first level.  Same machine, byte-compiled, elisp
  engine: para-create -26%, justify zh w=400 -41%, justify zh w=200
  -8% (cold), no case slower.  ekp-clear-caches flushes it (needed
  after font/theme changes, as before).
- cache keys (paragraph and width) now strip 'fontified' and other
  volatile bookkeeping properties: font-lock flipping 'fontified'
  as text scrolled halved the paragraph-cache hit rate in fontified
  buffers by aliasing one paragraph into several entries.
- fix: dp-cache results are now keyed by looseness too -- a cached
  looseness-0 layout was returned after 'ekp-looseness' changed
  (reproduced: 3-line stale result where a fresh run gives 4).
- fix: variable watchers invalidate the same-string fast path when a
  style variable changes -- (setq ekp-alignment 'center) kept
  returning the paragraph resolved under the previous style for
  callers reusing the same string object.
- render path fetches the dp plist once instead of three lookups.

4 new ERT tests (88 total); fuzz 300/300 with C parity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 01:21:15 +08:00
Kinneyzhang
bb7b21eb92 chore: keep internal working notes out of the public repository
.phrase/ (phase handoffs, agent role modules) and AGENTS.md (a
personal cross-project agent protocol referencing .phrase/) are
local working files, not project documentation.  They stay on disk,
untracked.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 01:11:47 +08:00
Kinneyzhang
bc4907e1fc perf(region): smarter lazy re-flow scheduling
- a dirty area larger than the lazy threshold (big paste, revert)
  is chunked into the background queue instead of freezing the
  command loop with one synchronous re-justification
- queued chunks intersecting the visible span move to the queue
  front on scroll (window-scroll-functions): scrolling into an
  unprocessed area no longer waits for the whole queue
- background ticks work under a time budget
  (ekp-auto-justify-tick-budget, default 5 ms): several small
  chunks per tick, guaranteed progress of at least one, peek-then-
  pop so an aborted justification never loses a chunk
- ticks also yield while an input-method composition is active

3 new ERT tests (84 total).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 01:11:05 +08:00
Kinneyzhang
f0c7927644 feat: whole-buffer commands, DWIM, M-q remap, org/markdown wiring
- new commands ekp-justify-buffer / ekp-unjustify-buffer
- ekp-justify-region / ekp-unjustify-region without an active region
  now act on the hard paragraph at point (DWIM) instead of erroring
- ekp-auto-justify-mode remaps fill-paragraph to the new
  ekp-refill-paragraph — plain M-q treated glue spaces and soft
  breaks as content and destroyed the original whitespace
- ekp-org-setup / ekp-markdown-setup wire the existing skip-face
  presets in one call; enabling the mode in an Org/Markdown buffer
  applies them automatically unless the user configured their own.
  The markdown setup also stops font-lock from managing the display
  property: refontification was stripping the pixel glue and
  wrecking the layout permanently.
- justify runs font-lock-ensure over the region first when
  face-based verbatim detection is configured — paragraphs jit-lock
  had never fontified (off-screen code blocks) were not skipped
- showcase/demo buffers disable undo recording (width sweeps
  generated thousands of useless undo entries)

5 new ERT tests (81 total).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 01:08:40 +08:00
Kinneyzhang
a12a3c1191 feat: coexist with the Emacs ecosystem — isearch, kill ring, windows
- isearch searches the logical text: entering isearch temporarily
  un-justifies (spans remembered as markers), exiting restores the
  layout.  Justified CJK lines contain real injected spaces, which
  made even same-line phrases unfindable.
- the kill ring receives the logical text via a buffer-local
  filter-buffer-substring-function: yanking justified text elsewhere
  transports words, not the source window's pixel layout.
- multi-window: the layout width is now the narrowest window showing
  the buffer (was: whichever window resized last, leaving the other
  window overflowing), and window-configuration-change-hook catches
  the buffer becoming displayed, splits, and closed windows.
- paragraphs containing field or read-only text (comint prompts)
  are never re-written; interactive justify/unjustify on read-only
  buffers signals instead of editing through inhibit-read-only.
- edit re-flows wait for an active input-method composition (quail
  preedit) to finish instead of re-writing the text under it.

4 new ERT tests (76 total).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 01:05:15 +08:00
Kinneyzhang
ede518f10f fix: editor-state integrity — saving, stickiness, modified flag, undo
Justification is a re-layout of the same logical text; it now behaves
that way toward the rest of the editor:

- saving a justified buffer writes the LOGICAL text: before-save-hook
  unjustifies (spans remembered as markers), after-save-hook restores
  the justified view; the deterministic pair stays off undo history.
  Previously soft newlines and break hyphens were silently persisted
  to disk and the original whitespace was unrecoverable after reopen.
- renderer marker properties (ekp-glue/soft-break/soft-hyphen/hidden/
  justified) are registered in text-property-default-nonsticky: text
  typed after a glue no longer inherits the marker and is no longer
  deleted as a synthesized space by the next re-flow.
- pure re-layout preserves buffer-modified-p when it was nil: no more
  lock files, auto-saves and "buffer modified" prompts from merely
  enabling ekp-auto-justify-mode.
- changes applied by undo are not re-dirtied (no timer war against
  the user's undo sequence).
- switching major modes tears justified state down cleanly
  (change-major-mode-hook), widening first so narrowing never leaves
  justified orphans.

5 new ERT tests (72 total).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 01:00:27 +08:00
Kinneyzhang
9dcaeb0ba3 chore: licensing, customize support, and packaging hygiene
- add COPYING (GPL-3.0-or-later) and license headers to all elisp
  and C sources; real author/maintainer info replaces placeholders
- convert user-facing options to defcustom under new group `ekp'
  (spacing internals managed by ekp-param-set stay defvars)
- autoload user commands: ekp-param-reset, ekp-clear-caches,
  ekp-c-module-load, ekp-c-module-build
- ekp--load-dicts: a missing dictionaries/ directory now only
  disables hyphenation instead of breaking (require 'ekp);
  ekp--split-with-hyphen degrades gracefully (and resolves the
  hyphenator once per call instead of once per word)
- package summary no longer redundantly says "for Emacs"
- Makefile: drop personal Windows EMACS_ROOT default; add guidance
- ekp_c/README.md: fix stale 1.1 version references (module is 1.4)
- DEVELOPER*.md: remove archive/ from file map (not in the repo)
- remove unreferenced 10 MB demo GIF

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 00:52:37 +08:00
Kinneyzhang
c7a89df6ce feat(showcase): pin the key cheat sheet to the header line
Keys are now always visible at the top (help-key-binding key caps),
and the live state readout — width, alignment, protrusion, indent,
parshape, effective engine, last re-flow ms — moves to the mode line,
so both are permanently on screen without fighting for space.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-27 00:11:26 +08:00
Kinneyzhang
5233baeea0 feat: interactive showcase — M-x ekp-showcase
One self-contained demo buffer for every feature, driven by single
keys, with per-reflow milliseconds in the header line:

  -/+  width ±20 px (arrows: ±4)   d  animated width sweep + fps report
  a    cycle 4 alignments          p  hanging punctuation
  i    first-line indent (2 em)    s  wedge parshape (Elisp 2D path)
  c    C engine vs pure Elisp      w  follow window width (auto-mode)

The sample text bundles a verbatim-protected code block, an inline
no-break atom, and NBSP-joined figures; the engine indicator honestly
shows when a feature (indent/parshape) bypasses C.  Launch:

  emacs -Q -L /path/to/emacs-kp -l tests/ekp-showcase.el -f ekp-showcase

Verified interactively in GUI Emacs (justify 480→360 px, protrusion +
indent on: state and timing update live).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 23:48:35 +08:00
Kinneyzhang
dc2f6da846 feat: visible-first lazy re-flow for large buffers; packaging polish
Buffers past ekp-auto-justify-lazy-threshold (20k chars) no longer
re-justify wholesale on a width change: the visible span updates
synchronously and the rest follows in polite background chunks
(run-with-timer ticks that yield to pending input).  Measured on a
300-paragraph, 132k-char article: perceived latency 332 ms → 15 ms,
background completion ~0.5 s.

Fixed in the process: adjacent chunk markers shared a boundary, and
the earlier chunk's delete+insert pushed the next chunk's start
marker back over the freshly inserted text — chunks grew linearly
(2.2k, 4.9k, 7.4k… chars) and background work went quadratic (14 s).
Start markers now use insertion-type t.  Regression test asserts the
chunked result equals the one-shot result property-for-property.

Also: org/markdown skip-face presets (ekp-region-org-skip-faces /
ekp-region-markdown-skip-faces), MELPA header hygiene (Version + URL
on the main file, stale Package-Requires dropped from ekp-hyphen.el),
readme notes.

67 ERT green; fuzz 300/300.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 23:23:25 +08:00
Kinneyzhang
c4d7fdf8bb perf: reclaim feature costs — hot-path caches for smooth reflow
Investigation: the apparent 7× bench regression was interpreted-mode
measurement (tests always deleted .elc); byte-compiled + C the real
P1/P2 cost was +70–90%.  Worktree A/B across phase commits confirmed
the features themselves added ~+20% interpreted.

Optimizations (all verified by the 66-test suite + 300-case fuzz):
- per-char memo table for ekp--str-type (get-char-code-property was
  the para-build hot spot)
- precomputed punctuation char lists (no per-call list rebuild)
- ekp--box-offsets memoized in the paragraph struct
- interned glue strings for the two dominant payload shapes
- rendered-output cache per (paragraph, width) inside dp-cache
  (capped at 64 widths per paragraph), so resize sweeps that revisit
  a width skip rendering entirely
- unjustify walks hop property boundaries instead of chars

Numbers (byte-compiled + C): justify zh w=200 53.8 ms — at parity
with the pre-feature record (57 ms) despite punctuation boxes and
break permissions; range zh 117 ms (+55%, inherent box-count cost).
Continuous reflow, 60-paragraph 26k-char article through the full
region layer: 187 ms → 73 ms per width change; incremental
single-paragraph re-justify after an edit: 62 ms → 17 ms.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 22:03:05 +08:00
Kinneyzhang
c703ce0226 docs: typography features, verbatim protection, engine notes
readme/readme_zh gain Typography and verbatim-protection sections
(alignment, hanging punctuation, parshape/first-line indent,
no-break characters, halfwidth kinsoku) plus the honest limitation
notes (mid-line CLREQ compression and left protrusion are not
renderable in Emacs).  DEVELOPER/DEVELOPER_ZH document the break-
permission layer, alignment extra-stretch, protrusion lockstep and
per-line specs (§5.1) and the C 1.4 API surface.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 21:44:56 +08:00
Kinneyzhang
4d9a018fbf feat: verbatim protection — code blocks and mode hooks
Paragraphs stay verbatim (never justified, byte-identical through
justify/auto-mode/unjustify) when they carry the ekp-verbatim text
property, wear a face from ekp-region-skip-faces (point it at
org-block / markdown-code-face and code blocks are exempt), or match
the buffer-local ekp-region-skip-predicate — three layers of the same
general mechanism, no mode-specific code.  Inline code inside prose
is the existing ekp-no-break atom (rigid, unbreakable, unhyphenated).

New commands: ekp-verbatim-region / ekp-clear-verbatim-region.
Core engine untouched: block protection lives entirely in the region
layer, so skipped text never even reaches the tokenizer.

Tests: 66 ERT green (property / face / predicate paths, exact
roundtrips around justified prose).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 21:40:43 +08:00
Kinneyzhang
720b1cd0e4 feat: parshape and first-line indentation (Elisp 2D path)
ekp-parshape gives each line its own (INDENT . WIDTH) — TeX \parshape
— with the last entry repeating; ekp-first-line-indent is sugar for
the CJK 2-em paragraph convention (t derives 2 em from the paragraph's
CJK font, or pass pixels).  Line-number-dependent widths require the
(position × line-count) DP, so both ride the loose engine and bypass
the C module, exactly like looseness.

Indents render as leading spacers marked ekp-glue, so buffer
roundtrips remain character- and property-exact.  Composes with
alignment modes (indent adds to the centering lead) and protrusion.

Tests: 63 ERT green; fuzz 300/300.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 21:37:58 +08:00
Kinneyzhang
f6aa64b3b1 feat: right-edge protrusion — hanging punctuation (C 1.4)
ekp-protrusion enables microtype-style character protrusion at the
right edge: a line ending in fullwidth punctuation (。、」…) hangs
the whitespace half of the glyph past the flush edge — visually
identical to CLREQ line-end punctuation compression — and latin
periods/commas/quotes and soft hyphens protrude by their configured
ratios (ekp-protrusion-ratios).

Mechanics: per-box tail protrusion, folded through trailing space
boxes into a per-gap array; each DP candidate widens its effective
target (lw = width + release), mirrored exactly in the renderer's
per-line effective width and in the C-result reconstruction, so DP,
rendering and both engines stay coherent.  Left-edge protrusion is
documented as out of scope (Emacs cannot render before the line
origin).  ekp-region reserves the protrusion width in auto layouts
(ekp-region-protrusion-reserve).

Off by default; all-zero arrays make it a no-op on the shared path.
C module 1.4: break-with-arrays 12→14 args (tail-protrudes array,
hyphen-protrude scalar), batch vectors 14 elements.

Tests: 59 ERT green (deterministic hang check, flush-when-off,
protrusion C parity); fuzz 300/300.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 21:32:54 +08:00
Kinneyzhang
57a3abe853 feat: alignment modes — ragged-right, ragged-left, center (C 1.3)
ekp-alignment selects the paragraph alignment. Non-justify modes keep
inter-word glue rigid (arrays and class params zeroed at para build)
while the DP gains a per-line flexibility ekp-ragged-stretch-pixel
(default ≈2 em): badness = 100·(shortfall/R)³, so Knuth-Plass still
globally minimizes raggedness — this is \raggedright with a finite
\rightskip stretch, not first-fit.

Rendering distributes each line's leftover by mode: trailing
(ragged-right), split evenly (center), leading (ragged-left); the
spacers are ekp-glue-marked, so buffer roundtrips stay exact.

C module 1.3: set-penalties gains an extra-stretch scalar (reset when
omitted); max_w widens by it, so the prefix-based flexibility picks it
up automatically. Cache keys include alignment and R.

Tests: 56 ERT green incl. per-mode invariants (fit, rigid interior
glue, flush edges, centered leftover ±1px) and four-mode C parity;
fuzz 300/300.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 21:22:19 +08:00
Kinneyzhang
ea96a6dc96 feat: no-break spans, NBSP/WORD JOINER support, verbatim atoms
- text property ekp-no-break turns any span into a rigid typesetting
  atom: no line break inside, no hyphenation, literal spacing (space
  runs inside the span become preserved space boxes instead of
  stretchable glue) — inline code, product names, numbers with units
- character-driven joins: NBSP, NARROW NBSP, FIGURE SPACE keep their
  neighbors together (they are boxes whose adjacent gaps are
  unbreakable and glue-free — the character supplies its spacing);
  WORD JOINER / ZWNBSP attach invisibly like combining marks
- new commands: ekp-no-break-region / ekp-allow-break-region
- overlong atoms fall back to single emergency lines (atomic-run
  handling from the break-permission layer); zero C changes needed

Tests: 52 ERT green incl. atomicity across widths, hyphenation
suppression, NBSP/WJ adjacency sweeps; fuzz 300/300 with C parity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 21:12:07 +08:00
Kinneyzhang
64eb2f38d5 refactor!: per-gap break permissions — punctuation as first-class boxes
Kinsoku moves from merge-based box attachment to a general break-
permission layer, the foundation for no-break spans, NBSP, verbatim
atoms and punctuation protrusion:

- tokenizer: every CJK char (punctuation included) is its own box;
  the hold/attach machinery is gone.  Fixes two latent bugs: a second
  consecutive closer (字。」) could start a line, and an opener held
  across spaces reordered box content vs the original string
- ekp--str-type: cjk-punct splits into cjk-open (Ps/Pi) / cjk-close
- ekp-para gains breaks-allowed (bool-vector) + forbidden-positions
  (sparse, for C); glue at unbreakable punct gaps is nws, so
  punctuation hugs its content and justification stretch no longer
  opens gaps at 「x or x。
- halfwidth kinsoku: boxes consisting purely of .,;:!?)]}’”»›… may
  not start a line ((&[{‘“«‹ may not end one) — an ASCII comma after
  a CJK char no longer dangles at line start (old behavior violated
  CLREQ)
- DP (elisp 1D, elisp loose 2D, C): forbidden gaps are skipped as
  candidates while the line keeps extending; the emergency fallback
  generalizes from single boxes to atomic runs (no permitted break
  inside), recording real gap counts so multi-box emergency lines
  render correctly
- C module 1.2: ekp-c-break-with-arrays 11→12 args
  (forbidden-positions), batch vectors 12 elements, version gate bumped

Hyphenation improvement: 「Hello / Hello」 previously failed the
word regexp as merged boxes and were never hyphenated; as separate
boxes they hyphenate normally.

Tests: 48 ERT green (split tests migrated to the new box contract;
new unit test for break permissions and a rendered-output kinsoku
sweep across widths); 300-case fuzz 0 failures with C 1.2 parity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 21:05:50 +08:00
Kinneyzhang
33981a647e docs: describe lossless renderer markers in developer docs
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 20:25:03 +08:00
Kinneyzhang
95ed2b32d1 feat: buffer-level justification — region commands and auto-justify mode
The renderer (shared by both engines) is now lossless: synthesized
glue carries the original text it replaced (ekp-glue), soft line
breaks carry the whitespace swallowed around the break
(ekp-soft-break), break hyphens are marked (ekp-soft-hyphen), and
paragraph-edge whitespace survives as zero-display ekp-hidden text.
Orphaned ekp--combine-glues-and-boxes / ekp--interleave removed.

New ekp-region.el:
- ekp-justify-region / ekp-unjustify-region: in-place justification
  with exact structural restore (character- and property-exact),
  robust to edits made while justified
- ekp-auto-justify-mode: keeps the buffer justified at the window
  width; debounced re-flow on window resize, incremental
  per-paragraph re-justification after edits (served by the
  paragraph cache)
- fix found in live GUI testing: buffer-local members of
  window-size-change-functions receive the WINDOW as argument and
  may run with an unrelated buffer current; the handler now resolves
  window and buffer explicitly (regression test included)

Tests: 47 ERT (36 core + 11 region) passing; 300-case fuzz 0
failures (C/elisp parity unchanged); byte-compile clean with
error-on-warn. Verified interactively in GUI Emacs: justified at
1403px maximized, auto-reflowed to 614px on frame resize.

docs: interactive-use section in readme.md / readme_zh.md
ci: byte-compile list includes ekp-region.el

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 20:23:03 +08:00
Kinneyzhang
d3972fdc26 ci: add GitHub Actions workflow
Two jobs on ubuntu-latest:
- elisp-only matrix (Emacs 29.1 floor + 30.1): byte-compile with
  error-on-warn, then the ERT suite (C-module tests auto-skip)
- C-module job (apt Emacs 29.3): build ekp.so, full ERT suite with
  C/elisp parity tests, then the 300-case property fuzz

Both paths were dry-run in a local ubuntu:24.04 container before
committing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 19:53:40 +08:00
Kinneyzhang
9afe491e1a chore: silence build warnings on non-macOS toolchains
- ekp_hyphen.c: check fgets return when skipping the dictionary
  encoding line (gcc -Wunused-result via glibc fortify); behavior
  unchanged — an empty file already fell through to EOF in the loops
- ekp-utils.el: declare-function for font-info, which does not exist
  in non-window-system builds (emacs-nox); call site is runtime-guarded
  by display-multi-font-p. Keeps byte-compile-error-on-warn green.

Verified: clean gcc build + 36/36 ERT + 300-case fuzz on Ubuntu 24.04
(Emacs 29.3, arm64), clean clang build + 36/36 ERT on macOS (30.2).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 19:53:39 +08:00
Kinneyzhang
112b3a0e52 refactor!: overhaul KP core — correctness, C parity, performance, tests, docs
- fix: ekp-param-set silently reset after first justify (now persists;
  ekp-param-reset added)
- fix: narrow-width CJK returned empty string (data loss); two-pass
  emergency-break strategy guarantees output for any input
- fix: K-P penalties never synced to C module; space-box metrics
  divergence between C and Elisp engines
- fix: para cache ignored ekp-latin-lang (stale hyphenation after
  language switch) and used collision-prone sxhash keys
- fix: fullwidth letters/digits misclassified as CJK punctuation
- fix: combining chars split from their base char in the tokenizer
- fix: punctuation-wrapped words (word!/(word)/word;) never hyphenated
- fix: renderer double-counted stripped space widths; negative glue
  clamped; batch/tty font detection no longer crashes
- feat: real looseness support via (position × line-count) DP
- perf: O(1) line metrics and gap counts via prefix arrays (inner loop
  previously allocated O(n) subsequences → O(n³) total); box measurement
  dedupe; eq fast-path para lookup; prebuilt per-para glue arrays
  → zh justify 7547ms → 96ms (compiled elisp) / 57ms (C);
    range-justify 68.5s → 0.48s / 34ms; C module itself 3–19× faster
- test: 36 batch-safe ERT tests + 300-case property fuzz (C/elisp
  byte-identical output, zero content loss) replacing ad-hoc suite
- docs: readme/readme_zh/DEVELOPER/DEVELOPER_ZH/ekp_c-README rewritten
  to match the implementation; phase handoff in .phrase/phases/

BREAKING: requires Emacs 29.1+; C module must be rebuilt (v1.1, new
arities); ekp-threshold-factor / ekp-flagged-penalty /
ekp-forced-break-penalty removed; Rust module stubs removed.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-26 18:44:02 +08:00
Geekinney
11437cb029
Merge pull request #10 from Kinneyzhang/copilot/fix-issue-9
fix: enforce kinsoku rules for CJK opening punctuation (Fixes #9)
2026-05-25 12:18:47 +08:00
copilot-swe-agent[bot]
7287bec474
chore: add *.elc to .gitignore and remove accidentally committed .elc
Agent-Logs-Url: https://github.com/Kinneyzhang/emacs-kp/sessions/d05069f4-db64-4601-9816-f582edad4a1b

Co-authored-by: Kinneyzhang <38454496+Kinneyzhang@users.noreply.github.com>
2026-05-24 15:23:13 +00:00
copilot-swe-agent[bot]
50d41ed30f
fix: enforce kinsoku rules for CJK opening punctuation in box splitting
Opening brackets ((, 「, 《, etc.) are now attached to the following
character instead of the preceding one, preventing them from appearing
at the end of a line which violates kinsoku typographic rules.

Fixes #9

Agent-Logs-Url: https://github.com/Kinneyzhang/emacs-kp/sessions/d05069f4-db64-4601-9816-f582edad4a1b

Co-authored-by: Kinneyzhang <38454496+Kinneyzhang@users.noreply.github.com>
2026-05-24 15:22:52 +00:00
Kinneyzhang
caf6576260 update 2026-02-20 11:52:54 +08:00
Geekinney
f7280df9c3
Merge pull request #8 from Kinneyzhang/copilot/analyze-repo-issues
refactor: eliminate code duplication and fix correctness bugs across Elisp and C modules
2026-02-20 11:44:32 +08:00
copilot-swe-agent[bot]
9b89115b5d fix: address code review feedback - improve docstrings and use defalias consistently
Co-authored-by: Kinneyzhang <38454496+Kinneyzhang@users.noreply.github.com>
2026-02-10 12:19:22 +00:00
copilot-swe-agent[bot]
af855e3394 refactor: Linus-style code quality improvements across Elisp and C modules
- Extract shared binary search from duplicate ekp--hyphenate-p/ekp--flagged-p
- Remove duplicate ekp-root-dir definition (shadowed ekp-utils.el version)
- Remove dead ekp-clear-caches from ekp-utils.el (cleared non-existent var)
- Replace hardcoded 'zsh' with shell-file-name for portability
- Deduplicate module reload: shared ekp--module-reload for Rust/C
- Add NULL guard for ekp_global in ekp_paragraph.c (crash prevention)
- Track actual thread count in pool to fix destroy joining wrong count
- Normalize line endings (CRLF→LF) in ekp-utils.el

Co-authored-by: Kinneyzhang <38454496+Kinneyzhang@users.noreply.github.com>
2026-02-10 12:18:00 +00:00
copilot-swe-agent[bot]
ddfa26fafc Initial plan 2026-02-10 12:11:27 +00:00
Kinneyzhang
262b5eefdc Makefile supports multiple plantform 2026-01-26 11:51:19 +08:00
Kinneyzhang
136e53fe8e improve C module 2026-01-26 01:24:47 +08:00
Kinneyzhang
f42c2bda00 delete unnecessary code 2026-01-25 21:32:34 +08:00
copilot-swe-agent[bot]
6d65902933 Fix: preserve all leading spaces in paragraph indentation
Co-authored-by: Kinneyzhang <38454496+Kinneyzhang@users.noreply.github.com>
2026-01-25 13:23:00 +00:00
copilot-swe-agent[bot]
5e3571f689 Fix: preserve paragraph indentation, strip only line-break artifact spaces
Co-authored-by: Kinneyzhang <38454496+Kinneyzhang@users.noreply.github.com>
2026-01-25 13:09:59 +00:00
copilot-swe-agent[bot]
8e39dc2b83 Fix: DP line metrics now excludes leading/trailing space box widths to match rendering
Co-authored-by: Kinneyzhang <38454496+Kinneyzhang@users.noreply.github.com>
2026-01-25 12:55:58 +00:00
copilot-swe-agent[bot]
c60ddb3903 Fix: DP algorithm now excludes leading/trailing space box widths to match rendering behavior
Co-authored-by: Kinneyzhang <38454496+Kinneyzhang@users.noreply.github.com>
2026-01-25 12:41:04 +00:00