etaf/scripts

Generic Emacs GUI verification

For an already running Emacs, use its existing server through emacsclient. Load the checkout and call the example entry point in an explicit buffer, show that buffer in the existing graphical frame, and capture only that owned window. Preserve the user's font and chrome. Do not start another daemon or frame for this workflow. Preserve application focus so the user can keep working in other apps. The emacsclient-render-capture skill supplies buffer selection without activation and before/after target guards.

This directory also owns reusable verification mechanisms; application scenarios live in their respective example repositories:

• emacs-gui-verifier.el defines Scenario, ordered Action, run-local Context, checkpoint sequencing, assertions, completion, and fail-closed evidence finalization. Its etaf-gui-verifier-measure-action entry records callback wall/CPU/GC time and GUI conditions without taking application focus.
• benchmark-ebox-resize.el drives continuous pixel-width sweeps of any existing mounted Ebox buffer, including ETAF apps, without taking over its observer. See the existing-server entry below.
• The legacy isolated runner run-emacs-gui-verification.sh owns one named daemon, explicit load paths, external application activation, recorder lifecycle, driver loading, report generation, and exact cleanup.
• record-emacs-window.swift uses macOS 15 ScreenCaptureKit to record only the frame owned by the supplied Emacs PID. It follows replacement window IDs and rejects changed pixel mappings. Its desktop-independent window filter works without activating Emacs. Its canvas is fixed at recording start: when the scenario includes resize, prepare the largest tested frame before starting the recorder. Smaller windows retain native pixels with padding; a window larger than the original canvas invalidates the recording.
• capture-emacs-window.sh supplies window-only checkpoint screenshots and rejects missing, ambiguous, or changing frame identities without activating Emacs. The runner compiles the video helper using the system Swift compiler; no package install is needed.

Concrete repositories provide adapter files that construct a Scenario and an entry function. They can run through the existing server. The following command instead starts the legacy isolated environment; use it only when that separate environment is explicitly intended:

scripts/run-emacs-gui-verification.sh run ADAPTER.el ENTRY \
  --load-path /path/to/provider \
  --run-dir /private/tmp/my-gui-run

In that isolated runner, fresh captures remain INCOMPLETE until their selected images and contact sheet are reviewed. Finalize the same evidence directory with:

scripts/run-emacs-gui-verification.sh review /private/tmp/my-gui-run

Only VERDICT=PASS completes that runner's evidence bundle. An existing-server run reports its actual interaction assertions and inspected screenshots separately. A failed recorder is never evidence of continuous capture, and neither screenshots nor recording establish an operation latency bound.

For an existing server, take a background screenshot with the observed process ID:

ETAF_GUI_CAPTURE_PID=12345 scripts/capture-emacs-window.sh -x /tmp/app-checkpoint.png

Keep the target frame open and non-minimized. Validate a known content change in before/after pixels to establish fresh background rendering. Hidden, minimized, off-Space and locked-session behavior is not covered by this procedure. Background Lisp callbacks and runtime timing do not require application activation; report them as background measurements, separately from foreground input-to-display latency. Use Emacs-local operations while another app is active, not global input.

Measuring an existing GUI action

Reuse etaf-gui-verifier-measure-action with the adapter's Context and Action:

;; Prepare the target control and fixture before entering the measured callback.
(etaf-gui-verifier-measure-action context action t)
(car (etaf-gui-verifier-context-measurements context))

The final t requires an already selected foreground target. It rejects a background, hidden, unknown or wrong target before running the action, and invalidates a sample if focus, target, font or geometry changes. It never raises Emacs. Omit that argument for background functional work; those samples are explicitly labeled and must not be reported as foreground latency.

Timing covers exactly Action.execute, excluding target inspection, checkpoints and the verifier's intentional settle waits. Keep fixture setup, control lookup and focus navigation outside an activation measurement. If the chosen boundary includes an explicit redisplay, put that call in execute and describe the boundary accordingly. Returning from redisplay still does not prove compositor presentation. Each record says presentation_measured=nil.

Context retains every measurement, including first use, warmups, failures and rejections. Failed callbacks retain their own timing and original condition; rejections have no duration. Checkpoints export only the most recent record. Report the full sample count and all-sample maximum as well as any explicitly labeled steady-state statistics. Compare identical payload/page size, viewport, loaded code, recording mode and foreground class; do not discard a slow group because another run is faster. The historical .omx/component-api-implementation/ current-workbench-sample.el probe is superseded by this measurement boundary.

For provider traces, scope etaf-performance-records by its numeric observer runtime ID, not buffer name, which can be reused after unmount. Pass the selected records explicitly to summary/export functions: an empty selection stays empty. Check operation IDs/counts as well as runtime ID when sampling the same mounted instance again; the recorder's bounded history is not a per-run sample store.

Continuous resize on the existing server

Use this entry for resize performance investigations instead of copying the historical .omx/component-api-implementation/*continuous-resize*.el probes. First show the intended buffer in the user's existing foreground GUI frame. Preserve its font and chrome. Compile changed product modules before loading them; make compile in this repository also compiles the benchmark tool.

emacsclient --eval '(progn
  (load "/path/to/etaf/scripts/benchmark-ebox-resize.elc" nil t t)
  (ebox-resize-benchmark-start "*Your app*" 480 1400
    :step 32 :rounds 3 :output "/tmp/resize-run-1.eldata"))'

The bounds are outer-frame pixels. Every round goes from narrow to wide and back, including both endpoints even when the step does not divide the range. The last measured request returns to the initial width. Consecutive equal widths are omitted because they request no resize; no warmup or slow sample is dropped. The function returns immediately. Poll or cancel this owned run with:

(ebox-resize-benchmark-status)
(ebox-resize-benchmark-cancel)

:delay (default 0.01 seconds) runs between completed publications; :timeout (default 5 seconds) bounds startup, publication and final settling. The tool waits for a preceding render's GC lease to end before recording the baseline, and for its own run's lease to end before accepting the final result. Unsupported sizes, changed targets/focus/font/height, publication failures and cancellation produce invalid/incomplete evidence. Advice and timers are removed on success, error, quit and timeout. Failure leaves the last applied width; it does not fight a user's subsequent window changes. No server, frame, application buffer, observer, profiler or GC policy is created/replaced to make the benchmark pass.

Use a new output path each time. UTF-8 .eldata reports retain all requests, all completed samples, actual published dimensions, node/text counts, GC, per-operation CPU time, code origins and hashes. Optional :phases '(FUNCTIONS…) records inclusive phase durations for loaded ebox-, tp- and etaf- provider functions. The driver and outer publication boundary cannot be selected as phases; nested durations must not be summed. Publication timing ends when the real public Ebox viewport call returns. Request→publication also includes event-loop delivery. Neither metric proves compositor presentation, absence of flicker, or every interaction's correctness; use the existing visual scenario/capture tools for those claims.

:valid t requires complete correctly attributed publication coverage. :within-limit t additionally requires every request→publication sample to meet :limit-ms (default 50). Inspect p95 and max, including GC samples. Compare payload counts at matching widths before making speedup claims. A different page size/content or loaded version is a different baseline.

Code origins and disk hashes alone cannot establish when a running function was loaded. After recompilation, :reload-files '("/checkout/ebox/ebox-viewport.elc" …) explicitly loads the chosen fresh bytecode before instrumentation. :code contains every explicitly loaded file's canonical path, source/bytecode hashes and load result, plus observed core function and benchmark driver origins; matching origins are marked :reloaded t. Choose the affected modules; do not reload application component declarations blindly. Without this option, report code origin observations separately from independent evidence of a fresh GUI load.

Run the tool's regression checks with make resize-benchmark-tests; they also run in the full make check gate. Application-specific data setup and interaction assertions remain in the example repositories, not this resize driver.