Skip to content

M12: add local Read out loud on Windows, macOS and Linux - #1476

Open
robertobulltech wants to merge 8 commits into
storytold:mainfrom
robertobulltech:codex/read-aloud
Open

robertobulltech wants to merge 8 commits into
storytold:mainfrom
robertobulltech:codex/read-aloud

Conversation

@robertobulltech

@robertobulltech robertobulltech commented Oct 12, 2026 •

Copy link
Copy Markdown

Main integration — 2026-10-17

Integrated upstream main at 924c4da4 in 91bc67e3, preserving the existing feature scope and branch history.

Local validation passed: formatting; strict all-target Clippy on affected crates; relevant library/integration tests; layering, asset attribution and parity gates; CLI without default features; dependency licence/advisory checks; and all 31 library crates on WASM with the pinned craft-fonts input. Native builds/tests use Rust 1.99, the low-disk profile, two build jobs and isolated Cargo targets. Existing ignored hardware/measurement probes remain explicit in the logs.

Preserved main’s automation read/write roots, accessibility additions, independent PDF oracle requirements and portable RPM dependency names alongside speech support. Added the ten missing Vietnamese labels and refreshed attribution hashes. Workflow actionlint and ALSA packaging YAML checks also passed.

Full-workspace Linux/Windows CI for this exact commit: https://github.com/robertobulltech/pdfcraft/actions/runs/39020796825. The run is still in progress at publication; workflow dispatch also starts the additional nightly mutation-fuzz job.

The earlier validation history and feature limitations below remain for context.


Read extracted PDF text aloud through local voices on Windows, macOS and Linux. The dialog and CLI/MCP tools expose page/range, installed voice, speed, Pause/Resume and Stop. English and Italian voice selection is independent of interface language; no automatic document-language detection is claimed.

Windows uses safe WinRT synthesis and MediaPlayer. macOS uses the system say executable; Linux requires espeak-ng. Unix synthesis produces bounded WAV audio in an owner-private temporary directory, followed by Rodio/CPAL playback with same-position pause/resume. The permissively licensed Hound decoder is selected explicitly; no copyleft speech-engine library, models, shell, cloud transport or runtime download is added. Text goes through stdin, embedded speech-command delimiters are escaped, and temporary text/audio is removed before playback or on cancellation/error.

The source PDF is immutable; preparation is off the UI thread and limited to 64 pages, 1 MiB and 4,096 Unicode-safe chunks. Closing, switching or changing the source stops speech. Automation includes prepare/start/control/status/voices/wait; CLI scripts can wait for audio completion. The voice/speed/dialog labels are localized in all 21 existing translation catalogs, with attribution hashes regenerated. Linux build/CI and deb/rpm audio prerequisites are updated.

Validation:

  • Earlier Windows full-workspace verification (before the latest upstream integration): 3,031 passed, 0 failed, 17 opt-in/environment-dependent tests ignored, with pinned craft-fonts. Strict all-targets Clippy, fmt and 31-crate wasm checks passed on Windows.
  • Asset attribution, layering, parity and cargo-deny checks.
  • English/Italian voice parsing (including names with spaces and regional English voices), UTF-8 stdin, bounded subprocess failure/timeout/cancellation and private-data cleanup.
  • Real Rodio sink consumed silently in memory: pause freezes position, resume advances, Stop empties playback.
  • GUI tests select an English voice under the Italian interface and an Italian voice under the English interface. CLI/MCP tests pass the selected voice and rate unchanged.
  • Previous real Windows muted playback check and synthetic CLI/MCP playback smoke checks passed.
  • New contributor-fork native workflow checks platform compilation/lint/tests and native English synthesis on Windows/macOS/Linux; native Italian synthesis on Linux and on macOS when the runner has that voice. Native workflow PASSED on Windows, macOS and Linux at commit ba1c212: https://github.com/robertobulltech/pdfcraft/actions/runs/38278639802. macOS/Linux both generated and decoded non-silent English AND Italian WAVs at 0.5x, 1x and 2x; Windows generated native English audio at those three rates. Platform Clippy/tests passed on all three. Unix owner-private 0700 directory and 0600 input-file permissions were checked natively. Audio handles remain on their creating speech worker; only injected backends require Send, respecting CoreAudio thread affinity without unsafe wrappers.

Physical Linux validation (2026-10-13):

  • Source 50cf2ce was built and tested on a physical CachyOS / KDE Plasma Wayland desktop with Rust 1.99.0, eSpeak NG 1.52.0, PipeWire 1.6.9 and USB Audio Speakers. Platform tests: 22 passed, 0 failed, 4 opt-in tests initially ignored. All three opt-in Unix speech checks were then explicitly run and passed: non-silent English/Italian WAV synthesis at 0.5x, 1x and 2x, and muted playback on the real audio device with position-preserving pause/resume and resource release on Stop. All three read-aloud UI regressions passed; strict platform Clippy and fmt passed.
  • Both audible English/Italian probes completed. The listener confirmed that both languages played correctly with pause and resume. Voice naturalness was reported poor; these results validate functionality, not speech quality.
  • Real CLI scripts and persistent compact MCP sessions read synthetic PDFs in English and Italian to completion. MCP exercised pause/resume/Stop and document-close cancellation. The native wgpu GUI selected an English voice under the Italian interface and an Italian voice under the English interface, with working pause/resume/Stop and cancellation on source switching/closing. PDF hashes stayed unchanged, neither GUI document became dirty, and CLI documents closed without discarding changes.
  • After integration with upstream b5467d93, the published merge at 30f4879 passed 320 targeted tests: 8 native-opening regressions, 299 affected UI tests, 10 batch tests and 3 speech-automation tests. Four opt-in UI checks remained ignored. Strict all-target Clippy on the affected UI/automation crates, fmt, layering, asset attribution, parity and all 31 Wasm crate checks passed. The Wasm check used a temporary, checksum-verified Rust target installation without changing the system toolchain.
  • Detailed physical Linux evidence and reproduction commands: https://github.com/robertobulltech/pdfcraft/blob/30f4879a09bd701b9886c0e428a815e3532ce513/docs/read-aloud.md#physical-linux-validation-2026-10-13

Remaining validation and limits:

  • Physical macOS desktop playback and audible English/Italian listening remain untested. Linux physical desktop playback was verified as described above; packaged/sandboxed Linux builds still need separate playback validation. Runner WAV synthesis plus silent sink tests alone do not prove speaker/device integration.
  • Linux eSpeak NG voice naturalness remains a limitation: the listener described the voice quality as very poor despite correct playback and controls.
  • This Windows PC has only Italian WinRT voices; its real muted control check uses those. Native English synthesis was verified separately on the Windows runner. The selected language must have an installed voice.
  • Linux needs an available espeak-ng executable and default audio output. In a sandboxed package the executable must be accessible inside the sandbox; host installation alone may not suffice. Shared release builders need ALSA development files.
  • Temporary-file deletion is not secure erasure; forced process termination can leave files in the private temporary directory for OS cleanup.
  • Parity remains partial: extracted-page ordering instead of semantic tag-tree order; scans need OCR; no sentence highlighting, automatic page navigation, pitch, persistent options, FreeBSD or web audio.

Integrated with upstream b5467d93 (0.7.1) in merge commit 30f4879. GitHub reports the PR as mergeable without conflicts; the post-integration verification scope is listed above. Conflict resolution preserves the bounded session-wide text cache, auto-tag Apply, folder batching and both sets of translations/documentation. Upstream native-opening improvements are retained: chosen tools survive preparation, conversion and password retries, startup intake waits for pending preparation, and shared state-based waits keep the stronger document-identity assertions. No release or upstream merge is performed by this PR.

Use system say and external espeak-ng synthesis with private temporary WAV
staging and safe playback. Keep speech voice independent from interface
language and cover English/Italian selection, bounded subprocesses and
same-position pause/resume. Add opt-in native synthesis checks and document
physical audio validation still required.

Generated with Codex for @robertobulltech.
@robertobulltech robertobulltech changed the title M12: add local Windows Read out loud with CLI/MCP controls M12: add local Read out loud on Windows, macOS and Linux Oct 12, 2026
Require Send only when an injected backend crosses into the worker, so
CoreAudio streams can stay thread-bound without unsafe wrappers. Keep
native backend factories before the test module for cross-platform Clippy.
Avoid a premature SIGPIPE when detecting an installed Italian macOS voice.

Generated with Codex for @robertobulltech.
Set 0700 atomically when creating temporary speech folders instead of
relying on tempfile defaults or the user's umask. The native permission
regression caught the issue before desktop playback validation.

Generated with Codex for @robertobulltech.
robertobulltech and others added 3 commits October 13, 2026 15:09
Merge current upstream main while retaining bounded session-wide text
caching, auto-tag Apply, folder batching, title-bar and link-consent
translations alongside read-aloud controls. Regenerate translation hashes
and preserve both sets of roadmap progress.

Preserve deferred tool commands across asynchronous opening and delay
startup downloads until pending opens finish. Existing UI regressions wait
for background completion while retaining their original assertions.

Generated with Codex for @robertobulltech.
Integrate upstream b5467d9 native-opening improvements while preserving speech controls and physical Linux validation.

Validation: 320 targeted tests passed, 4 opt-in UI checks ignored. Strict all-target Clippy, fmt, layering, assets, parity and all 31 Wasm crate checks passed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant