Compare commits

...
267 Commits
Author SHA1 Message Date
Dylan Byars 973a902332 fix: duplicate User-Agent header on wolfSSL requests breaks strict servers (aiohttp 400) (#2661) 2026-07-21 01:48:26 -04:00
Justin Mitchell cb9b971c26 Refactor SCOPE.md for improved formatting
Updated SCOPE.md to remove unnecessary line breaks and improve readability.
2026-07-21 01:20:26 -04:00
Justin MitchellandJulia Nguyen f42fab1c66 feat: Add touch coordinate mapping and RTOS task yielding (#2481)
Co-authored-by: Julia Nguyen <julia@uxj.io>
2026-07-20 16:31:07 -04:00
Justin Mitchell c9188a7347 chore: Clarify project scope and development priorities (#2149)
Expand scope to support multiple e-ink devices beyond X4. Add guiding
principle to fill gaps in stock firmware rather than duplicate
functionality. Establish current focus on memory/flash optimization and
code cleanup. Temporarily freeze new themes and network connectors to
consolidate codebase before multi-device expansion.
2026-07-20 16:02:02 -04:00
Leopoldo Pla Sempere e03aa16330 fix: select strongest AP for matching WiFi SSID (#2655) 2026-07-20 18:17:16 +03:00
Phạm Bình An 3319aa1720 fix(epub): preserve word continuation when splitting CJK text on MAX_WORD_SIZE (#2652) 2026-07-20 15:02:08 +03:00
CookieCaptainD 3dfbc0383f fix(i18n): add missing strings in PT-PT translation (#2632) 2026-07-20 14:12:06 +03:00
Thiago Kenji Okada 64364bec5d feat: add Nix development shell (#2645) 2026-07-20 00:57:37 +03:00
Stefan Blixten Karlsson 556b8aee5b fix: swedish translation (#2649) 2026-07-19 22:12:10 +03:00
b1d1d757ba feat: Slim dictionary (#2583)
Co-authored-by: Ryan Hitchman <hitchmanr@gmail.com>
Co-authored-by: Kurtis Grant <kurtis.b.grant@gmail.com>
Co-authored-by: kemonine <kemonine@kemonine.info>
Co-authored-by: DustinHu <hu.dustin@gmail.com>
Co-authored-by: Justin Mitchell <justin@jmitch.com>
2026-07-19 17:57:16 +03:00
Uri Tauber 5ba1d5747f fix: EndOfBookOptions fails to compile
ActivityManager.h forward-declares Activity but holds
std::unique_ptr<Activity> members. Instantiating that unique_ptr's
destructor requires the complete type, so any TU including
ActivityManager.h without Activity.h fails to compile.

Include Activity.h directly. Adding it to ActivityManager.h instead does
not work: Activity.h depends on HomeMenuItem, which ActivityManager.h
defines.
2026-07-19 16:45:42 +03:00
Thomas Symalla 9fe4dc5e38 feat: Add option to switch behavior for "back to browser / home" in Reader activity (#2366)
## Summary

It would be nice to switch back to the file list from an Reader activity
via a short back button press. This change adds an Reader option to
switch the default behavior, so a short back button press in the Reader
activity can now go back to the file list, and a long press on back goes
back to the home view. This does a fair bit of refactoring, introducing
a new constant for the ms limit.

* **What changes are included?**

- Changes to the translation
- Additional global Reader option 
- Refactoring of the back button behavior in the Reader activity
2026-07-19 08:29:13 +03:00
Justin Mitchell b1d037569b feat: Add kosync user registration and switch to crosspoint-sync server (#2587)
Implements createUser() endpoint to allow account creation via the
KOSync protocol. Changes default sync server from sync.koreader.rocks to
sync.crosspointreader.com with migration logic to preserve existing
users' server settings. Extends sync protocol to include position data
(spine index, page numbers, xpath) that crosspoint-sync supports while
remaining compatible with standard kosync servers.
2026-07-18 14:50:51 -04:00
Julia 9737cb335c fix: correct the settings enums for "blank" and "cover + custom" sleep screens (#2635) 2026-07-17 12:35:51 -04:00
Phạm Bình An fdffc2e5d9 fix: reduce CSS parse-time OOM risk in chapter layout (#2606) 2026-07-16 07:21:08 +03:00
a2db43d235 feat: enable CORS headers in the HTTP API (#2594)
Closes #2558.

Enables the Arduino WebServer's built-in CORS support
(`enableCORS(true)`), which adds
`Access-Control-Allow-Origin/Methods/Headers: *` to every response, and
answers preflight `OPTIONS` requests with `204` in `handleNotFound()` —
routes are registered per-method, so OPTIONS always lands there. The
AP-mode captive-portal redirect is untouched (the OPTIONS check runs
before it, and browsers don't send preflights for captive-portal
probes).

This lets web-based clients and PWAs served from other origins call the
JSON API (`/api/status`, `/api/files`, `/api/settings`, ...) directly
from the browser.

Overhead is three static response headers; no behavior change for the
built-in web UI.

Note: not yet tested on hardware.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: metoli <metoli@metoli-Mac-mini.local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-15 18:35:10 -04:00
rxmmahandkira 63093e606c feat: Back on home menu opens the most recent book (#2619)
Co-authored-by: kira <rammah@tuta.io>
2026-07-15 23:50:22 +03:00
a4ac3b2788 feat: add smart KOReader progress sync (#2192)
## Summary

Make KOReader progress sync a one-click flow for common cases while
keeping the existing manual mode available.

### Changes

- Add a **Sync Behavior** setting:
  - **Smart sync** for new configurations
  - **Ask every time** for manual control
- Preserve **Ask every time** when migrating an existing credential file
that predates this setting.
- In Smart mode:
  - upload local progress when no remote record exists;
  - show a short confirmation when already synced;
  - upload when local progress is further ahead;
  - apply remote progress when remote progress is further ahead.
- Probe both KOReader document-matching hashes before making the Smart
decision, while keeping uploads on the user's configured matching
method.
- Auto-return after successful Smart terminal states, with Back/Confirm
still available.
- Persist the setting alongside the current credential, matching-method,
and send-metadata fields.
- Document both behaviors in the user guide.

## Additional context

This pairs well with #2189 (smart Wi-Fi auto-connect), but does not
depend on it. It does not add background Wi-Fi or passive network
detection; sync is still triggered by the user from the reader menu.

Smart sync uses the furthest progress because CrossPoint does not
currently persist local progress timestamps. Users who prefer explicit
conflict resolution can select **Ask every time**.

## Verification

- `git diff --check`
- `platformio check --fail-on-defect low --fail-on-defect medium
--fail-on-defect high` — no defects
- `platformio run -e default` — firmware build successful
- The original implementation was manually smoke-tested on an X4 device.

---

### AI Usage

Did you use AI tools to help write this code? _**YES**_

AI tools were used to inspect the codebase, draft and review the
implementation, resolve the rebase against current `develop`, and
prepare the PR text. The resulting firmware was built locally.

---------

Co-authored-by: Alexander Hoffer <git@alexanderhoffer.com>
Co-authored-by: Alexander Hoffer <contact@alexanderhoffer.com>
2026-07-15 14:59:15 -04:00
Víctor Fernández 35a45f9c1e feat: add options to remember web upload settings & rename ebooks to {title} - {author} (#2534) 2026-07-15 14:43:42 -04:00
William Floyd f3cad2f9e1 fix(kosync): reload Epub before reading upload metadata (#2608) 2026-07-15 11:46:10 -04:00
Oscar Nogueira Neto 95d8cb712b feat: configurable OPDS download folder and filename format (#2571) 2026-07-15 10:53:27 -04:00
Mirza Arnaut b9867b0d09 feat(i18n): add Bosnian translation (#2616) 2026-07-15 10:49:42 -04:00
Uri Tauber 3dac4446b7 fix: remove duplicate sleep logic (#2492) 2026-07-15 10:48:08 -04:00
Uri Tauber f180069643 fix: ignore open-x4-sdk and fs_ (#2609) 2026-07-14 21:34:28 +03:00
Husam Younis a0ec5812b1 feat: Arabic translation YAML - PR 3/3 (#2599) 2026-07-14 20:33:42 +03:00
Phạm Bình An e84840b473 fix(css-parser): don't save unusable rules to RAM (#2604) 2026-07-14 17:00:13 +03:00
Tom-Inge Larsen 94ae750e89 feat(i18n): add Norwegian Bokmål translation (#2113) 2026-07-14 16:54:43 +03:00
metoli86andmetoli 8c5cd67f38 feat: complete Turkish translation (390/390) (#2592)
Co-authored-by: metoli <metoli@metoli-Mac-mini.local>
2026-07-14 16:50:03 +03:00
Justin Mitchell c5787d1cc1 fix: Use HALF_REFRESH for sleep and boot instead of FULL (#2588)
Changes all sleep screen display refreshes and boot screens from
FULL_REFRESH to HALF_REFRESH to match OEM firmware behavior. The stock
firmware uses a single-pass 0xD7 waveform for sleep screens, not the
multi-flash GC waveform (0xF7) that FULL_REFRESH triggers. For grayscale
mode, HALF_REFRESH is required because the gray nudge LUT is calibrated
against the pixel charge state left by the single-pass waveform; using
FULL_REFRESH causes blotchy noise in gray areas. Also adds X3 wb_gc
passive byte (the one-byte 0x54 to 0x00, matching stock's passive WB
table in the gc nudge bank): Every X3 grayscale render, reader AA,
covers, and the sleep-cover nudge, gets it.
2026-07-13 17:06:09 -04:00
Husam Younis 2a19eb9e0d feat: Arabic/Farsi/Urdu glyphs in built-in UI fonts - PR 2/3 (#2596) 2026-07-13 22:35:47 +03:00
Uri Tauber fae3423dec feat: add portuguese-PT.yaml (#2597) 2026-07-13 22:27:23 +03:00
Husam YounisandUri Tauber 932a472835 feat: Arabic/Farsi/Urdu bidi reordering and contextual shaping — PR 1/3 (#2541)
Co-authored-by: Uri Tauber <uritaube@gmail.com>
2026-07-13 16:02:21 +03:00
Jan Steinke 552b2683e6 fix: keep list item bullet inline with nested paragraph text (#2589) 2026-07-13 12:06:43 +03:00
Mr.Catfood bfa1d706b1 feat: Hidden wifi ssid support (#2360) 2026-07-13 02:33:48 -04:00
Jacob Latonis 73ab9ab20d feat: implement captive portal redirects for auto-loading the management page on hotspot mode (#2550) 2026-07-13 00:10:23 -04:00
Nick Perez 3a97f3e63e feat: Send optional document metadata with KOSync progress uploads (#1820) 2026-07-12 23:50:38 -04:00
Matthías Páll Gissurarson 39ea4b045f feat: render placeholders while waiting for images to render (#1003) 2026-07-12 21:40:07 +03:00
Justin Mitchell 444d87de82 fix: Add framebuffer release/realloc and improved lazy indexing (#2563) 2026-07-12 13:16:48 -04:00
Leopoldo Pla Sempere 859f6cb0d5 chore: update Spanish, Catalan, and Valencian Wi-Fi strings (#2578) 2026-07-12 12:13:06 +03:00
hooligan333andErica Jensen d8139e9605 perf: release CSS rule map after warm open (#2439)
Co-authored-by: Erica Jensen <erica@mailershaven.com>
2026-07-11 22:59:31 -04:00
hooligan333andErica Jensen b596f21e61 perf: stream NCX/NAV TOC into parser, drop temp-file round-trip (#2440)
Co-authored-by: Erica Jensen <erica@mailershaven.com>
2026-07-11 22:57:48 -04:00
hooligan333andErica Jensen 5fbc657aeb perf: always binary-search idref lookups in content.opf (#2433)
Co-authored-by: Erica Jensen <erica@mailershaven.com>
2026-07-11 22:56:28 -04:00
Jason Huebel bb06219c9c docs: add script to generate EPUB from USER_GUIDE.md (#2152) 2026-07-11 22:54:32 -04:00
Alexander Hoff ❍andAlexander Hoffer 5202bbf911 feat: auto-connect saved Wi-Fi networks (#2189)
Co-authored-by: Alexander Hoffer <git@alexanderhoffer.com>
2026-07-11 22:47:26 -04:00
hooligan333andErica Jensen e0253a8664 perf: reserve CSS rule map before loading from cache (#2435)
Co-authored-by: Erica Jensen <erica@mailershaven.com>
2026-07-11 22:24:43 -04:00
hooligan333andErica Jensen e142e88ce7 perf: drop per-image delay(50) on chapter build, retry getDimensions (#2434)
Co-authored-by: Erica Jensen <erica@mailershaven.com>
2026-07-11 22:23:52 -04:00
Timo 6add0b91bc feat: Add Finnish hyphenation (#2084) 2026-07-11 22:16:42 -04:00
Leopoldo Pla Sempere 287457f7dd fix: handle low-bit-depth, upscaled, and SVG EPUB images (#2503) 2026-07-11 21:45:53 -04:00
Justin Mitchell 3e627112f6 fix: oom exceptions for OPDS, KOSync, and OTA via wolfssl (#2475) 2026-07-12 01:53:26 +03:00
Stefan Blixten Karlsson 1f5669a08a fix: Swedish translation (#2577) 2026-07-11 23:20:20 +03:00
Ing. Jan Kaláb 0dc92c142c fix: update czech.yaml (#2574) 2026-07-11 21:09:09 +03:00
Leopoldo Pla Sempere 4b34a576eb chore: update Spanish, Catalan, and Valencian translations (#2566) 2026-07-10 17:45:40 +03:00
4fb843ef92 perf: skip redundant progress writes when position is unchanged (#2436)
Co-authored-by: Erica Jensen <erica@mailershaven.com>
Co-authored-by: Uri Tauber <uritaube@gmail.com>
2026-07-10 10:37:43 +03:00
Matteo Scopel a43061305a chore: update the Italian translation (#2559) 2026-07-08 22:22:47 +03:00
Uri Tauber 7ec07b4d9d fix: show "Failed to index" when failing to parse epub (#2556) 2026-07-08 21:22:35 +03:00
Uri Tauber c2c1badc7e chore: Refactor stores to use PersistableStore CRTP template (#2464) 2026-07-08 12:44:52 +03:00
Nick 5776911af0 fix: FontDecompressor OOM aborts on the render path (-fno-exceptions makes vector resize fatal) (#2526) 2026-07-07 09:06:18 +03:00
Pietro Campagnano 3c5c5e8aa7 feat: preview image files inline in web file browser (#2429) 2026-07-06 16:45:22 -04:00
Justin Mitchell 6f5c5a0900 fix: Flatten TextBlock word storage into single allocation (#2547) 2026-07-06 23:41:00 +03:00
Uri Tauber f2b00f6b93 chore: Replace product link with affiliate tracking link (#2401) 2026-07-06 23:13:27 +03:00
Uri TauberandBrooks Ilg 44ff313740 fix: render <br> between paragraphs as a visible section break (#2548)
Co-authored-by: Brooks Ilg <brooksmilg@gmail.com>
2026-07-06 20:10:31 +03:00
Jacob Latonis ca1b833126 fix: follow spec for zxing qr code generation (#2540) 2026-07-06 00:33:34 +03:00
Tom-Inge Larsen 3e1dd31e53 feat(reader): End of Book next-book suggestions (#2499) (#2532) 2026-07-04 23:12:59 +03:00
685d4e88f9 feat: Lazy incremental EPUB section indexing (#2452)
Co-authored-by: Uri Tauber <uritaube@gmail.com>
Co-authored-by: Julia Nguyen <julia@uxj.io>
2026-07-04 21:21:24 +03:00
Jakub Sypiański 79b4d63ee2 fix: settings persist on font clear, reserve() before push_back, minor (#2519) 2026-07-03 18:58:24 +03:00
Leopoldo Pla SempereandJiangoJ 57447a56a6 feat(epub): improve text-decoration support (#2397)
Co-authored-by: JiangoJ <jiangj2620@gmail.com>
2026-07-02 12:35:24 +03:00
Rahat Ahmed 5c86bfe1fd feat(network): add device serial number to web UI (#2506) 2026-07-02 00:53:55 +03:00
Justin Mitchell 4cdefb137c Update README with firmware flashing warnings and crosspet status
Removed warning about flashing Papyrix on USB-locked devices and updated crosspet status to unmaintained.
2026-07-01 14:22:27 -04:00
Justin Mitchell e7f3406704 fix: Add socket module import to build-sd-fonts script (#2504) (#2505) 2026-07-01 12:10:15 -04:00
Bastian 6f08a88b12 fix: changing translation in czech to fix overflowing navigation (#2502) 2026-07-01 11:58:24 -04:00
Tom-Inge LarsenandClaude Opus 4.8 e4d3981469 fix(reader): correct slider side-button direction and legend on X3 (#2402) (#2428)
## Summary

* **What is the goal of this PR?** Fixes #2402. On the X3, the "Time to
Sleep"
picker's 5-minute side buttons were inverted (left increased, right
decreased)
  and the on-screen legend didn't match the physical buttons.
* **What changes are included?**
  * Flip the large-step (±5 min) direction on X3 so the left side button
decreases and the right increases, matching the layout. X4 is unchanged.
* Add a device-specific step-hint string
(`STR_SLEEP_TIMER_STEP_HINT_X3`)
selected via `gpio.deviceIsX3()`. X3 shows `+/-: 1 min Side buttons: 5
min`;
    X4 keeps the original `Left/Right: 1 min  Up/Down: 5 min`.
* Same fix applied to both slider pickers that read the raw up/down side
    buttons: the Time-to-Sleep picker and the "Go to %" picker
(`EpubReaderPercentSelectionActivity`), each with its own X3 hint string
    (`STR_SLEEP_TIMER_STEP_HINT_X3`, `STR_PERCENT_STEP_HINT_X3`).

## Additional Context

* **Root cause:** the X3's side buttons sit one on each edge of the
screen
(power on top), whereas the X4 has a vertical up/down rocker on the
right
edge. So `BTN_UP` is physically the *left* button on X3 but the *top*
button
  on X4. The picker mapped up→+5 / down→−5 unconditionally, which reads
naturally on the X4 rocker but inverted on the X3's left/right buttons.
The
  static legend ("Up/Down: 5 min") had the same X3-only mismatch.
* The codebase already detects the device at runtime and handles this
layout
difference elsewhere (e.g. `LyraTheme::drawSideButtonHints` draws "Up on
left,
Down on right" for X3), so this reuses the same `gpio.deviceIsX3()`
signal.
* Swept the codebase for other side-button sliders:
`ClockOffsetActivity` already
uses the `Next/Previous` (`ButtonNavigator`) abstraction, which resolves
to
left=decrement / right=increment on X3, so it needs no change. List/page
navigation (Wi-Fi, KOReader sync, BMP viewer) and the keyboard cursor
are also
unaffected. The two slider pickers above were the only ones using raw
up/down.
* **X4 is untouched** — same direction, same legend wording as before.
* **Tested on X3 hardware:** left side button now decreases, right
increases, and
  the legend matches. X4 not yet tested on device (no unit on hand); its
  behaviour and strings are unchanged from master.
* **Translations:** the new `STR_SLEEP_TIMER_STEP_HINT_X3` and
`STR_PERCENT_STEP_HINT_X3` were added to all 26 languages, but the
non-English
  ones are AI-generated and would benefit from a native-speaker pass —
particularly Hebrew (RTL ordering with a leading `+/-`), Kazakh, and
Vietnamese.

---

### AI Usage

Did you use AI tools to help write this code? _**YES**_

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 11:06:31 -04:00
Bastian c9d27826f9 docs: update TOC in userguide (#2479) 2026-06-30 08:48:21 +03:00
Bastian 13bd00bf9d fix: sort sleep screen menu options to be more logically consistent (#2480) 2026-06-30 08:47:25 +03:00
Uri Tauberandpablohc fef39da23c feat: Selection Popup (#2358)
Co-authored-by: pablohc <pablonoviello@outlook.com>
2026-06-29 16:59:25 -04:00
Uri Tauber fd43ca2fe1 chore: Initial multi-core compatibility (#2294) 2026-06-29 21:57:21 +03:00
Pietro Campagnano 28255061fb fix(icons): align home menu icons with their labels (#2470) 2026-06-29 13:59:54 -04:00
Julia Nguyen 43ff1b6644 Merge remote-tracking branch 'origin/master' into develop 2026-06-29 12:29:58 -04:00
Егор Мартынов 2754a5ff01 feat: add Vollkorn font (#2473)
CI (build) / clang-format (push) Canceled after 0s
CI (build) / cppcheck (push) Canceled after 0s
CI (build) / build (push) Canceled after 0s
CI (build) / unit-tests (push) Canceled after 0s
CI (build) / Test Status (push) Canceled after 0s
2026-06-29 15:54:01 +03:00
Justin Mitchell 6edeed73c5 fix: x4 sleep/boot ghosting (#2471) 2026-06-29 12:04:33 +03:00
Justin Mitchell 8ac302018c fix: x4 ghosting by bumping sdk (#2469) 2026-06-28 22:38:35 +03:00
Rodrigo Matsuura a11517a62d fix: update Portuguese (Brasil) translations (#2458) 2026-06-28 15:07:52 +03:00
Bastian cbaa498ccc docs: adding quick resume option and quick resume on timeout to userguide (#2425) 2026-06-27 22:40:21 +03:00
Justin Mitchell ebebc6f202 chore: migrate from open-x4-sdk to freeink-sdk (#2449)
This PR moves us from the xteink openx4 SDK to the freeink sdk from
https://freeink.org. Out of the box there are NO changes needed in the
firmware to support this swap, it all magically works as is. However as
we support more than just the x3/x4 devices, this sdk allows us to pass
env vars into the build commands to include support for other devices.
As support for new hardware such as touch screens and bluetooth are
added the xteink builds decide at compile time if the libraries are used
or not. For example right now the freeinkui and icons libraries are in
the platform.io file but as they are not used anywhere, they won't be
included in the final build. Once the touch branch and sd themes branch
are merged in this sdk is required for them to function correctly. All
the docs for freeink are available at freeink.org/docs. x4/x3 is a
single binary build unlike other devices that will build unique binaries
for each device. Eventually we will want to remove a lot of the manual
isx3 type stuff from our firmware and go through the boardsupport api
the sdk provides as it will generalize everything into one common system
that any device can support. The upcoming touch branch does a lot of
this for us but this initial PR is JUST to get the sdk swapped over
without any code changes to show seamless integration without any
regressions.
2026-06-27 21:22:58 +03:00
Julia 970b2c6ca1 chore: release 1.4.1 (#2447)
Compile Release / build-release (push) Canceled after 0s
## Improvements

* Moved the File Manager breadcrumb into the Contents card header for a
cleaner, more consistent interface.
* Updated the Wireless Transfer section of the User Guide with clearer
instructions.
* Battery status bar indicator no longer changes position when adding or
removing bookmarks.

## Performance

* Optimized path normalization for faster file handling.
* Significantly improved bookmark rendering by removing unnecessary
XPath lookups.
* Optimized dithered rectangle drawing (fillRectDither) using a
byte-aligned rendering implementation, improving display performance on
supported devices.

## Bug Fixes

* Fixed an issue where the Inverted Orientation label was incorrectly
combined with the Color Filter label for Geman localization.
* Fixed excessive ghosting on the X3 cover screen during sleep
2026-06-26 17:40:04 -04:00
Julia Nguyen b6ce599b20 fix: address release review feedback 2026-06-26 16:22:20 -04:00
Juliaandcoderabbitai[bot] f54eab2725 fix: typo in STR_ORIENTATION_INVERTED Spanish translation
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-06-26 16:13:19 -04:00
Justin Mitchell 0d4c9ab91b Bump version to 1.4.1 2026-06-26 15:48:19 -04:00
Julia 0daa9db243 fix: keep status bar indicators stable when toggling bookmarks (#2444) 2026-06-26 19:26:33 +03:00
Uri TauberandRyan Mercado a2f2eea79e perf: Optimize fillRectDither with Byte-Aligned fillRectImpl (#2270)
## Summary

* **What is the goal of this PR?**
Replace the pixel-by-pixel `fillRectDither` implementation with a new
byte-aligned `fillRectImpl` that eliminates per-pixel
`rotateCoordinates` calls and Read-Modify-Write bitwise loops, yielding
a significant rendering speedup on ESP32 E-ink framebuffers.

Ported from @rhythmerc's crosspoint-reader fork commit 27ea625.

* **What changes are included?**

- **`GfxRenderer.cpp` — `fillRectDither` refactor:** The existing
`if/else if` chain is replaced with a `switch` statement that delegates
each `Color` case to the new `fillRectImpl<Color>()` template,
eliminating runtime branching.

- **`GfxRenderer.cpp` — new `fillRectImpl<C>()` template:** Core of the
optimization. Key behaviors:
    - Clips the rectangle in logical space upfront.
- Rotates only **2 opposing corner points** (top-left and bottom-right)
into physical framebuffer space instead of rotating every pixel
individually.
- Derives physical-space `byteStart`/`byteEnd` and precomputes
`headMask` / `tailMask` for MSB-first partial-byte boundaries,
performing RMW only on the edge bytes.
- **Solid fills (`Black` / `White`):** Uses `memset` for all interior
full-byte runs per row — no per-pixel writes.
- **Dithered fills (`LightGray` / `DarkGray`):** Precomputes both parity
variants of `blackMask` (even/odd `py`) **outside** the row loop,
eliminating the previously re-evaluated 8-bit construction loop on every
physical row. Interior full bytes are then written with a single
`memset(whiteMask)`.
- Uses `if constexpr` throughout to dispatch on `Color` at compile time,
generating zero runtime branches per template instantiation.

- **`GfxRenderer.h`:** Declares the new private `fillRectImpl<Color>()`
template method with an explanatory doc-comment.

- **Explicit template instantiations** added for all four active `Color`
variants (`Black`, `White`, `LightGray`, `DarkGray`).

## Additional Context

* **Performance:** The primary motivation is ESP32 E-ink framebuffer
performance. The old path called `rotateCoordinates` and did a full RMW
for every single pixel in the rectangle. The new path calls
`rotateCoordinates` exactly **twice** per fill regardless of rectangle
size, then operates at byte granularity — a complexity reduction from
O(W×H) coordinate transforms to O(1).
* **Dither correctness:** The `blackMask` precomputation relies on the
dither pattern having period 2 in both logical X and Y, which makes the
per-row byte pattern repeat with period 2 in `py`. Reviewers should
verify the `lxBase`/`lyBase` derivations for all four orientations
(`Portrait`, `PortraitInverted`, `LandscapeClockwise`,
`LandscapeCounterClockwise`) match the inverse of `rotateCoordinates`.
* **Edge case — single-byte rows:** When `byteStart == byteEnd`, the
head and tail masks are ANDed together into a single `rectMask` to avoid
double-masking the same byte. This path should be tested with narrow
rectangles (width < 8px).
* **No behavioral change for `Color::Clear`:** The `Clear` case exits
early via `if constexpr` and is a no-op, matching the original behavior.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< YES >**_

---------

Co-authored-by: Ryan Mercado <rmercado@firstdollar.com>
2026-06-26 12:16:25 -04:00
Ankit 86a9b9c4a2 docs: update user guide wireless transfer section (#2369)
Co-authored-by: Uri Tauber <uritaube@gmail.com>
Closes #1407
2026-06-26 08:25:52 +03:00
Julia Nguyen ba44978ac4 Merge branch 'master' into develop 2026-06-25 17:51:56 -04:00
Pietro Campagnano 555f76da88 feat: move file manager breadcrumb into contents card header (#2430) 2026-06-26 00:45:43 +03:00
Justin Mitchell 0a57c0a5a7 Fix X3 display ghosting on cover screen transitions
Force display resync on X3 when HALF refresh is requested to clear prior content before rendering. Add grayscale preconditioning for X3's UC81xx controller to even out single-pixel dithering artifacts that appear as speckle with its turbo BW waveform.
2026-06-25 17:42:19 -04:00
Uri Tauber a09aef0889 fix: Optimize Bookmark Rendering by Removing XPath Lookup (#2417)
## Summary

fix #2414 

### Root Cause

`updateBookmarkFlag()` was introduced in commit 1db1442 and is executed
on every render (every page turn). The function calls:

`ProgressMapper::toSavedProgress()`
→ `ChapterXPathResolver::findXPathForProgress()`

This path decompresses the current EPUB section content twice:

1. To count visible characters.
2. To resolve the corresponding XPath.

For larger sections (e.g. ~133 KB decompressed content), this adds
approximately **1 second of I/O overhead per page turn**, with the cost
increasing as chapter size grows.

### Fix

`updateBookmarkFlag()` only needs to determine whether a bookmark falls
within the currently displayed page range.

The required information is already available during rendering:

* `currentPage`
* `section->pageCount`
* `currentSpineIndex`

Instead of converting the current location to a saved progress object
(and resolving an XPath), the implementation now computes the current
page's progress range directly and compares bookmark percentages against
that range.

This is effectively the same percentage-based matching logic already
used as a fallback in `bookmarkMatchesProgress()` when XPath matching is
unavailable.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< PARTIALLY >**_
2026-06-25 09:20:24 -04:00
Bastian 8626d69f46 fix: small translation changes for german (#2420) 2026-06-25 09:29:05 +03:00
Uri Tauber fc89e57e69 perf: optimise normalisePath (#2162) 2026-06-25 09:01:50 +03:00
Julia 487613b082 fix: split inverted orientation label from color filter label (#2421)
## Summary

* **What is the goal of this PR?** 
* Fix the “Inverted” translation so reader orientation and color/filter
inversion can use separate labels.
* **What changes are included?**
* Adds `STR_ORIENTATION_INVERTED` for the inverted portrait orientation
option.
* Updates the reader orientation setting to use
`STR_ORIENTATION_INVERTED` instead of reusing `STR_INVERTED`.
* Leaves `STR_INVERTED` for the sleep cover filter and tilt page-turn
mode
  * Adds the new orientation string across all 26 locale YAML files.

## Additional Context

* The original issue was found by a user in German, where `STR_INVERTED`
was translated as `Hochformat 180°`, which made sense for orientation
but not for color filters.
* This is a UI-label-only change. It does not change persisted
orientation values or settings behavior.
* Reviewer note: non-English wording may still benefit from
native-speaker review, especially for the new orientation-specific
labels and to verify the interchangeable usage between inverted color
and inverted tilt page turn direction.
---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< YES >**_
2026-06-25 00:47:06 -04:00
JuliaandHusam Younis 8d5b119644 fix: sync master into develop (#2423)
Sync develop branch with master

Co-authored-by: Husam Younis <youhusam@gmail.com>
2026-06-24 22:07:13 -04:00
Husam Younis 7271c00d35 feat: Allow statusbar clock to be on the left (#2359) 2026-06-25 00:23:23 +03:00
Uri Tauber 6e8dbd7f23 chore: Update version to 1.4.0 (#2283)
Compile Release / build-release (push) Canceled after 0s
2026-06-24 18:30:32 +03:00
Uri Tauber c4b1d9644a fix: correct font size selection (#2410) 2026-06-24 17:01:10 +03:00
Justin Mitchell f6e59aab72 feat: Add displayGrayscaleBase method for differential refresh
(#2334)

Introduces a new display method that prepares the framebuffer as a base
frame for grayscale overlays. On X3 panels, this uses the OEM
differential base waveform (AA-pre-BW) without forcing a resync. Other
panels fall back to normal display with configurable refresh mode.


Dependent upon matching SDK commit to work
2026-06-24 07:53:19 -04:00
Uri Tauber 9ad2da0950 fix: add seven missing hebrew translations (#2409) 2026-06-24 09:34:40 +03:00
Uri TauberandJulia Nguyen 1db1442319 fix: several bookmarks UX improvments (#2372)
## Summary

This PR enhances the EPUB reader's bookmark system with two
complementary improvements: a per-page bookmark indicator icon and
toggle behavior on the existing long-press action.

---

### What Changed

**Bookmark Toggle (was: add-only)**

The long-press Confirm action now toggles bookmarks rather than always
adding. `addBookmark()` checks whether a bookmark with the same xpath
already exists in the in-memory cache:
- If found → removes it and shows "Bookmark removed."
- If not found → adds it and shows "Bookmark added."

A new `STR_BOOKMARK_REMOVED` translation string was added to support the
removal message.

**Bookmark Icon Indicator**

A `BookmarkIcon` is now drawn at the top-right corner of the page
whenever the current page has a bookmark. `updateBookmarkFlag()` is
called at render time to determine whether the current page is
bookmarked.

**In-Memory Bookmark Cache**

Bookmarks are now loaded into `cachedBookmarks` on `onEnter()` rather
than being re-read from disk on every toggle. All subsequent add/remove
operations work against this cache and flush to disk, avoiding redundant
file reads on each bookmark action.

**Faster bookmarks list**

Previously, calculating "page X/Y" for each entry required decompressing
the entire spine item. We now persist `si`/`pc`/`pp` (spine index, page
count, and page progress) in the bookmark JSON when saving, and restore
them when loading. This avoids the expensive `toCrossPoint()` loop in
`onEnter()`, significantly reducing the cost of initializing the
bookmarks list.

---

### Files Changed

- `EpubReaderActivity.cpp` — `addBookmark()` toggle logic,
`updateBookmarkFlag()` (new), icon rendering in `renderContents()`,
cache initialization in `onEnter()`
- `EpubReaderActivity.h` — new fields: `currentPageBookmarked`,
`bookmarkRemoved`, `cachedBookmarks`; new method declaration
`updateBookmarkFlag()`

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< PARTIALLY >**_

---------

Co-authored-by: Julia Nguyen <julia@uxj.io>
2026-06-23 14:55:01 -04:00
Leopoldo Pla Sempere 362dcb2a65 feat: Smooth progressive JPEG cover upscales in BMP conversion (#2214) 2026-06-22 13:55:12 -04:00
Julia d1abcc00a2 feat: render grayscale epub images without text aa (#2393) 2026-06-22 17:15:27 +03:00
Zeph 0ce874f8d4 feat: footnote returns to original position on deep sleep to avoid losing reading position (#2394) 2026-06-22 15:28:42 +03:00
kygia 282514f755 fix(epub): flush displaced anchor before overwrite (#2336) (#2382)
## Summary

* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)

Fixes footnote links landing in start section instead of the specific
foot note target.

* **What changes are included?**

ChapterHtmlSlimParser.cpp: call flushPendingAnchor() before overwriting
pendingAnchorId. First id got lost due to consecutive non-block elements
carry ids, the first id was lost and the reader had no page to jump to,
so it defaulted to page 0.

## Additional Context

Tested with the epubs attached to to #2336. Need to clear .crosspoint/
cache after flashing so the anchor map gets rebuilt with the fix.

* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
  specific areas to focus on).

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? 

Partially assisted with writing documentation.
2026-06-21 16:43:39 -04:00
SurprisedDuck d1e4650e19 fix: don't justify-stretch a leading no-break space (#2185) (#2298) 2026-06-21 16:12:14 +03:00
Justin Mitchell 370f87ea01 feat: add Spacing Modifier Letters range to font presets (#2194) 2026-06-21 09:14:42 +03:00
Leopoldo Pla Sempere 8b9a73a735 chore: update Spanish, Catalan, and Valencian translations (#2384) 2026-06-21 00:26:30 +03:00
Julia 29c69b3d8f fix: automatically connect to wifi for clock sync (#2379)
## Summary

* **What is the goal of this PR?** 
* Make manual clock sync work even when the device is not already
connected to Wi-Fi, so the user can start the sync flow directly from
settings instead of being blocked by connection state.
* **What changes are included?**
* `ClockSyncActivity` now launches the normal Wi-Fi selection flow
before syncing when the device is offline.
* After Wi-Fi selection succeeds, clock sync resumes automatically and
performs the existing forced NTP sync.
* If clock sync had to bring Wi-Fi up, the activity disconnects and uses
the existing silent restart cleanup path to avoid leaving the device in
a fragmented post-Wi-Fi heap state.
* The completion UI now only advertises Back, matching the updated input
handling.

## Additional Context

* Existing behavior is unchanged when Wi-Fi is already connected: the
activity syncs immediately.
* Cancelling Wi-Fi selection exits the clock sync flow.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< YES >**_
2026-06-20 22:09:28 +03:00
Matteo Scopel 4b5a84dd81 chore: update the Italian translation (#2376) 2026-06-20 22:08:35 +03:00
muhasandmuhas b9874c8114 fix(lang): [russian] new string (#2374)
Co-authored-by: muhas <mail@muhas.name>
2026-06-19 13:15:15 +03:00
Paul Delestrac b1131795d8 feat: add live font preview pane to font selection screen (#2349)
## Summary
* Adds a live font preview pane to the font selection screen so users
can see how a font looks before committing to it.
* Changes
* A preview pane occupying the top 30% of the font selection screen,
rendering sample pangram text in the previewed font
* A two-step confirm flow: first press (enter button) previews the font,
second press selects it
* Back restores the original font settings, so browsing has no side
effects
* Layout dimensions cached in `onEnter()` to avoid redundant
recalculation between `loop()` and `render()`

## Additional Context
* Preview sample text is hardcoded English; didn't want to use AI for
translation as I would not be able to verify the output in most
languages...
* The preview pane reduces visible list height; this is compensated by
passing the reserved height into `getNumberOfItemsPerPage`

---
### AI Usage
Did you use AI tools to help write this code? **PARTIALLY**
AI use for assisting in coding and in writing the PR description.
2026-06-18 18:16:28 -04:00
Pedro CardosoandClaude Opus 4.8 5e990b3991 fix: prevent progress.bin corruption from interrupted writes (#2275) (#2305)
## Summary

* **What is the goal of this PR?**
Fixes #2275. A book could get stuck reopening on an old page, with
progress no
longer saving and neither "Delete Book Cache" nor "Clear Reading Cache"
able to
fix it. Root cause: `progress.bin` was written truncate-in-place, so an
interrupted write (power loss, or a crash mid-SPI during sleep) left it
with a
broken FAT cluster chain that the firmware could neither rewrite nor
delete —
  recovery required `fsck`/manual deletion on a host PC.

  Confirmed in the SDK: `SDCardManager::openFileForWrite` opens with
`O_RDWR | O_CREAT | O_TRUNC`, so the canonical file is zeroed before the
few
progress bytes are rewritten — exactly the window that corrupts the FAT
chain.

* **What changes are included?**
  * New shared helper `ProgressFile::writeAtomic()`
    (`src/activities/reader/ProgressFile.h`): writes progress to
    `progress.bin.tmp`, flushes and closes it, then `remove`s the old
`progress.bin` and `rename`s the temp into place. An interrupted write
now
only ever damages the throwaway temp; the canonical file is never torn.
  * All three readers route their progress saves through the helper:
EPUB (`EpubReaderUtils.h`), `TxtReaderActivity`, `XtcReaderActivity` —
they
    all shared the identical vulnerable pattern.
* Minor: `EpubReaderUtils::saveProgress` now takes `const Epub&` (clears
a
    cppcheck `constParameterReference` finding).

## Additional Context

* **Crash-safe, not metadata-atomic.** On FAT the replace is `remove` +
`rename`
(two directory ops; SdFat's `rename` won't overwrite, hence
remove-first). A
crash between them leaves *neither* file, which reads as "no saved
progress" on
next launch — a harmless reset to an old page, never a
corrupt/unclearable file.
  The guarantee is that `progress.bin` is never half-written.
* **Prevents, does not repair.** This stops new corruption on healthy
cards. It
cannot fix an already-corrupted `progress.bin` (removing it may itself
fail at
the FAT level) — those still need `fsck`/manual deletion, as in the
issue's
  workaround.
* **Known follow-up (out of scope here):** a crash *while writing the
temp* can
leave an orphan `progress.bin.tmp`. It's harmless and self-healing (the
next
save overwrites it, and it never blocks reading progress), but a
boot-time
  orphan-`.tmp` cleanup would be a tidy follow-up.
* **Focus areas for review:** the close-before-rename ordering in
  `ProgressFile.h` and the remove-before-rename rationale.

## Verification

* `./bin/clang-format-fix` — clean
* `pio check --fail-on-defect low --fail-on-defect medium
--fail-on-defect high` — no defects
* `pio run` — SUCCESS (RAM 30.9%, Flash 78.8%; footprint essentially
unchanged)
* Tested on a **Xteink X4** device: open book, turn pages, sleep/exit,
reopen —
  progress now restores to the navigated page across all three readers
  (EPUB / TXT / XTC).

---

### AI Usage

Did you use AI tools to help write this code? _**YES**_

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 18:01:56 -04:00
Uri Tauber ff1951c715 fix: correct behaviour for prev/next side buttons (#2373)
## Summary

* **What is the goal of this PR?** fix #2365
2026-06-18 13:13:35 -04:00
Ankit 121c0690b0 fix: use STR_SELECT instead of STR_OPEN in bookmark button hint (#2371) 2026-06-18 19:58:47 +03:00
16b0853654 fix(epub): NFC-normalize EPUB text so NFD diacritics render correctly (#2277)
Co-authored-by: Uri Tauber <uritaube@gmail.com>
Co-authored-by: Julia <julia@uxj.io>
2026-06-18 13:00:01 +03:00
Uri Tauber 7d639cf880 fix: submodule pointer (#2368) 2026-06-17 22:35:12 +03:00
Justin MitchellandUri Tauber 22f3575064 feat: Support for Korean line breaks and glyph spacing (#2288)
Co-authored-by: Uri Tauber <uritaube@gmail.com>
2026-06-17 17:27:03 +03:00
darkbubluandClaude Opus 4.7 d4069aeae5 feat: long-press Confirm launches KOReader sync from EPUB (#1808)
## Summary
- Hold the menu (Confirm) button for ≥1s while reading an EPUB to launch
the existing `KOReaderSyncActivity` directly — replaces the three-step
path (open reader menu → scroll to Sync → confirm) with a single
gesture.
- Reuses `ReaderUtils::GO_HOME_MS` (same 1s threshold used by long-press
Back) and the existing `KOREADER_STORE.hasCredentials()` guard.
- Adds a Controls picker **"Long-press Menu"** (`longPressMenuFunction`,
default **Bookmark**) that **cycles through the available functions**
bound to the long-press gesture: `KOSync → Disabled → Bookmark`. The
field name and `LONG_PRESS_MENU_FUNCTION` enum are intentionally general
so future actions (dictionary lookup, table of contents, etc.) can be
appended without another schema migration. The setting is **not** a
binary toggle.
- Existing menu Sync entry still works — both call sites share one
extracted helper (`launchKOReaderSync`); no logic duplication.
- Short-press Confirm release is gated on duration so the reader menu
does not also open after a long press that *acts*, mirroring the
existing long-press Back pattern.
- **No-credentials fall-through:** `launchKOReaderSync()` now returns
whether it acted. When the function is set to KOSync but no KOReader
credentials are stored, the long-press is a no-op that **falls through
to open the reader menu** — so the menu stays reachable instead of the
hold silently swallowing the gesture. The release is only suppressed
when sync actually launched or surfaced a save error.

## Test plan
- [x] `pio run` succeeds clean for the `default` ESP32-C3 environment.
- [x] On-device, value **KOSync**, valid KOReader credentials:
long-press Confirm ≥1s → sync screen launches; get + update progress
return HTTP 200; release does **not** also open the reader menu; returns
to the same page.
- [x] On-device, value **KOSync**, **no** credentials: long-press
Confirm falls through and **opens the reader menu** (regression fix);
short-press also opens the menu.
- [x] On-device: menu → Sync still launches the same screen (shared
helper) and syncs (200/200).
- [x] On-device, value **Bookmark**: long-press drops a bookmark and
does **not** also open the menu.
- [x] On-device, value **Disabled**: long-press Confirm opens the menu
on release; no sync, no bookmark.
- [x] On-device: long-press Back still goes to the file browser
(unchanged path).
- [x] Heap: epub is released before the TLS handshake (frees ~16 KB);
min free heap stayed ~84 KB during sync, well above the safe floor. No
panics/OOM across the session.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-17 17:10:26 +03:00
Mauvis LedfordandJulia 54d5a788a5 feat: add drag-and-drop to upload modal (#2290)
Co-authored-by: Julia <julia@uxj.io>
2026-06-17 16:53:31 +03:00
Ankit f66220a245 fix: enable Shift key for URL keyboard input (#2178) (#2357) 2026-06-17 16:32:36 +03:00
KymAndriyandKymAndriy 2266060e84 fix: Ukrainian translation (#2354)
Co-authored-by: KymAndriy <test@notamail.ua>
2026-06-16 14:02:23 +03:00
Julia 90039d7f4d fix(koreader): resolve element XPath progress against visible body text (#2308) 2026-06-15 22:03:32 -04:00
rafaelmsseandRafael Santos 16eb66d7cf fix: add missing HTML 4.01 named entities (#2352)
Co-authored-by: Rafael Santos <rmsantos@applaudostudios.com>
2026-06-15 22:25:37 +03:00
RoninandUri Tauber 02bab00be6 fix: swap reader menu navigation direction in CCW/inverted (#2321) (#2341)
Co-authored-by: Uri Tauber <uritaube@gmail.com>
2026-06-15 22:19:34 +03:00
Mauvis Ledford 55a56914d7 fix(i18n): localize home empty-state strings (#2342) 2026-06-14 21:40:05 +03:00
Uri Tauber 8634cb8ad3 fix: Restore first-line paragraph indentation (#2320) 2026-06-12 11:55:00 -04:00
Julia 21caebd6e4 fix: use Noto Sans as punctuation fallback when generating downloadable SD fonts (#2331) 2026-06-12 11:54:34 -04:00
Matteo Scopel 6f05e401a1 feat: add the Libre Baskerville font family (#2088)
## Summary

This PR adds the [Libre Baskerville
](https://github.com/impallari/Libre-Baskerville) font family to the
fonts available for download in CrossPoint.


## Additional Context

Baskerville is a classic font family, widely available in some form or
the other in most ereaders and widely used in publishing and in the
academy.

Libre Baskerville is a libre (SIL Open Font License v1) implementation
of it, which makes it possible for CrossPoint to ship it without issues.


![Vita-e-destino_ch30_p1_13pct_99220.bmp](https://github.com/user-attachments/files/28070672/Vita-e-destino_ch30_p1_13pct_99220.bmp)


---

### AI Usage

Did you use AI tools to help write this code? _**NO**_
2026-06-11 19:10:41 +03:00
Zach Nelson f2e3d117dc fix: Hanging indent causes overlapping words (#2324) 2026-06-11 18:23:49 +03:00
Zach Nelson 9202522a39 refactor: Drop redundant self-class prefix on applyDirectionToEntry (#2325)
## Summary

Non-functional cleanup: `ChapterHtmlSlimParser::` prefix is redundant
when calling `applyDirectionToEntry` within `ChapterHtmlSlimParser`
methods.

---

### AI Usage

Did you use AI tools to help write this code? _**NO**_
2026-06-11 10:38:00 -04:00
Uri Tauber 301f1d1a38 fix: Compile Error: Duplicate _order values found (#2323) 2026-06-11 15:18:53 +03:00
Hoang Manh LinhandClaude Opus 4.8 1f83087aa5 feat(fonts): add Vietnamese glyph coverage to the built-in UI font (#2280)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 15:11:50 +03:00
Hoang Manh LinhandClaude Opus 4.8 68138fffdb feat(i18n): add Vietnamese translation (#2279)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-11 15:10:45 +03:00
MartinandUri Tauber eb98735115 feat: Slovak translation add (#2251)
Co-authored-by: Uri Tauber <uritaube@gmail.com>
2026-06-10 18:49:39 +03:00
Uri Tauber b94b58756f fix: german traslation for STR_INVERTED (#2315) 2026-06-10 16:33:54 +03:00
muhas ca6ebc56fb fix(lang): russian.yaml new string (#2309) 2026-06-10 08:10:26 +03:00
Uri Tauber 2ea042a68a fix: skip <span> anchors (#2303)
## Summary

* **What is the goal of this PR?** Prevent heap memory exhaustion caused
by thousands of machine-generated anchor IDs injected by epub converters
like Kobo KePub.
* Fix #2292.

* **What changes are included?**
* **Element-type filter (Layer 1):** Introduced the
`isNonNavigableInlineElement()` function in `ChapterHtmlSlimParser.cpp`
to automatically skip recording IDs on `<span>` elements, as they are
purely inline wrappers used for tracking and lack navigable meaning.
* **Hard cap (Layer 2):** Added the `MAX_ANCHORS_PER_CHAPTER = 1024`
constant to act as a fallback safety net against unbounded heap growth
from unknown future ID-injection patterns on non-span elements.
* **TOC Safety net:** Ensured that IDs matching known Table of Contents
(TOC) entries explicitly bypass both the `<span>` filter and the
1024-anchor cap, guaranteeing that chapter page-break and core
navigation logic are never compromised.


---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< PARTIALLY >**_
2026-06-09 08:58:21 -04:00
Lorenzo Santini 9c12b90737 feat: add power button short press for quick access to footnotes (#1658) 2026-06-08 14:45:41 -04:00
Leopoldo Pla Sempere 7f01eab62e docs: refresh cache formats and web server workflows (#2233)
## Summary

* **What is the goal of this PR?** Update project documentation to match
the current master implementation for cache formats, i18n, file
transfer/web server workflows, SD-card fonts, and root user-facing docs.
* **What changes are included?** Refreshes `book.bin`/`section.bin` docs
for v6/v25, updates File Transfer/Calibre/WebDAV/API docs, documents 24
UI languages and JSON language persistence, updates root
README/USER_GUIDE cache and network details, and syncs the tracked
CLAUDE skill doc cache-version notes.

## Additional Context

* Docs-only change. Verified with `git diff --check origin/master..HEAD`
and stale-reference greps for old cache versions, removed i18n APIs, old
WiFi screen wording, and raw `Serial.printf` examples. No firmware build
was run.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**NO**_
2026-06-07 15:47:44 -04:00
Zach Nelson 3a1e9f3023 refactor: Dedupe wifi scan in-place without std::map (#2262)
## Summary

`std::map` heap-allocates each node. Avoid using it for wifi SSID
dedupe, and instead just dedupe in-place with the existing `networks`
vector. Removed dead `ipAddress` member in `WifiNetworkInfo` struct.

---

### AI Usage

Did you use AI tools to help write this code? _**PARTIALLY**_
2026-06-07 08:27:20 -04:00
SurprisedDuck fad1a801a4 fix: scale SUP/SUB underline to match 50%-scaled glyphs (#2255) 2026-06-06 18:55:47 -04:00
Julia 67936cb3af fix(epub): decode footnote href path before spine lookup (#2271) 2026-06-05 18:38:13 -04:00
Uri Tauber fd5b8078c6 chore: add t5s3 fork (#2268) 2026-06-05 18:50:22 +03:00
Julia bd101b2af8 fix(epub): decode percent-encoded internal asset paths so assets render correctly (#2249)
## Summary

* **What is the goal of this PR?** 
* Fix EPUBs where internal file references are written in URL-style
escaped form, like spaces appearing as `%20`, so the reader can find the
right files instead of treating those references as missing.
* **What changes are included?**
* Added a small shared helper that converts those escaped EPUB-internal
paths back into their normal filenames before we try to look them up.
* Applied that cleanup step across the EPUB parsing flow wherever we
resolve internal references, including cover images, manifest items, TOC
links, spine entries, and inline HTML images.
* Kept the change narrowly focused on EPUB-internal asset resolution
rather than changing broader URL or networking behavior.

## Additional Context

* The user-facing bug here is that some books package their internal
filenames in an escaped form, so a file like `Chapter 1.xhtml` may be
referenced more like `Chapter%201.xhtml`. The reader was treating that
escaped text as the literal filename, which means otherwise-valid books
could lose images, covers, or chapter targets because the lookup no
longer matched the real file inside the EPUB.
* Risk is intentionally low. The helper only rewrites valid `%XX` escape
sequences and leaves malformed input alone, so it should improve
compatibility with escaped filenames without broadening the parser’s
behavior in unrelated cases.

## Local Testing Performed
* This was tested on my device with the user-provided optimized epub
that was not rendering images within the text prior to this fix (cover
image and chapter headers were rendering fine):

[orv_main_baseline.epub.zip](https://github.com/user-attachments/files/28529545/orv_main_baseline.epub.zip)
* This was also tested by the user with a local build and the affected
epub and confirmed to be working

## Steps for Testing
* Try to open the affected epub (linked above) or any epub that has
similar percent-encoding on a build prior to this fix.
* Apply this fix, clear book cache, and re-open the affected book.
* Images should render properly.
---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< YES >**_
2026-06-05 10:31:20 -04:00
Uri Tauber 19d51ec08c fix: german translation for STR_INVERTED (#2269) 2026-06-05 12:22:29 +03:00
Uri Tauber ea231ffd18 Revert "Update german.yaml" (#2267) 2026-06-05 11:22:36 +03:00
Uri Tauber a2d4c6a139 Update german.yaml
Fix #2266
2026-06-05 11:17:22 +03:00
Zach Nelson b5b1f650e2 perf: Minimize string allocations in CSS parsing (#2263)
## Summary

Use `std::string_view` and case-insensitive comparisons to avoid string
allocations during CSS parsing.

**Hot path:** `resolveStyle` (called per HTML start tag during chapter
rendering) now does zero heap allocations. Previously it allocated a
normalized tag string, a vector of class strings, and a composite key
per class. For a chapter with ~2000 tags × 2 classes each, that's ~12
000 small short-lived allocations eliminated per page render — primarily
a heap-fragmentation win on the ESP32-C3's ~380KB RAM.

**Cold path:** CSS load no longer allocates per-rule selector vectors or
per-token strings; `splitOnChar`/`splitWhitespace` are gone, replaced
with callback-based tokenization (`forEachDelimitedToken`).

**Behavioral notes:**
- The selector `unordered_map` now uses an ASCII-case-insensitive
hash/equal. Selectors are stored with their original case rather than
pre-lowercased; the observable lookup result is unchanged.
- `stripTrailingImportant` is now case-insensitive (per CSS spec;
previously matched only lowercase `!important`).

**Cache compatibility:** `CSS_CACHE_VERSION` unchanged. Old caches
(lowercase selectors) load correctly under the new lookup; new caches
will contain verbatim-case selectors — both forms work.

---

### AI Usage

Did you use AI tools to help write this code? _**PARTIALLY**_
2026-06-04 18:05:32 -04:00
Eloren1 f04b8aa9a9 fix: apply progress bar offset from top instead of bottom edge (#2250) 2026-06-04 17:54:57 +03:00
Justin Mitchell d9bcef7a58 fix: Replace full-image cache buffer with streaming band buffer to reduce memory usage (#2230) 2026-06-04 10:46:22 -04:00
Uri TauberandJulia a60f31cdd4 fix: Crash on invalid font filename (#2253)
Co-authored-by: Julia <julia@uxj.io>
2026-06-04 10:34:34 -04:00
Uri TauberandJulia f055fdd774 fix: long-press back should move to the start of chapter (#2243)
Co-authored-by: Julia <julia@uxj.io>
2026-06-04 16:35:38 +03:00
Julia bb078bae09 fix: KOReader sync drift when syncing at chapter start (#2245) 2026-06-04 15:23:52 +03:00
Matteo Scopel 2d65808302 chore: small Italian fixes (#2252) 2026-06-03 14:43:43 +03:00
Matheus Martins 6e75e5a6be docs: Fix section numbering and table of contents in USER_GUIDE.md (#2244) 2026-06-02 22:12:00 +03:00
Uri Tauber db94a86fba fix: Skip Underline Calculations During Font Cache Scan Pass (#2237)
## Summary

This PR optimizes the text rendering process by skipping underline style
calculations and measurements during the initial font cache scanning
phase. This prevents excessive and unnecessary SD card reads on pages
with heavy use of underlines (e.g., Table of Contents pages).

### **The Problem**

During the first rendering pass (the font cache scan pass used to
collect text for prewarming), `GfxRenderer::drawText()` early-returns
after recording text as expected. However, `TextBlock::render()`
continued past this point to execute the underline decoration logic.

Because underline calculation calls `getTextWidth()` and
`getTextAdvanceX()`, it triggered immediate glyph lookups via
`EpdFont::getGlyph()`. Since the SD card font had not been prewarmed yet
at this stage, the lookups fell back to the `glyphMissHandler`,
resulting in hundreds of individual, slow SD card reads into a limited
8-slot ring buffer.

### **The Fix**

1. **Exposed Scan State:** Added `GfxRenderer::isFontCacheScanning()` to
safely check if the font cache manager is currently in
text-collection/scan mode.
2. **Bypassed Underline Logic:** Modified `TextBlock::render()` to check
this state and skip underline measurement and drawing entirely while
scanning is active.

> [!NOTE]
> The text itself is still properly captured for prewarming via
`drawText()`. The underlines will be safely and efficiently calculated
and drawn during the actual render pass after the fonts have been
completely prewarmed.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< YES >**_
2026-06-02 09:47:24 -04:00
Uri Tauber b12839d1d4 chore: Move opendyslexic from flash to SD (#2231) 2026-06-01 13:49:57 -04:00
Justin Mitchell 50e4d550fb Fix ghosting on pages following images in grayscale (#2226) 2026-05-31 23:33:26 -04:00
Julia 03f73fadc7 fix: avoid zip-wide css scan for large epubs (#2213)
## Summary

* **What is the goal of this PR?** 
* This PR ports over Crossink's handling of large EPUBs that helps
prevent crashes during open after the book metadata cache is built.

* **What changes are included?**
* Removes the post-indexing ZIP-wide CSS discovery pass that built an
in-memory map of every ZIP entry.
* Reuses `content.opf` parsing to collect declared CSS files without
writing spine entries again.
* Temporarily releases the loaded book metadata cache while rebuilding
CSS for cached books.
* Parses CSS before reloading `book.bin` after a fresh cache build,
leaving more heap available during CSS rule parsing.

## Additional Context

* User reported their EPUB opening fine on Crossink but would crash on
Crosspoint. Verified this claim on my own devices.
* The crash this addresses happened after `book.bin` was successfully
built, when CSS discovery allocated a large `unordered_map` for ~3k EPUB
ZIP entries.
* Tradeoff: CSS files not declared in `content.opf` are no longer
discovered by scanning the full ZIP. This avoids the high-risk memory
allocation but improperly formatted EPUBs (ones that don't declare their
CSS styles in `content.opf` will render without styling and fallback to
inline styles.
* User provided epub that was crashing prior to this change:
https://www.mediafire.com/file/g57ea4mj13iunvh/Quang+%C3%82m+Chi+Ngo%E1%BA%A1i+-+Nh%C4%A9+C%C4%83n.epub/file

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< YES >**_
2026-05-31 18:31:06 -04:00
Wilhelm Schuster 0d64980d25 fix: update German translation (#2224) 2026-06-01 00:51:52 +03:00
Nathanael Maher 4fe80ba7a5 fix: bookmark percentage always 0% and page number starts at 0 (#2188)
## Summary

* Fix the issue described in #2176 where bookmark percentages are always
0%
* Also fixed the page number rendering to +1 because they are 0 index
based, but should be rendered starting from 1

---

### AI Usage

Did you use AI tools to help write this code? _**< NO >**_
2026-05-31 17:37:21 -04:00
Wilhelm Schuster b229428a17 chore: remove unused sleep timer interval strings (#2223) 2026-06-01 00:26:17 +03:00
Matteo Scopel 36d0e52e4b chore: new Italian fixes (#2218) 2026-05-31 17:39:15 +03:00
Uri Tauber 94c0ba466c feat: Hebrew localization (#2068) 2026-05-31 16:57:51 +03:00
muhas da2e4eabb0 fix(lang): Russian translate for new strings (#2217) 2026-05-31 16:05:47 +03:00
Matteo Scopel 3392b3e35a chore: update the Italian translation (#2207) 2026-05-30 23:42:19 +03:00
Matheus Martins 52e7446757 fix(docs): Section numbering in USER_GUIDE.md (#2195) 2026-05-30 23:34:44 +03:00
Jeremy Klein d83605e003 docs: add Claude Code contributor skills for firmware code quality (#2212) 2026-05-30 23:30:18 +03:00
WuTofuandJulia ebf0413fe1 feat: add custom sleep timer picker (#2206)
## Summary

* **What is the goal of this PR?**
Mainly fixes #2196, but also allows users to set a sleep time of 1~30
min or to never sleep.

* **What changes are included?**

1. Revert changes from #1948 and #2137 (Sorry @Uri-Tauber)
2. Cherrypick changes from
https://github.com/uxjulia/CrossInk/commit/0cb91c3dd1c63f0559d4392fadc4a8340af5bdde
and
https://github.com/uxjulia/CrossInk/commit/4e6bd634b6ede51ff1a7107ba3e8efcd152bc6ee
for a timeout interval picker UI, the custom sleep time feature, and
migrate the existing setting in v1.3.0, `sleepTimeout`, to
`sleepTimeoutMinutes` (Thanks! @uxjulia)
3. Add a "Never" at the far right of the timeout interval picker

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**PARTIALLY**_

---------

Co-authored-by: Julia <julia@uxj.io>
2026-05-30 08:28:07 -04:00
Julia 66d9e4403a fix: redesign X3's UTC offset picker so it is easier to use (#2205)
## Summary

* **What is the goal of this PR?**
Improve the X3 clock UTC offset picker so it is clearer which part of
the offset will change when the user presses up/down.

* **What changes are included?**
Replaces the single underlined `UTC +/- H:MM` string with separate
editable fields for the sign, hour, and minutes.
The editable fields are spaced out, outlined, and the currently selected
field is highlighted with a light gray background.

## Additional Context

* This is a render-only UI change in `ClockOffsetActivity`; it does not
change how UTC offsets are stored, clamped, saved, or applied.
* The selected field still cycles with Confirm, and up/down still
adjusts the active sign/hour/minute value.

Screenshot of **current design** showing the sign (+/-) highlighted:
<img width="264" height="396" alt="old utc offset"
src="https://github.com/user-attachments/assets/c88d69ea-089c-47da-b5b3-ed0ebda5b112"
/>


Screenshot of **new design** showing the sign (+/-) highlighted:
<img width="264" height="396" alt="utc offset fix"
src="https://github.com/user-attachments/assets/f51da8b3-9aab-4f48-a9d1-21c828c01499"
/>

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< YES >**_
2026-05-29 19:37:38 -04:00
Leopoldo Pla Sempere 174a24ba42 fix: complete Spanish, Catalan and Valencian strings (#2203) 2026-05-29 16:09:14 +03:00
Stefan Blixten Karlsson 9897025ded fix: swedish translation for strings added by #1337 and #2105 (#2197) 2026-05-29 11:13:57 +03:00
Uri TauberandZach Nelson f5bc554ae7 feat: add RTL support in epub and txt readers (#1700)
Co-authored-by: Zach Nelson <zach@zdnelson.com>
2026-05-29 03:25:17 -04:00
Angad Kandhari cc2079a578 docs: sync book.bin/section.bin version refs to source (#2169) 2026-05-29 03:07:57 -04:00
Djef0o 75a35f0578 chore: Update font size translation in French YAML (#2187) 2026-05-29 00:39:52 +03:00
Jeremy Klein f872f1f549 fix(docs): Remove fork with upstream attribution issues. (#2177)
Due to attribution issues with jpirnay's crosspoint fork, I am removing
it from our blessed list of forks.

I will not tolerate stolen code.
2026-05-27 22:21:10 -04:00
Justin Mitchell 02573dc592 chore: bump open-x4-sdk to 344c479 (restore X3 4-level grayscale LUT) (#2174) 2026-05-27 18:56:59 -04:00
muhasandmuhas dfefbfe9b5 feat(lang): Russian translate for new string (#2173)
Co-authored-by: muhas <mail@muhas.name>
2026-05-28 00:22:51 +03:00
36a3a0cc3a feat: epub bookmarks (#1337)
Co-authored-by: vedi0boy <nate@origin8publishing.com>
Co-authored-by: Uri Tauber <uritaube@gmail.com>
2026-05-27 15:52:01 -04:00
Uri Tauber bbb3e06eb1 fix: BOOK_CACHE_VERSION jump (#2161) 2026-05-26 22:13:09 +03:00
Vadim KaushanandClaude Sonnet 4.6 213972badc fix: navigate to TOC anchor when selecting sub-chapters (#1981)
Chapter selection previously only used the spine index, causing
navigation to always land on page 0 of the spine item. Sub-chapters that
share a spine file but differ by anchor (e.g. `chapter.xhtml#sec2`) were
silently ignored. Now the TOC anchor is passed through `ChapterResult`
and applied via the existing `pendingAnchor` mechanism.

## Summary

* This PR implements navigation to sub-chapters which didn't work
correctly previously. If a sub-chapter of the current top-level chapter
was selected, nothing happened. If a sub-chapter of another top-level
chapter was selected, reader switched to the beginning of the top-level
chapter.
* In addition, chapters now always start from a new page. This fixes
anchor to page calculation for the cases when the actual chapter content
doesn't fit on the page where the corresponding ToC anchor was found.

## Additional Context

* I might misuse `pendingAnchor` here which was previously used for
footnote navigation, please double check. I'm open to suggestions for
improvements.
* Note that the chapter selected by default when
`EpubReaderChapterSelectionActivity` opens is still wrong. I'm going to
fix this separately. This PR addresses only navigation to the selected
chapter.
* I tested this PR on my X4 and verified that navigation to a different
sub-chapter works correctly, both inside and outside the current spine.
* Some of the changes were borrowed from
https://github.com/crosspoint-reader/crosspoint-reader/pull/1455
---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**PARTIALLY**_

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-26 12:14:32 -05:00
c5e861d71c feat: <sup> and <sub> support (#2131)
## Summary

* **What is the goal of this PR?** Support for `<sup>` and `<sub>` tags.

## Additional Context

This isn't my work, but @jpirnay 's (missing you here, man!). I migrated
his work from
https://github.com/jpirnay/crosspoint-reader/commit/bcd8c32cf26447ccc792cfedd2fdbfce4fee5210
with some micro-optimizations.

Screenshots: 

[Subscript-and-Superscript-Tests_ch2_p1_10pct_55632.bmp](https://github.com/user-attachments/files/28196936/Subscript-and-Superscript-Tests_ch2_p1_10pct_55632.bmp)

[Subscript-and-Superscript-Tests_ch3_p1_21pct_77473.bmp](https://github.com/user-attachments/files/28196937/Subscript-and-Superscript-Tests_ch3_p1_21pct_77473.bmp)


---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< YES >**_

---------

Co-authored-by: jpirnay <jens@pirnay.com>
Co-authored-by: Julia <julia@uxj.io>
2026-05-26 12:38:40 -04:00
Zach Nelson 34e923d722 chore: Pin PNGdec lib_dep to 1.1.6 (#2154) 2026-05-26 01:06:24 -04:00
Jacob Latonis f4e7eaa198 feat: implement CSS enumeration through OPF directory when missing from manifest (#2148) 2026-05-25 21:23:19 -04:00
Zach Nelson 94d4b0c7bb ci: Cache PlatformIO packages between runs (#2142) 2026-05-25 17:03:10 -07:00
Zach Nelson 6dbfc29c7e test: Migrate unit tests to gtest, integrate with CI (#2144) 2026-05-25 15:51:53 -07:00
Jeremy Klein 4ee406897b feat: tiled grayscale rendering to drop the storeBwBuffer peak (#2106)
Tiled grayscale rendering to drop the storeBwBuffer peak largest
contiguous free block (the value that actually drives OOM on the C3)
from ~114 KB to ~82-90 KB.

This renders each grayscale plane band-by-band into a small (~8 KB)
scratch and
streams each band straight to controller RAM (community-sdk
writeGrayscalePlaneStrip), leaving the BW framebuffer intact. No save,
no
restore; controller RAM is re-synced for the next differential turn
directly
from the live framebuffer.

Three writers honor the active band target so per-band re-rendering
stays cheap
and correct:

- drawPixel (text) redirects writes to the band scratch and clips to it.
- renderCharImpl skips glyphs whose physical y-extent is outside the
band before
the bitmap decode (glyphIntersectsStrip), so the per-band re-render
doesn't
  pay N x glyph decode.
- DirectPixelWriter (images) writes the band scratch via getWriteTarget
instead
  of the framebuffer. Without this, image pixels wrote the live BW frame
directly and cleanup re-synced that corruption, leaving thin outlines
after
  navigating away from an image.

Controller specifics live in the SDK (X4 setRamArea windowing, X3 PTL);
the
reader checks supportsStripGrayscale() and is otherwise
controller-agnostic.

Measured on hardware (X4 and X3, text and images, visually correct):

- Grayscale scratch ~8 KB vs ~50 KB save; largest contiguous free block
held at
  full size during grayscale instead of dropping ~25-32 KB.
- X4 text page about +25 ms/page; X3 page time is dominated by its
intrinsic
  grayscale refresh, not tiling.

Depends on community-sdk #13 (the writeGrayscalePlaneStrip API). The
submodule
bump here points at that branch, so until #13 merges the submodule won't
resolve
from upstream and CI will fail there; keeping this a draft until then.
Will
rebase onto master and re-point the submodule to the merged SDK commit
once #13
lands.

Did you use AI tools to help write this code? partial
2026-05-25 18:03:01 -04:00
Joseph DiGiovanni 67973166e3 feat: Allow disabling side buttons in reader (#2105) 2026-05-25 18:00:27 -04:00
Jason HuebelandUri Tauber 50bc9325a4 docs: update USER_GUIDE.md for v1.1.0–v1.3.0 features (#2134)
Co-authored-by: Uri Tauber <uritaube@gmail.com>
2026-05-26 00:00:00 +03:00
WuTofu d736e52bdd fix: clear cache when deleting folders in FileBrowserActivity (#1892) 2026-05-25 16:47:35 -04:00
OrbisAI Security 91d17d9f48 perf: right-size (#2093) 2026-05-25 16:46:49 -04:00
Leopoldo Pla Sempere d5c28fa1f1 fix: refresh Catalan, add Valencian locale, and complete Spanish clock strings (#2063) 2026-05-25 16:41:48 -04:00
Leopoldo Pla Sempere f69650fb86 fix: validate OPF cover items as images (#2062) 2026-05-25 16:40:41 -04:00
Boris Faure e3298c6e43 docs: update presets on sd-card-fonts (#2110) 2026-05-25 16:31:38 -04:00
Chun Ming Lee 56ba9e4337 feat: add image auto-cropping in web epub optimizer (#2139) 2026-05-25 16:31:05 -04:00
Stefan Blixten Karlsson b56d3d853d fix: update Swedish translations and remove unused strings (#2090) 2026-05-25 16:30:22 -04:00
Zach Nelson e9120888fa refactor: Drop FsFile alias, use HalFile in downstream code (#2141) 2026-05-25 16:26:54 -04:00
Eloren1 d7797baff2 fix: Fill the gap under the screen for the progress bar (#2138) 2026-05-25 15:10:45 -04:00
Zach Nelson b53ac3d52c chore: Minor cleanup flagged by newer gcc (#2140)
## Summary

Minor cleanup flagged by newer gcc:
- Removed unused variables
- Removed unimplemented function declaration
- `static` -> `inline` to avoid per-TU duplication

---

### AI Usage

Did you use AI tools to help write this code? _**NO**_
2026-05-25 11:39:28 -05:00
Uri Tauber 38210be820 fix: SLEEP_TIMEOUT enum mismatch (#2137)
## Summary

* **What is the goal of this PR?** fixes #2132.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< NO >**_
2026-05-25 09:51:52 -05:00
Jeremy Klein 88b10c82a3 fix: serialize SdFat FsFile close through HalStorage mutex (#2135)
SdFat's SdSpiCard tracks SPI bus state with an unsynchronized
m_spiActive bool. When two tasks call into SdFat concurrently they can
confuse that state machine, ending with one task calling
SPIClass::endTransaction() against a paramLock the other task holds.
That trips FreeRTOS's xTaskPriorityDisinherit assert (tasks.c:5156,
pxTCB == pxCurrentTCBs[0]) and panics the system.

HalStorage already serialized every explicit method call via
storageMutex, but HalFile's destructor was `= default`, which let the
underlying SdFat FsFile destructor run close() outside any lock
(DESTRUCTOR_CLOSES_FILE=1). Any task that destructed a HalFile while
another task was mid-SD-op would race the unsynchronized state.

Move the locking discipline into HalFile::Impl::~Impl: an explicit
close() under StorageLock, then the FsFile member destructor's redundant
close() is a no-op. HalFile's special members can stay = default.

Switch storageMutex to xSemaphoreCreateRecursiveMutex so openFileForRead
and openFileForWrite can hold the lock while assigning to a HalFile&
out-param whose prior Impl needs locked teardown. Priority inheritance
still applies to recursive mutexes.

Also documented the no-bypass rule in CLAUDE.md: never call SdFat /
SdSpiCard / FsBaseFile / SDCardManager directly, never define
HAL_STORAGE_IMPL outside HalStorage.cpp.

Addresses my admittedly synthetic repro for #2047

Did you use AI tools to help write this code? partial
2026-05-24 21:48:33 -04:00
Jeremy Klein 19954aa1b5 refactor: route OTA version check through HttpDownloader (#2076)
With both OtaUpdater and HttpDownloader on esp_http_client
(https://github.com/crosspoint-reader/crosspoint-reader/pull/2074,
https://github.com/crosspoint-reader/crosspoint-reader/pull/2075),
checkForUpdate no longer needs its own client and event handler to fetch
the release JSON. It streams the response straight into
ReleaseJsonParser through a new HttpDownloader::fetchUrl(url,
DataCallback) overload, dropping the duplicate esp_http_client setup,
the HTTP_EVENT_ON_DATA handler, and the totalBytesReceived file global.

DataCallback hands body chunks to a callback without buffering. The
naive alternative, collecting the ~32KB JSON into a std::string, aborts
under -fno-exceptions: the growing allocation collides with the TLS
session's heap mid-fetch and operator new calls abort().

The OTA install path stays on esp_https_ota (flash-write streaming),
which has no HttpDownloader equivalent.

HttpDownloader.h must precede the lwip (esp_http_client) headers in
OtaUpdater.cpp, or Arduino/SdFat macros collide with lwip.


Did you use AI tools to help write this code? partial
2026-05-24 21:44:30 -04:00
Jeremy Klein 2823a4a2cd refactor: move HttpDownloader onto esp_http_client (#2075)
Based on the learnings from
https://github.com/crosspoint-reader/crosspoint-reader/pull/2074 , I
wanted to bring the same buffer savings to the rest of our HTTP Client
stack. That being said, HttpDownloader (fonts/OPDS) used the Arduino
HTTPClient.

HttpDownloader was the last consumer of the Arduino HTTPClient +
NetworkClientSecure stack. OtaUpdater already runs on esp_http_client,
so this drops the parallel HTTP/TLS implementation. It also fixes a
class of OPDS/font download failures: HTTPClient's setTimeout is uint16
and truncates, and its short per-read deadline killed slow or chunked
responses (the -11 / incomplete-data errors).

What changed:

- Rewrote fetchUrl/downloadToFile around esp_http_client with a
streaming open() -> fetch_headers() -> read() loop, manual redirect
following, and is_complete_data_received() as the completeness gate.
Body bytes go straight to the sink (OPDS parser stream, std::string, or
file), so nothing buffers the payload.

- HTTPS is now verified against the CA bundle instead of
NetworkClientSecure::setInsecure(). esp-tls is built with
CONFIG_ESP_TLS_INSECURE off, so an unverified handshake can't be set up
anyway; the model is public servers over verified https and local
servers over plain http (transport is chosen from the URL scheme).

** Self-signed https servers are no longer supported, by design. **

- timeout_ms is 60s; esp_http_client's timeout is uint32, so unlike
HTTPClient it doesn't silently truncate.

- HTTP buffers are 4096 (rx) / 1024 (tx). 4096 holds real OPDS server
headers; the GitHub release CDN sends more and logs a non-fatal
truncation warning, but the headers we read (Location, Content-Length)
come first and survive.

- Removed the now-unused UrlUtils::isHttpsUrl and a stale HTTPClient
comment in FontDownloadActivity.

Validated on device: OPDS browse and a 3.4 MB book download over
verified https, GitHub font downloads (crc-checked), redirect handling
matching curl, and slow/erroring servers surfaced correctly.


Did you use AI tools to help write this code? partial
2026-05-24 00:03:56 -04:00
Danila Yudin 929f290042 fix: close leaked resource handles (#2040) 2026-05-23 21:26:13 +03:00
Leopoldo Pla Sempere 0f021ea5f1 fix: normalize Wi-Fi spelling across remaining locales (#2094) 2026-05-23 21:01:12 +03:00
Justin Mitchell 99ab8b2772 fix: improves Edge case font/glyph handling (#2100) 2026-05-23 20:40:19 +03:00
Julia 7accc607af feat: ports hr tag rendering from crossink (#2117) 2026-05-23 09:47:16 -04:00
Julia f39ba7037f fix(settings): preserve quick resume timeout preference (#2101)
## Summary

### **What is the goal of this PR?**
This fixes an unintended settings side effect when cycling the `Sleep
Screen` option through `Quick Resume`.

Previously, selecting `Sleep Screen = Quick Resume` globally forced
`Quick Resume on Timeout = ON` and left it enabled even after the user
toggled `Sleep Screen` to another option within the same settings
session. Now the auto-enable behavior is scoped to the Settings screen
session:

- If `Quick Resume on Timeout` was already `ON` when entering Settings,
it stays `ON`.
- If it was `OFF`, selecting `Sleep Screen = Quick Resume` temporarily
turns it `ON`.
- If the user then switches away from `Quick Resume`, it turns back
`OFF`.

### **What changes are included?**

- Removes the global logic that permanently forced `Quick Resume on
Timeout` to `ON` whenever `Sleep Screen` was set to `Quick Resume`, even
if it was just due to toggling through the options.
- Adds Settings-screen session tracking so `Quick Resume on Timeout` is
only auto-enabled while the user has `Sleep Screen = Quick Resume`.
- Restores `Quick Resume on Timeout` back to `OFF` when the user
switches away, but only if it was `OFF` when they entered Settings.
- Preserves existing `ON` timeout preferences.
- Same behavior applies to the web settings

## Additional Context

- Tested this on device and via the settings UI
---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it helps set the right
context for reviewers.

Did you use AI tools to help write this code? _**< YES  >**_
2026-05-21 21:06:51 -04:00
WuTofu 2dd491b62e refactor: unify book cache clearing for epub, txt, and xtc files (#1875) 2026-05-21 14:44:10 +03:00
Matteo Scopel dc404b41c7 chore: fix the Italian translation (#2095) 2026-05-21 13:08:50 +03:00
Jeremy Klein 4ffc2a7e7e fix: sleep from a WiFi activity instead of silent-rebooting (#2092)
Unify the two splash-skip signals (RTC silent-reboot flag, SD
seamless-sleep flag) into one BootResume enum driving a single switch.
Storage unchanged; behavior-preserving apart from the fix.

Holding power to sleep from a WiFi activity (Font Download, OPDS, web
server, Calibre, KOReader sync) rebooted to home instead of sleeping.
goToSleep() runs the outgoing activity's onExit(), and those activities
call silentRestart() to clear heap fragmentation, so the heap-defrag
reboot fired before deep sleep could start.

enterDeepSleep() now latches deepSleepInProgress before goToSleep();
silentRestart()/silentRestartToReader() no-op while it's set. Deep sleep
is a full chip reset on wake, so it already clears the fragmentation the
reboot existed for.



Did you use AI tools to help write this code? partial
2026-05-21 00:03:46 -04:00
Julia c44555007b feat(settings): modify "Page as Sleep Screen" to "Quick Resume" options (#2089)
## Summary

**What is the goal of this PR?**

Adds a clearer Quick Resume sleep-screen flow. The previous “Page as
Sleep Screen” behavior is now exposed as a dedicated `Sleep Screen >
Quick Resume `option, with the timeout-only behavior controlled by a
renamed `Quick Resume on Timeout `setting.

**What changes are included?**

- Adds `Quick Resume` as a new `Sleep Screen` option.
- Renames the old `Page as Sleep Screen` setting to `Quick Resume on
Timeout`.
- Changes that setting’s choices from `Never / After Timeout / Always`
to `OFF / ON`.
- Makes `Quick Resume on Timeout = ON` equivalent to the old `After
Timeout` behavior.
- Makes `Sleep Screen > Quick Resume` equivalent to the old `Always`
behavior.
- Automatically forces `Quick Resume on Timeout` to `ON` when `Sleep
Screen` is set to `Quick Resume`.
- Renames internal setting references from `seamlessSleepScreen` to
`quickResumeSleepScreen`.
- Updates translations for the renamed setting label.

**Additional Context**

- This is mostly a settings/labeling restructure around existing
behavior, not a new rendering path.
- The runtime quick-resume behavior still uses the existing saved
framebuffer / last-screen sleep flow.
- Review focus areas:
  - Sleep entry behavior from manual sleep vs timeout sleep.
- The automatic dependency where selecting `Sleep Screen > Quick Resume`
sets `Quick Resume on Timeout` to `ON`.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< YES >**_

---
**New `Quick Resume` option for `Sleep Screen` will automatically set
`Quick Resume on Timeout` to `ON`**:
<img width="480" height="800" alt="quick resume"
src="https://github.com/user-attachments/assets/94c553fd-a122-47a8-add9-f29694f55566"
/>

**Example where a different sleep screen setting like `Cover` can be
used in combination with the `Quick Resume on Timeout` setting**:
<img width="480" height="800" alt="cover + quick resume"
src="https://github.com/user-attachments/assets/dd18ce18-230b-4b78-808b-ac85f5e7d5d8"
/>
2026-05-20 22:43:48 -04:00
Vadim Kaushan d9aa5b4de1 fix: take orientation into account for border generation in ScreenshotUtil (#1977)
## Summary

Previously `ScreenshotUtil` used physical display size to draw a border
around the screen contents. Because of this, in landscape orientation
the border was shown as a broken square. This PR changes border drawing
to use logical screen size instead of a physical display size to take
orientation into account.

## Additional Context

* Tested on X4 in all 4 reading orientations. Behavior is now correct,
however it doesn't look perfect on my X4: the border is much closer to
the physical top side of the display than to the other sides. This might
be related to assembly variation during manufacturing, but it might as
well be related to the way a eink controller is connected to the display
(controller supports bigger display sizes, so an offset may be present).

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**NO**_
2026-05-20 15:48:06 -04:00
jpirnayandArthur Tazhitdinov a7aa4c55d8 fix: Prefer epub format over derived formats when downloading from opds server (#1480)
## Summary

* **What is the goal of this PR?** Prefer epub format over kepub or
other formats offered from an OPDS server
* **What changes are included?**

## Additional Context

Should address #1419 

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**<  NO >**_

---------

Co-authored-by: Arthur Tazhitdinov <lisnake@gmail.com>
2026-05-20 11:46:11 -05:00
Jeremy Klein 3800179595 fix: wire through silent restart clear resume state with sdk (#2033)
## Summary

After a silent reboot, there was a small window where the esp32 would
listen for button presses but the full refresh would hold the event
loop. This gave a UX experience where the silent reboot had completed to
the home screen, a user taps select (at any time during the process),
and they find themselves unexpectedly in a book.

## Additional Context

This must land after
https://github.com/crosspoint-reader/community-sdk/pull/11 and will need
the submodule SHA changes included in. 

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? partially
2026-05-20 08:58:43 -04:00
Jeremy Klein 0b9d1a7d23 fix: keep wifi OTA off the heap floor (#2074)
OTA install streams the full 5.8MB image over a multi-minute TLS session
while wifi/LWIP already holds the big internal arena. Measured on
device, the arena bottomed out at ~7.7KB free with the largest
contiguous block down to ~2.4KB; for 80% of the download there wasn't
even a contiguous 8KB block. It finishes on a clean heap but tips into
OOM for anyone carrying more pre-OTA fragmentation.

Two avoidable drains, both in OtaUpdater:

- The esp_http_client RX/TX buffers were 8192/8192 on both the version
check and the install. RX only has to hold response headers (bodies
stream through the parser / OTA writer) and TX only carries our GET, so
trim both to 4096/1024. 4096 still fits the github->CDN redirect
headers; the 512 IDF default truncates them, which is why they got
oversized in the first place.

- installUpdate fired the progress callback every ~100ms perform
iteration, waking the render task on every tick. Its framebuffer work
fights the TLS session for the same arena, and epd can't really repaint
faster than a percent anyway. Throttle it to whole-percent changes.

On device, combined: floor 7.7KB -> 19KB, worstcase contiguous block
2.4KB -> 21KB, zero sub 8KB iterations across the whole download.

KOReaderSyncClient already uses small buffers; HttpDownloader is on the
Arduino HTTPClient stack with no equivalent knob, so neither changed.



Did you use AI tools to help write this code? partial, heap-exploration
assisted by Claude.
2026-05-20 08:46:49 -04:00
Justin Mitchell 252a64fa97 chore: Add funding badge for contributors in README (#2072) 2026-05-19 22:42:45 -04:00
Jeremy Klein 75764bc6eb fix: 0 -> 1, even more deep-sleep fix (#2073)
setTimeout(0) on the serial could trigger a subtle but obnoxious
underflow.

Eat a milli, save a reset button.


Did you use AI tools to help write this code? no
2026-05-19 22:40:44 -04:00
Dave Allie 0ddefdc0c9 chore: Remove FUNDING.yml (#2071)
## Summary

* I am no longer maintaining or running the project, so avoiding
collecting money for nothing
* This will likely be updated by Justin in the near future with
different details
* See
https://github.com/crosspoint-reader/crosspoint-reader/discussions/2070
2026-05-20 11:33:32 +10:00
Eloren1andClaude Sonnet 4.6 41e6e15229 feat: Seamless sleep/wake screens for displaying book pages during deep sleep (#2064)
<img width="605" height="454" alt="image"
src="https://github.com/user-attachments/assets/bfd84afe-3b58-436e-9a5d-539af3ec3d4e"
/>

Actualized "Last" sleep screen setting from previous PRs, rebranded as a
~~`Seamless Sleep`~~ `Page as Sleep Screen` option with more
improvements.


https://github.com/user-attachments/assets/59029ba6-007e-4841-abfa-f680d7e98b79

---

New option: `Page as Sleep Screen` - `Never (default)`, `After Timeout`,
`Always`

When enabled, it seamlessly sleeps on timeout or power off, making a
fast refresh for the moon icon. When waking up, we still show the last
page, instead of the boot screen, making it fully seamless.

I tried different icons such as "refresh arrow" and others, but they
looked not as nice as 3 simple dots.

With this mode, the device turns off 4 seconds faster. And has 6 seconds
less delay when turning back on. Much more responsive.

Previously, even a 10-minute timeout sometimes wasn't enough, and I'd
worry about seeing the book cover. It's now easier to use a shorter
sleep timeout: if I get distracted during a reading session but don't
want to stop, the new screen is much more inviting to come back to.

---

Did you use AI tools to help write this code? _**PARTIALLY**_.

---

Test v1.3.0 firmware.bin file
[download](https://github.com/user-attachments/files/28015193/firmware.zip)

Based on PRs #410 and #495

Closes #400, #1649

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-19 18:45:28 -04:00
Danila Yudin 082b5f295b docs: update delete endpoint reference (#1940)
## Summary

Update the `/delete` endpoint reference to match the current web server
handler.

## Details

The endpoint docs still described a `type` form field and the old
`Deleted successfully` response. The handler now accepts either `path`
for a single item or `paths` as a JSON array for multi-delete, infers
file versus folder from the SD card entry, and returns `All items
deleted successfully` when all deletes complete.

This updates the curl examples, parameter table, success response, and
error response list to match the implemented behavior.

## Validation

- Compared the documented parameters and response strings with
`CrossPointWebServer::handleDelete()`
- `git diff --check`
2026-05-19 17:39:04 -05:00
Jackson beb7876ccf feat: Make page turn naturally follow orientation (#2023) 2026-05-19 23:13:27 +03:00
Uri Tauber b69111bea5 refactor: Consolidate theme rendering into ThemeMetrics (#1868) 2026-05-19 22:50:26 +03:00
Jeremy Klein 86a9510957 fix: stabilize deep sleep wake on USB power (#2060)
When the device went into deep sleep while plugged into USB, a
power-button press would occasionally not wake it. The display held its
last frame, the chip stayed in deep sleep, and recovery required an
unplug + reset + hold-power cycle. On battery the symptom never surfaced
because the power button physically re-energises the chip.

Two peripherals were holding power domains alive across the deep sleep
boundary and interfering with the configured GPIO wake on the power
button:

1. HWCDC. Once Serial is initialized, the USB Serial/JTAG peripheral
keeps its power domain configured even with TX timeout at zero and no
host draining. Tear it down with Serial.end() in
HalPowerManager::startDeepSleep, gated by ENABLE_SERIAL_LOG to match the
Serial.begin site. This hit me if I was charging off my computer.

2. WiFi. enterDeepSleep had no WiFi teardown, so sleeping from any
network-using activity left the modem domain alive. Call
WiFi.disconnect(true) + WiFi.mode(WIFI_OFF) when WiFi is active. Wake
from deep sleep is effectively a chip reset, so no WiFi state needs to
survive. While this doesn't cause higher power drain, it apparently was
causing issues where I'd occasionally have the chip hang on sleep
transition from a wifi activity.

Confirmed on device.


Did you use AI tools to help write this code? partial
2026-05-19 12:04:38 -04:00
Jeremy Klein acc1ed4358 fix: guard DC writes in JPEGDEC MCU_SKIP path (#2058)
EIGHT_BIT_GRAYSCALE decode of a 3-component progressive JPEG calls
JPEGDecodeMCU_P with MCU_SKIP for Cb and Cr after every Y MCU. The
existing safe-pMCU patch redirects the wild pointer to &sMCUs[0] but
leaves the DC store unguarded, so each chroma skip overwrites the
just-decoded Y DC with the chroma DC predictor. Output reads sMCUs[0],
gets the trailing Cr DC (~0), and renders an all-black image.

Add `if (iMCU >= 0)` guards to the two pMCU[0] writes (main DC store and
successive-approximation update). The pointer redirect stays as the AC
wild-pointer defense; the new guards stop the silent corruption at
sMCUs[0]. The two fixes are independent and both required.

fixes the progressive 8bit grayscale jpeg regression in 1.3.0

Did you use AI tools to help write this code? partial
2026-05-19 09:52:29 -04:00
Uri Tauber 08461c08e7 fix: small QoL: return to the last selected menu location (#1629)
## Summary

* **What is the goal of this PR?** Small UX improvement to the Home
screen by preserving the last selected cursor position when returning to
it.

It supersedes #985 and #1103, which are both significantly outdated and
hundreds of commits behind master.

---

### AI Usage

Did you use AI tools to help write this code? _**< YES >**_
2026-05-19 08:41:39 -05:00
Justin Mitchell dac7fef49d fix: Update documentation with new features and links (#1991) 2026-05-19 07:32:30 +03:00
Jeremy Klein a14c8e762d perf: shrink HomeActivity cover cache from 48KB framebuffer to 16KB region (#2035)
On-device repro showed the cover snapshot pinning ~52KB of contiguous
heap (cloning the full 48KB framebuffer with malloc overhead). MaxAlloc
on Home was 61KB; nothing was leaving headroom for HTTPS, which needs
30-50KB contiguous for the mbedTLS handshake.

Add region-aware framebuffer helpers to GfxRenderer that translate a
logical rect through rotateCoordinates and copy only the byte range that
contains the rotated rect. HomeActivity records the tile rect it passes
to drawRecentBookCover and caches only that subregion.

Measured on device (X3, Portrait):

  Idle on Home    | Free 102K -> 139K  | MaxAlloc 61K -> 115K
  Mid-EPUB-read   | Free  81K -> 134K  | MaxAlloc 70K -> 115K
  Cover cache     |        ~52K -> ~16K (per allocation)

Works in all four orientations because the bounds helper samples the
four logical corners through the existing rotation, so the cached byte
range always covers the pixels the theme could have drawn into.

Savings will vary with theme, but should be significant across all.
2026-05-18 22:41:02 -04:00
Jeremy Klein a525606d7f fix: USB serial logs now flow on cold+warm boot without jiggle (#2034)
The "logs only flow if you unplug and replug the USB cable at the right
moment" symptom traced to two interacting problems with the ESP32-C3 USB
Serial/JTAG controller (HWCDC):

1. Serial.begin was gated on gpio.isUsbConnected(). That check sampled
USB state at one specific microsecond during boot. If USB enumeration on
the host hadn't completed by that moment (common after a reset that
auto- reconnects a moment later), Serial was never initialized and
stayed dead until the next boot where the timing happened to win.

2. HWCDC writes block for up to the configured TX timeout (default 250
ms) when the host has the port open but isn't actively draining — a
state the macOS USB CDC stack enters intermittently after reconnect. The
firmware then appears to hang on logging until a USB unplug+replug
cycles the peripheral and flushes the TX FIFO.

Fix: move the Serial init to the very top of setup() with a 250 ms stall
before Serial.begin (lets the USB peripheral power-on and host
enumeration complete on cold boot), and call logSerial.setTxTimeoutMs(0)
so writes drop bytes harmlessly when the host is slow instead of
stalling the firmware. Both warm reboot and cold power-on now produce
logs immediately.

Did you use AI tools to help write this code? partial
2026-05-18 22:26:25 -04:00
KemoNine df53faab91 feat: allow removing book from recent list (#2045)
## Summary

Add ability to long press 'confirm' on a book in the recent books list
to be prompted to remove it from the list.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? *Yes, Claude*

---------
2026-05-18 22:16:33 -04:00
Justin Mitchell f2adefe729 chore: Update open-x4-sdk submodule for faster page turns on x3 (#2055) 2026-05-18 21:41:40 -04:00
KemoNine 6a98c2d865 feat: add setting that allows removing books from recent list when read (#2043) 2026-05-18 21:13:41 -04:00
Justinian cfd3a381ed feat: X3 clock display with DS3231 RTC and NTP sync (#1612) 2026-05-18 21:06:56 -04:00
Jeremy Klein 151bf1dae4 fix: silent-restart on exit from KOReader auth and OTA update (#2036)
PR #1908 silent-restarts on exit from any wifi-using activity to defuse
LWIP/mbedTLS heap fragmentation, but two of the wifi-using paths slipped
through that audit:

  KOReaderAuthActivity (Settings -> KOReader sync -> Authenticate)
  OtaUpdateActivity    (Settings -> Check for update, back-out paths)

Both used WiFi.disconnect + WiFi.mode(WIFI_OFF) on exit and returned
control to Settings, leaving ~50KB of contiguous heap stranded for the
rest of the session.

Mirror the FontDownloadActivity pattern: if WiFi was activated,
disconnect and silentRestart. OTA's success path is unchanged:
SHUTTING_DOWN already calls plain ESP.restart() so the new firmware
boots normally; only the cancel/fail/no-update back-out paths now go
through silentRestart().


Did you use AI tools to help write this code? partial
2026-05-18 21:04:11 -04:00
KemoNine 06d28d6ffa feat: port crossink 'read book move' feature to crosspoint (#2032) 2026-05-18 12:39:50 -04:00
CaptainFrito 8a11f44571 feat: Themed reader menus (#1072) 2026-05-18 17:35:27 +03:00
WuTofu 061c7688a4 chore: add 3-minute sleep option (#1948) 2026-05-18 00:17:24 +03:00
muhasandmuhas 510ee10153 feat: update Russian translation (#2017)
Co-authored-by: muhas <mail@muhas.name>
2026-05-17 15:17:56 +03:00
zgredexandJustin Mitchell c6d116024c fix: harden EPUB optimiser UI gating, size reporting, and picker teardown (#1947)
Co-authored-by: Justin Mitchell <justin@jmitch.com>
2026-05-17 12:12:56 +03:00
Matteo Scopel 85e08f9a93 feat: add the Domitian font family (#2016) 2026-05-17 12:08:16 +03:00
Justin Mitchell 0af0ad5a17 fix: bump open-x4-sdk to clear grayscale state after AA cleanup (#2022)
Pulls in community-sdk PR #9, which clears inGrayscaleMode inside
cleanupGrayscaleBuffers() after the restored BW frame is written back
into RED RAM. Without this, the next BW page turn would still see the
flag set and trigger a redundant grayscaleRevert() refresh, producing
visible ghosting on the X4 with text anti-aliasing enabled.

Regression introduced by SDK commit 0a8ada2 (factory LUT grayscale
support), which removed a redundant inGrayscaleMode guard in
grayscaleRevert() and so caused the cleanup to actually run for the
first time.

Bypassing rules to avoid this going stale and all nightly builds being broken for x4 users
2026-05-17 03:45:11 -04:00
KemoNine 93e81daf41 fix: prune books missing form sd card in recent books list (#1959) 2026-05-16 22:03:49 +03:00
mvidelatraduc a3e51f9b1e chore: Update spanish.yaml (#2011) 2026-05-16 21:55:27 +03:00
Blue 90d4c885e1 fix: update URL-encoded image during EPUB optimization (#1985)
## Summary

* **What is the goal of this PR?**  
Fix EPUB optimization when XHTML image references are URL-encoded.

* **What changes are included?**  
The optimizer already converts image files to `.jpg`, but XHTML files
could still reference the original URL-encoded image path, for example:

```html
<img src="images/wensday%201%20full%202.png">
````

The optimized EPUB then contained the converted file:

```text
images/wensday 1 full 2.jpg
```

but the XHTML still pointed to the old `.png`, so CrossPoint failed to
extract/render the image.

The issue was that the previous replacement logic matched only the plain
filename form, such as:

```text
wensday 1 full 2.png
```

but not the URL-encoded form:

```text
wensday%201%20full%202.png
```

This PR updates XHTML image `src` attributes through the existing
DOMParser pass by decoding and resolving the image path before matching
it against renamed images.

After this fix, the optimized EPUB correctly rewrites the XHTML image
reference to the generated `.jpg`, and the image renders correctly.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**PARTIALLY**_

---

Please let me know if you have questions,
Thank you!
2026-05-15 21:30:49 -05:00
Kira ee947b06d1 fix: prevent card overflow on screens (#1943)
## Summary

prevent card overflow

## Additional Context

<img width="1920" height="1080" alt="bug"
src="https://github.com/user-attachments/assets/df84e233-908e-4ce1-8289-d0e9b579bc13"
/>
<img width="1920" height="1080" alt="Bug"
src="https://github.com/user-attachments/assets/cfd20f51-6421-4271-9a62-9c1987cc0dd0"
/>
<img width="1920" height="1080" alt="fix"
src="https://github.com/user-attachments/assets/4fbe83fe-5376-4393-bd45-a825f24e19e3"
/>
<img width="1920" height="1080" alt="fix2"
src="https://github.com/user-attachments/assets/4397848e-d8fa-4c51-ab4d-c32b2fcf33d1"
/>


---

### AI Usage

Did you use AI tools to help write this code? _**NO**_
2026-05-15 21:29:38 -05:00
marcinoktawian 28b907321e fix: use power button held time for shutdown logic (#1890)
## Summary

* **What is the goal of this PR?**
Fix incorrect power button long-press detection during shutdown/wake
verification by introducing dedicated power button timing logic.
* **What changes are included?**
* Added getPowerButtonHeldTime() to HalGPIO as a wrapper over input
manager logic
* Replaced generic getHeldTime() usage with power-button-specific timing
in verifyPowerButtonWakeup()
* Ensures shutdown/wake decision is based only on actual power button
hold duration, not any-button timing
  * Minor header update for new API exposure in HalGPIO.h
## Additional Context

This fixes a bug where holding another button while briefly pressing the
power button could incorrectly trigger shutdown behavior due to shared
timing state (getHeldTime()).

The change isolates power button timing to prevent cross-button
interference and makes shutdown logic reliable during multi-button
interactions.

No behavioral changes are expected outside of power-button handling
logic.

**Dependencies**
- SDK PR: https://github.com/crosspoint-reader/community-sdk/pull/3

This PR requires the `community-sdk` submodule to be updated after the
SDK change is merged.

- Fixes: #1881

---

### AI Usage
Did you use AI tools to help write this code? _**PARTIALLY**_
2026-05-15 21:28:52 -05:00
Stefan Blixten Karlsson 77afea4d95 feat: Add swedish hyphenation (#1637)
## Summary

* Add swedish hyphenation using scripts/update_hypenation.sh
* Add hyphenation test data using the Swedish translation of Andy Weir's
Project Hail Mary

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**NO**_
2026-05-15 21:27:52 -05:00
WuTofu 2f342508bc fix: several QoL updates for SD font's UI (#1965)
## Summary

* **What is the goal of this PR?**  
Improve the UI based on feedback from someone on discord

> Downloading ALL fonts feature.
> 1.1 Disable sleep when downloading, in my case went directly to sleep
just right after downloading.
> 1.2 It would be great to have and overall progress indicator as we
only have the indication of each font family
> 1.3 Any cancel or pause function might come in handy in case battery
is running out and then resume or retry with pending fonts

* **What changes are included?**  
- Now the UI can show overall progress across every file being
downloaded in the batch, not just progress inside the current family.
- Extended `HttpDownloader::downloadToFile()` to accept a cancel flag
and abort the download.
- Rendered a cancel button in the font download UI while a download is
in progress.
- `preventAutoSleep()` in `FontDownloadActivity.h` now returns true for
`state_ == COMPLETE` and `state_ == ERROR` in addition to
`LOADING_MANIFEST` and `DOWNLOADING`

## Additional Context

Not very satisfied with how `HttpDownloader.cpp` is right now, might try
to refactor it after v1.3.0

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**PARTIALLY**_
2026-05-15 21:25:48 -05:00
luca 7bb1f7ed76 fix: update Italian translation (#1970)
## Summary
* **What is the goal of this PR?** Update the Italian translation.
* **What changes are included?** Took the latest `english.yaml` as
reference and updated `italian.yaml` accordingly, translating new
strings and revising existing ones where needed. Specific changes can be
inspected from the diff.

## Additional Context
* Nothing special to flag — happy to adjust any wording the reviewer
disagrees with.

---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it helps set the right
context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY**_ — Claude
provided a first-pass draft; I revised and rewrote a substantial portion
by hand.
2026-05-15 20:52:42 -05:00
Jeremy Klein 7acc31bc34 fix: silent-reboot on wifi activity exit to clear heap fragmentation (#1908)
WiFi/LWIP/netif teardown scatters long-lived allocations across the
heap, leaving ~50KB of contiguous space unrecoverable without a reboot.

Reboot the SoC on exit from any wifi-using activity to guarantee a clean
heap. An RTC_NOINIT flag survives the reboot and tells setup() to skip
the boot splash and route the user back where they came from:
  - File transfer / Calibre / OPDS / Font download -> home
  - KOReader sync -> currently-open EPUB

Activities check WiFi.getMode() before rebooting, so backing out of the
network mode menu without joining doesn't trigger a cycle. KOSync also
esp_wifi_stop()s after the sync result so the radio is off while the
user reads it; full teardown happens at the reboot.


## Additional Context

The silent reboot skips the booting splash screen - it visibly looks
like a screen refresh. This does cause a disconnection/reconnection blip
for developers actively pulling logs over serial, but `pio device
monitor` and the like successfully reconnect and feed in the early boot
serial.
as an example: 
```
[256676] [DBG] [ACT] Exiting activity: KOReaderSync
[256706] [DBG] [MAIN] Silent restart (target=reader)

ESP-ROM:esp32c3-api1-20210207
Build:Feb  7 2021
rst:0xc (RTC_SW_CPU_RST),boot:0xf (SPI_FAST_FLASH_BOOT)
Saved PC:0x403872bc
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd72a0,len:0x990
load:0x403cbf10,len:0xac8
load:0x403ce710,len:0x4d28
entry 0x403cbf10
[22] [INF] [MAIN] Hardware detect: X4
[29] [SD] SD card detected
[43] [DBG] [CPS] Settings loaded from file
[58] [DBG] [KRS] Loaded KOReader credentials for user: jeremydk
[69] [DBG] [OPS] Loaded 1 OPDS servers from file
[69] [DBG] [UI] Using Lyra theme
[70] [DBG] [MAIN] Starting CrossPoint version 1.2.0-dev-detached-bde75787

...

[203] [DBG] [ACT] Entering activity: Reader
[211] [DBG] [EBP] Loading ePub: /Halting State - Charles Stross.epub
[221] [DBG] [BMC] Loaded cache data: 51 spine, 41 TOC entries
[246] [DBG] [CSS] Loaded 41 rules from cache
[247] [DBG] [EBP] Loaded ePub: /Halting State - Charles Stross.epub
```
---
### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _PARTIALLY_
2026-05-15 19:27:54 -05:00
938ca3fac1 chore: Update version to 1.3.0 (#1827)
## Summary

This release adds SD card fonts — the most-requested feature since
launch — brings the X3 to first-class status, redesigns the on-screen
keyboard, overhauls OPDS, and ships SD-card firmware updates. 144
changes from 53 contributors, 32 of whom are new to the project.

**🔠 SD Card Fonts**
Custom fonts are here. A complete font subsystem lets you install and
use fonts beyond the three built-in families. A new `.cpfont` binary
format packs multiple styles (regular, bold, italic, bold-italic) into a
single file per size, with on-demand glyph loading from the SD card. A
two-pass prewarm renderer bulk-reads glyphs per page, achieving
near-flash performance for Latin text and viable CJK rendering. Fonts
can be downloaded over WiFi directly from the device, uploaded via the
web interface, or copied manually to the SD card. The build pipeline
ships a 17-family font library (serif, sans, mono, accessibility) with
CI distribution via a dedicated crosspoint-fonts repository. As a bonus,
CJK characters no longer get spurious hyphens at line breaks, and an
advance-table cache eliminates 30+ second stalls during CJK section
indexing.

**📱 X3 Comes of Age**
The X3 graduates from initial bring-up to a proper target. Grayscale
antialiasing is sharper, EPUB images render correctly, OTA updates work,
and sleep screen dimensions are dialed in. The headline addition:
gyroscope-based tilt page turning via the QMI8658 IMU — tilt the device
to turn pages hands-free. SD-card firmware update support and X3
bootloader compatibility mean users can update without a USB connection.

**⌨️ Redesigned On-Screen Keyboard**
The keyboard has been completely redesigned with improved layout, better
key feedback, and a fix for the space key barely moving the cursor. Text
entry across WiFi setup, OPDS search, and KOSync login is noticeably
smoother.

**👁️ Focus Reading**
A new reading mode bolds the initial characters of each word (similar to
Bionic Reading) to create artificial fixation points, helping improve
reading speed and focus. The bolding ratio is 45%, with a minimum of 1
character and a maximum of 9, applied dynamically during indexing.

**📚 OPDS Overhaul**
OPDS gains in-catalog search with next/prev page navigation, support for
multiple servers, correct handling of relative paths and query
parameters (fixing CopyParty compatibility), and KOReader-compatible
download filenames.

**🔤 Text Rendering Refinements**
Combining marks (diacritics) now use font metrics for positioning
instead of heuristics, proportional numeral spacing is supported, and
differential rounding eliminates uneven inter-glyph gaps. Hyphenation
now recognizes ISO 639-2 language codes, nested block-level CSS styles
are tracked correctly, and horizontal CSS insets are capped at 2em to
prevent runaway margins. Bookerly has been replaced with Noto Serif for
licensing reasons.

**🎨 New Theme: RoundedRaff**
A new rounded theme joins the theme picker, with fixes for sleep cover
crop grid artifacts.

**🔋 Battery & Power**
Battery percentage smoothing on the X4 eliminates jittery readings. A
short press on the power button can be set to trigger a manual screen
refresh — handy for clearing ghosting.

**📶 WiFi & Networking**
WiFi connections now self-heal from transient drops without manual
intervention, and a dBm signal strength indicator appears during web
server sessions. WiFi networks can be edited directly from the web UI.

**🔄 KOSync**
Reading position sync is significantly more accurate. The old
character-offset approach frequently landed on the wrong paragraph after
syncing between devices — the new xpath-based mapping syncs at the
paragraph level, matching KOReader's own behavior. A separate fix
switches the HTTP layer to `esp_http_client`, and the reader now
releases ~65KB of EPUB heap before the TLS handshake — together these
eliminate the out-of-memory crashes that plagued KOSync on large books.

**🛡️ Stability**
Two memory leaks patched, a wild pointer crash in JPEGDEC MCU_SKIP
handling fixed, boot loops with large XTC files eliminated, legacy XTC
headers supported, the OTA updater now streams GitHub release JSON
instead of buffering it in RAM, and a JPEG downscaler y-axis scale
factor bug is corrected.

**🌐 Languages**
Slovenian is new. Russian, Ukrainian, Swedish, Italian, and Spanish
translations received significant updates.

---

Also in this release: **SD-card firmware updates without USB**, **file
extensions in the file browser**, **full path bar navigation**,
**end-of-book navigation improvements**, **XTC status bar**, **smarter
"Cover + Custom" sleep screens**, **set sleep cover from the BMP
viewer**, **orientation-aware popups**, **page turn buttons that follow
orientation**, **long-press delete for directories**, **context-aware
screenshot filenames with book title**, **crash reason displayed on
boot**, **empty line rendering in the TXT reader**, **wallpaper recency
buffer to prevent clustering**, **font family deletion from the
device**, **next/prev labels in the BMP viewer**, **non-breaking space
justification fix**, **README guidance for USB-locked third-party Xteink
units**, and a long tail of web UI polish, i18n memory optimizations,
and code quality improvements.

## What's Changed

### Features

* feat: add SD card font support with on-device download and web
management by @adriancaruana, @znelson, @itsthisjustin, @jpirnay, and
@mcrosson
* feat: Initial support for the x3 by @itsthisjustin in
https://github.com/crosspoint-reader/crosspoint-reader/pull/875
* feat: X3 grayscale antialiasing improvements by @juicecultus in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1607
* feat: X3 gyroscope-based tilt page turning via QMI8658 IMU by
@juicecultus in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1636
* feat(update): SD-card firmware update + X3 bootloader compatibility by
@eunchurn in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1786
* feat: self-heal from transient WiFi loss, add dBm indicator during
WebServerActivity by @jeremydk in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1780
* feat: edit wifi networks in webui by @osteotek in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1743
* feat: add OPDS search support & next/prev page navigation by @rxmmah
in https://github.com/crosspoint-reader/crosspoint-reader/pull/1462
* feat: Support for multiple OPDS servers by @osteotek in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1209
* feat: Adjust Navigation at End of Book by @nscheung in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1425
* feat: Display file extensions in File Browser by @CaptainFrito in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1019
* feat: show full path bar in file browser by @zgredex in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1411
* feat: enable manual screen refresh on power button short press by
@bdeshi in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1626
* feat: Rework "Cover + Custom" sleep screens to show covers only when
currently reading by @iandchasse in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1256
* feat: Set sleep cover from BMP viewer by @el in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1104
* feat: show crash reason on boot by @ngxson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1453
* feat: Support for proportional numeral spacing by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1414
* feat: add orientation-aware popups for reader activities by @mrtnvgr
in https://github.com/crosspoint-reader/crosspoint-reader/pull/1428
* feat: smooth battery percentage for x4 by @jonvex in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1635
* feat: context-aware screenshot filenames with book title by
@jonstieglitz in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1589
* feat(theme): add roundedraff theme and fix sleep cover crop grid
artifacts by @bunsoootchi in
https://github.com/crosspoint-reader/crosspoint-reader/pull/918
* feat: Page turn button orientation change by @mchuck in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1069
* feat: Status bar for XTC files by @leecming82 in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1849
* feat: enhance long press action to delete both files and directories
by @WuTofu in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1803
* feat: Added Slovenian translation by @thehijacker in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1551
* feat: focus reading by @vjapolitzer in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1670
* feat: add next / prev labels to bmp viewer by @Telemaniaka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1852
* feat: add font family deletion functionality by @WuTofu in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1919
* feat: separate into "Download All" and "Update All" in font manager by
@WuTofu in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1955
* feat: verify CRC32 checksum for font files by @WuTofu in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1904
* feat: increase default weight of Bitter font for improved rendering by
@uxjulia in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1922
* feat: allow unnamed intervals by @steka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1903

### Fixes

* fix: epub images not rendering correctly on x3 by @itsthisjustin in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1572
* fix: OTA update on x3 and progress bar on x4 and x3 by @itsthisjustin
in https://github.com/crosspoint-reader/crosspoint-reader/pull/1805
* fix: boot looping when opening large XTC files by @itsthisjustin in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1648
* fix: Wild pointer crash in JPEGDEC MCU_SKIP handling by @itsthisjustin
in https://github.com/crosspoint-reader/crosspoint-reader/pull/1627
* fix: two small memory leaks by @Uri-Tauber in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1628
* fix: use esp_http_client for KOSync to prevent TLS OOM on ESP32-C3 by
@trilwu in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1381
* fix: Read GH release JSON as stream in OTA updater by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1810
* fix: support legacy XTC file headers where pageTableOffset=48 by
@uxjulia in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1816
* fix: Use font metrics for combining mark positioning by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1310
* fix: Use differential rounding for consistent inter-glyph spacing by
@znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1413
* fix: Support hyphenation for EPUBs using ISO 639-2 language codes by
@znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1461
* fix: Track block style stack for nested styles by @daveallie in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1582
* fix: cap per-side horizontal CSS inset at 2em by @rhoopr in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1694
* fix: increase loadable epub size by @CSCMe in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1638
* fix: Switch to xpath map for paragraph level syncing in KOSync by
@itsthisjustin in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1686
* fix: free Epub RAM and simplify KOSync navigation via ActivityManager
by @wylanswets in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1860
* fix: improve KOSync bidirectional position matching accuracy by
@wylanswets in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1897
* fix: Fix failing very first wifi connection attempt by @jpirnay in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1521
* fix: avoid skipping chapter after screenshot by @Mraulio in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1625
* fix: back navigation from BMPViewer by @Telemaniaka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1597
* fix: Fix ghosting on exit of BMPViewer by @jpirnay in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1432
* fix: make footnotes consider orientation for gutters by @Telemaniaka
in https://github.com/crosspoint-reader/crosspoint-reader/pull/1665
* fix: footnote link text by @steka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1666
* fix: Erroneous navigation with long filenames in footnote links by
@CSCMe in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1723
* fix: prevent wallpaper clustering with 16-entry recency buffer by
@zgredex in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1606
* fix: webserver /delete API backward compatibility by @DianaNites in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1475
* fix: relative opds paths and query param with copyparty by @philips in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1535
* fix: use same file name as KOReader for OPDS downloads by @spfenwick
in https://github.com/crosspoint-reader/crosspoint-reader/pull/1286
* fix: pressing space barely moves input cursor (#1729) by @pablohc in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1733
* fix: keyboard feedback #1644 by @pablohc in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1697
* fix: pluralize folder/file counts correctly in file list summary by
@fain182 in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1701
* fix: rendering bug of scrollbar in RoundedRaff theme by @Uri-Tauber in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1814
* fix: two roundedraff bugs by @Uri-Tauber in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1851
* fix: overlap in download font list layout by @pablohc in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1900
* fix: remove duplicate 'Download Fonts' menu entry and improve
navigation by @zgredex in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1893
* fix: Add common ligatures to SD font conversion ranges by @znelson
* fix: capture instantiateVariableFont return value by @jpirnay in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1911
* fix: Roundraff theme home menu offset with no recent books by @znelson
in https://github.com/crosspoint-reader/crosspoint-reader/pull/1845
* fix: Missing navigation button labels in Roundedraff theme by
@Uri-Tauber in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1905
* fix: gracefully resolve fonts missing variants by @Uri-Tauber in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1921
* fix: distribute justifyExtra to non-breaking space tokens by
@prawnwhoyawns in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1783
* fix: remove percent rendering from activities by @mcrosson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1901
* fix: Restore performance in fontconvert_sdcard.py by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1924
* fix: Prepare SD card font caches from txt reader by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1973
* fix: make script help paths lightweight by @sabraman in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1937
* fix: Replaced Bookerly with Noto Serif for licensing reasons by
@znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1736
* fix: incorrect y-axis scale factor in jpeg nearest-neighbor downscaler
by @WuTofu in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1807
* fix: display empty lines in txt reader by @Uri-Tauber in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1841
* fix: short-press power action triggered after screenshot combo release
by @pablohc in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1853
* fix: correct Russian auto-turn translations by @a-ignatev in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1566
* fix: Update Ukrainian translations for footnotes (issue 1409) by
@mirus-ua in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1585
* fix: missing swedish translations by @steka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1667
* fix: Add swedish keyboard translations by @steka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1726
* fix: swedish translations by @steka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1762
* fix: swedish translation by @steka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1829
* fix: swedish translation by @steka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1888
* fix: Polish translation by @th0m4sek in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1909
* fix: Ukrainian-translation by @KymAndriy in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1946
* fix: Ukrainian translation by @KymAndriy in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1939
* fix: python requirements files by @steka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1768
* fix: missing requirement by @steka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1896
* fix: Use LOG_ macros in loc functions by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1794

### Internal

* refactor: redesign on-screen keyboard by @pablohc in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1644
* refactor: replace picojpeg with JPEGDEC for cover art conversion by
@jpirnay in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1517
* refactor: Refactor drawArc / fillArc for faster execution by @jpirnay
in https://github.com/crosspoint-reader/crosspoint-reader/pull/1540
* perf: replace i18n pointer tables with offset tables, strip unused
strings by @jpirnay in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1408
* refactor: Store only unique localization strings in offset buffers by
@znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1802
* refactor: Move language setting into JSON settings by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1796
* refactor: Use C++20 'requires' in ActivityResult constructor by
@znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1420
* refactor: Use default member initializers for JpegContext and
PngContext by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1435
* refactor: logPrintf and predefined log level strings by @CSCMe in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1546
* refactor: RAII scoped open/close for ZipFile by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1433
* refactor: Deduplicated BMP header writing in Xtc by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1439
* refactor: Added shared XML parser teardown helper by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1438
* refactor: Removed redundant FsFile close() calls by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1434
* refactor: Deduplicate battery drawing code and fix Lyra charging
indicator by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1437
* refactor: Deduplicate Roundraff battery drawing by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1847
* refactor: Simplify sort in GfxRenderer::fillPolygon by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1817
* refactor: Avoid vector for page turn rates list by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1818
* refactor: Use std::size instead of sizeof/sizeof by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1819
* refactor: Use fixed-size integers for BookMetadataCache data by
@znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1844
* refactor: Simplify isReaderActivity bookkeeping by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1838
* refactor: Simplify XtcReaderActivity with detectPageTurn by @znelson
in https://github.com/crosspoint-reader/crosspoint-reader/pull/1837
* refactor: change ukrainian translation to adaptation and add missing
lines by @KymAndriy in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1828
* chore: drop JPEGDEC patch in favour of upstream fix by @martinbrook in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1465
* chore: clang-format.fix.ps1 script: Add .venv to list of path
exclusions by @jpirnay in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1515
* chore: Updating sleep screen dimensions for X3 by @jensechu in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1688
* chore: Clarify X3 RTC in SCOPE.md by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1687
* chore: Improved Italian translations by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1685
* chore: change ukrainian translation to adaptation by @KymAndriy in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1684
* chore: Update spanish.yaml by @mvidelatraduc in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1717
* chore: One Italian translation tweak by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1718
* chore: git pre-commit hook for format fix by @osteotek in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1730
* chore: Update SDK to fork in CrossPoint org by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1836
* chore: Added RAM to firmware_size_history.py script by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1830
* chore: Updated docs to reflect DESTRUCTOR_CLOSES_FILE=1 by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1878
* feat: cap compressed group size at 64 KB by @jpirnay in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1913
* fix: build-script bug fixes for fontconvert{,_sdcard}.py by @jpirnay
in https://github.com/crosspoint-reader/crosspoint-reader/pull/1910
* feat: include short SHA in CROSSPOINT_VERSION by @osteotek in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1728
* feat: show long branch names by @steka in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1727
* feat: enable pio build cache by @Uri-Tauber in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1769
* style: put page name first in browser titles by @fain182 in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1703
* style: unify page headers across web UI by @fain182 in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1702
* style: move file type badges into Type column by @fain182 in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1793
* style: align action buttons vertically with page title by @fain182 in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1795
* docs: Update README with firmware flashing instructions by @ryneches
in https://github.com/crosspoint-reader/crosspoint-reader/pull/1654
* docs: fix typos by @kianmeng in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1705
* docs: update README.md to reflect the current state of crosspoint by
@Uri-Tauber in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1812
* docs: Add documentation for USB-locked Xteink devices by
@itsthisjustin in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1990
* docs: expand first use of OPDS acronym and provide a wikipedia link by
@sizezero in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1824
* docs: fix KOReader sync guide link by @sabraman in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1930
* docs: fix hyphenation updater script name by @sabraman in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1931
* fix: sd font download urls in docs by @mcrosson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1945
* fix: sd font folder paths in documentation by @mcrosson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1944
* chore: Add verbose mode to build-sd-fonts.py by @znelson in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1923

## New Contributors
* @a-ignatev made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1566
* @CSCMe made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1546
* @thehijacker made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1551
* @Telemaniaka made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1597
* @Mraulio made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1625
* @rxmmah made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1462
* @bdeshi made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1626
* @DianaNites made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1475
* @ryneches made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1654
* @zgredex made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1411
* @jonvex made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1635
* @KymAndriy made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1684
* @jensechu made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1688
* @kianmeng made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1705
* @philips made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1535
* @fain182 made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1701
* @mvidelatraduc made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1717
* @bunsoootchi made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/918
* @rhoopr made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1694
* @spfenwick made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1286
* @trilwu made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1381
* @jonstieglitz made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1589
* @uxjulia made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1816
* @mchuck made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1069
* @sizezero made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1824
* @leecming82 made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1849
* @jeremydk made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1780
* @WuTofu made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1803
* @wylanswets made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1860
* @sabraman made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1930
* @prawnwhoyawns made their first contribution in
https://github.com/crosspoint-reader/crosspoint-reader/pull/1783
* @mcrosson made their first contribution as co-author on SD card font
support

**Full Changelog**:
https://github.com/crosspoint-reader/crosspoint-reader/compare/1.2.0...release/1.3.0

---------

Co-authored-by: Justin Mitchell <justin@jmitch.com>
Co-authored-by: Chun Ming Lee <95391408+leecming82@users.noreply.github.com>
Co-authored-by: Uri Tauber <uritaube@gmail.com>
2026-05-15 15:51:43 -05:00
Uri TauberandZach Nelson 0fbe6ff469 fix: update README.md to reflect the current state of crosspoint (#1812)
## Summary

As noted in
[#1680](https://github.com/crosspoint-reader/crosspoint-reader/discussions/1680#discussioncomment-16661106),
the README hasn't been updated in a while and has fallen behind the
actual firmware. This PR brings it up to date.

Beyond the feature list, I added a section acknowledging community forks
worth knowing about. I also took some deliberate editorial choices
around how CrossPoint is framed — I think it has the potential to be
more than just "an alternative Xteink firmware", and the wording
reflects that.

A note on process: I wrote the bulk of the text myself, but used AI
tools to scan the codebase and catch features I might have missed, and
to clean up my English (I'm fluent but not a native speaker). If any
line reads as unnatural or AI-sounding, please flag it — I'd rather fix
it than leave it.

---

One thing outside the scope of this PR: I think the cover photo could
use a refresh, ideally replaced with a small gallery showing different
CrossPoint screens. If you have a professional camera and an Xteink
device and want to help with that, let me know.

---------

Co-authored-by: Zach Nelson <zach@zdnelson.com>
2026-05-15 15:38:22 -05:00
Justin Mitchell 43b20bd8be fix: Add documentation for USB-locked Xteink devices (#1990)
Document the Xteink Unlocker tool requirement for third-party purchased
xteink units that ship with USB flashing locked. Include warnings about
bricking risks when flashing unsupported firmwares (e.g. Papyrix) on
locked devices, as they may permanently lock the device with no recovery
path.
2026-05-15 10:44:47 -05:00
Zach NelsonandJustin Mitchell b186529120 fix: Prepare SD card font caches from txt reader (#1973)
## Summary

SD card font fixes:
- `TxtReaderActivity` needs to call `renderer.ensureSdCardFontReady` to
build the advance lookup table to support rendering with SD card fonts.
This revealed that `TxtReaderActivity` was inconsistently performing
layout with `getTextWidth`, when the renderer actually uses
`getTextAdvanceX`, which can lead to minor inconsistencies in alignment.
- Avoid allocating one big `allText` string in
`ParsedText::layoutAndExtractLines`. Instead, pass the vector of word
strings directly to `SdCardFont::buildAdvanceTable`, where the algorithm
just needs to iterate codepoints anyway.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**PARTIALLY**_

---------

Co-authored-by: Justin Mitchell <justin@jmitch.com>
2026-05-15 09:50:41 -05:00
IjonFryderykandClaude Opus 4.6 5fa5a71ba2 feat: Add Polish hyphenation support (#1590)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-14 16:32:59 +03:00
Zach Nelson aa43bd3569 chore: Removed unused icon header files (#1975)
## Summary

Deleted two unused header files containing binary icon data.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**NO**_
2026-05-14 07:34:35 -05:00
Zach Nelson 8377ac9e31 refactor: Added utils for non-throwing memory allocation and scoped cleanup (#1832)
## Summary

Pared down version of #1418.

Following up on b5df6cb2b5. Added
lib/Memory/Memory.h with:
- `makeUniqueNoThrow<T>` a `nothrow` wrapper for `std::make_unique` that
return `nullptr` on OOM instead of calling `abort()` (the behavior of
bare `new` with `-fno-exceptions`)
- `ScopedCleanup` a helper to call a cleanup lambda on scope exit.

These utilities help to write code that handles OOM scenarios
gracefully, and consistently cleans up resources on scope exit.

JpegToBmpConverter.cpp has been converted to use these utilities. Other
files can be converted later.

This will simplify some of the SD card font resource management in
#1327.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**NO**_
2026-05-13 21:19:47 -05:00
Fabio Barbonanddrbourbon 71518b3336 fix: KOSync authentication with Calibre-Web-Automated (#1951)
## Summary

* **What is the goal of this PR?** Fix regression of
Calibre-Web-Automated authentication
* **What changes are included?** Trivial fix using ESP HTTP Client
primitives

## Additional Context

Calibre-Web-Automated KOSync stopped working after refactoring HTTP
client to use `ESP HTTP Client`, resulting in a _Server error (try again
later)_ error.

---

### AI Usage

Did you use AI tools to help write this code? _**NO**_

---------

Co-authored-by: drbourbon <fabio@MacBook-Air-di-Fabio.local>
2026-05-13 08:32:18 -05:00
Zach Nelson 30b14f2ecf refactor: Removed SdCardFontGlobals.h (#1962)
## Summary

First of several changes to decouple and clean up SD card fonts
integration. This change eliminates SdCardFontGlobals.h:
- Simply declare the `extern SdCardFontSystem sdFontSystem` in
SdCardFontSystem.h.
- `ActivityManager::goToReader` should not care about loading SD card
fonts. Instead do the same work in `ReaderActivity::onEnter`, after the
previous activity has exited and after ReaderActivity has validated the
file path.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**NO**_
2026-05-13 08:26:56 -05:00
Zach Nelson 6f7f4c592a refactor: Eliminated relative path includes (#1961)
## Summary

Relative includes can hide inappropriate dependency relationships. In
this case, I found that lib/KOReaderSync/KOReaderCredentialStore.cpp was
dependent on src/JsonSettingsIO.h -- a lib -> app dependency, the
opposite direction dependencies should flow in this project.

This change replaces all relative includes with root-relative includes,
and corrects the KOReaderCredentialStore dependency by moving its JSON
settings serialization local to the KOReaderSync library.

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**PARTIALLY**_
2026-05-13 08:26:30 -05:00
zgredex bc57e5d64b fix: jump page on hold in font family and language selection (#1925)
## Summary
- Holding the navigation button in **Settings → Reader → Font family**
now advances the selection by a full visible page instead of one item at
a time.
- Same fix applied to **Settings → Language**, which had the same
one-item-only behavior.
- Mirrors the pattern already used in font download (ceb3fed) and
chapter selection screens.

## Test plan
- [ ] Settings → Reader → Font family: tap moves by one item; hold jumps
a page (wraps at ends)
- [ ] Settings → Language: tap moves by one item; hold jumps a page
(wraps at ends)
2026-05-12 17:37:56 -04:00
Chun Ming LeeandUri Tauber db3bb850b2 fix: handle fallbacks for advance table and prewarm (#1929)
## Summary
- Fixes #1928 by having the prewarm and advance table functions resolve
fallback styles
---

### AI Usage
Did you use AI tools to help write this code?  YES - Codex

---------

Co-authored-by: Uri Tauber <uritaube@gmail.com>
2026-05-12 22:11:37 +03:00
Uri Tauber 30a209de8b fix: Characters from unsupported characterset are overlapping (#1958)
## Summary

* **What is the goal of this PR?** fix #1956.

## Additional Context



[NixOS-and-Flakes-Book_ch3_p7_4pct_130044.bmp](https://github.com/user-attachments/files/27648392/NixOS-and-Flakes-Book_ch3_p7_4pct_130044.bmp)

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< YES >**_
2026-05-12 14:00:17 -05:00
431 changed files with 62709 additions and 61895 deletions
+33
View File
@@ -0,0 +1,33 @@
# CrossPoint Reader: Claude Code skills
Project skills for Claude Code. Claude auto-discovers them and loads one when the
task matches its `description`; you do not invoke them by hand. They encode how
this project wants C/C++ written: the judgment calls and self-review gates that
keep the firmware small, stable, and reviewable.
These are written for capable agents, not beginners. They are principle- and
decision-focused on purpose. They deliberately avoid line-number citations,
which drift; they anchor on durable names (APIs, types, macros, files).
This is separate from `.skills/SKILL.md`, the GitHub coding-agent guide that
mirrors CLAUDE.md. CLAUDE.md stays the always-loaded rule set; these skills are
the applied decision procedures that load on demand and add the judgment layer
CLAUDE.md does not carry.
| Skill | Loads when you are... |
|---|---|
| `heap-discipline` | allocating memory: new/malloc/vector/string, buffers, caches |
| `control-flow-clarity` | writing branching logic, state flags, modes, if/else ladders |
| `hal-and-abstractions` | touching storage, input, display, settings, i18n, rendering |
| `scope-discipline` | adding a feature, activity, lib, setting, or dependency |
| `refactor-for-review` | refactoring, cleaning up, or preparing a change for PR |
Each skill ends with a self-review checklist Claude runs against its own diff
before handing it back. Reviewing a PR? Those checklists double as a fast rubric.
## Maintaining these
Edit the `SKILL.md` under each directory. Keep them tight. Do not restate
CLAUDE.md; add the judgment CLAUDE.md cannot afford to carry. Trigger quality
lives in the `description` field: it must name the situations that should pull
the skill in, in the words a contributor's task would use.
@@ -0,0 +1,56 @@
---
name: control-flow-clarity
description: Branching and state-modeling clarity in C/C++. Use when writing or refactoring logic that branches on discrete values: if/else-if ladders, status flags, mode or state ints, or anything that should be an enum plus an exhaustive switch. Covers enum class over magic ints, exhaustive switch over nested if, early-return guard clauses, table dispatch, and when each is the right call.
---
# Control-Flow Clarity
The goal: a reviewer verifies correctness by reading, not by tracing. Branching
that mirrors the problem's shape is self-evident; branching that encodes it in
ad-hoc ints and nesting forces the reader to reconstruct intent.
## Core moves
- **Model a closed set of states/modes as an `enum class`, not ints or bools.**
A variable kept honest by a comment ("0 = hidden, 1 = showing, 2 = confirm")
is a latent bug. Make it an enum and the comment becomes the type.
- **Dispatch on an enum with an exhaustive `switch`, no `default`.** This
codebase relies on it: omitting `default` lets the compiler flag the
unhandled case when someone adds an enum value. A `default:` that swallows the
unknown case throws that safety away. Add `default` only when "every other
value does nothing" is a deliberate, documented decision.
- **Replace nested `if/else-if` ladders that branch on one discriminant with a
`switch`.** If each branch only maps input to a value, prefer a lookup table
(`static constexpr` array) over both.
- **Prefer early-return guard clauses over nested success bodies.** Handle the
error/empty/skip cases first and return; keep the main path at the left
margin.
## When NOT to switch
- Branches test unrelated conditions, not one discriminant: a guarded `if`
sequence is honest; a switch would be forced.
- Two outcomes on a genuine boolean: keep the `if`.
- The discriminant is an open or unbounded set (arbitrary ints, strings): table
or map, not a switch.
## Enum hygiene
- `enum class` by default for type safety. Plain `enum` only when values must
implicitly convert (e.g. a value that doubles as a UI dropdown index), and
then give it a trailing `_COUNT` sentinel for safe bounds/iteration, matching
the existing settings enums.
- Name the discriminant after what it selects, not its storage:
`Orientation orientation`, not `uint8_t mode`.
- No magic numeric codes for states. If you write a comment mapping numbers to
meanings, you owe an enum.
## Self-review
- [ ] No int/bool standing in for a closed set of modes; it is an `enum class`.
- [ ] Enum dispatch is an exhaustive `switch` with no catch-all `default` (or
the `default` is a documented deliberate choice).
- [ ] No nested if/else-if ladder on a single discriminant that should be a
switch or a table.
- [ ] Error/skip cases are early-return guards; the happy path is not buried.
- [ ] No magic numbers where a named enum or `constexpr` would state the intent.
@@ -0,0 +1,59 @@
---
name: hal-and-abstractions
description: Layering and abstraction discipline for the firmware. Use when touching storage, input, display, settings, i18n, or rendering, or any code that could reach into the SDK. Covers routing through the HAL (HalStorage / HalGPIO / HalDisplay) instead of raw SDK classes, MappedInputManager logical buttons instead of raw GPIO indices, UITheme/GUI for all rendering, the singleton macros, tr() for user-facing text, and where a new abstraction boundary belongs.
---
# HAL and Abstractions
CLAUDE.md lists the HAL classes and the SdFat-concurrency reason they exist.
This is when and how to route through them, and where to draw a new boundary.
## Route through the layer, always
- **SD card I/O:** `Storage` (HalStorage) and `HalFile`. Never `SdFat`,
`FsFile`, `SdSpiCard`, `FsBaseFile`, or `SDCardManager` directly. The HAL
serializes every SD access through one mutex; bypassing it races the SPI state
machine and panics FreeRTOS (CLAUDE.md has the failure mode). This is a
correctness boundary, not a style preference.
- **Display:** `HalDisplay` over `EInkDisplay`. **Input:** `HalGPIO` over
`InputManager`.
- **Rendering:** everything through the `GUI` macro (UITheme) and the renderer's
oriented metrics. No hardcoded fonts, colors, coordinates, or 800/480
literals; ask the renderer for width/height and use the oriented viewable
area.
- **Input in activities:** `MappedInputManager::Button` logical enums
(`Button::Confirm`, `Button::PageForward`, ...). Never raw `HalGPIO::BTN_*`
indices outside `ButtonRemapActivity`. Logical buttons survive user remapping
and orientation; raw indices do not.
- **Shared state:** the singleton macros (`SETTINGS`, `APP_STATE`, `GUI`,
`Storage`, `I18N`), not threaded pointers.
## User-facing text
Every string a user reads goes through `tr(STR_*)`. Add the key to the English
YAML, regenerate with `scripts/gen_i18n.py`, then use the `StrId`. Log lines
(`LOG_*`) stay hardcoded.
## Drawing a new boundary
When you need an SDK capability the HAL does not expose yet, **add the method to
the HAL; do not reach around it.** The new method inherits the mutex, logging,
and error contract the rest of the HAL carries. A one-off direct SDK call in an
activity is exactly the layering violation the mutex discipline cannot tolerate.
Keep abstractions thin. A wrapper that only renames an SDK call without adding
the mutex, logging, or an error contract is dead weight. Add a layer only when
it carries one of those contracts or hides a real implementation choice.
## Self-review
- [ ] No direct SdFat / FsFile / SDCardManager / EInkDisplay / InputManager use
outside `lib/hal`.
- [ ] File access uses `HalFile`; no `.close()` on a local handle
(DESTRUCTOR_CLOSES_FILE); members closed in `onExit`.
- [ ] Input uses `MappedInputManager::Button`, not raw `BTN_*` indices.
- [ ] Rendering goes through GUI/UITheme and oriented metrics; no 800/480 or
hardcoded fonts/coords.
- [ ] User-facing strings use `tr(STR_*)`; new keys added to YAML and
regenerated.
- [ ] Any new SDK capability is exposed as a HAL method, not called inline.
+65
View File
@@ -0,0 +1,65 @@
---
name: heap-discipline
description: Memory allocation discipline for the ESP32-C3 (~380KB RAM, no PSRAM, single 48KB framebuffer). Use whenever writing or reviewing code that allocates: new / malloc / std::vector / std::string, buffers, caches, or anything held across a loop or an activity lifecycle. Covers makeUniqueNoThrow vs raw new/malloc, fragmentation avoidance, reserve-before-push_back, alloc-once-reuse, stack vs heap sizing, and the chunked grayscale buffer pattern.
---
# Heap Discipline (ESP32-C3)
CLAUDE.md states the allocation rules. This is the procedure you run while
writing the code and the gate you run before handing it back.
The constraint that makes every call matter: ~380KB RAM, no PSRAM, one 48KB
framebuffer. **Fragmentation, not total usage, is what kills this device.**
Free-heap can read fine while the largest free block is too small for the next
allocation. Optimize for not leaving holes, not just for using fewer bytes.
## Allocation decision procedure
Ask in order; stop at the first yes.
1. **Stack?** Local, bounded, under ~256 bytes total: plain array/struct. No
heap, no fragmentation. Keep frames lean; the task stack is small.
2. **Compile-time constant?** `static constexpr` lives in flash, costs zero DRAM.
3. **Allocated once and reused for an activity's lifetime?** Allocate in
`onEnter`, hold in a member, release in `onExit`. Never per-frame, never
per-iteration.
4. **Dynamic and fallible?** `makeUniqueNoThrow<T>(...)` /
`makeUniqueNoThrow<T[]>(n)` from `lib/Memory/Memory.h`. Null-check, `LOG_ERR`
with the size, return false. It frees on every exit path.
5. **A C/SDK API takes ownership and frees it itself?** Only then raw
`new (std::nothrow)` / `malloc`, with a comment naming who frees it.
Bare `new` / `new[]` is never correct here: under `-fno-exceptions` it calls
`abort()` on OOM instead of returning null.
## Fragmentation rules
- `std::vector`: `reserve(n)` before any `push_back` loop. Each growth is
alloc-copy-free (three heap ops) and leaves a hole. Unknown n: estimate high.
- No repeated `new`/`delete` or growing containers inside a loop or render path.
Hoist the allocation out of the loop.
- Large contiguous blocks fragment worst. Full-screen-class buffers use the
chunked `storeBwBuffer` / `restoreBwBuffer` path in `GfxRenderer` so they
never demand one contiguous 48KB block. Reuse that path. Do not malloc a
second full-screen buffer.
- `std::string` / Arduino `String`: acceptable on cold paths (file I/O, one-shot
setup). Banned on hot/render paths. Build text with a stack `char[]` +
`snprintf`; if a `std::string` is unavoidable, `reserve` it first.
## Justify every allocation
Per CLAUDE.md's evidence rule: when you add a heap allocation, state in one line
why stack/static/reuse was rejected and the worst-case size. If you cannot name
the size, you cannot budget it, and you should not allocate it.
## Self-review before handoff
- [ ] No bare `new`/`new[]`. Every fallible alloc is `makeUniqueNoThrow`, or a
raw alloc with an explicit owner comment.
- [ ] Every allocation is null-checked with `LOG_ERR` before the error return.
- [ ] No allocation inside a loop or render path that could be hoisted.
- [ ] Every `push_back` loop has a preceding `reserve`.
- [ ] Anything allocated in `onEnter` is released in `onExit`; member `HalFile`
closed there too.
- [ ] No second full-screen buffer; grayscale uses store/restoreBwBuffer.
- [ ] Each new allocation carries a one-line size + why-not-stack/static note.
@@ -0,0 +1,59 @@
---
name: refactor-for-review
description: Producing small, single-concern, reviewable changes. Use when refactoring, cleaning up, restructuring, decomposing, or preparing a change for PR, especially in this multi-contributor AI-assisted codebase that is prone to sprawl diffs. Covers one-concern-per-commit, extracting helpers without widening scope, not bundling unrelated edits, decomposing oversized activities, comment hygiene, and a pre-handoff self-review checklist.
---
# Refactor for Review
This is a multi-contributor, AI-assisted codebase, and the dominant failure mode
is the sprawl diff: a one-line intent that touches thirty files. The goal is a
change a reviewer can verify in one sitting. Cleaner structure that makes the
next change easier is the win, not lines added.
## One concern per change
- A commit/PR does one thing. A bug fix is not also a rename is not also a
reformat. If you spot an unrelated improvement mid-change, leave it or capture
it separately; do not fold it in.
- When the working tree has bundled two changes, separate them with the
copy-affected-files-aside, reset, re-apply one concern, restore the rest
pattern, not by committing the tangle.
- Refactor and behavior change do not ride together. A pure refactor must not
alter behavior; a behavior change should not drag a refactor along. If both
are needed: two commits, refactor first.
## Keep the diff narrow
- Extract a helper to remove real duplication or to name a concept, not to chase
abstraction. Three-plus copies, or a block that needs a name to be understood:
extract. Two similar lines: leave them.
- No "while I'm here" scope creep. A signature or type change that ripples to
many call sites is its own PR: map every caller first, update them in one
topological pass, and land it separately, not as a rider on a feature.
- Match the surrounding code: comment density, naming, idiom. The diff should
read like the file, not like a different author.
## Decompose oversized units
An activity or function that has outgrown one screen of responsibility (multiple
unrelated state machines, or a file far larger than its siblings) is a
decomposition candidate. Extract a cohesive sub-responsibility into its own
unit, as a standalone behavior-preserving refactor, verified on its own, never
mixed into a feature change.
## Comments earn their place
Comments explain why: an invariant, a defense, a past incident, a non-obvious
constraint. Never what the next line already says. Delete narration, phase-marker
comments ("now we loop over..."), and restated function names. If a comment and
the code it sits on say the same thing, the comment is the thing to cut.
## Self-review before handoff
- [ ] The change does exactly one thing; nothing unrelated rode along.
- [ ] Refactor and behavior change are not mixed in one commit.
- [ ] No "while I'm here" creep; rename/signature ripples are split out.
- [ ] Extractions remove real duplication or name a real concept, not
speculative abstraction.
- [ ] New comments say why, not what; no narration or phase markers.
- [ ] A reviewer can understand the diff without running it.
+54
View File
@@ -0,0 +1,54 @@
---
name: scope-discipline
description: Feature-scope discipline for a dedicated e-reader (not a Swiss Army knife). Use when adding a feature, a new activity, a new lib, a setting, or a dependency, or when a request would grow the firmware's surface. Covers the SCOPE.md test, the RAM-cost vs reading-benefit gate, preferring no-code or existing-mechanism solutions, awareness of the existing activity surface, and how to push back on out-of-scope asks.
---
# Scope Discipline
The mission: do one thing exceptionally well, focused reading on constrained
hardware. `SCOPE.md` is the source of truth for what is in and out. Read it
before adding surface. This is the gate to run before writing a new feature.
## The gate
Before adding a feature, activity, lib, setting, or dependency, answer in order:
1. **Is it in `SCOPE.md`?** Explicitly out: interactive apps (notepad,
calculator, games), active connectivity (RSS, news, browser), media/audio
playback. If it is out, say so and stop.
2. **Does it materially improve focused reading?** If the benefit is
"nice to have" or serves a different use case, it is out. This is not a PDA.
3. **What does it cost in RAM and in the largest-free-block budget?** A feature
that adds steady-state RAM or a large transient allocation needs a reading
benefit that clearly outweighs it. Quantify with `firmware_size_history.py`
and `script_profile_mem.sh` rather than guessing.
4. **Can it be done with no new code?** Prefer an existing activity, an existing
setting, or a doc over a new code path. The cheapest feature is the one
already built.
If a request fails the gate, push back with the specific reason and the
`SCOPE.md` basis, and offer the in-scope alternative. Make the call and say why;
do not just hand over a menu.
## Surface awareness
The firmware already carries dozens of activities. Each new one is permanent
RAM, permanent maintenance, and another thing every future refactor must not
break. Default to extending an existing activity or setting before adding a new
screen. New top-level surface needs a real justification, not "it would be
convenient."
## Settings are not free
A new setting is a field to persist, migrate, validate, translate, and render,
plus combinatorial test burden. Add one only when users genuinely need the
choice; otherwise pick a sensible fixed default.
## Self-review
- [ ] Checked against `SCOPE.md`; not on the out-of-scope list.
- [ ] Stated the concrete reading benefit, not a generic "useful."
- [ ] Named the RAM/size cost (measured, not guessed) and why the benefit wins.
- [ ] Checked whether an existing activity/setting/doc already covers it.
- [ ] New setting (if any) is justified by a real user need, not added
"just in case."
+1 -2
View File
@@ -1,2 +1 @@
github: [daveallie]
ko_fi: daveallie
custom: ["https://app.royalty.dev/crosspoint-reader/crosspoint-reader"]
+13
View File
@@ -0,0 +1,13 @@
blank_issues_enabled: false
contact_links:
- name: Scope or Roadmap Question (start here if unsure)
url: https://github.com/crosspoint-reader/crosspoint-reader/discussions
about: |
Not sure if your idea fits CrossPoint's scope? Start a Discussion before filing an issue.
See SCOPE.md and ROADMAP.md for what is in, out, and currently paused.
- name: Read the Scope document
url: https://github.com/crosspoint-reader/crosspoint-reader/blob/master/SCOPE.md
about: The authoritative list of what CrossPoint will and will not accept.
- name: Read the Roadmap
url: https://github.com/crosspoint-reader/crosspoint-reader/blob/master/ROADMAP.md
about: Current phase, what is being closed out, and what comes next.
@@ -0,0 +1,80 @@
name: Feature Request
description: Propose a new feature, enhancement, or change to CrossPoint
title: "Short, descriptive title of the request"
labels: ["enhancement", "needs-scope-review"]
body:
- type: markdown
attributes:
value: |
Thanks for taking the time to propose a change to CrossPoint!
**Before you continue, please read [SCOPE.md](../blob/master/SCOPE.md) and [ROADMAP.md](../blob/master/ROADMAP.md).**
CrossPoint is intentionally narrow. Most rejected proposals are rejected for scope reasons that are already
documented. The checklist below exists to save both of us time.
If you are not sure whether your idea fits, open a [Discussion](../../discussions) first instead of filing
this issue.
- type: checkboxes
id: scope-check
attributes:
label: Scope Self-Check (required)
description: Please confirm each of the following. If any are unchecked, your issue will likely be closed.
options:
- label: I have read SCOPE.md and ROADMAP.md.
required: true
- label: This is **not** a new theme or theming change (themes are temporarily closed pending the move to SD-loaded themes).
required: true
- label: This is **not** a new external network connector (sync engine, cloud storage, remote file access, OPDS extensions beyond what exists, or any new "talk to a server" feature).
required: true
- label: This is **not** an interactive app (game, calculator, notepad), writing/authoring tool, RSS/news/browser feature, media playback feature, or PDF rendering.
required: true
- label: The stock firmware does **not** already handle this well.
required: true
- label: No other popular CrossPoint fork already handles this well (or, if one does, I explain below why CrossPoint still needs it).
required: true
- type: textarea
id: problem
attributes:
label: Problem this solves
description: What user-facing problem or reading-experience gap does this address? Be concrete.
placeholder: e.g., "When reading in landscape, paragraph breaks are inconsistent because..."
validations:
required: true
- type: textarea
id: stock-gap
attributes:
label: Why the stock firmware (and other forks) do not already solve this
description: Explain which existing solutions you checked and why they fall short. This is the core scope filter.
validations:
required: true
- type: textarea
id: proposal
attributes:
label: Proposed change
description: A short description of what you would build or change. Focus on user impact; implementation details can come later.
validations:
required: true
- type: textarea
id: tradeoffs
attributes:
label: Memory / flash / complexity cost
description: |
CrossPoint runs on 380KB of RAM. Roughly how much DRAM, flash, or code complexity does this add?
"Don't know" is a valid answer, but please attempt an estimate.
placeholder: e.g., "Adds ~2KB flash for the new font tables, no DRAM impact at runtime."
validations:
required: false
- type: textarea
id: additional-context
attributes:
label: Additional context
description: Anything else relevant (links, screenshots, related discussions).
validations:
required: false
+18
View File
@@ -3,10 +3,28 @@
* **What is the goal of this PR?** (e.g., Implements the new feature for file uploading.)
* **What changes are included?**
## Scope Check
CrossPoint is intentionally narrow. See [SCOPE.md](../blob/master/SCOPE.md) and [ROADMAP.md](../blob/master/ROADMAP.md).
Please confirm:
- [ ] I have read SCOPE.md and ROADMAP.md.
- [ ] This PR is **not** a new built-in theme (themes are temporarily closed pending the move to SD-loaded themes).
- [ ] This PR is **not** a new external network connector (sync engine, cloud storage, remote file access, etc.).
- [ ] This PR is **not** an interactive app, writing tool, RSS/news/browser, media playback, or PDF feature.
- [ ] The stock firmware does not already handle this well, **and** no other popular CrossPoint fork already does
(or, if one does, I explain why CrossPoint still needs it below).
- [ ] If this PR touches `freeink-sdk/`, `lib/hal/`, the bootloader, OTA, or recovery code, I have coordinated with
the relevant maintainer.
**If this PR was opened against the previous (broader) scope and was already in flight under Phase 0, link the
relevant Discussion or issue so reviewers can see the history.**
## Additional Context
* Add any other information that might be helpful for the reviewer (e.g., performance implications, potential risks,
specific areas to focus on).
* Memory / flash impact, if known.
---
+47 -1
View File
@@ -53,6 +53,13 @@ jobs:
- name: Install PlatformIO Core
run: uv pip install --system -U https://github.com/pioarduino/platformio-core/archive/refs/tags/v6.1.19.zip
- name: Cache PlatformIO packages
uses: actions/cache@v4
with:
path: ~/.platformio
key: pio-${{ runner.os }}-${{ hashFiles('platformio.ini') }}
restore-keys: pio-${{ runner.os }}-
- name: Run cppcheck
run: pio check --fail-on-defect low --fail-on-defect medium --fail-on-defect high
@@ -76,10 +83,21 @@ jobs:
- name: Install PlatformIO Core
run: uv pip install --system -U https://github.com/pioarduino/platformio-core/archive/refs/tags/v6.1.19.zip
- name: Cache PlatformIO packages
uses: actions/cache@v4
with:
path: ~/.platformio
key: pio-${{ runner.os }}-${{ hashFiles('platformio.ini') }}
restore-keys: pio-${{ runner.os }}-
- name: Build CrossPoint
# Both envs in ONE pio invocation: a second `pio run` can wipe the whole
# .pio/build tree (project-checksum mismatch after the S3 toolchain
# installs into the restored cache), deleting the default firmware.bin
# before the artifact upload.
run: |
set -euo pipefail
pio run | tee pio.log
pio run -e default -e sticky | tee pio.log
- name: Extract firmware stats
@@ -102,6 +120,33 @@ jobs:
path: .pio/build/default/firmware.bin
if-no-files-found: error
unit-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
with:
submodules: recursive
- name: Install build tools
run: |
sudo apt-get update
sudo apt-get install -y cmake ninja-build
- name: Cache googletest source
uses: actions/cache@v4
with:
path: build/test/_deps/googletest-src
key: ${{ runner.os }}-googletest-${{ hashFiles('test/CMakeLists.txt') }}
- name: Configure
run: cmake -S test -B build/test -G Ninja -DCMAKE_BUILD_TYPE=Release
- name: Build
run: cmake --build build/test
- name: Run tests
run: ctest --test-dir build/test --output-on-failure -j
# This job is used as the PR required actions check, allows for changes to other steps in the future without breaking
# PR requirements.
test-status:
@@ -110,6 +155,7 @@ jobs:
- build
- clang-format
- cppcheck
- unit-tests
if: always()
runs-on: ubuntu-latest
steps:
+7
View File
@@ -25,6 +25,13 @@ jobs:
- name: Install PlatformIO Core
run: uv pip install --system -U https://github.com/pioarduino/platformio-core/archive/refs/tags/v6.1.19.zip
- name: Cache PlatformIO packages
uses: actions/cache@v4
with:
path: ~/.platformio
key: pio-${{ runner.os }}-${{ hashFiles('platformio.ini') }}
restore-keys: pio-${{ runner.os }}-
- name: Build CrossPoint
run: pio run -e gh_release
+7
View File
@@ -25,6 +25,13 @@ jobs:
- name: Install PlatformIO Core
run: uv pip install --system -U https://github.com/pioarduino/platformio-core/archive/refs/tags/v6.1.19.zip
- name: Cache PlatformIO packages
uses: actions/cache@v4
with:
path: ~/.platformio
key: pio-${{ runner.os }}-${{ hashFiles('platformio.ini') }}
restore-keys: pio-${{ runner.os }}-
- name: Extract env
run: |
echo "SHORT_SHA=${GITHUB_SHA::7}" >> $GITHUB_ENV
+6
View File
@@ -2,6 +2,8 @@
.idea
.DS_Store
.vscode
open-x4-sdk
fs_
lib/EpdFont/fontsrc
lib/I18n/I18nKeys.h
lib/I18n/I18nStrings.h
@@ -19,3 +21,7 @@ build
lib/EpdFont/scripts/downloaded_fonts/
lib/EpdFont/scripts/instanced_fonts/
lib/EpdFont/scripts/output/
# Claude Code: share project skills with the team, keep local agent state
# (worktrees, scheduled-task locks, settings.local, scout CLEANUP.md) out.
.claude/*
!.claude/skills/
+4 -3
View File
@@ -1,3 +1,4 @@
[submodule "open-x4-sdk"]
path = open-x4-sdk
url = https://github.com/crosspoint-reader/community-sdk.git
[submodule "freeink-sdk"]
path = freeink-sdk
url = https://github.com/Free-Ink/freeink-sdk.git
branch = main
+71 -31
View File
@@ -7,7 +7,7 @@ Mission: Provide a lightweight, high-performance reading experience focused on E
* Role: Senior Embedded Systems Engineer (ESP-IDF/Arduino-ESP32 specialized).
* Primary Constraint: 380KB RAM is the hard ceiling. Stability is non-negotiable.
* Evidence-Based Reasoning: Before proposing a change, you MUST cite the specific file path and line numbers that justify the modification.
* Anti-Hallucination: Do not assume the existence of libraries or ESP-IDF functions. If you are unsure of an API's availability for the ESP32-C3 RISC-V target, check the open-x4-sdk or official docs first.
* Anti-Hallucination: Do not assume the existence of libraries or ESP-IDF functions. If you are unsure of an API's availability for the ESP32-C3 RISC-V target, check the freeink-sdk source or the FreeInk SDK docs (https://freeink.org/llms.txt for an LLM-readable index) first.
* No Unfounded Claims: Do not claim performance gains or memory savings without explaining the technical mechanism (e.g., DRAM vs IRAM usage).
* Resource Justification: You must justify any new heap allocation (new, malloc, std::vector) or explain why a stack/static alternative was rejected.
* Verification: After suggesting a fix, instruct the user on how to verify it (e.g., monitoring heap via Serial or checking a specific cache file).
@@ -58,6 +58,7 @@ find src -name "*.cpp" -o -name "*.h" | xargs clang-format -i
6. `constexpr` First: Compile-time constants and lookup tables must be `constexpr`, not just `static const`. This moves computation to compile time, enables dead-branch elimination, and guarantees flash placement. Use `static constexpr` for class-level constants.
7. `std::vector` Pre-allocation: Always call `.reserve(N)` before any `push_back()` loop. Each growth event allocates a new block (2×), copies all elements, then frees the old one — three heap operations that fragment DRAM. When the final size is unknown, estimate conservatively.
8. SPIFFS Write Throttling: Never write a settings file on every user interaction. Guard all writes with a value-change check (`if (newVal == _current) return;`). Progress saves during reading must be debounced — write on activity exit or every N page turns, not on every turn. SPIFFS sectors have a finite erase cycle limit.
9. `new` is not nothrow on ESP32: With `-fno-exceptions`, bare `new` that fails calls `abort()` — it does NOT return `nullptr`. Always use `new (std::nothrow)` and null-check the result, or use `makeUniqueNoThrow<T>()` from `lib/Memory/Memory.h`. Never write bare `new` for any fallible allocation.
---
@@ -126,7 +127,7 @@ These flags in `platformio.ini` fundamentally affect firmware behavior:
* lib/hal/: Hardware Abstraction Layer (HalDisplay, HalGPIO, HalStorage)
* lib/I18n/: Internationalization (translations in `translations/*.yaml`, generated string tables)
* src/activities/: UI logic using the Activity Lifecycle (onEnter, loop, onExit)
* open-x4-sdk/: Low-level SDK (EInkDisplay, InputManager, BatteryMonitor, SDCardManager)
* freeink-sdk/: Low-level SDK (EInkDisplay, InputManager, BatteryMonitor, SDCardManager)
* .crosspoint/: SD-based binary cache for EPUB metadata and pre-rendered layout sections
### Hardware Abstraction Layer (HAL)
@@ -151,14 +152,19 @@ These flags in `platformio.ini` fundamentally affect firmware behavior:
#include <HalStorage.h>
// Use Storage singleton (defined via macro)
FsFile file;
HalFile file;
if (Storage.openFileForRead("MODULE", "/path/to/file.bin", file)) {
// Read from file
// No file.close() needed — DESTRUCTOR_CLOSES_FILE=1 handles it at scope exit
}
```
**Usage**: See example above. Uses `FsFile` (SdFat), NOT Arduino `File`. Do NOT add `file.close()` for local variables (see DESTRUCTOR_CLOSES_FILE above).
**Usage**: Use `HalFile` (the mutex-wrapping handle), NOT raw SdFat `FsFile` or Arduino `File`. Do NOT add `file.close()` for local variables (see DESTRUCTOR_CLOSES_FILE above).
**SdFat is not thread-safe; all SD access MUST go through HalStorage**:
- SdFat's `SdSpiCard` tracks SPI bus state with an unsynchronized `m_spiActive` bool. Two tasks calling SdFat concurrently can confuse that state machine and end with one task calling `SPIClass::endTransaction()` against a paramLock the *other* task is holding. That trips FreeRTOS's `xTaskPriorityDisinherit` assert (`tasks.c:5156, pxTCB == pxCurrentTCBs[0]`) and panics the system. See SdFat issue #518.
- `HalStorage` serializes everything via `storageMutex`. Downstream code uses `HalFile` (declared in `<HalStorage.h>`); every method call (read, write, seek, close) takes the mutex. `HalFile`'s destructor also takes the mutex before letting the underlying SdFat `FsFile` close.
- **Never** call into `SdFat` / `SdSpiCard` / `FsBaseFile` / `SDCardManager` / raw `FsFile` directly — that bypasses the mutex.
---
@@ -266,43 +272,78 @@ When a template is necessary, limit instantiations: use explicit template instan
**Rules**: NO exceptions, NO abort(), ALWAYS log before error return
### Acceptable malloc/free Patterns
### Heap Buffer Allocation
**Source**: [src/activities/home/HomeActivity.cpp:166](../src/activities/home/HomeActivity.cpp), [lib/GfxRenderer/GfxRenderer.cpp:439-440](../lib/GfxRenderer/GfxRenderer.cpp)
**Prefer `makeUniqueNoThrow` over `malloc`.** Both are nothrow (return `nullptr` on OOM rather than calling `abort()`), but `malloc` requires a manual `free` on every return path — a common source of leaks. `makeUniqueNoThrow<uint8_t[]>(size)` from `lib/Memory/Memory.h` frees automatically when it goes out of scope.
Despite "prefer stack allocation," malloc is acceptable for:
1. **Large temporary buffers** (> 256 bytes, won't fit on stack)
2. **One-time allocations** during activity initialization
3. **Bitmap rendering buffers** (variable size, used briefly)
**Pattern**:
**Preferred pattern**:
```cpp
// Allocate
auto* buffer = static_cast<uint8_t*>(malloc(bufferSize));
#include <Memory.h>
auto buffer = makeUniqueNoThrow<uint8_t[]>(bufferSize);
if (!buffer) {
LOG_ERR("MODULE", "malloc failed: %d bytes", bufferSize);
return false; // Handle allocation failure
LOG_ERR("MODULE", "OOM: %d bytes", bufferSize);
return false;
}
// Use buffer
processData(buffer, bufferSize);
processData(buffer.get(), bufferSize);
// freed automatically — no manual free needed, no leak on early return
```
// Free immediately after use
free(buffer);
buffer = nullptr;
**`malloc` or `new (std::nothrow)` are still acceptable** when the buffer must be passed to a C API that takes ownership and frees it itself (e.g., certain SDK callbacks). In that case follow the manual pattern:
```cpp
auto* buffer = static_cast<uint8_t*>(malloc(bufferSize)); // or new (std::nothrow) uint8_t[bufferSize]
if (!buffer) {
LOG_ERR("MODULE", "OOM: %d bytes", bufferSize);
return false;
}
sdkApiThatTakesOwnership(buffer, bufferSize); // SDK calls free() / delete[]
```
**Rules**:
- **ALWAYS check for nullptr** after malloc
- **Free immediately** after use (don't hold across multiple operations)
- **Set to nullptr** after free (avoid use-after-free)
- **Document size**: Comment why stack allocation was rejected
- **Prefer `makeUniqueNoThrow`** — automatic cleanup eliminates leak risk on error paths
- **ALWAYS check for nullptr** after any allocation and `LOG_ERR` before returning false
- **Raw allocation only** when a C API takes ownership; document why in a comment
**Examples in codebase**:
- Memory utilities: [Memory.h](../lib/Memory/Memory.h) (`makeUniqueNoThrow`)
- Cover image buffers: [HomeActivity.cpp:166](../src/activities/home/HomeActivity.cpp)
- Text chunk buffers: [TxtReaderActivity.cpp:259](../src/activities/reader/TxtReaderActivity.cpp)
- Bitmap rendering: [GfxRenderer.cpp:439-440](../lib/GfxRenderer/GfxRenderer.cpp)
- OTA update buffer: [OtaUpdater.cpp:40](../src/network/OtaUpdater.cpp)
### Heap Allocation with `new`: Always Use `makeUniqueNoThrow`
**CRITICAL**: With `-fno-exceptions`, bare `new` on OOM calls `abort()` — it does NOT return `nullptr`. Always use `makeUniqueNoThrow` from `lib/Memory/Memory.h`, which wraps `new (std::nothrow)` and returns a `std::unique_ptr` that is null on OOM and automatically frees on scope exit.
**Preferred pattern**:
```cpp
#include <Memory.h>
auto obj = makeUniqueNoThrow<MyClass>(args);
if (!obj) { LOG_ERR("MOD", "OOM: MyClass"); return false; }
auto buf = makeUniqueNoThrow<uint8_t[]>(size);
if (!buf) { LOG_ERR("MOD", "OOM: %d bytes", size); return false; }
// Pass to C APIs via .get(); unique_ptr frees automatically on return
someApi(buf.get(), size);
```
**`new (std::nothrow)` directly is acceptable** when the object must be passed to a C API that takes ownership and calls `delete` itself:
```cpp
auto* obj = new (std::nothrow) MyClass(args);
if (!obj) { LOG_ERR("MOD", "OOM: MyClass"); return false; }
sdkApiThatTakesOwnership(obj); // SDK calls delete
```
**Rules**:
- **Prefer `makeUniqueNoThrow`** — automatic cleanup eliminates leak risk on error paths
- **NEVER use bare `new`** — always `makeUniqueNoThrow` or `new (std::nothrow)`
- **ALWAYS `LOG_ERR` before returning false** on OOM
- **Use `.get()`** to pass the raw pointer to C-style APIs; ownership stays with the `unique_ptr`
- **`new (std::nothrow)` directly only** when a C API takes ownership; document why in a comment
**Examples in codebase**:
- Memory utilities: [Memory.h](../lib/Memory/Memory.h) (`makeUniqueNoThrow`)
---
@@ -416,7 +457,6 @@ void onExit() { /* free: vTaskDelete, free buffer, close member FsFiles */ Act
**All fonts are loaded as global static objects** at firmware startup:
- Noto Serif: 12, 14, 16, 18pt (4 styles each: regular, bold, italic, bold-italic)
- Noto Sans: 12, 14, 16, 18pt (4 styles each)
- OpenDyslexic: 8, 10, 12, 14pt (4 styles each)
- Ubuntu UI fonts: 10, 12pt (2 styles)
**Total**: ~80+ global `EpdFont` and `EpdFontFamily` objects
@@ -856,8 +896,8 @@ rm -rf /path/to/sd/.crosspoint/epub_<hash>/sections/
**Source**: `lib/Epub/Epub/Section.cpp`, `lib/Epub/Epub/BookMetadataCache.cpp`
**Current Versions** (as of docs/file-formats.md):
- `book.bin`: **Version 5** (metadata structure)
- `section.bin`: **Version 12** (layout structure)
- `book.bin`: **Version 7** (metadata structure)
- `section.bin`: **Version 25** (layout structure)
**Version Increment Rules**:
1. **ALWAYS increment version** BEFORE changing binary structure
@@ -867,7 +907,7 @@ rm -rf /path/to/sd/.crosspoint/epub_<hash>/sections/
**Example** (incrementing section format version):
```cpp
// lib/Epub/Epub/Section.cpp
static constexpr uint8_t SECTION_FILE_VERSION = 13; // Was 12, now 13
static constexpr uint8_t SECTION_FILE_VERSION = 26; // Was 25, now 26
// Add new field to structure
struct PageLine {
+197 -107
View File
@@ -1,122 +1,196 @@
# CrossPoint Reader
Firmware for the **Xteink X4** e-paper display reader (unaffiliated with Xteink).
Built using **PlatformIO** and targeting the **ESP32-C3** microcontroller.
[![Fund contributors](https://img.shields.io/badge/%F0%9F%91%91_Fund_contributors-royalty.dev-BB953A?style=for-the-badge&labelColor=1a1a1a)](https://app.royalty.dev/crosspoint-reader/crosspoint-reader)
CrossPoint Reader is a purpose-built firmware designed to be a drop-in, fully open-source replacement for the official
Xteink firmware. It aims to match or improve upon the standard EPUB reading experience.
CrossPoint is open-source e-reader firmware - community-built, fully hackable, free forever. It's maintained by a growing community of developers and readers who believe your device should do what you want - not what a manufacturer decided for you.
![](./docs/images/cover.jpg)
**Now running on:** ESP32C3-based Xteink [X4](https://www.xteink.com/products/xteink-x4) and [X3](https://www.xteink.com/products/xteink-x3).
## Motivation
![CrossPoint Reader running on Xteink device](./docs/images/cover.jpg)
E-paper devices are fantastic for reading, but most commercially available readers are closed systems with limited
customisation. The **Xteink X4** is an affordable, e-paper device, however the official firmware remains closed.
CrossPoint exists partly as a fun side-project and partly to open up the ecosystem and truly unlock the device's
potential.
> If you're planning to buy an Xteink device, consider purchasing an **X3/X4 Developer Edition** through https://crosspointreader.com. CrossPoint receives a small share of each sale, helping fund development costs.
CrossPoint Reader aims to:
* Provide a **fully open-source alternative** to the official firmware.
* Offer a **document reader** capable of handling EPUB content on constrained hardware.
* Support **customisable font, layout, and display** options.
* Run purely on the **Xteink X4 hardware**.
## What can CrossPoint do?
This project is **not affiliated with Xteink**; it's built as a community project.
- **Reader engine**: EPUB 2/3 rendering with embedded-style option, image handling, hyphenation, kerning, chapter navigation, footnotes, bookmarks, dictionary lookups ([StarDict](docs/dictionary.md)), go-to-percent, auto page turn, orientation control, focus reading, KOReader progress sync and more.
## Features & Usage
- **Various formats**: native handling for `.epub`, `.xtc/.xtch`, `.txt`, and `.bmp`.
- [x] EPUB parsing and rendering (EPUB 2 and EPUB 3)
- [x] Image support within EPUB
- [x] Saved reading position
- [x] File explorer with file picker
- [x] Basic EPUB picker from root directory
- [x] Support nested folders
- [ ] EPUB picker with cover art
- [x] Custom sleep screen
- [x] Cover sleep screen
- [x] Wifi book upload
- [x] Wifi OTA updates
- [x] KOReader Sync integration for cross-device reading progress
- [x] Configurable font, layout, and display options
- [ ] User provided fonts
- [ ] Full UTF support
- [x] Screen rotation
- **Screenshots.**
Multi-language support: Read EPUBs in various languages, including English, Spanish, French, German, Italian, Portuguese, Russian, Ukrainian, Polish, Swedish, Norwegian, [and more](./USER_GUIDE.md#supported-languages).
- **Custom fonts**: install your favorite fonts on the SD card.
See [the user guide](./USER_GUIDE.md) for instructions on operating CrossPoint, including the
[KOReader Sync quick setup](./USER_GUIDE.md#367-koreader-sync-quick-setup).
- **Tilt page turn (X3 only)**.
For more details about the scope of the project, see the [SCOPE.md](SCOPE.md) document.
- **Library workflow**: folder browser, hidden-file toggle, long-press delete, recent books, SD-cache management.
## Installing
- **Wireless workflows**:
### Web (latest firmware)
- File transfer web UI
- EPUB Optimizer
- Web settings UI/API (edit many device settings from browser)
- WebSocket fast uploads
- WebDAV handler
- AP mode (hotspot) and STA mode (join existing Wi-Fi), both with QR helpers
- Calibre wireless connect flow
- OPDS browser with saved servers (up to 8), search, pagination, and direct download
- OTA update checks and installs from GitHub releases
1. Connect your Xteink X4 to your computer via USB-C and wake/unlock the device
2. Go to https://xteink.dve.al/ and click "Flash CrossPoint firmware"
- **Customization**: multiple themes (Classic, Lyra, Lyra Extended, RoundedRaff), sleep screen modes, front/side button remapping, status bar controls, power-button behavior, refresh cadence, and more.
To revert back to the official firmware, you can flash the latest official firmware from https://xteink.dve.al/, or swap
back to the other partition using the "Swap boot partition" button here https://xteink.dve.al/debug.
- **Localization**: 24 UI languages and counting. RTL support.
### Web (specific firmware version)
### Coming soon:
1. Connect your Xteink X4 to your computer via USB-C
2. Download the `firmware.bin` file from the release of your choice via the [releases page](https://github.com/crosspoint-reader/crosspoint-reader/releases)
3. Go to https://xteink.dve.al/ and flash the firmware file using the "OTA fast flash controls" section
- More themes.
To revert back to the official firmware, you can flash the latest official firmware from https://xteink.dve.al/, or swap
back to the other partition using the "Swap boot partition" button here https://xteink.dve.al/debug.
- Much more! stay tuned.
### Command line (specific firmware version)
---
## USB-locked devices (Xteink Unlocker)
Some Xteink units purchased from third-party stores (e.g. AliExpress) ship with USB flashing locked from the factory.
If your device is locked, you will need to use the **Xteink Unlocker** tool available at
https://crosspointreader.com/#unlock-tool before you can flash CrossPoint.
**You do not need this tool if you bought your device directly from xteink.com.** Those units are not locked.
**Not sure if your device is locked?** Power it on, connect the USB-C cable, and try flashing via the web flasher first (see
[Install firmware](#install-firmware) below). If the browser's serial device picker does not show your device, try a different
USB port or browser before assuming the device is locked. Only reach for the unlocker if the device still doesn't appear.
> ### ⚠️ WARNING: READ THIS BEFORE USING THE UNLOCKER ⚠️
>
> **The only officially supported firmwares in the unlock tool are CrossPoint and CrossInk.**
>
> Flashing any other firmware on a USB-locked device may **permanently brick the device** or leave it **permanently
> stuck on that firmware with no recovery path**. Once USB flashing is re-locked, your only way back is via OTA, and if
> the firmware you flashed doesn't support OTA, **there is no way out**.
## Install firmware
### Web installer (recommended)
1. Connect your device to your computer via USB-C and wake/unlock the device
2. Go to https://crosspointreader.com/#flash-tools, select device (X3 or X4), and choose an official CrossPoint release.
### Web installer (specific version)
1. Connect your device to your computer via USB-C and wake/unlock the device
2. Download a `firmware.bin` from [Releases](https://github.com/crosspoint-reader/crosspoint-reader/releases), local build, or continuous integration artifact.
3. Go to https://crosspointreader.com/#flash-tools, select device (X3 or X4), click "Custom .bin" and upload a `firmware.bin`.
### Revert to Official Firmware
To revert to the official firmware, you can also flash the latest official firmware using https://crosspointreader.com/#flash-tools.
### Command line
1. Install [`esptool`](https://github.com/espressif/esptool):
1. Install [`esptool`](https://github.com/espressif/esptool) :
```bash
pip install esptool
```
2. Download the `firmware.bin` file from the release of your choice via the [releases page](https://github.com/crosspoint-reader/crosspoint-reader/releases)
3. Connect your Xteink X4 to your computer via USB-C.
4. Note the device location. On Linux, run `dmesg` after connecting. On MacOS, run :
2. Download `firmware.bin` from the [releases page](https://github.com/crosspoint-reader/crosspoint-reader/releases).
3. Connect your device via USB-C.
4. Find the device port. On Linux, run `dmesg` after connecting. On macOS:
```bash
log stream --predicate 'subsystem == "com.apple.iokit"' --info
```
5. Flash the firmware :
5. Flash:
```bash
esptool.py --chip esp32c3 --port /dev/ttyACM0 --baud 921600 write_flash 0x10000 /path/to/firmware.bin
```
Change `/dev/ttyACM0` to the device for your system.
Adjust `/dev/ttyACM0` to match your system.
### Manual
See [Development](#development) below.
See [Development quick start](#development-quick-start) below.
## Development
---
## Custom SD-card fonts
Convert your own TTF/OTF files into `.cpfont` files that load from the SD card. No firmware reflash is needed.
1. Go to https://crosspointreader.com/fonts and open the "SD-card font builder" form.
2. Upload up to four styles (regular, bold, italic, bold-italic), set the family name, point sizes, and Unicode range.
3. Download the generated `.cpfont` files.
4. Copy them to your SD card under `/fonts/YourFont/` (or `/.fonts/YourFont/` to hide the folder).
5. Select the font on the device from the font settings.
Conversion runs the firmware repo's `lib/EpdFont/scripts/fontconvert_sdcard.py` script unmodified, so output matches a local host build.
---
## Documentation
- [User Guide](./USER_GUIDE.md)
- [Web server usage](./docs/webserver.md)
- [Web server endpoints](./docs/webserver-endpoints.md)
- [Project scope](./SCOPE.md)
- [Contributing docs](./docs/contributing/README.md)
- [Touch and UI development](./docs/contributing/touch-and-ui.md) - FreeInkUI components for new screens, the touch bridge for existing ones, and build envs for the non-Xteink touch devices
---
## Development quick start
### Prerequisites
* **PlatformIO Core** (`pio`) or **VS Code + PlatformIO IDE**
* Python 3.8+
* USB-C cable for flashing the ESP32-C3
* Xteink X4
- [pioarduino](https://github.com/pioarduino/pioarduino) or VS Code + pioarduino plugin
- Python 3.8+
- `clang-format` 21
- USB-C cable supporting data transfer
### Checking out the code
### Setup
CrossPoint uses PlatformIO for building and flashing the firmware. To get started, clone the repository:
```
```bash
git clone --recursive https://github.com/crosspoint-reader/crosspoint-reader
cd crosspoint-reader
# Or, if you've already cloned without --recursive:
# if cloned without --recursive:
git submodule update --init --recursive
```
### Flashing your device
### Nix/NixOS
Connect your Xteink X4 to your computer via USB-C and run the following command.
Nix/NixOS users can enter the development shell with either `nix develop` (flakes) or `nix-shell`:
```sh
```bash
nix develop -f nix
# or
nix-shell nix
```
To flash a connected ESP32-C3 device, enable PlatformIO's udev rules in your NixOS configuration:
```nix
services.udev.packages = with pkgs; [ platformio-core.udev ];
```
After rebuilding the system configuration, reconnect the device or reload udev rules.
### Build / flash / monitor
```bash
pio run --target upload
```
### Contributor pre-PR checks
```bash
./bin/clang-format-fix
pio check -e default
pio run -e default
```
### Debugging
After flashing the new features, its recommended to capture detailed logs from the serial port.
@@ -126,7 +200,9 @@ First, make sure all required Python packages are installed:
```python
python3 -m pip install pyserial colorama matplotlib
```
after that run the script:
After that run the script:
```sh
# For Linux
# This was tested on Debian and should work on most Linux systems.
@@ -135,63 +211,77 @@ python3 scripts/debugging_monitor.py
# For macOS
python3 scripts/debugging_monitor.py /dev/cu.usbmodem2101
```
Minor adjustments may be required for Windows.
---
## Internals
CrossPoint Reader is pretty aggressive about caching data down to the SD card to minimise RAM usage. The ESP32-C3 only
has ~380KB of usable RAM, so we have to be careful. A lot of the decisions made in the design of the firmware were based
on this constraint.
CrossPoint Reader is pretty aggressive about caching data down to the SD card to minimise RAM usage. The ESP32-C3 only has ~380KB of usable RAM, so we have to be careful. A lot of the decisions made in the design of the firmware were based on this constraint.
### Data caching
The first time chapters of a book are loaded, they are cached to the SD card. Subsequent loads are served from the
cache. This cache directory exists at `.crosspoint` on the SD card. The structure is as follows:
```
```text
.crosspoint/
├── epub_12471232/ # Each EPUB is cached to a subdirectory named `epub_<hash>`
│ ├── progress.bin # Stores reading progress (chapter, page, etc.)
│ ├── cover.bmp # Book cover image (once generated)
│ ├── book.bin # Book metadata (title, author, spine, table of contents, etc.)
── sections/ # All chapter data is stored in the sections subdirectory
├── 0.bin # Chapter data (screen count, all text layout info, etc.)
├── 1.bin # files are named by their index in the spine
├── epub_<hash>/ # one directory per book, named by content hash
│ ├── progress.bin # reading position (chapter, page, etc.)
│ ├── cover.bmp # generated cover image
│ ├── book.bin # metadata: title, author, spine, TOC
── css_rules.cache # parsed CSS rule cache
├── img_* # rendered image cache files
└── sections/ # per-chapter layout cache
│ ├── 0.bin
│ ├── 1.bin
│ └── ...
── epub_189013891/
├── settings.json # device settings
── state.json # resume/runtime state
└── recent.json # recent books list
```
Deleting the `.crosspoint` directory will clear the entire cache.
Due the way it's currently implemented, the cache is not automatically cleared when a book is deleted and moving a book
file will use a new cache directory, resetting the reading progress.
Removing `/.crosspoint` clears all cached metadata and forces a full regeneration on next open. Book deletes, overwrites, and moves done through the firmware or web UI clear or re-key matching caches; manual SD-card edits may leave stale cache directories behind.
For more details on the internal file structures, see the [file formats document](./docs/file-formats.md).
---
## Contributing
Contributions are very welcome!
Contributions are welcome. If you're new to the codebase, start with the [contributing docs](./docs/contributing/README.md). For things to work on, check the [ideas discussion board](https://github.com/crosspoint-reader/crosspoint-reader/discussions/categories/ideas) — leave a comment before starting so we don't duplicate effort.
If you are new to the codebase, start with the [contributing docs](./docs/contributing/README.md).
If you're looking for a way to help out, take a look at the [ideas discussion board](https://github.com/crosspoint-reader/crosspoint-reader/discussions/categories/ideas).
If there's something there you'd like to work on, leave a comment so that we can avoid duplicated effort.
Everyone here is a volunteer, so please be respectful and patient. For more details on our governance and community
principles, please see [GOVERNANCE.md](GOVERNANCE.md).
### To submit a contribution:
1. Fork the repo
2. Create a branch (`feature/dithering-improvement`)
3. Make changes
4. Submit a PR
Everyone here is a volunteer, so please be respectful and patient. For governance and community expectations, see [GOVERNANCE.md](./GOVERNANCE.md).
---
CrossPoint Reader is **not affiliated with Xteink or any manufacturer of the X4 hardware**.
## Community forks
Huge shoutout to [**diy-esp32-epub-reader** by atomic14](https://github.com/atomic14/diy-esp32-epub-reader), which was a project I took a lot of inspiration from as I
was making CrossPoint.
One of the best things about open source is that anyone can take the code in a different direction. If you need something outside CrossPoint's [scope](./SCOPE.md), check out the community forks:
- [CrossInk](https://github.com/uxjulia/CrossInk) — Typography and reading tracking: Bionic Reading (bolds word stems to create fixation points), guide dots between words, improved paragraph indents, and replaces the default fonts with ChareInk/Lexend/Bitter.
- [papyrix-reader](https://github.com/bigbag/papyrix-reader) — Adds FB2 and MD format support. Actively maintained with Arabic script support. Custom themes via SD card.
- ~~[crosspet](https://github.com/trilwu/crosspet) — A Vietnamese fork that adds a Tamagotchi-style virtual chicken that grows based on your reading milestones (pages read, streaks, care). Also: Flashcards, Weather, Pomodoro timer, and mini-games.~~ (Unmaintained)
- [crosspoint-reader-cjk](https://github.com/aBER0724/crosspoint-reader-cjk) — Purpose-built for Chinese, Japanese, and Korean reading.
- [inx](https://github.com/obijuankenobiii/inx) — Completely reimagines the user interface with tabbed navigation.
- ~~[PlusPoint](https://github.com/ngxson/pluspoint-reader) — custom JS apps support.~~ (Unmaintained)
- [crosspoint-reader-papers3](https://github.com/juicecultus/crosspoint-reader-papers3) — Crosspoint port for M5Stack Paper S3.
- [t5s3-reader](https://github.com/ShallowGreen123/t5s3-reader) — Crosspoint port for LilyGo T5 ePaper S3 / T5S3 4.7-inch e-paper device.
**Note:** Many of these features will make their way into CrossPoint over time. We maintain a slower pace to ensure rock-solid stability and squash bugs before they reach your device.
Want to build your own device? Be sure to check out the [de-link](https://github.com/iandchasse/de-link) project.
---
CrossPoint Reader is **not affiliated with Xteink or any device manufacturer**.
Huge shoutout to [diy-esp32-epub-reader](https://github.com/atomic14/diy-esp32-epub-reader), which inspired this project.
+88
View File
@@ -0,0 +1,88 @@
# CrossPoint Reader Roadmap
This roadmap describes how CrossPoint is moving through the tighter scope defined in [SCOPE.md](SCOPE.md). It is
intentionally phased: Phase 0 closed out the commitments already in flight before locking down to the stricter
"fill gaps the stock firmware leaves" delineator.
Phases are sequential. We do not start the next phase until the prior one is wrapped or explicitly carried over.
---
## Phase 0 - Close Out Legacy Scope Items — **COMPLETE**
**Goal:** Land the work that was already in motion under the prior, broader scope so contributors are not left
hanging, and so we enter the stricter phases with a clean slate.
**Landed in Phase 0:**
* **RTL support PRs.** The in-flight right-to-left work was reviewed, iterated, and merged.
* **Dictionary PR.** The offline dictionary lookup work was reviewed and merged.
* **Bookmarks** feature. First-class navigation markers in EPUBs.
* ~~**Transparent sleep screens.**~~ Shelved; not picked back up under the stricter phases.
Phase 0 is closed. The tighter scope in [SCOPE.md](SCOPE.md) is now fully enforced. "But it was on the old roadmap"
is not a valid argument for accepting a PR.
---
## Phase 1 - Consolidation, Footprint, and Multi-Device Support — **IN PROGRESS**
**Goal:** Reduce memory and flash usage, clean up the codebase, and land the SDK / HAL generalization work so
CrossPoint runs cleanly on ESP32-based e-reader hardware beyond Xteink (X3 / X4), including ESP32-S3 class devices.
**Focus areas:**
* DRAM and heap fragmentation reduction across the reader core.
* Flash footprint reduction (dead code, redundant strings, oversized tables).
* Refactors that tighten the HAL / SDK boundary.
* Pluggable per-device SDK layers (display, input, storage, battery) and per-device build configuration without
forking the reader core.
* Documentation for adding a new ESP32 e-reader target.
* E-ink driver refinement (ghosting, partial update behavior).
**Closed during this phase:** new themes built into firmware, new external network connectors (sync engines, cloud
storage, remote file access).
---
## Phase 2 - Languages, Fonts, and Themes
**Goal:** With the codebase smaller and portable, make reading great in every language: multi-language support,
better font support with custom fonts, UI translations, and themes loaded from the SD card instead of consuming
flash.
**Focus areas:**
* Multi-language reading support (underserved languages, complex script support where realistic on ESP32 hardware).
* Better font support and custom fonts.
* UI languages and localization.
* Moving themes off-firmware to SD-loaded assets (see SCOPE.md Section 6).
* **Moving hyphenation files off-firmware.** Hyphenation rules vary per language and the files are large (German
alone is ~200KB). Today these eat flash budget that should be available for the reader core. The plan is to build
a downloader analogous to the existing font downloader and store the dictionaries on SD / SPIFFS, loading on
demand. This unlocks better hyphenation for long-word languages (German, Finnish, Norwegian, etc.) without paying
the flash cost up front.
This phase depends on Phase 1 cleanup landing first; otherwise we generalize a moving target.
---
## Out of Roadmap
The following are explicitly *not* on the roadmap. They may live in other CrossPoint forks; they will not be picked
up here:
* Interactive apps (games, calculators, notepads).
* Writing / authoring tools.
* Active connectivity features (RSS, news, browsers).
* PDF rendering as a first-class format.
See [SCOPE.md](SCOPE.md) for the full rationale.
---
## How This Roadmap Changes
* Phase boundaries are decided by maintainers, not by individual PRs.
* If a phase needs to be extended or an item carried over, that is documented here with a short note.
* Proposals for new phases or reordering should go through a Discussion first.
+80 -42
View File
@@ -1,62 +1,100 @@
# Project Vision & Scope: CrossPoint Reader
The goal of CrossPoint Reader is to create an efficient, open-source reading experience for the Xteink X4. We believe a
dedicated e-reader should do one thing exceptionally well: **facilitate focused reading.**
The goal of CrossPoint Reader is to create an efficient, open-source reading experience for ESP32-based e-reader devices. Xteink hardware (X3, X4) is where the project started and remains a primary target, but CrossPoint is explicitly broadening to support the wider ecosystem of small ESP32 e-ink readers. We believe a dedicated e-reader should do one thing exceptionally well: **facilitate focused reading.**
## 1. Core Mission
To provide a lightweight, high-performance firmware that maximizes the potential of the X4, prioritizing legibility and
usability over "swiss-army-knife" functionality.
To provide a lightweight, high-performance firmware that maximizes the potential of ESP32-based e-reader hardware, prioritizing legibility, performance, and usability over "swiss-army-knife" functionality.
## 2. Scope
CrossPoint is **not** a kitchen-sink firmware, and it is **not** Xteink-only. We want clean, maintainable code that the community can build on, and that runs across the range of ESP32 e-reader devices (ESP32-C3, ESP32-S3, and adjacent variants). Every accepted change should make that goal easier, not harder. Device-specific code should live behind the HAL / SDK boundary so the reader core stays portable.
## 2. Guiding Principle: Fill Gaps the Stock Firmware Leaves
CrossPoint exists to do the things the stock firmware does poorly or not at all. New work is evaluated against that delineator:
* **Does the stock firmware already do this well?** We should hit that bar or surpass it
* **Is another popular CrossPoint fork already solving this well?** If yes, we generally defer to that fork if it's not part of the core reading experience. e.g. stats
* **Does this directly improve the reading experience or the firmware's long-term maintainability?** If no, it is out of scope.
## 3. Current Focus
We are intentionally narrowing scope to consolidate the codebase as we open it up to more ESP32 e-reader devices.
During this period, the priorities are:
* **Memory footprint:** Reducing DRAM usage and heap fragmentation. The ESP32-C3 is the tightest target and sets the ceiling, but the gains benefit every ESP32 variant we run on.
* **Flash footprint:** Trimming binary size to leave room for additional device targets and features.
* **Code cleanup:** Refactoring, removing dead code, tightening abstractions, and improving readability.
* **Reading experience:** EPUB parsing and rendering, typography, hyphenation, line spacing, font handling, and legibility improvements.
### Temporarily Closed Areas
PRs in the following areas will be closed until this notice is lifted. Adding these now makes the cleanup and multi-device work materially harder:
* **New themes.** The existing theming surface is frozen.
* **New external network connectors.** This includes sync engines, cloud storage clients, OPDS extensions beyond what exists, remote file access, and any new "talk to a server" feature. We now have our own CrossPoint KOSync server which gives us a way to sync to 3rd party systems like Hardcover at an API level instead of bloating the firmware. If you're interested in helping here, the sync server is also open source.
If you are unsure whether your idea falls into one of these categories, open a Discussion first.
## 4. Scope
### In-Scope
*These are features that directly improve the primary purpose of the device.*
*Features that directly improve the core reading experience or the firmware's maintainability.*
* **User Experience:** E.g. User-friendly interfaces, and interactions, both inside the reader and navigating the
firmware. This includes things like button mapping, book loading, and book navigation like bookmarks.
* **Document Rendering:** E.g. Support for rendering documents (primarily EPUB) and improvements to the rendering
engine.
* **Format Optimization:** E.g. Efficiently parsing EPUB (CSS/Images) and other documents within the device's
capabilities.
* **Typography & Legibility:** E.g. Custom font support, hyphenation engines, and adjustable line spacing.
* **E-Ink Driver Refinement:** E.g. Reducing full-screen flashes (ghosting management) and improving general rendering.
* **Library Management:** E.g. Simple, intuitive ways to organize and navigate a collection of books.
* **Local Transfer:** E.g. Simple, "pull" based book loading via a basic web-server or public and widely-used standards.
* **Language Support:** E.g. Support for multiple languages both in the reader and in the interfaces.
* **Reference Tools:** E.g. Local dictionary lookup. Providing quick, offline definitions to enhance comprehension
without breaking focus.
* **Clock Display (device dependent):**
| Device | Scope |
| -- | -- |
| X3 | The X3 uses a dedicated DS3231 RTC, which maintains accurate time across sleep cycles and can be treated as a reliable wall clock. |
| X4 | The X4 relies on the ESP32-C3's internal RTC, which drifts significantly during deep sleep. NTP sync could correct this, with an appropriate user experience around connecting to the internet on wake or on demand. This causes some tension with the **Active Connectivity** section below, so please open a discussion about this UX if it's a feature you would find useful. |
* **EPUB Rendering & Optimization:** Improvements to the rendering engine, CSS/image handling, and parsing performance.
* **Typography & Legibility:** Custom font support, hyphenation, line and paragraph spacing, margins.
* **E-Ink Driver Refinement:** Reducing full-screen flashes (ghosting management) and improving general rendering.
* **Reading UX:** Bookmarks, progress tracking, button mapping, page navigation, and other in-reader interactions.
* **Library Management:** Simple, intuitive ways to organize and navigate a local book collection.
* **Memory, Flash, and Code Quality:** Refactors and cleanups that reduce resource use or improve maintainability, even without a user-visible feature.
### Out-of-Scope
*These items are rejected because they compromise the device's stability or mission.*
*Rejected because they compromise the device's stability, maintainability, or core mission.*
* **Interactive Apps:** No Notepads, Calculators, or Games. This is a reader, not a PDA.
* **Active Connectivity:** No RSS readers, News aggregators, or Web browsers. Background Wi-Fi tasks drain the battery
and complicate the single-core CPU's execution.
* **Media Playback:** No Audio players or Audio-books.
* **Complex Annotation:** No typed out notes. These features are better suited for devices with better input
capabilities and more powerful chips.
* **Interactive Apps:** No notepads, calculators, or games. These belong in other forks and are not part of CrossPoint's focus.
* **Writing / Authoring Tools:** No typed notes, journals, or editors. Input hardware and RAM are wrong for this, and other forks already explore this space.
* **Active Connectivity:** No RSS readers, news aggregators, or web browsers. Background Wi-Fi drains the battery and complicates the single-core CPU.
* **PDF Rendering:** PDFs are fixed-layout documents, so rendering them requires displaying pages as images rather than reflowable text, resulting in constant panning and zooming that makes for a poor reading experience on e-ink. Out of scope on the current hardware class.
### In-scope — Technically Unsupported
## 5. Calls to Action
*These features align with CrossPoint's goals but are impractical on the current hardware or produce poor UX.*
These are the areas where contributor help is most valuable right now. If you want to take one of these on, open a Discussion or issue first so we can coordinate.
* **PDF Rendering:** PDFs are fixed-layout documents, so rendering them requires displaying pages as images rather than reflowable text — resulting in constant panning and zooming that makes for a poor reading experience on e-ink.
### Theme System: Move Themes Off-Firmware
## 3. Idea Evaluation
We want to abstract themes out of the firmware entirely so they no longer consume flash, and instead load from the SD card. This directly supports the current focus on flash footprint and code cleanup.
While I appreciate the desire to add new and exciting features to CrossPoint Reader, CrossPoint Reader is designed to be
a lightweight, reliable, and performant e-reader. Things which distract or compromise the device's core mission will not
be accepted. As a guiding question, consider if your idea improve the "core reading experience" for the average user,
and, critically, not distract from that reading experience.
* **Status:** [@itsthisjustin](https://github.com/itsthisjustin) plans to take this on eventually but is very open to someone else claiming it sooner.
* **Why it matters:** Every built-in theme costs flash that we would rather spend on rendering, fonts, or future device support. SD-loaded themes also let users customize without rebuilding firmware. It also leads to SD font loading for better language support in the UI.
* **How to claim:** Comment on the relevant Discussion (or open one) before starting.
> **Note to Contributors:** If you are unsure if your idea fits the scope, please open a **Discussion** before you start
> coding!
### Identifying Other Stock-Firmware Gaps
We want help cataloguing things the stock firmware (and other popular CrossPoint forks) handle poorly or not at all, so future work has a clear target list. Particularly interested in:
* **RTL (right-to-left) text support:** Arabic, Hebrew, Persian, and similar scripts.
* **Languages with poor stock and fork coverage:** Especially those that need shaping, complex layout, or non-Latin font work that nobody is handling well today.
* **Other gaps:** Rendering edge cases, accessibility issues, input quirks, anything stock does badly and existing forks have not fixed.
If you can read or use the device in one of these languages, your feedback (even without code) is genuinely useful. Open a Discussion with concrete examples (screenshots, sample EPUBs, expected vs actual behavior) and we will prioritize from there.
## 6. Funding and Contributor Sustainability
CrossPoint uses [Royalty.dev](https://royalty.dev) (yes, a product built by [@itsthisjustin](https://github.com/itsthisjustin)) to fund contributors. There has been some tension in the community around this, so the intent is being clarified here directly.
**Why we do this:**
* To maintain long-term interest from contributors and maintainers, in direct response to substantial community requests for a way to give back.
* To motivate contributors to invest in the *core* project rather than spinning up competing forks.
* To help pay for new ESP32 devices so we can port CrossPoint to additional hardware.
* To give the project a credible long-term path to sustainability.
**How it works:**
* Funds are distributed automatically to contributors based on impact to the codebase and tenure on the project.
* Over **$600** was raised in the first few hours after opening up funding, which is a signal the demand is real.
* The exact scoring methodology is published at <https://app.royalty.dev/transparency>.
**This is not fixed in stone.** The weighting, eligibility, and distribution rules can be tweaked as we learn what works for this project. If you have concerns or suggestions about how funds are allocated, open a Discussion. The goal is a system that fairly recognizes the people doing the work, not a perfect one on day one.
+327 -76
View File
@@ -5,6 +5,7 @@ Welcome to the **CrossPoint** firmware. This guide outlines the hardware control
- [CrossPoint User Guide](#crosspoint-user-guide)
- [1. Hardware Overview](#1-hardware-overview)
- [Button Layout](#button-layout)
- [Taking a Screenshot](#taking-a-screenshot)
- [2. Power \& Startup](#2-power--startup)
- [Power On / Off](#power-on--off)
- [First Launch](#first-launch)
@@ -14,31 +15,47 @@ Welcome to the **CrossPoint** firmware. This guide outlines the hardware control
- [3.3 Browse Files Screen](#33-browse-files-screen)
- [3.4 Recent Books Screen](#34-recent-books-screen)
- [3.5 File Transfer Screen](#35-file-transfer-screen)
- [3.5.1 Calibre Wireless Transfers](#351-calibre-wireless-transfers)
- [3.5.1 Calibre Wireless Transfers](#351-calibre-wireless-transfers)
- [Installing the Plugin in Calibre](#installing-the-plugin-in-calibre)
- [Configuring the CrossPoint Plugin in Calibre](#configuring-the-crosspoint-plugin-in-calibre)
- [Uploading Books](#uploading-books)
- [Removing a Book](#removing-a-book)
- [3.6 Settings](#36-settings)
- [3.6.1 Display](#361-display)
- [3.6.2 Reader](#362-reader)
- [3.6.3 Controls](#363-controls)
- [3.6.4 System](#364-system)
- [3.6.5 OPDS Servers (Multiple Libraries)](#365-opds-servers-multiple-libraries)
- [3.6.6 Web Settings (WiFi + OPDS)](#366-web-settings-wifi--opds)
- [3.6.6 Web Settings (Wi-Fi + OPDS)](#366-web-settings-wi-fi--opds)
- [3.6.7 KOReader Sync Quick Setup](#367-koreader-sync-quick-setup)
- [Option A: CrossPoint Sync Server (`sync.crosspointreader.com`, default)](#option-a-crosspoint-sync-server-synccrosspointreadercom-default)
- [Option B: Legacy Public KOReader Server (`sync.koreader.rocks`)](#option-b-legacy-public-koreader-server-synckoreaderrocks)
- [Option C: Self-Hosted Server (Docker Compose)](#option-c-self-hosted-server-docker-compose)
- [Syncing While Reading](#syncing-while-reading)
- [3.7 Sleep Screen](#37-sleep-screen)
- [Cover settings](#cover-settings)
- [Custom images](#custom-images)
- [3.8 Custom Fonts (SD Card)](#38-custom-fonts-sd-card)
- [4. Reading Mode](#4-reading-mode)
- [Page Turning](#page-turning)
- [Chapter Navigation](#chapter-navigation)
- [Auto Page Turn](#auto-page-turn)
- [Tilt Page Turn (X3 only)](#tilt-page-turn-x3-only)
- [Footnote Navigation](#footnote-navigation)
- [System Navigation](#system-navigation)
- [Supported Languages](#supported-languages)
- [5. Chapter Selection Screen](#5-chapter-selection-screen)
- [5. Reader Menu](#5-reader-menu)
- [5.1 Chapter Selection](#51-chapter-selection)
- [5.2 Bookmarks](#52-bookmarks)
- [6. Current Limitations \& Roadmap](#6-current-limitations--roadmap)
- [7. Troubleshooting Issues \& Escaping Bootloop](#7-troubleshooting-issues--escaping-bootloop)
## 1. Hardware Overview
The device utilises the standard buttons on the Xteink X4 (in the same layout as the manufacturer firmware, by default):
### Button Layout
| Location | Buttons |
| --------------- | ---------------------------------------------------- |
| **Bottom Edge** | **Back**, **Confirm**, **Left**, **Right** |
@@ -47,6 +64,7 @@ The device utilises the standard buttons on the Xteink X4 (in the same layout as
Button layout can be customized in the **[Controls Settings](#363-controls)**.
### Taking a Screenshot
When the Power Button and Volume Down button are pressed at the same time, it will take a screenshot and save it in the folder `screenshots/`.
Alternatively, while reading a book, press the **Confirm** button to open the reader menu and select **Take screenshot**.
@@ -83,11 +101,12 @@ See [Reading Mode](#4-reading-mode) below for more information.
### 3.3 Browse Files Screen
The Browse Files screen acts as a file and folder browser.
The Browse Files screen acts as a file and folder browser. The full path to the current directory is shown at the top of the screen. File extensions are displayed alongside each filename, and directories are shown with brackets (e.g. `[folder-name]`). Hidden directories (those beginning with `.`) are also visible.
* **Navigate List:** Use **Left** (or **Volume Up**), or **Right** (or **Volume Down**) to move the selection cursor up and down through folders and books. You can also long-press these buttons to scroll a full page up or down.
* **Open Selection:** Press **Confirm** to open a folder or read a selected book.
* **Delete Files:** Hold and release **Confirm** to delete the selected file. You will be given an option to either confirm or cancel deletion. Folder deletion is not supported.
* **Open Selection:** Press **Confirm** to open a folder or start reading a selected book. Selecting a `.bmp` file will open the image viewer.
* **Delete Files or Folders:** Hold and release **Confirm** to delete the selected file or folder. You will be given an option to either confirm or cancel. Multiple files can be selected for deletion in a single operation.
* **Rename or Move:** Files can be renamed or moved to a different folder from within the browse screen.
### 3.4 Recent Books Screen
@@ -95,24 +114,62 @@ The Recent Books screen lists the most recently opened books in a chronological
### 3.5 File Transfer Screen
The File Transfer screen allows you to upload new e-books to the device. When you enter the screen, you'll be prompted with a WiFi selection dialog and then your X4 will start hosting a web server.
The File Transfer screen allows you to upload and manage files on the device. When you enter the screen, choose **Join a Network**, **Calibre Wireless**, or **Create Hotspot**. The reader then starts the web server for the selected mode.
See the [webserver docs](./docs/webserver.md) for more information on how to connect to the web server and upload files.
See the [web server docs](./docs/webserver.md) for more information on how to connect to the web server and upload files.
The web interface also supports **WebDAV**, allowing you to mount the device as a network drive and manage files directly from your computer's file manager.
Download links for files already on the device are available in the web interface, so you can retrieve books or screenshots over Wi-Fi without connecting a cable.
A **Wi-Fi signal strength indicator** (dBm) is displayed on-screen during joined-network web server sessions.
> [!TIP]
> Advanced users can also manage files programmatically or via the command line using `curl`. See the [webserver docs](./docs/webserver.md) for details.
> Advanced users can also manage files programmatically or via the command line using `curl`. See the [web server docs](./docs/webserver.md) for details.
> [!TIP]
> If your EPUBs have compatibility issues, you can run the built-in **EPUB Optimizer** directly from the device to clean up and reprocess books for better rendering.
### 3.5.1 Calibre Wireless Transfers
CrossPoint supports sending books from Calibre using the CrossPoint Reader device plugin.
1. Install the plugin in Calibre:
- Head to https://github.com/crosspoint-reader/calibre-plugins/releases to download the latest version of the crosspoint_reader plugin.
- Download the zip file.
- Open Calibre → Preferences → Plugins → Load plugin from file → Select the zip file.
2. On the device: File Transfer → Connect to Calibre → Join a network.
3. Make sure your computer is on the same WiFi network.
4. In Calibre, click "Send to device" to transfer books.
#### Installing the Plugin in Calibre
If you don't already have the plugin installed:
1. Head to https://github.com/crosspoint-reader/calibre-plugins/releases to download the latest version of the crosspoint_reader plugin.
2. Download the zip file.
3. Open Calibre → Preferences → Plugins → Load plugin from file → Select the zip file.
4. Restart Calibre.
#### Configuring the CrossPoint Plugin in Calibre
1. In Calibre select Preferences.
2. In the Preferences dialog select Plugins.
3. In Plugins search for "crosspoint".
4. Click on "Customize plugin".
5. Update the value for "Host" to match the IP for your device.
6. Leave the other settings as they are.
7. [optional] Modify the "Upload path" to point to a subfolder other than the root "/" folder. Enter this as a path relative to the root folder. Example: `/mybooks`
8. Restart Calibre.
<img width="420" height="385" alt="Image" src="https://github.com/user-attachments/assets/01fc7e33-a9a7-48ba-9e26-2e68d1f9daec" />
#### Uploading Books
To upload a book using the CrossPoint plugin in Calibre:
1. On the device: File Transfer -> Calibre Wireless, then join a network.
2. Select one or more books.
3. Right-click on that selection.
4. Select "Send to Device" > "Send to main memory"
The CrossPoint plugin will connect to your device, create a folder for the book's author in the root folder (or the folder you configured for the plugin), then copy the book into that folder.
<img width="783" height="310" alt="Image" src="https://github.com/user-attachments/assets/741b0909-2e1d-4f16-8af0-2c43fbda5ce6" />
#### Removing a Book
Books cannot be removed from your device through Calibre. Use the web interface instead.
### 3.6 Settings
@@ -121,96 +178,158 @@ The Settings screen allows you to configure the device's behavior. There are a f
#### 3.6.1 Display
- **Sleep Screen**: Which sleep screen to display when the device sleeps:
- "Dark" (default) - The default dark Crosspoint logo sleep screen
- "Light" - The same default sleep screen, on a white background
- "Custom" - Custom images from the SD card; see [Sleep Screen](#37-sleep-screen) below for more information
- "Cover" - The book cover image (Note: this is experimental and may not work as expected)
- "None" - A blank screen
- "Cover + Custom" - The book cover image, falls back to "Custom" behavior
- "Cover + Custom" - The book cover image while actively reading, falls back to "Custom" behavior otherwise
- "Quick resume" - The text of the last page read will be displayed on the sleep screen and a moon icon is shown on the edge of the screen. Waking up the device will return to the same page of the opened book. This is useful for quickly resuming reading without waiting for the device to fully wake up and load the book.
- **Sleep Screen Cover Mode**: How to display the book cover when "Cover" sleep screen is selected:
- "Fit" (default) - Scale the image down to fit centered on the screen, padding with white borders as necessary
- "Crop" - Scale the image down and crop as necessary to try to fill the screen (Note: this is experimental and may not work as expected)
- **Sleep Screen Cover Filter**: What filter will be applied to the book cover when "Cover" sleep screen is selected:
- "None" (default) - The cover image will be converted to a grayscale image and displayed as it is
- "Contrast" - The image will be displayed as a black & white image without grayscale conversion
- "Inverted" - The image will be inverted as in white & black and will be displayed without grayscale conversion
- **Quick Resume on Timeout**: Whether to enable the "Quick Resume" sleep screen when the device goes to sleep due to inactivity (System > Time to Sleep). This is useful for quickly resuming reading without waiting for the device to fully wake up and load the book. This overwrites the Sleep Screen Cover Mode when enabled.
- **Status Bar**: Configure the status bar displayed while reading:
- "None" - No status bar
- "No Progress" - Show status bar without reading progress
- "Full w/ Percentage" - Show status bar with book progress (as percentage)
- "Full w/ Book Bar" - Show status bar with book progress (as bar)
- "Book Bar Only" - Show book progress (as bar)
- "Full w/ Chapter Bar" - Show status bar with chapter progress (as bar)
- **Hide Battery %**: Configure where to suppress the battery percentage display in the status bar; the battery icon will still be shown:
- "Never" (default) - Always show battery percentage
- "In Reader" - Show battery percentage everywhere except in reading mode
- "Always" - Always hide battery percentage
- **Refresh Frequency**: Set how often the screen does a full refresh while reading to reduce ghosting; options are every 1, 5, 10, 15, or 30 pages.
- **UI Theme**: Set which UI theme to use:
- "Classic" - The original Crosspoint theme
- "Lyra" - The new theme for Crosspoint featuring rounded elements and menu icons
- "Lyra Extended" - Lyra, but displays 3 books instead of 1 on the **[Home Screen](#31-home-screen)**
- "RoundedRaff" - A rounded theme with additional visual styling
- **Sunlight Fading Fix**: Configure whether to enable a software-fix for the issue where white X4 models may fade when used in direct sunlight:
- "OFF" (default) - Disable the fix
- "ON" - Enable the fix
> [!NOTE]
> A battery charging indicator is shown on the battery icon whenever the device is actively charging.
#### 3.6.2 Reader
- **Reader Font Family**: Choose the font used for reading:
- "Noto Serif" (default) - Google's serif font
- "Noto Sans" - Google's sans-serif font
- "Open Dyslexic" - Font designed for readers with dyslexia
- **Reader Font Size**: Adjust the text size for reading; options are "Small", "Medium" (default), "Large", or "X Large".
- **Reader Line Spacing**: Adjust the spacing between lines; options are "Tight", "Normal" (default), or "Wide".
- **Reader Screen Margin**: Controls the screen margins in Reading Mode between 5 and 40 pixels in 5-pixel increments.
- **Reader Paragraph Alignment**: Set the alignment of paragraphs; options are "Justified" (default), "Left", "Center", or "Right".
- **Embedded Style**: Whether to use the EPUB file's embedded HTML and CSS stylisation and formatting; options are "ON" or "OFF".
- **Hyphenation**: Whether to hyphenate text in Reading Mode; options are "ON" or "OFF".
- **Reading Orientation**: Set the screen orientation for reading EPUB files:
- "Portrait" (default) - Standard portrait orientation
- "Landscape CW" - Landscape, rotated clockwise
- "Inverted" - Portrait, upside down
- "Landscape CCW" - Landscape, rotated counter-clockwise
- **Extra Paragraph Spacing**: Set how to handle paragraph breaks:
- "ON" - Vertical space will be added between paragraphs in Reading Mode
- "OFF" - Paragraphs will not have vertical space added, but will have first-line indentation
- **Dictionary**: Select the StarDict dictionary used for word lookups while reading, or "None" to disable lookups. *(Only shown when at least one dictionary folder exists under `/dictionaries/` on the SD card — see [docs/dictionary.md](docs/dictionary.md) for setup and usage.)*
- **Text Anti-Aliasing**: Whether to show smooth grey edges (anti-aliasing) on text in reading mode. Note this slows down page turns slightly.
- **Images**: Whether to display embedded images (JPG/PNG) found in EPUB files; options are "ON" (default) or "OFF".
- **Focus Reading**: Bolds the first part of each word to create visual fixation points, similar to Bionic Reading. This can help improve reading speed and focus; options are "ON" or "OFF" (default).
#### 3.6.3 Controls
- **Remap Front Buttons**: A menu for customising the function of each bottom edge button.
- **Side Button Layout (reader)**: Swap the order of the up and down volume buttons from "Prev/Next" (default) to "Next/Prev". This change is only in effect when reading.
- **Side Button Layout (reader)**: Swap the order of the up and down volume buttons from "Prev/Next" (default) to "Next/Prev". You can also disable them entirely. This change is only in effect when reading.
- **Long-press Chapter Skip**: Set whether long-pressing page turn buttons skips to the next/previous chapter:
- "Chapter Skip" (default) - Long-pressing skips to next/previous chapter
- "Page Scroll" - Long-pressing scrolls a page up/down
- **Long-press Menu**: Selects the function bound to holding the menu button (Confirm) while reading an EPUB. **Cycles through the available functions** each time the setting is selected — additional functions may be added in future releases, so this is not a binary on/off toggle. A short press of Confirm always opens the reader menu as normal:
- "Bookmark" (default) - Hold Confirm (~0.4 second) to drop a bookmark at the current page.
- "KOSync" - Hold Confirm (~1 second) to launch KOReader sync directly.
- "Dictionary" - Hold Confirm (~0.4 second) to start dictionary word selection on the current page (see [docs/dictionary.md](docs/dictionary.md)).
- "Disabled" - Long-press is ignored; only short-press opens the reader menu.
- **Short Power Button Click**: Controls the effect of a short click of the power button:
- "Ignore" (default) - Require a long press to turn off the device
- "Sleep" - A short press puts the device into sleep mode
- "Page Turn" - A short press in reading mode turns to the next page; a long press turns the device off
- "Footnotes" - A short press in reading mode opens the footnotes submenu; if only one footnote is present on the page, the referenced page is opened directly. The short press on the power button can be used to select the footnote in the submenu, and to go back to the original page after finish reading the footnote (like the back button).
- "Refresh" - A short press triggers a manual full-screen refresh, useful for clearing ghosting
- **Quick-return from footnotes**: Toggles on and off the quick return functionality from the footnotes. When the functionality it's active, a short press of the power button will act as the back button from the footnotes page.
#### 3.6.4 System
- **Time to Sleep**: Set the duration of inactivity before the device automatically goes to sleep; options are 1, 5, 10 (default), 15 or 30 minutes.
- **Time to Sleep**: Set the duration of inactivity before the device automatically goes to sleep; options are 1, 3, 5, 10 (default), 15 or 30 minutes.
- **Wi-Fi Networks**: Connect to Wi-Fi networks for file transfers and firmware updates.
- **KOReader Sync**: Options for setting up KOReader for syncing book progress. **Smart sync** is the default for new configurations and auto-resolves simple push/pull decisions. Existing credential files retain **Ask every time** when migrated; you can switch Sync Behavior at any time if you prefer manual confirmation.
- **WiFi Networks**: Connect to WiFi networks for file transfers and firmware updates.
- **KOReader Sync**: Options for setting up KOReader for syncing book progress.
- **OPDS Servers**: Manage one or more OPDS [(Open Publication Distribution System)](https://en.wikipedia.org/wiki/Open_Publication_Distribution_System) libraries for browsing and downloading books. See [OPDS Servers (Multiple Libraries)](#365-opds-servers-multiple-libraries) below.
- **Clear Reading Cache**: Clear the internal SD card cache.
- **Check for updates**: Check for Crosspoint firmware updates over WiFi.
- **Language**: Set the system language (see **[Supported Languages](#supported-languages)** for more information).
- **Check for updates**: Check for Crosspoint firmware updates over Wi-Fi. Firmware can also be updated without a USB connection by placing a `firmware.bin` file on the SD card.
- **Language**: Set the UI language. CrossPoint supports 24 languages: English, Spanish, French, German, Czech, Brazilian Portuguese, Russian, Swedish, Romanian, Catalan, Ukrainian, Belarusian, Italian, Polish, Finnish, Danish, Dutch, Turkish, Kazakh, Hungarian, Lithuanian, Slovenian, Valencian, and Hebrew.
- **Manage Fonts**: Browse, download, and manage custom font families installed from the SD card. See [Custom Fonts (SD Card)](#38-custom-fonts-sd-card) for more information.
#### 3.6.5 OPDS Servers (Multiple Libraries)
CrossPoint supports saving multiple OPDS servers and switching between them when browsing catalogs.
1. Open **Settings -> System -> OPDS Servers**.
2. Select **Add Server** to create a new entry, or select an existing server to edit it.
3. Configure these fields:
- **Server Name**: Optional display name (for example, "Home Calibre" or "Public Catalog").
- **OPDS Server URL**: Full catalog root URL (for Calibre Content Server, usually ends with `/opds`).
- **Username / Password**: Optional credentials for authenticated servers.
- **Server Name**: Optional display name (for example, "Home Calibre" or "Public Catalog").
- **OPDS Server URL**: Full catalog root URL (for Calibre Content Server, usually ends with `/opds`).
- **Username / Password**: Optional credentials for authenticated servers.
4. Use **Delete Server** inside a server entry to remove it.
Behavior notes:
@@ -224,31 +343,59 @@ You can also manage OPDS servers from the web interface while in File Transfer m
2. Open `http://<device-ip>/settings`.
3. Use the **OPDS Servers** card to add, edit, or delete entries.
For web-based WiFi network management, see [Web Settings (WiFi + OPDS)](#366-web-settings-wifi--opds).
For web-based Wi-Fi network management, see [Web Settings (Wi-Fi + OPDS)](#366-web-settings-wi-fi--opds).
#### 3.6.6 Web Settings (WiFi + OPDS)
#### 3.6.6 Web Settings (Wi-Fi + OPDS)
While in **File Transfer** mode, the web settings page includes management cards for both **WiFi Networks** and **OPDS Servers**.
While in **File Transfer** mode, the web settings page includes management cards for both **Wi-Fi Networks** and **OPDS Servers**.
1. On device: open **File Transfer** and connect to WiFi.
1. In a browser, open `http://<device-ip>/settings` or `http://crosspoint.local`.
1. In **WiFi Networks**, add, edit, or delete saved network entries (SSID + optional password).
1. In **OPDS Servers**, add, edit, or delete OPDS catalogs.
1. On device: open **File Transfer** and connect through **Join a Network** or **Create Hotspot**.
2. In a browser, open `http://<device-ip>/settings` or `http://crosspoint.local`.
3. In **Wi-Fi Networks**, add, edit, or delete saved network entries (SSID + optional password).
4. In **OPDS Servers**, add, edit, or delete OPDS catalogs.
Behavior notes:
- Passwords are never shown back in the web UI after saving.
- Leaving Password blank while editing keeps the existing saved password unchanged.
- The web UI can save hidden-network SSIDs, but connecting to hidden networks still depends on device-side WiFi connection flow.
- The web UI can save hidden-network SSIDs, but connecting to hidden networks still depends on the device-side Wi-Fi connection flow.
#### 3.6.7 KOReader Sync Quick Setup
CrossPoint can sync reading progress with KOReader-compatible sync servers.
It also interoperates with KOReader apps/devices when they use the same server and credentials.
##### Option A: Free Public Server (`sync.koreader.rocks`)
##### Option A: CrossPoint Sync Server (`sync.crosspointreader.com`, default)
1. Register a user once (only if needed):
When **Sync Server URL** is left empty, CrossPoint uses the free CrossPoint sync server at `https://sync.crosspointreader.com`. It speaks the standard KOReader sync protocol (so KOReader apps can use it too) and additionally stores an exact spine/page position for lossless CrossPoint-to-CrossPoint sync.
1. On each CrossPoint device:
- Go to **Settings -> System -> KOReader Sync**.
- Set **Username** and **Password** (enter the plain password; CrossPoint computes MD5 internally, and use the same values on all devices).
- Leave **Sync Server URL** empty (or set it to `https://sync.crosspointreader.com`).
- On the first device, run **Sign Up** once to create the account directly from the device. On every other device, just run **Authenticate**.
Accounts are per server. Existing `sync.koreader.rocks` credentials do not exist on the CrossPoint server; either sign up again with the same username/password or use Option B to keep using the legacy server.
##### Option B: Legacy Public KOReader Server (`sync.koreader.rocks`)
Use this if you already sync KOReader devices against the official public server.
1. On each CrossPoint device:
- Go to **Settings -> System -> KOReader Sync**.
- Set **Sync Server URL** to `https://sync.koreader.rocks` (required; an empty URL now points at the CrossPoint server instead).
- Set **Username** and **Password** to your existing KOReader Sync credentials.
- Run **Authenticate**.
2. If you do not have an account yet, run **Sign Up** on the device, or register once with curl:
```bash
USERNAME="user"
@@ -261,21 +408,9 @@ curl -i "https://sync.koreader.rocks/users/create" \
--data "{\"username\":\"$USERNAME\",\"password\":\"$PASSWORD_MD5\"}"
```
Already have KOReader Sync credentials? Skip registration; basic sync only requires using the same existing username/password on all devices.
When this returns `HTTP 402` with `{"code":2002,"message":"Username is already registered."}`, pick a different username or use that existing account.
2. On each CrossPoint device:
- Go to **Settings -> System -> KOReader Sync**.
- Set **Username** and **Password** (enter the plain password; CrossPoint computes MD5 internally, and use the same values on all devices).
- Set **Sync Server URL** to `https://sync.koreader.rocks`, or leave it empty (both use the same default KOReader sync server).
- Run **Authenticate**.
3. While reading, press **Confirm** to open the reader menu, then select **Sync Progress**.
- Choose **Apply Remote** to jump to remote progress.
- Choose **Upload Local** to push current progress.
##### Option B: Self-Hosted Server (Docker Compose)
##### Option C: Self-Hosted Server (Docker Compose)
1. Start a sync server:
@@ -315,7 +450,7 @@ curl -H "Accept: application/vnd.koreader.v1+json" "http://<server-ip>:17200/hea
```
3. Register a user once.
CrossPoint authenticates against KOReader Sync (`koreader/kosync`) using an MD5 key, so register using the MD5 of your password:
CrossPoint authenticates against KOReader Sync (`koreader/kosync`) using an MD5 key, so register using the MD5 of your password:
> [!WARNING]
> Sending a reusable MD5-derived password over plain HTTP is insecure.
@@ -337,29 +472,36 @@ curl -i "http://<server-ip>:17200/users/create" \
If this returns `HTTP 402` with `{"code":2002,"message":"Username is already registered."}`, the account already exists.
4. On each CrossPoint device:
- Go to **Settings -> System -> KOReader Sync**.
- Set **Username** and **Password** (enter the plain password; CrossPoint computes MD5 internally, and use the same values on all devices).
- Set **Sync Server URL** to `http://<server-ip>:17200`.
- Run **Authenticate**.
If you use the HTTPS listener, use `https://<server-ip>:7200` (`curl -k` only for self-signed certificate testing).
5. While reading, press **Confirm** to open the reader menu, then select **Sync Progress**.
- Choose **Apply Remote** to jump to remote progress.
- Choose **Upload Local** to push current progress.
##### Syncing While Reading
Once any of the options above is set up, press **Confirm** while reading to open the reader menu, then select **Sync Progress**. Alternatively, set **Settings -> Controls -> Long-press Menu** to **KOSync** and hold Confirm to launch sync directly.
- With **Sync Behavior** set to **Ask every time**, choose **Apply Remote** to jump to remote progress or **Upload Local** to push current progress.
- With **Sync Behavior** set to **Smart sync**, CrossPoint auto-resolves simple cases: upload when no remote progress exists, confirm and leave both unchanged when local and remote progress are already synchronized, upload when local progress is further ahead, or apply remote when remote progress is further ahead.
### 3.7 Sleep Screen
The **Sleep Screen** setting controls what is displayed when the device goes to sleep:
| Mode | Behavior |
|------|----------|
| **Dark** (default) | The CrossPoint logo on a dark background. |
| **Light** | The CrossPoint logo on a white background. |
| **Custom** | A custom image from the SD card (see below). Falls back to **Dark** if no custom image is found. |
| **Cover** | The cover of the currently open book. Falls back to **Dark** if no book is open. |
| **Cover + Custom** | The cover of the currently open book. Falls back to **Custom** behavior if no book is open. |
| **None** | A blank screen. |
| Mode | Behavior |
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------- |
| **Dark** (default) | The CrossPoint logo on a dark background. |
| **Light** | The CrossPoint logo on a white background. |
| **Custom** | A custom image from the SD card (see below). Falls back to **Dark** if no custom image is found. |
| **Cover** | The cover of the currently open book. Falls back to **Dark** if no book is open. |
| **Cover + Custom** | The cover of the currently open book, shown only while actively reading. Falls back to **Custom** behavior when not reading. |
| **None** | A blank screen. |
#### Cover settings
@@ -377,10 +519,30 @@ To use custom sleep images, set the sleep screen mode to **Custom** or **Cover +
> [!TIP]
> For best results:
>
> - Use uncompressed BMP files with 24-bit color depth
> - X4: Use a resolution of 480x800 pixels to match the device's screen resolution.
> - X3: Use a resolution of 528x792 pixels to match the device's screen resolution.
> [!TIP]
> You can set an image as the sleep screen cover directly from the BMP image viewer in the **[Browse Files](#33-browse-files-screen)** screen.
---
### 3.8 Custom Fonts (SD Card)
CrossPoint supports loading additional fonts from the SD card, extending beyond the two built-in families (Noto Serif, Noto Sans). Custom fonts can include extended Unicode coverage, enabling CJK (Chinese, Japanese, Korean) and other scripts.
There are three ways to install fonts:
1. **Download from device (recommended):** Go to **Settings -> System -> Manage Fonts**, browse the available font families, and select one to download over Wi-Fi.
2. **Upload via web interface:** While in **File Transfer** mode, open the web UI in a browser and navigate to the **Fonts** tab to upload `.cpfont` files.
3. **Manual SD card copy:** Download font files from the [crosspoint-fonts repository](https://github.com/crosspoint-reader/crosspoint-fonts) and copy them to `/.fonts/` (preferred) or `/fonts/` on your SD card.
Once installed, custom fonts appear in **Settings → Reader → Font Family** alongside the built-in fonts.
See [docs/sd-card-fonts.md](./docs/sd-card-fonts.md) for full installation details and SD card folder structure.
---
## 4. Reading Mode
@@ -388,6 +550,7 @@ To use custom sleep images, set the sleep screen mode to **Custom** or **Cover +
Once you have opened a book, the button layout changes to facilitate reading.
### Page Turning
| Action | Buttons |
| ----------------- | ------------------------------------ |
| **Previous Page** | Press **Left** _or_ **Volume Up** |
@@ -398,55 +561,143 @@ The role of the volume (side) buttons can be swapped in the **[Controls Settings
If the **Short Power Button Click** setting is set to "Page Turn", you can also turn to the next page by briefly pressing the Power button.
### Chapter Navigation
* **Next Chapter:** Press and **hold** the **Right** (or **Volume Down**) button briefly, then release.
* **Previous Chapter:** Press and **hold** the **Left** (or **Volume Up**) button briefly, then release.
This feature can be disabled in the **[Controls Settings](#363-controls)** to help avoid changing chapters by mistake.
### Auto Page Turn
Auto Page Turn automatically advances pages at a set interval, useful for hands-free reading. This feature can be enabled and configured from the **[Reader Menu](#5-reader-menu)** while reading an EPUB.
### Tilt Page Turn (X3 only)
On the **Xteink X3**, the gyroscope can be used to turn pages by tilting the device. This feature is available in the Controls settings.
### Footnote Navigation
When reading an EPUB that contains footnotes, you can navigate to the footnote text by selecting the footnote reference in the book. From the footnote, you can return to your original reading position.
If the device goes to sleep or you close the book while viewing a footnote, the book reopens to your original reading position, not the footnote.
### System Navigation
* **Return to Home:** Press the **Back** button to close the book and return to the **[Home](#31-home-screen)** screen.
* **Return to Browse Files:** Press and hold the **Back** button to close the book and return to the **[Browse Files](#33-browse-files-screen)** screen.
* **Chapter Menu:** Press **Confirm** to open the **[Table of Contents/Chapter Selection](#5-chapter-selection-screen)** screen.
* **Reader Menu:** Press **Confirm** to open the **[Reader Menu](#5-reader-menu)**, which includes chapter navigation, reading options, and more.
* **Long-press Confirm (configurable):** Holding **Confirm** runs the function chosen by the **Long-press Menu** setting in **[Controls Settings](#363-controls)** — "Bookmark" (default) drops a bookmark, "KOSync" launches KOReader Sync, "Dictionary" starts a word lookup, "Disabled" does nothing. A short press always opens the Reader Menu.
### Supported Languages
CrossPoint renders text using the following Unicode character blocks, enabling support for a wide range of languages:
* **Latin Script (Basic, Supplement, Extended-A):** Covers English, German, French, Spanish, Portuguese, Italian, Dutch, Swedish, Norwegian, Danish, Finnish, Polish, Czech, Hungarian, Romanian, Slovak, Slovenian, Turkish, and others.
* **Cyrillic Script (Standard and Extended):** Covers Russian, Ukrainian, Belarusian, Bulgarian, Serbian, Macedonian, Kazakh, Kyrgyz, Mongolian, and others.
* **Latin Script (Basic, Supplement, Extended-A/B):** Covers English, German, French, Spanish, Portuguese, Italian, Dutch, Swedish, Norwegian, Danish, Finnish, Polish, Czech, Hungarian, Romanian, Slovak, Slovenian, Turkish, Catalan, and others.
* **Cyrillic Script (Standard and Extended):** Covers Russian, Ukrainian, Belarusian, Bulgarian, Serbian, Macedonian, Kazakh, Kyrgyz, Mongolian, and others.
* **Vietnamese:** Supported via extended Latin glyph coverage in the built-in reader fonts.
What is not supported: Chinese, Japanese, Korean, Vietnamese, Hebrew, Arabic, Greek and Farsi.
What is not supported with built-in reader fonts: Chinese, Japanese, Korean, Arabic, Greek, Hebrew, and Farsi. However, **CJK, Hebrew, Greek, and other extended scripts can be enabled by installing custom SD card fonts** — see [Custom Fonts (SD Card)](#38-custom-fonts-sd-card).
---
## 5. Chapter Selection Screen
## 5. Reader Menu
Accessible by pressing **Confirm** while inside a book.
Press **Confirm** while reading to open the Reader Menu. From here you can access reading utilities and navigation options without leaving the book.
1. Use **Left** (or **Volume Up**), or **Right** (or **Volume Down**) to highlight the desired chapter.
2. Press **Confirm** to jump to that chapter.
3. *Alternatively, press **Back** to cancel and return to your current page.*
Available options include:
- **Select Chapter** Open the table of contents to jump to a specific chapter (see [Chapter Selection](#51-chapter-selection) below).
- **Footnotes** Navigate to the footnotes for the current section *(only shown in books that contain footnotes)*.
- **Look Up** Select a word on the current page and show its dictionary definition (see [docs/dictionary.md](docs/dictionary.md)). Requires a dictionary to be selected in **Settings → Reader → Dictionary**.
- **Reading Orientation** Cycle through screen orientations without leaving the reader.
- **Auto Turn (Pages Per Minute)** Cycle through automatic page turn speed options for hands-free reading.
- **Go to %** Jump to a specific position in the book by percentage.
- **Take screenshot** Save a screenshot of the current page to the `screenshots/` folder.
- **Show page as QR** Display a QR code encoding the current reading position.
- **Go Home** Close the book and return to the Home screen.
- **Sync Progress** Push or pull reading progress with a KOReader sync server (see [KOReader Sync Quick Setup](#367-koreader-sync-quick-setup)).
- **Delete Book Cache** Clear the cached layout data for the current book, forcing a re-index on next open.
Press **Back** at any time to close the menu and return to your current page.
### 5.1 Chapter Selection
Accessible by selecting **Chapters** from the Reader Menu.
1. Use **Left** (or **Volume Up**), or **Right** (or **Volume Down**) to highlight the desired chapter.
2. Press **Confirm** to jump to that chapter.
3. *Alternatively, press **Back** to cancel and return to your current page.*
---
### 5.2 Bookmarks
Bookmarks can be created to quickly save and restore your place in a book.
To create a bookmark, hold **Confirm** for about half a second while inside a book. A popup will appear letting you know a bookmark was created. The popup message will automatically disappear in a couple of seconds.
To open bookmarks, press **Confirm** while inside a book. Then navigate to the **Bookmarks** menu. Bookmarks can be opened by navigating to them and pressing **Confirm**, which will redirect you to that place in the book. You can delete bookmarks by holding **Confirm** for about 0.7 seconds, and then pressing **Confirm** again to confirm deletion, or **Back** to cancel.
Bookmarks are stored in the `.crosspoint/bookmarks` folder in the JSON format.
## 6. Current Limitations & Roadmap
Please note that this firmware is currently in active development. The following features are **not yet supported** but are planned for future updates:
* **Images:** Embedded images in e-books will not render.
* **Cover Images:** Large cover images embedded into EPUB require several seconds (~10s for ~2000 pixel tall image) to convert for sleep screen and home screen thumbnail. Consider optimizing the EPUB with e.g. https://github.com/bigbag/epub-to-xtc-converter to speed this up.
* **Unsupported Image Formats:** Most JPG and PNG images in EPUBs render correctly. GIFs and progressive JPEGs are not supported and will fall back to an `[Image]` placeholder.
*
* **Dictionary Lookup:** Inline word lookup is not yet implemented.
---
## 7. Troubleshooting Issues & Escaping Bootloop
If an issue or crash is encountered while using Crosspoint, feel free to raise an issue ticket and attach the serial monitor logs. The logs can be obtained by connecting the device to a computer and starting a serial monitor. Either [Serial Monitor](https://www.serialmonitor.org/) or the following command can be used:
If an issue or crash is encountered while using Crosspoint, feel free to raise an issue ticket and attach the logs.
**Crash reports on SD card:** After a crash, CrossPoint automatically saves a crash report to the SD card (no USB connection needed). Check the root of the SD card for a crash log file and include it with any bug report.
**Serial monitor logs:** For more detailed debugging, connect the device to a computer and run the custom debugging monitor script (requires Python 3 with `pyserial`, `colorama`, and `matplotlib`; install via `pip3 install pyserial colorama matplotlib`):
```
pio device monitor
python3 scripts/debugging_monitor.py
```
The script auto-detects the serial port. You can also specify one explicitly:
```
python3 scripts/debugging_monitor.py /dev/ttyACM0 # Linux
python3 scripts/debugging_monitor.py /dev/tty.usbmodem1 # macOS
python3 scripts/debugging_monitor.py COM7 # Windows
```
**Features:**
- Color-coded log output by category (errors, memory, display, EPUB parsing, etc.)
- Live memory usage graph (free RAM, total RAM, max contiguous allocation) updated every second
- Interactive command prompt — type a command and press Enter to send it to the device
- Screenshot capture — saves the current display to `screenshot.bmp` when triggered by the device
**Options:**
| Option | Description |
| -------------------- | --------------------------------------------------------- |
| `--baud RATE` | Baud rate (default: 115200) |
| `--filter KEYWORD` | Show only lines containing the keyword (case-insensitive) |
| `--suppress KEYWORD` | Hide lines containing the keyword (case-insensitive) |
**Examples:**
```
# Show only memory-related log lines
python3 scripts/debugging_monitor.py --filter MEM
# Hide noisy SD card log lines
python3 scripts/debugging_monitor.py --suppress "[SD]"
```
Press **Ctrl-C** or close the graph window to exit.
If the device is stuck in a bootloop, press and release the Reset button. Then, press and hold on to the configured Back button and the Power Button to boot to the Home Screen.
There can be issues with broken cache or config. In this case, delete the `.crosspoint` directory on your SD card (or consider deleting only `settings.bin`, `state.bin`, or `epub_*` cache directories in the `.crosspoint/` folder).
There can be issues with broken cache or config. In this case, delete the `.crosspoint` directory on your SD card (or consider deleting only `settings.json`, `state.json`, or `epub_*` cache directories in the `.crosspoint/` folder).
+1
View File
@@ -47,6 +47,7 @@ git ls-files --exclude-standard ${GIT_LS_FILES_FLAGS} \
| grep -v -E '^lib/EpdFont/builtinFonts/' \
| grep -v -E '^lib/Epub/Epub/hyphenation/generated/' \
| grep -v -E '^lib/uzlib/' \
| grep -v -E '^lib/miniz/third_party/' \
| xargs -r "${CLANG_FORMAT_BIN}" -style=file -i
# Restore strict pipeline failure handling for the rest of the script.
set -o pipefail
+2 -2
View File
@@ -4,7 +4,7 @@
.DESCRIPTION
Formats all C/C++ source and header files in the repository, excluding
generated, vendored, and build directories (open-x4-sdk, builtinFonts,
generated, vendored, and build directories (freeink-sdk, builtinFonts,
hyphenation tries, uzlib, .pio, *.generated.h).
The clang-format binary path is resolved once and cached in
@@ -92,7 +92,7 @@ function Resolve-ClangFormat {
$clangFormat = Resolve-ClangFormat
$exclude = @(
'open-x4-sdk'
'freeink-sdk'
'lib\EpdFont\builtinFonts'
'lib\Epub\Epub\hyphenation\generated'
'lib\uzlib'
+1
View File
@@ -7,5 +7,6 @@ It is written for software developers who may be new to embedded development.
- [Architecture Overview](./architecture.md)
- [Development Workflow](./development-workflow.md)
- [Testing and Debugging](./testing-debugging.md)
- [Touch and UI Development](./touch-and-ui.md)
If you are new, start with [Getting Started](./getting-started.md).
+40 -23
View File
@@ -8,17 +8,18 @@ At a high level, it is firmware that uses an activity-driven application archite
```mermaid
graph TD
A[Hardware: ESP32-C3 + SD + E-ink + Buttons] --> B[open-x4-sdk HAL]
B --> C[src/main.cpp runtime loop]
C --> D[Activities layer]
C --> E[State and settings]
D --> F[Reader flows]
D --> G[Home/Library/Settings flows]
D --> H[Network/Web server flows]
F --> I[lib/Epub parsing + layout + hyphenation]
I --> J[SD cache in .crosspoint]
D --> K[GfxRenderer]
K --> L[E-ink display buffer]
A[Hardware: ESP32-C3 + SD + E-ink + Buttons] --> B[freeink-sdk]
B --> C[lib/hal wrappers]
C --> D[src/main.cpp runtime loop]
D --> E[Activities layer]
D --> F[State and settings]
E --> G[Reader flows]
E --> H[Home/Library/Settings flows]
E --> I[Network/Web server flows]
G --> J[lib/Epub parsing + layout + hyphenation]
J --> K[SD cache in .crosspoint]
E --> L[GfxRenderer]
L --> M[E-ink display buffer]
```
## Runtime lifecycle
@@ -58,7 +59,7 @@ Top-level activity groups:
- `src/activities/home/`: home and library navigation
- `src/activities/reader/`: EPUB/XTC/TXT reading flows
- `src/activities/settings/`: settings menus and configuration
- `src/activities/network/`: WiFi selection, AP/STA mode, file transfer server
- `src/activities/network/`: Wi-Fi selection, AP/STA mode, file transfer server
- `src/activities/boot_sleep/`: boot and sleep transitions
## Reader and content pipeline
@@ -73,10 +74,11 @@ flowchart LR
C -->|EPUB| D[lib/Epub/Epub]
C -->|XTC| E[lib/Xtc reader]
C -->|TXT| F[lib/Txt reader]
D --> G[Parse OPF/TOC/CSS]
G --> H[Layout pages/sections]
H --> I[Write section and metadata caches]
I --> J[Render current page via GfxRenderer]
D --> G[Parse OPF/TOC and collect CSS refs]
G --> H[Build/load book.bin and css_rules.cache]
H --> I[Layout pages/sections]
I --> J[Write section cache]
J --> K[Render current page via GfxRenderer]
```
Why caching matters:
@@ -98,7 +100,7 @@ flowchart TD
D --> E[Locate container and OPF]
E --> F[Build or load BookMetadataCache]
F --> G[Load TOC and spine]
G --> H[Load or parse CSS rules]
G --> H[Load CSS cache or parse manifest/base-dir CSS]
H --> I[EpubReaderActivity]
I --> J{Section cache exists for current settings?}
@@ -121,7 +123,12 @@ flowchart TD
Notes:
- "section cache exists" depends on cache-busting parameters such as font and layout-related settings
- CSS files are collected from the OPF manifest and, when needed, discovered by
streaming ZIP paths under the OPF content base directory; the firmware avoids
preloading the full ZIP central directory for large books.
- "section cache exists" depends on cache-busting parameters such as font,
viewport size, paragraph alignment, hyphenation, embedded CSS, image rendering,
and Focus Reading settings
- rendering favors reusing precomputed layout data to keep page turns responsive on constrained hardware
- progress/session state is persisted so the reader can reopen at the last position after reboot/sleep
@@ -138,14 +145,18 @@ Typical persisted areas on SD:
/.crosspoint/
epub_<hash>/
book.bin
css_rules.cache
progress.bin
cover.bmp
sections/*.bin
settings.bin
state.bin
img_* cache files
settings.json
state.json
```
For binary cache formats, see `docs/file-formats.md`.
`sections/*.bin` contains rendered pages plus anchor, paragraph, and list-item
lookup tables used for TOC/footnote jumps and KOReader sync refinement. For
binary cache formats, see `docs/file-formats.md`.
## Networking architecture
@@ -153,14 +164,18 @@ Network file transfer is controlled by `src/activities/network/CrossPointWebServ
Modes:
- STA: join existing WiFi network
- STA: join existing Wi-Fi network
- AP: create hotspot
- Calibre Wireless: STA flow specialized for Calibre plugin uploads
Server behavior:
- HTTP server on port 80
- WebSocket upload server on port 81
- WebDAV handler on the HTTP server
- UDP discovery listener for upload clients
- file operations backed by SD storage
- browser APIs for file management, settings, fonts, OPDS servers, and saved Wi-Fi networks
- activity requests faster loop responsiveness while server is running
Endpoint reference: `docs/webserver-endpoints.md`.
@@ -170,6 +185,7 @@ Endpoint reference: `docs/webserver-endpoints.md`.
Some sources are generated and should not be edited manually.
- `scripts/build_html.py` generates `src/network/html/*.generated.h` from HTML files
- `scripts/gen_i18n.py` generates `lib/I18n/I18nKeys.h`, `I18nStrings.h`, and `I18nStrings.cpp`
- `scripts/generate_hyphenation_trie.py` generates hyphenation headers under `lib/Epub/Epub/hyphenation/generated/`
When editing related source assets, regenerate via normal build steps/scripts.
@@ -179,9 +195,10 @@ When editing related source assets, regenerate via normal build steps/scripts.
- `src/`: app orchestration, settings/state, and activity implementations
- `src/network/`: web server and OTA/update networking
- `src/components/`: theming and shared UI components
- `lib/hal/`: hardware abstraction wrappers around freeink-sdk
- `lib/Epub/`: EPUB parser, layout, CSS handling, and hyphenation
- `lib/`: supporting libraries (fonts, text, filesystem helpers, etc.)
- `open-x4-sdk/`: hardware SDK submodule (display, input, storage, battery)
- `freeink-sdk/`: hardware SDK submodule (display, input, storage, battery). Docs: https://freeink.org/docs
- `docs/`: user and technical documentation
## Embedded constraints that shape design
+221
View File
@@ -0,0 +1,221 @@
# Touch and UI Development
CrossPoint now runs on touch devices (Seeed Sticky, M5Paper, LilyGo T5) alongside the button-only Xteink X3/X4. Every screen must work with both input styles. There are two supported ways to get there:
1. **New screens: build them with FreeInkUI components.** Touch hit-testing, tap highlighting, long-press, and button focus navigation come with the component; you never hand-roll coordinate math.
2. **Existing screens and in-flight features: use the MappedInputManager touch bridge.** A small set of helpers adds tap/hold/swipe support to hand-rolled rendering without restructuring the activity.
If you are starting a new activity, use FreeInkUI. If you have a feature branch with a hand-rolled screen already working on buttons, use the bridge; do not rewrite mid-flight.
---
## Path 1: New screens with FreeInkUI
FreeInkUI (`freeink-sdk/libs/ui/FreeInkUI`, namespace `freeink::ui`) is an immediate-mode component library. The core idea:
- While a component renders, it registers its tappable areas ("hit rects") into the frame's interaction buffer via `Frame::hit(rect, action, value, inputMask, state)`.
- Each loop you build an `InputSnapshot` from the input manager and route it against that buffer. If a tap (or mapped button press) lands in a registered rect, the routed `ActionEvent` tells you which action fired and with what value.
- Components query `frame.stateFor(action, value)` while painting, so touch-down highlight and focus states render correctly without any per-activity code.
You get touch, hold highlighting, long-press, minimum touch-target sizing, and orientation-aware coordinates for free. The same action IDs fire from physical buttons, so one code path serves both input styles.
### Component inventory
All under `freeink-sdk/libs/ui/FreeInkUI/include/components/`:
| Category | Components |
|---|---|
| Controls | `button`, `checkbox`, `slider`, `progress-bar`, `header` |
| Lists | `list` (virtualized), `table`, `dropdown`, `radio-group`, `setting-row`, `toggle-row`, `stepper-row` |
| Keyboard | `keyboard` (QWERTY/AZERTY/QWERTZ/ES layouts), `key-grid` |
| Overlays | `popup`, `option-dialog`, `context-menu`, `message-panel`, `toast` |
| Bars | `status-bar`, `tab-bar`, `reader-chrome`, `battery-indicator`, `gesture-bar`, `tap-zones` |
| Media | `book-card`, `cover-grid`, `cover-carousel`, `metric-card` |
| Text | `text-field`, `text-area` |
### Integration skeleton
The in-tree reference is [`src/activities/util/KeyboardEntryActivity.cpp`](../../src/activities/util/KeyboardEntryActivity.cpp); it drives the FreeInkUI keyboard component inside a normal Activity. The shape:
```cpp
#include <FreeInkUIGfxRenderer.h>
namespace fui = freeink::ui;
// Render step: draw the component and (re)register its hit rects.
fui::GfxRendererTarget target(renderer); // adapts GfxRenderer to FreeInkUI's DrawTarget
target.setFont(fui::GfxRendererTarget::FONT_BODY, UI_12_FONT_ID);
const fui::DeviceContext device = target.deviceContext(); // orientation, safe area, touch sizing
fui::Frame<48> frame(target, device, fui::InputSnapshot{}, interactions); // 48 = max hit rects
fui::KeyboardProps props;
props.layout = &currentLayout();
props.keyAction = ACTION_KEY; // one action id; the key is the event value
props.inputMask = fui::InputTouch | fui::InputLongPress;
fui::keyboard(frame, kbRect, props); // draws AND registers hit rects
```
```cpp
// Input step (each loop): feed touch state, act on routed events.
int tx, ty, tapX, tapY, hx, hy;
const bool pressedDown = mappedInput.wasScreenTouchDown(tx, ty);
const bool tapped = mappedInput.wasScreenTapped(tapX, tapY);
const bool inContact = mappedInput.isScreenTouchHeld(hx, hy);
const auto result = touchRouter.update(interactions, pressedDown, tx, ty,
tapped, tapX, tapY, inContact, millis());
if (result.event) {
activateValue(result.event.value, result.event.longPress);
requestUpdate();
} else if (result.activeChanged) {
requestUpdate(); // repaint touch-down highlight
}
```
Notes:
- `Frame<N>` is templated on the max number of hit rects; the buffer is stack/static, no heap.
- `fui::TouchHoldRouter` (see `components/keyboard/keyboard.h`) synthesizes long-press while the finger is still down and swallows the eventual release. Use it whenever a component distinguishes tap from long-press.
- For simpler screens without long-press, `FreeInkApp` / `Screen<N>` (`FreeInkApp.h`) is the ergonomic layer: it owns the interaction buffer, offers a top-to-bottom builder (`takeTop`, footer helpers, etc.), and its `render(input)` call routes the snapshot and dispatches registered action callbacks in one step. `snapshotFrom(input, device)` in `FreeInkUIInputManager.h` builds the `InputSnapshot` with orientation-aware touch mapping.
- Everything stays allocation-free and works on the button-only devices unchanged: physical buttons route through the same interaction table via each hit rect's `inputMask`.
---
## Path 2: Adding touch to existing hand-rolled screens
For activities that draw their own rows, menus, and buttons, `MappedInputManager` ([src/MappedInputManager.h](../../src/MappedInputManager.h)) exposes bridge helpers. They all return **logical screen coordinates** (orientation already applied via `GfxRenderer::tapToLogical`); never touch raw normalized panel coordinates or the SDK `InputManager` directly.
| Helper | Use for |
|---|---|
| `wasScreenTapped(x, y)` | A completed tap (press + release), with logical coords |
| `wasScreenTouchDown(x, y)` | Touch-down (held > 90 ms, not yet released): draw selection highlight |
| `isScreenTouchHeld(x, y)` | Live contact position while the finger is down (drag tracking) |
| `wasTapInRect(x, y, w, h)` | One-off hit test on a rectangle (a single button, a banner) |
| `wasListItemTapped(index, count, selected, listTop, listHeight, hasSubtitle)` | Taps on a standard UITheme list; does the row/paging math for you |
| `wasListItemTouchedDown(...)` | Same geometry, touch-down phase (highlight before activate) |
| `rowTouch(row, top, rowStep, rowCount, xStart, xEnd, rowHeight)` | Any custom band of equal-height rows; returns `RowTouch::None/Down/Tap` |
| `colTouch(col, left, colStep, colCount, yStart, yEnd, colWidth)` | Horizontal button bands (dialogs, prompts) |
| `wasSwipe()` | Raw swipe direction if you need one beyond the global gestures |
| `hasTouch()` | True when the device has a touch panel (rarely needed; helpers simply never fire without one) |
### Pattern A: standard themed list
One call per loop; UITheme owns the row geometry ([EpubReaderBookmarksActivity.cpp:123](../../src/activities/reader/EpubReaderBookmarksActivity.cpp)):
```cpp
int tapped = -1;
if (mappedInput.wasListItemTapped(tapped, bookmarks.size(), selectorIndex, listY, listHeight, true)) {
selectorIndex = tapped;
openBookmark();
return;
}
```
### Pattern B: custom rows with hold highlight
For non-theme row layouts, use `rowTouch` and distinguish `Down` (highlight) from `Tap` (activate), as in [EpubReaderFootnotesActivity.cpp](../../src/activities/reader/EpubReaderFootnotesActivity.cpp):
```cpp
int row = -1;
const auto touch = mappedInput.rowTouch(row, listTop, lineHeight, visibleCount,
contentX, contentX + contentWidth);
if (touch != MappedInputManager::RowTouch::None) {
const int touched = scrollOffset + row;
if (touch == MappedInputManager::RowTouch::Down) {
if (selectedIndex != touched) { selectedIndex = touched; requestUpdate(); }
} else { // RowTouch::Tap
selectedIndex = touched;
activateSelection();
}
return;
}
```
The `Down` state exists because e-ink repaints are slow: highlight on touch-down gives immediate feedback, activation happens on release.
### Global gestures: do not reimplement these
Three gestures are handled once, for every screen. Activities must not add their own edge-swipe handling:
| Gesture | Trigger | Where it is handled |
|---|---|---|
| Back | Right-swipe starting in the left 25% of the screen | Folded into `Button::Back`, so the existing `wasPressed(Button::Back)` in your activity already fires |
| Home | Up-swipe starting in the bottom 14% | `ActivityManager::loop()`; pops to Home (activities can override via `handleHomeGesture()`) |
| Menu | Down-swipe starting in the top 14% | Activities that have a menu check `wasMenuGesture()` themselves (the reader does this) |
Because the back gesture arrives as `Button::Back`, most button-era activities gain back-swipe support with zero changes. That is the bar to aim for: bridge helpers should make touch an additive layer over the button flow, not a second input state machine.
### Bridge rules
- Handle touch in `loop()` next to the existing button handling, one helper call per interaction zone, and `return` after consuming an event (mirrors the button pattern).
- Never call the SDK `InputManager` or read GPIO directly; the HAL rule from the main guide applies to touch too.
- Coordinates from the helpers are logical and orientation-correct on all four rotations; test at least Portrait and one Landscape mode before PR.
- Nothing to clean up in `onExit()`; the helpers are stateless from the activity's point of view.
---
## Building and testing on non-Xteink devices
Each MCU family is its own binary: X3/X4 are ESP32-C3, Sticky and LilyGo T5 are ESP32-S3, M5Paper v1.1 is a classic ESP32. The Sticky env ships in `platformio.ini` (`pio run -e sticky`). Envs for other devices go in **`platformio.local.ini`**, a gitignored file that PlatformIO merges over `platformio.ini` (see `extra_configs`). Create it next to `platformio.ini`; personal envs, ports, and debug flags live there and never get committed.
Both envs below extend the repo's `[base]`, so they build against the `freeink-sdk` submodule with all the normal deps and scripts.
### M5Paper v1.1 (classic ESP32, IT8951 panel)
```ini
[env:m5paper_v11]
extends = base
board = esp32dev
board_build.mcu = esp32
board_build.flash_mode = qio
; CP2104 UART bridge: 921600 drops out on macOS after the stub baud switch
upload_speed = 460800
build_unflags =
${base.build_unflags}
; classic ESP32 has UART serial, not USB CDC; Logging.h keys off these
-DARDUINO_USB_MODE=1
-DARDUINO_USB_CDC_ON_BOOT=1
build_flags =
${base.build_flags}
-DFREEINK_DEVICE_M5PAPER=1
; the 63KB 540x960 framebuffer lives in PSRAM (FREEINK_FB_PSRAM auto-on)
-DBOARD_HAS_PSRAM
-DCROSSPOINT_VERSION=\"${crosspoint.version}-m5paper\"
-DENABLE_SERIAL_LOG
-DLOG_LEVEL=2
; touch-first device: hide front-button hint labels
-DCROSSPOINT_SHOW_BUTTON_HINTS=0
; archive-scan-order workaround: without these a full relink drops Wire's i2c symbols
-Wl,-u,i2cInit
-Wl,-u,i2cSlaveInit
```
### LilyGo T5 S3 (ESP32-S3, controller-less panel via LovyanGFX)
```ini
[env:lilygo_t5s3]
extends = base
board = esp32-s3-devkitc1-n16r8
board_build.mcu = esp32s3
build_flags =
${base.build_flags}
-DFREEINK_DEVICE_LILYGO=1
; board injects the parallel-bus pins + PMIC power hooks (BoardT5S3)
-DFREEINK_LGFX_EPD_CONFIG=lilygoT5S3LgfxConfig
-DCROSSPOINT_VERSION=\"${crosspoint.version}-lilygo\"
-DENABLE_SERIAL_LOG
-DLOG_LEVEL=2
-DCROSSPOINT_SHOW_BUTTON_HINTS=0
lib_deps =
${base.lib_deps}
; LgfxEpdConfig for the T5 S3 (pins, PCA9535/TPS65185 power sequence)
BoardT5S3=symlink://freeink-sdk/libs/hardware/BoardT5S3
; LovyanGFX Panel_EPD drives the controller-less ED047TC1 panel
m5stack/M5GFX @ 0.2.20
```
Then `pio run -e m5paper_v11 -t upload` (or `-e lilygo_t5s3`). Gotchas worth knowing:
- **Flash mode matters.** The M5Paper is `qio`; the X4-family standalone envs need `dio`. A wrong flash-mode header boots into a `partition 0 invalid magic number 0xffff` loop even though esptool verified the write.
- **One `FREEINK_DEVICE_*` flag per env** selects the board profile (pins, panel, touch controller) from the SDK's `BoardConfig`. See `freeink-sdk/platformio.sample.ini` for reference envs of every supported device.
- **Serial logs:** `[base]` does not enable logging; without `-DENABLE_SERIAL_LOG` a non-default env prints nothing.
- No touch hardware on your desk? The X4 build still exercises the same code paths through buttons; touch-specific behavior (tap zones, gestures) needs a real device.
+51
View File
@@ -0,0 +1,51 @@
# Dictionary
Look up words while reading an EPUB using an offline StarDict dictionary stored on the SD card.
## Supported Format
The reader supports **StarDict** dictionaries. When searching for dictionaries online, look for "StarDict format" or files with `.dict`, `.idx`, and `.ifo` extensions.
A dictionary folder must contain:
- `.idx` — word index (required, **must be uncompressed** — a `.idx.gz` will not work; decompress it on your computer with `gzip -d` first)
- `.dict` or `.dict.dz` — definition data (`.dict.dz` is supported as-is; entries are decompressed on the fly during lookup)
- `.ifo` — metadata (optional)
Not supported: `.syn` synonym files (ignored), dictionaries with 64-bit index offsets (`idxoffsetbits=64` in the `.ifo` — rare, and rejected with an error), and HTML-formatted definitions render as raw markup rather than styled text.
## Setting Up a Dictionary
1. Copy your dictionary folder(s) to `/dictionaries/` on the SD card — one dictionary per folder, e.g. `/dictionaries/webster/webster.idx` + `webster.dict.dz`. A hidden `/.dictionaries/` folder (dot-prefixed) works the same way, for keeping it out of the file browser.
2. Open **Settings → Reader → Dictionary** on the device.
3. Select a dictionary from the list, or **None** to disable lookups.
The Dictionary setting only appears when at least one usable dictionary folder exists. Folders containing more than one dictionary (multiple `.idx` stems) are skipped as ambiguous.
## Looking Up a Word
Two ways to start a lookup while reading:
- Open the reader menu (**Confirm**) and choose **Look Up**.
- Or set **Settings → Controls → Long-press Menu** to "Dictionary", then hold **Confirm** (~0.4s) on the reading page.
One word on the page becomes highlighted:
1. Use **Left/Right** to move between words in reading order, and the side **Up/Down** buttons to jump between lines.
2. Press **Confirm** to look up the highlighted word.
3. Press **Back** to return to the reader.
On the very first lookup with a dictionary (and again if the dictionary file changes), the reader shows *"Indexing dictionary…"* while it builds a small `.qidx` sidecar file next to the `.idx`. This takes a few seconds for large dictionaries and makes all subsequent lookups fast. The sidecar can be deleted safely at any time — it will simply be rebuilt.
### How Lookup Works
1. **Direct match** — the word is found as-is (case-insensitive) in the dictionary index. Surrounding punctuation is ignored.
2. **Stemming** — on a miss, common English word forms are retried automatically: possessives and plurals (`dogs``dog`, `stories``story`) and verb endings (`walked``walk`, `running``run`, `making``make`).
3. **Not found** — a short popup appears and you return to word selection.
## The Definition Screen
When a word is found, the definition screen shows the matched headword at the top and the definition text below, with a page counter for long definitions.
- **Left/Right** or side **Up/Down** — previous / next page
- **Back** — return to word selection
+186 -81
View File
@@ -1,22 +1,26 @@
# File Formats
These formats describe the SD-card cache files under `/.crosspoint/epub_<hash>/`.
All POD fields are written in the ESP32 little-endian representation used by
`Serialization.h`; strings are length-prefixed UTF-8.
## `book.bin`
### Version 3
### Version 7
ImHex Pattern:
`book.bin` stores EPUB metadata plus lookup tables for spine and TOC entries.
The current firmware writes this version from `BookMetadataCache`.
ImHex pattern:
```c++
import std.mem;
import std.string;
import std.core;
// === Configuration ===
#define EXPECTED_VERSION 3
#define EXPECTED_VERSION 7
#define MAX_STRING_LENGTH 65535
// === String Structure ===
struct String {
u32 length [[hidden, comment("String byte length")]];
if (length > MAX_STRING_LENGTH) {
@@ -29,74 +33,56 @@ fn format_string(String s) {
return s.data;
};
// === Metadata Structure ===
struct Metadata {
String title [[comment("Book title")]];
String author [[comment("Book author")]];
String language [[comment("Book language code")]];
String coverItemHref [[comment("Path to cover image")]];
String textReferenceHref [[comment("Path to guided first text reference")]];
} [[comment("Book metadata information")]];
// === Spine Entry Structure ===
};
struct SpineEntry {
String href [[comment("Resource path")]];
u32 cumulativeSize [[comment("Cumulative size in bytes"), color("FF6B6B")]];
s16 tocIndex [[comment("Index into TOC (-1 if none)"), color("4ECDC4")]];
} [[comment("Spine entry defining reading order")]];
// === TOC Entry Structure ===
u32 cumulativeSize [[comment("Cumulative uncompressed spine size through this entry")]];
s16 tocIndex [[comment("Index into TOC, or inherited/previous TOC index when no direct entry exists")]];
};
struct TocEntry {
String title [[comment("Chapter/section title")]];
String href [[comment("Resource path")]];
String anchor [[comment("Fragment identifier")]];
u8 level [[comment("Nesting level (0-255)"), color("95E1D3")]];
s16 spineIndex [[comment("Index into spine (-1 if none)"), color("F38181")]];
} [[comment("Table of contents entry")]];
// === Book Bin Structure ===
u8 level [[comment("Nesting level")]];
s16 spineIndex [[comment("Index into spine (-1 if none)")]];
};
struct BookBin {
// Header
u8 version [[comment("Format version"), color("FFD93D")]];
// Version validation
u8 version;
if (version != EXPECTED_VERSION) {
std::error(std::format("Unsupported version: {} (expected {})", version, EXPECTED_VERSION));
}
u32 lutOffset [[comment("Offset to lookup tables"), color("6BCB77")]];
u16 spineCount [[comment("Number of spine entries"), color("4D96FF")]];
u16 tocCount [[comment("Number of TOC entries"), color("FF6B9D")]];
u32 lutOffset [[comment("Offset to lookup tables")]];
u16 spineCount;
u16 tocCount;
// Metadata section
Metadata metadata [[comment("Book metadata")]];
Metadata metadata;
// Validate LUT offset alignment
u32 currentOffset = $;
if (currentOffset != lutOffset) {
std::warning(std::format("LUT offset mismatch: expected 0x{:X}, got 0x{:X}", lutOffset, currentOffset));
}
// Lookup Tables
u32 spineLut[spineCount] [[comment("Spine entry offsets"), color("4D96FF")]];
u32 tocLut[tocCount] [[comment("TOC entry offsets"), color("FF6B9D")]];
u32 spineLut[spineCount] [[comment("Spine entry offsets")]];
u32 tocLut[tocCount] [[comment("TOC entry offsets")]];
// Data Entries
SpineEntry spines[spineCount] [[comment("Spine entries (reading order)")]];
TocEntry toc[tocCount] [[comment("Table of contents entries")]];
SpineEntry spines[spineCount];
TocEntry toc[tocCount];
};
// === File Parsing ===
BookBin book @ 0x00;
// Validate we've consumed the entire file
u32 fileSize = std::mem::size();
u32 parsedSize = $;
if (parsedSize != fileSize) {
std::warning(std::format("Unparsed data detected: {} bytes remaining at offset 0x{:X}", fileSize - parsedSize, parsedSize));
}
@@ -104,20 +90,45 @@ if (parsedSize != fileSize) {
## `section.bin`
### Version 8
### Version 30
ImHex Pattern:
Each file in `sections/*.bin` stores one laid-out spine section. The header is
also the cache-busting key: if any layout-affecting setting differs from the
current reader settings, the section is discarded and rebuilt.
Version 30 is binary-identical to version 29. The version was bumped because
Arabic contextual shaping changed text measurement (`getTextAdvanceX` now
measures the shaped visual text), so word positions cached by v29 no longer
match what `drawText` renders.
Version 28 introduced serialized word style bits for underline, strikethrough,
superscript, and subscript. The format also includes:
- cache-busting fields for paragraph alignment, hyphenation, embedded CSS,
image rendering mode, and Focus Reading
- page offset LUT
- anchor-to-page map for fragment and footnote navigation
- paragraph and list-item LUTs used by KOReader sync page refinement
- optional per-word Focus Reading split metadata
- per-page footnote entries
- serialized word style bits for underline, strikethrough, superscript, and
subscript
- flat TextBlock word storage (v29): per-word arrays plus one shared
NUL-terminated text blob, replacing v28's length-prefixed word strings. The
on-disk order mirrors the in-RAM arena so the firmware reads a whole block
payload with a single allocation and a single SD read
ImHex pattern:
```c++
import std.mem;
import std.string;
import std.core;
// === Configuration ===
#define EXPECTED_VERSION 8
#define EXPECTED_VERSION 30
#define MAX_STRING_LENGTH 65535
// === String Structure ===
#define FOOTNOTE_NUMBER_LEN 32
#define FOOTNOTE_HREF_LEN 96
struct String {
u32 length [[hidden, comment("String byte length")]];
@@ -131,90 +142,184 @@ fn format_string(String s) {
return s.data;
};
// === Page Structure ===
enum StorageType : u8 {
PageLine = 1
enum PageElementTag : u8 {
TAG_PageLine = 1,
TAG_PageImage = 2,
TAG_PageHorizontalRule = 3
};
enum WordStyle : u8 {
REGULAR = 0,
BOLD = 1,
ITALIC = 2,
BOLD_ITALIC = 3
BOLD_ITALIC = 3,
UNDERLINE = 4,
STRIKETHROUGH = 8,
SUP = 16,
SUB = 32
};
enum BlockStyle : u8 {
enum TextAlign : u8 {
JUSTIFIED = 0,
LEFT_ALIGN = 1,
CENTER_ALIGN = 2,
RIGHT_ALIGN = 3,
NONE = 4
};
struct BlockStyle {
TextAlign alignment;
bool textAlignDefined;
s16 marginTop;
s16 marginBottom;
s16 marginLeft;
s16 marginRight;
s16 paddingTop;
s16 paddingBottom;
s16 paddingLeft;
s16 paddingRight;
s16 textIndent;
bool textIndentDefined;
bool isRtl;
bool directionDefined;
};
struct TextBlock {
u16 wordCount;
u8 hasFocus;
u16 textBytes [[comment("Total size of text[], including one NUL per word")]];
if (wordCount > 0) {
u16 textOff[wordCount] [[comment("Byte offset of word i's text within text[]")]];
s16 wordXPos[wordCount];
if (hasFocus != 0) {
u16 wordFocusSuffixX[wordCount] [[comment("Suffix x offset from word start")]];
}
WordStyle wordStyle[wordCount];
if (hasFocus != 0) {
u8 wordFocusBoundary[wordCount] [[comment("UTF-8 byte boundary between bold prefix and suffix")]];
}
char text[textBytes] [[comment("All words back to back, each NUL-terminated")]];
}
BlockStyle blockStyle;
};
struct ImageBlock {
String imagePath;
s16 width;
s16 height;
};
struct PageLine {
s16 xPos;
s16 yPos;
u16 wordCount;
String words[wordCount];
u16 wordXPos[wordCount];
WordStyle wordStyle[wordCount];
BlockStyle blockStyle;
s16 xPos;
s16 yPos;
TextBlock block;
};
struct PageImage {
s16 xPos;
s16 yPos;
ImageBlock image;
};
struct PageHorizontalRule {
s16 xPos;
s16 yPos;
u16 width;
u8 thickness;
};
struct PageElement {
u8 pageElementType;
if (pageElementType == 1) {
PageElementTag pageElementType;
if (pageElementType == TAG_PageLine) {
PageLine pageLine [[inline]];
} else if (pageElementType == TAG_PageImage) {
PageImage pageImage [[inline]];
} else if (pageElementType == TAG_PageHorizontalRule) {
PageHorizontalRule horizontalRule [[inline]];
} else {
std::error(std::format("Unknown page element type: {}", pageElementType));
}
};
struct FootnoteEntry {
char number[FOOTNOTE_NUMBER_LEN];
char href[FOOTNOTE_HREF_LEN];
};
struct Page {
u16 elementCount;
PageElement elements[elementCount] [[inline]];
u16 footnoteCount;
FootnoteEntry footnotes[footnoteCount];
};
// === Section Bin Structure ===
struct AnchorEntry {
String anchor;
u16 page;
};
struct AnchorMap {
u16 count;
AnchorEntry entries[count];
};
struct ParagraphLut {
u16 count;
u16 paragraphIndex[count];
};
struct SectionBin {
// Header
u8 version [[comment("Format version"), color("FFD93D")]];
// Version validation
u8 version;
if (version != EXPECTED_VERSION) {
std::error(std::format("Unsupported version: {} (expected {})", version, EXPECTED_VERSION));
}
// Cache busting parameters
s32 fontId;
float lineCompression;
bool extraParagraphSpacing;
u8 paragraphAlignment;
u16 viewportWidth;
u16 vieportHeight;
u16 viewportHeight;
bool hyphenationEnabled;
bool embeddedStyle;
u8 imageRendering;
bool focusReadingEnabled;
u16 pageCount;
u32 lutOffset;
u32 pageLutOffset;
u32 anchorMapOffset;
u32 paragraphLutOffset;
u32 listItemLutOffset;
Page page[pageCount];
Page pages[pageCount];
// Validate LUT offset alignment
u32 currentOffset = $;
if (currentOffset != lutOffset) {
std::warning(std::format("LUT offset mismatch: expected 0x{:X}, got 0x{:X}", lutOffset, currentOffset));
if (currentOffset != pageLutOffset) {
std::warning(std::format("Page LUT offset mismatch: expected 0x{:X}, got 0x{:X}", pageLutOffset, currentOffset));
}
// Lookup Tables
u32 lut[pageCount];
u32 pageLut[pageCount] [[comment("Page data offsets")]];
if (anchorMapOffset != 0) {
AnchorMap anchorMap @ anchorMapOffset;
}
if (paragraphLutOffset != 0) {
ParagraphLut paragraphLut @ paragraphLutOffset;
}
if (listItemLutOffset != 0 && paragraphLutOffset != 0) {
u16 listItemIndex[paragraphLut.count] @ listItemLutOffset;
}
};
// === File Parsing ===
SectionBin section @ 0x00;
SectionBin book @ 0x00;
// Validate we've consumed the entire file
u32 fileSize = std::mem::size();
u32 parsedSize = $;
if (parsedSize != fileSize) {
std::warning(std::format("Unparsed data detected: {} bytes remaining at offset 0x{:X}", fileSize - parsedSize, parsedSize));
}
+1 -1
View File
@@ -11,7 +11,7 @@ Focus Reading is a reading aid that bolds the first portion of each word, guidin
1. Open **Settings > Reader**
2. Toggle **Focus Reading** on
Toggling the setting will trigger a re-index of your current book, the same as when changing font settings. Once indexing is complete, page turns proceed as normal. No changes are made to your EPUB files.
Toggling the setting invalidates affected EPUB section caches for the current layout, the same as changing font settings. Sections are rebuilt on demand, then page turns proceed as normal. No changes are made to your EPUB files.
## Examples
+42 -23
View File
@@ -5,17 +5,29 @@ This guide explains the multi-language support system in CrossPoint Reader.
## Supported Languages
- English
- French
- German
- Portuguese
- Spanish
- Swedish
- Czech
- Russian
- Ukrainian
- Polish
- Danish
- Turkish
- Español
- Français
- Deutsch
- Čeština
- Português (Brasil)
- Русский
- Svenska
- Română
- Català
- Українська
- Беларуская
- Italiano
- Polski
- Suomi
- Dansk
- Nederlands
- Türkçe
- Қазақша
- Magyar
- Lietuvių
- Slovenščina
- Valencià
- עברית
---
@@ -108,7 +120,9 @@ This automatically:
#### 3. Use in code
```cpp
#include <CrossPointSettings.h>
#include <I18n.h>
#include <Logging.h>
// Using the tr() macro (recommended)
renderer.drawText(font, x, y, tr(STR_MY_NEW_STRING));
@@ -175,7 +189,7 @@ The YAML files use UTF-8 encoding. Special characters are automatically converte
// tr(id) - Get translated string without StrId:: prefix
const char* text = tr(STR_SETTINGS_TITLE);
renderer.drawText(font, x, y, tr(STR_BROWSE_FILES));
Serial.printf("Status: %s\n", tr(STR_CONNECTED));
LOG_INF("I18N", "Status: %s", tr(STR_CONNECTED));
// I18N - Shorthand for I18n::getInstance()
I18N.setLanguage(Language::ES);
@@ -191,34 +205,39 @@ const char* text = tr(STR_SETTINGS_TITLE); // Macro (recommended)
const char* text = I18N.get(StrId::STR_SETTINGS_TITLE); // Direct call
const char* text = I18N[StrId::STR_SETTINGS_TITLE]; // Operator overload
// Set language
// Set runtime language
I18N.setLanguage(Language::ES);
// Get current language
Language lang = I18N.getLanguage();
// Save language setting to file
I18N.saveSettings();
// Load language setting from file
I18N.loadSettings();
// Get character set for font subsetting (static method)
const char* chars = I18n::getCharacterSet(Language::FR);
// Persist a user language choice
SETTINGS.language = static_cast<uint8_t>(Language::ES);
SETTINGS.saveToFile();
```
---
## File Storage
Language settings are stored in:
The selected language is stored with the rest of the device settings in:
```text
/.crosspoint/settings.json
```
The JSON field is `language`, stored as a stable language code string such as
`"EN"`, `"DE"`, or `"HE"` rather than a raw enum value.
Older firmware versions used:
```text
/.crosspoint/language.bin
```
This file contains:
- Version byte
- Current language selection (1 byte)
On load, current firmware migrates that legacy file into `settings.json` and
renames it to `language.bin.bak`.
---
Binary file not shown.

After

Width:  |  Height:  |  Size: 234 KiB

+30 -8
View File
@@ -9,15 +9,15 @@ There are three ways to install fonts:
### Option 1: Download from device (recommended)
1. Connect your CrossPoint reader to WiFi
1. Connect your CrossPoint reader to Wi-Fi
2. Go to **Settings > System > Manage Fonts**
3. Browse available font families and tap to download
4. Downloaded fonts appear immediately in **Settings > Reader > Font Family**
### Option 2: Upload via web browser
1. Connect your CrossPoint reader to WiFi
2. Open the web interface in your browser (shown on the WiFi screen)
1. Start **File Transfer** and connect through **Join Network** or **Create Hotspot**
2. Open the web interface URL shown on the reader
3. Navigate to the **Fonts** tab
4. Upload `.cpfont` files using the upload form
@@ -91,14 +91,36 @@ To convert your own TrueType/OpenType fonts:
| Preset | Coverage |
|--------|----------|
| `ascii` | U+0020-U+007E (Basic Latin) |
| `latin-ext` | European languages (Latin + Extended-A/B) |
| `ascii` | U+0020U+007E (Basic Latin) |
| `latin1` | U+0080U+00FF (Latin-1 Supplement) |
| `latin-ext` | European languages (Latin + Extended-A/B + punctuation + ligatures) |
| `greek` | Greek + Extended Greek |
| `cyrillic` | Cyrillic + Supplement |
| `hebrew` | Hebrew + Alphabetic Presentation Forms |
| `georgian` | Georgian + Georgian Supplement |
| `armenian` | Armenian |
| `ethiopic` | Ethiopic + Extended |
| `vietnamese` | Vietnamese subset (ơ/ư and combining marks) |
| `punctuation` | General punctuation (U+2000U+206F) |
| `cjk` | CJK Unified Ideographs + Hiragana + Katakana + Fullwidth |
| `hangul` | Korean Hangul syllables |
| `builtin` | Matches built-in Bookerly coverage exactly |
| `hangul` | Korean Hangul syllables + Jamo + Compatibility Jamo |
| `cherokee` | Cherokee (historic + supplement block) |
| `tifinagh` | Tifinagh |
| `symbols` | Math, currency, arrows, box-drawing, misc symbols, dingbats |
| `reading` | Literary fiction coverage: Latin, Greek, Cyrillic, math/symbol blocks, supplemental punctuation, and CJK quote marks |
| `builtin` | Matches the firmware's built-in font conversion intervals |
Combine presets with commas: `--intervals latin-ext,greek,cyrillic`
Install custom fonts via WiFi upload or manual SD card copy.
You can also specify arbitrary Unicode ranges directly:
`--intervals latin-ext,(0x2100-0x214F)`
To list all presets with codepoint counts:
python3 lib/EpdFont/scripts/fontconvert_sdcard.py --list-presets
### Additional options
`--force-autohint` — force FreeType's auto-hinter instead of the font's native hinting (useful when a font's built-in hints produce poor results at small sizes).
Install custom fonts via the web interface or manual SD card copy.
+6 -2
View File
@@ -1,7 +1,9 @@
# Translators
Below is a list of users and languages CrossPoint may support in the future.
Note because a language is below does not mean there is official support for the language at this time.
Below is a list of translator credits for languages with known contributors.
Official UI language support is determined by the YAML files in
`lib/I18n/translations/`; see [i18n.md](./i18n.md) for the current supported
language list.
## Contributing
@@ -37,6 +39,7 @@ If you'd like to add your name to this list, please open a PR adding yourself an
- [Skrzakk](https://github.com/Skrzakk)
- [pablohc](https://github.com/pablohc)
- [DaniPhii](https://github.com/DaniPhii)
- [lpla](https://github.com/lpla)
## Swedish
- [dawiik](https://github.com/dawiik)
@@ -47,6 +50,7 @@ If you'd like to add your name to this list, please open a PR adding yourself an
## Catalan
- [angeldenom](https://github.com/angeldenom)
- [lpla](https://github.com/lpla)
## Finnish
- [plahteenlahti](https://github.com/plahteenlahti)
+13 -10
View File
@@ -1,6 +1,6 @@
# Troubleshooting
This document show most common issues and possible solutions while using the device features.
This document shows common issues and possible solutions while using the device features.
- [Troubleshooting](#troubleshooting)
- [Cannot See the Device on the Network](#cannot-see-the-device-on-the-network)
@@ -14,25 +14,27 @@ This document show most common issues and possible solutions while using the dev
**Solutions:**
1. Verify both devices are on the **same WiFi network**
- Check your computer/phone WiFi settings
- Confirm the CrossPoint Reader shows "Connected" status
1. Verify both devices are on the correct network
- Check your computer/phone Wi-Fi settings
- In **Join Network** mode, your computer/phone and CrossPoint Reader must be on the same Wi-Fi network
- In **Create Hotspot** mode, your computer/phone must be connected to the `CrossPoint-Reader` hotspot
2. Double-check the IP address
- Make sure you typed it correctly
- Include `http://` at the beginning
- Try the displayed IP address if `http://crosspoint.local/` does not resolve
3. Try disabling VPN if you're using one
4. Some networks have "client isolation" enabled - check with your network administrator
4. Some networks have "client isolation" enabled - use Create Hotspot mode or check with your network administrator
### Connection Drops or Times Out
**Problem:** WiFi connection is unstable
**Problem:** Wi-Fi connection is unstable
**Solutions:**
1. Move closer to the WiFi router
1. Move closer to the Wi-Fi router, or use Create Hotspot mode for a direct connection
2. Check signal strength on the device (should be at least `||` or better)
3. Avoid interference from other devices
4. Try a different WiFi network if available
4. Try a different Wi-Fi network if available
### Upload Fails
@@ -40,10 +42,11 @@ This document show most common issues and possible solutions while using the dev
**Solutions:**
1. Ensure the file is a valid `.epub` file
2. Check that the SD card has enough free space
1. Check that the SD card has enough free space
2. Check that the filename is valid for the SD card filesystem
3. Try uploading a smaller file first to test
4. Refresh the browser page and try again
5. If WebSocket upload fails repeatedly, refresh the page and retry with the HTTP fallback path
### Saved Password Not Working
+400 -231
View File
@@ -1,72 +1,36 @@
# Webserver Endpoints
This document describes all HTTP and WebSocket endpoints available on the CrossPoint Reader webserver.
This document describes the HTTP, WebSocket, WebDAV, and discovery endpoints
available while CrossPoint Reader is in File Transfer or Calibre Wireless mode.
- [Webserver Endpoints](#webserver-endpoints)
- [Overview](#overview)
- [HTTP Endpoints](#http-endpoints)
- [GET `/` - Home Page](#get----home-page)
- [GET `/files` - File Browser Page](#get-files---file-browser-page)
- [GET `/api/status` - Device Status](#get-apistatus---device-status)
- [GET `/api/files` - List Files](#get-apifiles---list-files)
- [POST `/upload` - Upload File](#post-upload---upload-file)
- [POST `/mkdir` - Create Folder](#post-mkdir---create-folder)
- [POST `/delete` - Delete File or Folder](#post-delete---delete-file-or-folder)
- [WebSocket Endpoint](#websocket-endpoint)
- [Port 81 - Fast Binary Upload](#port-81---fast-binary-upload)
- [Network Modes](#network-modes)
- [Station Mode (STA)](#station-mode-sta)
- [Access Point Mode (AP)](#access-point-mode-ap)
- [Notes](#notes)
- HTTP server: port 80
- WebSocket upload server: port 81
- UDP discovery listener: port 8134
- WebDAV: port 80, handled by the same HTTP server
Examples use `crosspoint.local`. If mDNS does not resolve on your network, use
the IP address shown on the device screen.
## Overview
## HTTP Pages
The CrossPoint Reader exposes a webserver for file management and device monitoring:
| Method | Path | Purpose |
|--------|------|---------|
| `GET` | `/` | Home/status page |
| `GET` | `/files` | File manager page |
| `GET` | `/settings` | Web settings page |
| `GET` | `/fonts` | SD-card font manager page |
| `GET` | `/js/jszip.min.js` | JavaScript asset used by the file manager |
- **HTTP Server**: Port 80
- **WebSocket Server**: Port 81 (for fast binary uploads)
## Device Status
---
### `GET /api/status`
## HTTP Endpoints
### GET `/` - Home Page
Serves the home page HTML interface.
**Request:**
```bash
curl http://crosspoint.local/
```
**Response:** HTML page (200 OK)
---
### GET `/files` - File Browser Page
Serves the file browser HTML interface.
**Request:**
```bash
curl http://crosspoint.local/files
```
**Response:** HTML page (200 OK)
---
### GET `/api/status` - Device Status
Returns JSON with device status information.
**Request:**
```bash
curl http://crosspoint.local/api/status
```
**Response (200 OK):**
Response:
```json
{
"version": "1.0.0",
@@ -74,258 +38,463 @@ curl http://crosspoint.local/api/status
"mode": "STA",
"rssi": -45,
"freeHeap": 123456,
"uptime": 3600
"uptime": 3600,
"device": "X4"
}
```
| Field | Type | Description |
| ---------- | ------ | --------------------------------------------------------- |
| `version` | string | CrossPoint firmware version |
| `ip` | string | Device IP address |
| `mode` | string | `"STA"` (connected to WiFi) or `"AP"` (access point mode) |
| `rssi` | number | WiFi signal strength in dBm (0 in AP mode) |
| `freeHeap` | number | Free heap memory in bytes |
| `uptime` | number | Seconds since device boot |
| Field | Type | Description |
|-------|------|-------------|
| `version` | string | Firmware version |
| `ip` | string | Device IP address |
| `mode` | string | `"STA"` for joined Wi-Fi or `"AP"` for hotspot mode |
| `rssi` | number | Wi-Fi RSSI in dBm; `0` in AP mode |
| `freeHeap` | number | Free heap in bytes |
| `uptime` | number | Seconds since boot |
| `device` | string | `"X3"` or `"X4"` hardware detection |
---
## File Management
### GET `/api/files` - List Files
### `GET /api/files`
Returns a JSON array of files and folders in the specified directory.
Lists files and folders under a directory.
**Request:**
```bash
# List root directory
curl http://crosspoint.local/api/files
# List specific directory
curl "http://crosspoint.local/api/files?path=/Books"
```
**Query Parameters:**
Query parameters:
| Parameter | Required | Default | Description |
| --------- | -------- | ------- | ---------------------- |
| `path` | No | `/` | Directory path to list |
| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| `path` | No | `/` | Directory to list |
Response:
**Response (200 OK):**
```json
[
{"name": "MyBook.epub", "size": 1234567, "isDirectory": false, "isEpub": true},
{"name": "Notes", "size": 0, "isDirectory": true, "isEpub": false},
{"name": "document.pdf", "size": 54321, "isDirectory": false, "isEpub": false}
{"name":"MyBook.epub","size":1234567,"isDirectory":false,"isEpub":true},
{"name":"Notes","size":0,"isDirectory":true,"isEpub":false}
]
```
| Field | Type | Description |
| ------------- | ------- | ---------------------------------------- |
| `name` | string | File or folder name |
| `size` | number | Size in bytes (0 for directories) |
| `isDirectory` | boolean | `true` if the item is a folder |
| `isEpub` | boolean | `true` if the file has `.epub` extension |
Hidden dotfiles are omitted unless the device setting `showHiddenFiles` is
enabled. `System Volume Information` and `XTCache` are always hidden/protected.
**Notes:**
- Hidden files (starting with `.`) are automatically filtered out
- System folders (`System Volume Information`, `XTCache`) are hidden
### `GET /download`
---
Downloads a file from the SD card.
### POST `/upload` - Upload File
Uploads a file to the SD card via multipart form data.
**Request:**
```bash
# Upload to root directory
curl -X POST -F "file=@mybook.epub" http://crosspoint.local/upload
curl -OJ "http://crosspoint.local/download?path=/Books/MyBook.epub"
```
# Upload to specific directory
Query parameters:
| Parameter | Required | Description |
|-----------|----------|-------------|
| `path` | Yes | File path to download |
Protected dotfiles, `System Volume Information`, and `XTCache` cannot be
downloaded. EPUB files are served as `application/epub+zip`; other files use
`application/octet-stream`.
### `POST /upload`
Uploads a file with HTTP multipart form data.
```bash
curl -X POST -F "file=@mybook.epub" "http://crosspoint.local/upload?path=/Books"
```
**Query Parameters:**
Query parameters:
| Parameter | Required | Default | Description |
| --------- | -------- | ------- | ------------------------------- |
| `path` | No | `/` | Target directory for the upload |
| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| `path` | No | `/` | Destination directory |
**Response (200 OK):**
```
Successful response:
```text
File uploaded successfully: mybook.epub
```
**Error Responses:**
Notes:
| Status | Body | Cause |
| ------ | ----------------------------------------------- | --------------------------- |
| 400 | `Failed to create file on SD card` | Cannot create file |
| 400 | `Failed to write to SD card - disk may be full` | Write error during upload |
| 400 | `Failed to write final data to SD card` | Error flushing final buffer |
| 400 | `Upload aborted` | Client aborted the upload |
| 400 | `Unknown error during upload` | Unspecified error |
- Existing files with the same name are overwritten.
- EPUB cache data for the uploaded path is cleared after a successful upload.
- HTTP upload uses a 4 KB write buffer before flushing to the SD card.
**Notes:**
- Existing files with the same name will be overwritten
- Uses a 4KB buffer for efficient SD card writes
### `POST /mkdir`
---
Creates a folder.
### POST `/mkdir` - Create Folder
Creates a new folder on the SD card.
**Request:**
```bash
curl -X POST -d "name=NewFolder&path=/" http://crosspoint.local/mkdir
```
**Form Parameters:**
Form parameters:
| Parameter | Required | Default | Description |
| --------- | -------- | ------- | ---------------------------- |
| `name` | Yes | - | Name of the folder to create |
| `path` | No | `/` | Parent directory path |
| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| `name` | Yes | - | New folder name |
| `path` | No | `/` | Parent folder |
**Response (200 OK):**
```
Folder created: NewFolder
```
### `POST /rename`
**Error Responses:**
Renames a file.
| Status | Body | Cause |
| ------ | ----------------------------- | ----------------------------- |
| 400 | `Missing folder name` | `name` parameter not provided |
| 400 | `Folder name cannot be empty` | Empty folder name |
| 400 | `Folder already exists` | Folder with same name exists |
| 500 | `Failed to create folder` | SD card error |
---
### POST `/delete` - Delete File or Folder
Deletes a file or folder from the SD card.
**Request:**
```bash
# Delete a file
curl -X POST -d "path=/Books/mybook.epub&type=file" http://crosspoint.local/delete
# Delete an empty folder
curl -X POST -d "path=/OldFolder&type=folder" http://crosspoint.local/delete
curl -X POST -d "path=/Books/old.epub&name=new.epub" http://crosspoint.local/rename
```
**Form Parameters:**
Form parameters:
| Parameter | Required | Default | Description |
| --------- | -------- | ------- | -------------------------------- |
| `path` | Yes | - | Path to the item to delete |
| `type` | No | `file` | Type of item: `file` or `folder` |
| Parameter | Required | Description |
|-----------|----------|-------------|
| `path` | Yes | Existing file path |
| `name` | Yes | New file name, not a path |
**Response (200 OK):**
```
Deleted successfully
Only files can be renamed through this endpoint. The old EPUB cache path is
cleared before the rename.
### `POST /move`
Moves a file into an existing folder.
```bash
curl -X POST -d "path=/Books/mybook.epub&dest=/Read" http://crosspoint.local/move
```
**Error Responses:**
Form parameters:
| Status | Body | Cause |
| ------ | --------------------------------------------- | ----------------------------- |
| 400 | `Missing path` | `path` parameter not provided |
| 400 | `Cannot delete root directory` | Attempted to delete `/` |
| 400 | `Folder is not empty. Delete contents first.` | Non-empty folder |
| 403 | `Cannot delete system files` | Hidden file (starts with `.`) |
| 403 | `Cannot delete protected items` | Protected system folder |
| 404 | `Item not found` | Path does not exist |
| 500 | `Failed to delete item` | SD card error |
| Parameter | Required | Description |
|-----------|----------|-------------|
| `path` | Yes | Existing file path |
| `dest` | Yes | Existing destination folder |
**Protected Items:**
- Files/folders starting with `.`
- `System Volume Information`
- `XTCache`
Only files can be moved through this endpoint. The old EPUB cache path is
cleared before the move.
---
### `POST /delete`
## WebSocket Endpoint
Deletes one or more files or empty folders.
### Port 81 - Fast Binary Upload
A WebSocket endpoint for high-speed binary file uploads. More efficient than HTTP multipart for large files.
**Connection:**
```bash
curl -X POST -d "path=/Books/mybook.epub" http://crosspoint.local/delete
curl -X POST -d 'paths=["/Books/old.epub","/OldFolder"]' http://crosspoint.local/delete
```
Form parameters:
| Parameter | Required | Description |
|-----------|----------|-------------|
| `path` | Yes, unless `paths` is provided | Single path to delete |
| `paths` | Yes, unless `path` is provided | JSON array of paths to delete |
Protected items cannot be deleted. Non-empty folders are rejected. EPUB cache
data for deleted files is cleared.
## Settings API
### `GET /api/settings`
Returns a streamed JSON array of editable settings. Each item contains common
fields plus type-specific fields.
```bash
curl http://crosspoint.local/api/settings
```
Example item:
```json
{
"key": "fontSize",
"name": "Font Size",
"category": "Reader",
"type": "enum",
"value": 1,
"options": ["Small", "Medium", "Large"]
}
```
Types:
| Type | Extra fields |
|------|--------------|
| `toggle` | `value` (`0` or `1`) |
| `enum` | `value`, `options` |
| `value` | `value`, `min`, `max`, `step` |
| `string` | `value` |
The font-family setting includes SD-card font families when they are installed.
### `POST /api/settings`
Applies a partial settings update from a JSON object.
```bash
curl -X POST \
-H "Content-Type: application/json" \
-d '{"fontSize":2,"showHiddenFiles":1}' \
http://crosspoint.local/api/settings
```
Successful response:
```text
Applied 2 setting(s)
```
## Font Management API
### `GET /api/fonts`
Lists installed SD-card font families.
```bash
curl http://crosspoint.local/api/fonts
```
Response:
```json
{
"maxFamilies": 128,
"families": [
{
"name": "Literata",
"sizes": [12, 14, 16, 18],
"files": [
{"name": "Literata_12.cpfont", "size": 123456}
]
}
]
}
```
### `POST /api/fonts/upload`
Uploads one `.cpfont` file into a family folder.
```bash
curl -X POST \
-F "family=Literata" \
-F "file=@Literata_12.cpfont" \
http://crosspoint.local/api/fonts/upload
```
The handler validates the family name, `.cpfont` filename, and `CPFONT` magic
bytes before accepting the file.
Successful response:
```json
{"ok":true}
```
### `POST /api/fonts/delete`
Deletes an installed font family.
```bash
curl -X POST \
-H "Content-Type: application/json" \
-d '{"family":"Literata"}' \
http://crosspoint.local/api/fonts/delete
```
Successful response:
```json
{"ok":true}
```
## OPDS Server API
### `GET /api/opds`
Lists saved OPDS servers. Passwords are never returned.
```bash
curl http://crosspoint.local/api/opds
```
Response:
```json
[
{
"index": 0,
"name": "My Catalog",
"url": "http://calibre.local:8080/opds",
"username": "reader",
"hasPassword": true
}
]
```
### `POST /api/opds`
Adds or updates an OPDS server. Include `index` to update an existing entry.
If `password` is omitted during an update, the existing password is preserved.
```bash
curl -X POST \
-H "Content-Type: application/json" \
-d '{"name":"My Catalog","url":"http://calibre.local:8080/opds","username":"reader","password":"secret"}' \
http://crosspoint.local/api/opds
```
### `POST /api/opds/delete`
Deletes an OPDS server by index.
```bash
curl -X POST \
-H "Content-Type: application/json" \
-d '{"index":0}' \
http://crosspoint.local/api/opds/delete
```
## Wi-Fi Credential API
### `GET /api/wifi`
Lists saved Wi-Fi networks. Passwords are never returned.
```bash
curl http://crosspoint.local/api/wifi
```
Response:
```json
[
{
"index": 0,
"ssid": "HomeWiFi",
"hasPassword": true,
"isLastConnected": true
}
]
```
### `POST /api/wifi`
Adds or updates a saved Wi-Fi network. Include `index` to update an existing
entry. If `password` is omitted during an update, the existing password is
preserved.
```bash
curl -X POST \
-H "Content-Type: application/json" \
-d '{"ssid":"HomeWiFi","password":"secret"}' \
http://crosspoint.local/api/wifi
```
### `POST /api/wifi/delete`
Deletes a saved Wi-Fi network by index.
```bash
curl -X POST \
-H "Content-Type: application/json" \
-d '{"index":0}' \
http://crosspoint.local/api/wifi/delete
```
## WebSocket Upload
### Port 81
The WebSocket path is used for fast binary uploads from the file manager and
Calibre plugin workflows.
Connection:
```text
ws://crosspoint.local:81/
```
**Protocol:**
Protocol:
1. **Client** sends TEXT message: `START:<filename>:<size>:<path>`
2. **Server** responds with TEXT: `READY`
3. **Client** sends BINARY messages with file data chunks
4. **Server** sends TEXT progress updates: `PROGRESS:<received>:<total>`
5. **Server** sends TEXT when complete: `DONE` or `ERROR:<message>`
1. Client sends text: `START:<filename>:<size>:<path>`
2. Server replies `READY`
3. Client sends binary chunks
4. Server sends `PROGRESS:<received>:<total>` every 64 KB or at completion
5. Server sends `DONE` when complete or `ERROR:<message>` on failure
**Example Session:**
Example session:
```
Client -> "START:mybook.epub:1234567:/Books"
Server -> "READY"
Client -> [binary chunk 1]
Client -> [binary chunk 2]
Server -> "PROGRESS:65536:1234567"
Client -> [binary chunk 3]
```text
Client -> START:mybook.epub:1234567:/Books
Server -> READY
Client -> [binary chunk]
Server -> PROGRESS:65536:1234567
...
Server -> "PROGRESS:1234567:1234567"
Server -> "DONE"
Server -> DONE
```
**Error Messages:**
Error messages include:
| Message | Cause |
| --------------------------------- | ---------------------------------- |
| `ERROR:Failed to create file` | Cannot create file on SD card |
| `ERROR:Invalid START format` | Malformed START message |
| `ERROR:No upload in progress` | Binary data received without START |
| `ERROR:Write failed - disk full?` | SD card write error |
| Message | Cause |
|---------|-------|
| `ERROR:Upload already in progress` | A second upload was started before the first completed |
| `ERROR:Invalid START format` | Malformed START message or invalid size token |
| `ERROR:Failed to create file` | Destination file could not be opened |
| `ERROR:No upload in progress` | Binary data arrived without a matching START |
| `ERROR:Upload overflow` | Client sent more bytes than declared |
| `ERROR:Write failed - disk full?` | SD write failed |
**Example with `websocat`:**
```bash
# Interactive session
websocat ws://crosspoint.local:81
Incomplete WebSocket uploads are deleted on disconnect or error.
# Then type:
START:mybook.epub:1234567:/Books
# Wait for READY, then send binary data
## WebDAV
The same HTTP server registers a WebDAV-compatible handler for file manager clients.
Supported methods:
```text
OPTIONS, GET, HEAD, PUT, DELETE, PROPFIND, MKCOL, MOVE, COPY, LOCK, UNLOCK
```
**Notes:**
- Progress updates are sent every 64KB or at completion
- Disconnection during upload will delete the incomplete file
- Existing files with the same name will be overwritten
Notes:
---
- `PUT` writes to a temporary `.davtmp` file first, then renames it into place.
- Protected paths are rejected.
- `LOCK` and `UNLOCK` are accepted for client compatibility only. The server
does not implement full WebDAV Class 2 locking semantics such as persistent
locks or lock discovery.
## UDP Discovery
The server listens on UDP port `8134`. When it receives the text payload
`hello`, it replies to the sender with:
```text
crosspoint (on <hostname>);81
```
The final field is the WebSocket upload port.
## Network Modes
The device can operate in two network modes:
### Station Mode (STA)
- Device connects to an existing WiFi network
- IP address assigned by router/DHCP
- `mode` field in `/api/status` returns `"STA"`
- `rssi` field shows signal strength
- Device joins an existing 2.4 GHz Wi-Fi network.
- `crosspoint.local` is advertised with mDNS when available.
- `/api/status` returns `"mode": "STA"` and RSSI in dBm.
### Access Point Mode (AP)
- Device creates its own WiFi hotspot
- Default IP is typically `192.168.4.1`
- `mode` field in `/api/status` returns `"AP"`
- `rssi` field returns `0`
---
- Device creates an open hotspot named `CrossPoint-Reader`.
- The device shows a Wi-Fi QR code and URL QR code.
- The fallback IP is typically `192.168.4.1`.
- `/api/status` returns `"mode": "AP"` and `"rssi": 0`.
## Notes
### Calibre Wireless
- These examples use `crosspoint.local`. If your network does not support mDNS or the address does not resolve, replace it with the specific **IP Address** displayed on your device screen (e.g., `http://192.168.1.102/`).
- All paths on the SD card start with `/`
- Trailing slashes are automatically stripped (except for root `/`)
- The webserver uses chunked transfer encoding for file listings
Calibre Wireless starts the same web server in STA mode and displays setup
instructions plus WebSocket upload progress on the device screen.
+103 -185
View File
@@ -1,235 +1,153 @@
# Web Server Guide
This guide explains how to connect your CrossPoint Reader to WiFi and use the built-in web server to upload files from your computer or phone.
This guide explains how to use CrossPoint Reader's built-in web server for file
transfer, device settings, Wi-Fi/OPDS management, and SD-card font management.
## Overview
CrossPoint Reader includes a built-in web server that allows you to:
The web server is available while the device is in **File Transfer** or
**Calibre Wireless** mode. It can:
- Upload files wirelessly from any device on the same WiFi network
- Browse and manage files on your device's SD card
- Create folders to organize your library
- Delete files and folders
- Upload, download, rename, move, and delete files on the SD card
- Create folders
- Edit many device settings from a browser
- Manage saved Wi-Fi networks and OPDS servers
- Upload and delete `.cpfont` SD-card font families
- Accept WebDAV clients and Calibre wireless uploads
## Prerequisites
The server does not require authentication. Use it only on trusted private
networks or in hotspot mode when you control who is connected.
- Your CrossPoint Reader device
- A WiFi network
- A computer, phone, or tablet connected to the **same WiFi network**
## Starting File Transfer
---
1. From the Home screen, select **File Transfer**.
2. Choose one of the available modes:
## Step 1: Accessing the WiFi Screen
| Mode | Use when |
|------|----------|
| **Join Network** | You want the reader to join an existing Wi-Fi network. |
| **Calibre Wireless** | You want to receive books from the CrossPoint Calibre plugin workflow. |
| **Create Hotspot** | You want the reader to create its own open Wi-Fi network. |
1. From the main menu or file browser, navigate to the **Settings** screen
2. Select the **WiFi** option
3. The device will automatically start scanning for available networks
## Join Network Mode
---
1. Select **Join Network**.
2. If you have saved Wi-Fi credentials, CrossPoint first tries the last
connected network, then other visible saved networks in signal-strength
order. Press **Back** to cancel or **Confirm** to stop auto-connect and show
the network list.
3. If the network list is shown, pick a 2.4 GHz Wi-Fi network from the scan
results.
4. Enter the password if prompted.
5. Save credentials if you want the reader to reconnect automatically next time.
## Step 2: Connecting to WiFi
After connection, the reader shows:
### Viewing Available Networks
- The connected SSID
- A QR code for the web URL
- The direct IP URL, for example `http://192.168.1.102/`
- The mDNS fallback URL, usually `http://crosspoint.local/`
Once the scan completes, you'll see a list of available WiFi networks with the following indicators:
Use either URL from a phone, tablet, or computer on the same network.
- **Signal strength bars** (`||||`, `|||`, `||`, `|`) - Shows connection quality
- **`*` symbol** - Indicates the network is password-protected (encrypted)
- **`+` symbol** - Indicates you have previously saved credentials for this network
## Create Hotspot Mode
<img src="./images/wifi/wifi_networks.jpeg" height="500">
1. Select **Create Hotspot**.
2. Connect your phone or computer to the open Wi-Fi network:
### Selecting a Network
```text
CrossPoint-Reader
```
1. Use the **Left/Right** (or **Volume Up/Down**) buttons to navigate through the network list
2. Press **Confirm** to select the highlighted network
3. Open the URL shown on the reader. `http://crosspoint.local/` is preferred
when supported; the fallback IP is typically `http://192.168.4.1/`.
### Entering Password (for encrypted networks)
The reader displays one QR code for joining the hotspot and another QR code for
opening the web interface.
If the network requires a password:
## Calibre Wireless Mode
1. An on-screen keyboard will appear
2. Use the navigation buttons to select characters
3. Press **Confirm** to enter each character
4. When complete, select the **Done** option on the keyboard
Calibre Wireless starts the same web server in station mode, then displays setup
instructions and upload progress on the reader. Use this mode with the
CrossPoint Calibre plugin or other clients that speak the documented WebSocket
upload protocol.
<img src="./images/wifi/wifi_password.jpeg" height="500">
For Calibre OPDS browsing, add `/opds` to the catalog URL when configuring an
OPDS server.
**Note:** If you've previously connected to this network, the saved password will be used automatically.
## Web Interface
### Connection Process
The browser UI has four primary pages.
The device will display "Connecting..." while establishing the connection. This typically takes 5-10 seconds.
### Home
### Saving Credentials
If this is a new network, you'll be prompted to save the password:
- Select **Yes** to save credentials for automatic connection next time (NOTE: These are stored in plaintext on the device's SD card. Do not use this for sensitive networks.)
- Select **No** to connect without saving
---
## Step 3: Connection Success
Once connected, the screen will display:
- **Network name** (SSID)
- **IP Address** (e.g., `192.168.1.102`)
- **Web server URL** (e.g., `http://192.168.1.102/`)
<img src="./images/wifi/wifi_connected.jpeg" height="500">
**Important:** Make note of the IP address - you'll need this to access the web interface from your computer or phone.
---
## Step 4: Accessing the Web Interface
### From a Computer
1. Ensure your computer is connected to the **same WiFi network** as your CrossPoint Reader
2. Open any web browser (Chrome is recommended)
3. Type the IP address shown on your device into the browser's address bar
- Example: `http://192.168.1.102/`
4. Press Enter
### From a Phone or Tablet
1. Ensure your phone/tablet is connected to the **same WiFi network** as your CrossPoint Reader
2. Open your mobile browser (Safari, Chrome, etc.)
3. Type the IP address into the address bar
- Example: `http://192.168.1.102/`
4. Tap Go
---
## Step 5: Using the Web Interface
### Home Page
The home page displays:
- Device status and version information
- WiFi connection status
- Current IP address
- Available memory
Navigation links:
- **Home** - Returns to the status page
- **File Manager** - Access file management features
<img src="./images/wifi/webserver_homepage.png" width="600">
The Home page shows firmware status, network mode, IP address, device type,
uptime, and free heap.
### File Manager
Click **File Manager** to access file management features.
The File Manager page can:
#### Browsing Files
- Browse SD-card folders
- Upload files, using WebSocket upload when available and HTTP upload as a fallback
- Create folders
- Download files
- Rename files
- Move files into existing folders
- Delete one or more selected files or empty folders
- The file manager displays all files and folders on your SD card
- **Folders** are highlighted in yellow and indicated with a 📁 icon
- **EPUB Files** are highlighted in green and indicated with a 📗 icon
- **All Other Files** are not highlighted and indicated with a 📄 icon
- Click on a folder name to navigate into it
- Use the breadcrumb navigation at the top to go back to parent folders
Existing files with the same name are overwritten by uploads. When EPUB files
are overwritten, moved, renamed, or deleted through the web server, the matching
book cache is cleared so stale metadata is not reused.
<img src="./images/wifi/webserver_files.png" width="600">
### Settings
#### Uploading Files
The Settings page exposes many firmware settings in the browser. It also has
cards for:
1. Click the **📤 Upload** button in the top-right corner
2. Click **Choose File** and select a file from your device
3. Click **Upload**
4. A progress bar will show the upload status
5. The page will automatically refresh when the upload is complete
- Saved Wi-Fi networks
- OPDS servers
<img src="./images/wifi/webserver_upload.png" width="600">
Passwords are accepted when adding or editing entries, but saved passwords are
not returned by the API.
#### Creating Folders
### Fonts
1. Click the **📁 New Folder** button in the top-right corner
2. Enter a folder name (must not contain characters \" * : < > ? / \\ | and must not be . or ..)
3. Click **Create Folder**
The Fonts page lists installed SD-card font families and lets you upload
`.cpfont` files. Upload files from one font family at a time. The server validates
the font family name, filename, and `.cpfont` magic bytes before accepting the
upload.
This is useful for organizing your library by genre, author, series or file type.
Installed fonts appear in **Settings > Reader > Font Family** after the font
registry refreshes.
#### Deleting Files and Folders
## Command Line Use
1. Click the **🗑️** (trash) icon next to any file or folder
2. Confirm the deletion in the popup dialog
3. Click **Delete** to permanently remove the item
Power users can use `curl`, WebDAV clients, or WebSocket clients while the web
server is running.
**Warning:** Deletion is permanent and cannot be undone!
**Note:** Folders must be empty before they can be deleted.
#### Moving Files
1. Click the **📂** (folder) icon next to any file
2. Enter a folder name or select one from the dropdown
3. Click **Move** to relocate the file
**Note:** Typing in a nonexistent folder name will result in the following error: "Failed to move: Destination not found"
#### Renaming Files
1. Click the **✏️** (pencil) icon next to any file
2. Enter a file name (must not contain characters \" * : < > ? / \\ | and must not be . or ..)
3. Click **Rename** to permanently rename the file
---
## Command Line File Management
For power users, you can manage files directly from your terminal using `curl` while the device is in File Upload mode. Detailed documentation can be found [here](./webserver-endpoints.md).
Endpoint details are documented in [webserver-endpoints.md](./webserver-endpoints.md).
## Security Notes
- The web server runs on port 80 (standard HTTP)
- **No authentication is required** - anyone on the same network can access the interface
- The web server is only accessible while the WiFi screen shows "Connected"
- The web server automatically stops when you exit the WiFi screen
- For security, only use on trusted private networks
- The HTTP server runs on port 80.
- The WebSocket upload server runs on port 81.
- There is no authentication.
- Anyone on the same network can access the web interface while it is running.
- The server stops when you exit File Transfer or Calibre Wireless mode.
- Hotspot mode creates an open network for connectivity fallback; disconnect when done.
---
## Tips
## Technical Details
- **Supported WiFi:** 2.4GHz networks (802.11 b/g/n)
- **Web Server Port:** 80 (HTTP)
- **Maximum Upload Size:** Limited by available SD card space
- **Browser Compatibility:** All modern browsers (Chrome, Firefox, Safari, Edge)
---
## Tips and Best Practices
1. **Organize with folders** - Create folders before uploading to keep your library organized
2. **Check signal strength** - Stronger signals (`|||` or `||||`) provide faster, more reliable uploads
3. **Upload multiple files** - You can select and upload multiple files at once; the manager will queue them and refresh when the batch is finished
4. **Use descriptive names** - Name your folders clearly (e.g., "SciFi", "Mystery", "Non-Fiction")
5. **Keep credentials saved** - Save your WiFi password for quick reconnection in the future
6. **Exit when done** - Press **Back** to exit the WiFi screen and save battery
---
## Exiting WiFi Mode
When you're finished uploading files:
1. Press the **Back** button on your CrossPoint Reader
2. The web server will automatically stop
3. WiFi will disconnect to conserve battery
4. You'll return to the previous screen
Your uploaded files will be immediately available in the file browser!
---
1. Use **Create Hotspot** when no trusted network is available.
2. Prefer `crosspoint.local` when available, but keep the displayed IP address as a fallback.
3. Move closer to the router if upload progress stalls in Join Network mode.
4. Upload custom fonts through the Fonts page or copy them to `/.fonts/` or `/fonts/` on the SD card.
5. Exit File Transfer mode when finished to conserve battery.
## Related Documentation
- [User Guide](../USER_GUIDE.md) - General device operation
- [Troubleshooting](./troubleshooting.md) - Troubleshooting
- [README](../README.md) - Project overview and features
- [User Guide](../USER_GUIDE.md)
- [Webserver Endpoints](./webserver-endpoints.md)
- [SD Card Fonts](./sd-card-fonts.md)
- [Troubleshooting](./troubleshooting.md)
Submodule
+1
Submodule freeink-sdk added at 566fce3d4f
+8 -4
View File
@@ -44,16 +44,17 @@ void EpdFont::getTextBounds(const char* string, const int startX, const int star
continue;
}
const int raiseBy = isCombining ? combiningMark::raiseAboveBase(glyph->top, glyph->height, lastBaseTop) : 0;
const combiningMark::Anchor anchor = combiningMark::anchorFor(cp);
const int raiseBy = isCombining ? combiningMark::raiseAboveBase(anchor, glyph->top, glyph->height, lastBaseTop) : 0;
if (!isCombining && prevCp != 0) {
const auto kernFP = getKerning(prevCp, cp); // 4.4 fixed-point kern
lastBaseX += fp4::toPixel(prevAdvanceFP + kernFP);
}
const int glyphBaseX =
isCombining ? combiningMark::centerOver(lastBaseX, lastBaseLeft, lastBaseWidth, glyph->left, glyph->width)
: lastBaseX;
const int glyphBaseX = isCombining ? combiningMark::anchorOver(anchor, lastBaseX, lastBaseLeft, lastBaseWidth,
glyph->left, glyph->width)
: lastBaseX;
const int glyphBaseY = startY - raiseBy;
*minX = std::min(*minX, glyphBaseX + glyph->left);
@@ -101,6 +102,9 @@ static uint8_t lookupKernClass(const EpdKernClassEntry* entries, const uint16_t
}
int8_t EpdFont::getKerning(const uint32_t leftCp, const uint32_t rightCp) const {
if (utf8IsCjkBreakable(leftCp) || utf8IsCjkBreakable(rightCp)) {
return 0;
}
if (!data->kernMatrix) {
return 0;
}
+76 -11
View File
@@ -36,24 +36,71 @@ namespace combiningMark {
constexpr int MIN_GAP_PX = 1;
/// Compute the cursor-X at which to render a combining mark so its bitmap
/// is visually centered over the base glyph's bitmap.
constexpr int centerOver(int baseCursorPos, int baseLeft, int baseWidth, int markLeft, int markWidth) {
return baseCursorPos + baseLeft + baseWidth / 2 - markWidth / 2 - markLeft;
/// Placement of a mark relative to its base glyph. The default heuristic —
/// centered over the base, raised clear of its top — suits Latin diacritics
/// and Arabic harakat, but misplaces the Hebrew niqqud whose identity depends
/// on position: dagesh sits inside the letter body, the shin/sin dots
/// distinguish the letter by sitting over its right/left arm, and holam hangs
/// over the left corner. "Native" anchors keep the glyph's font-designed
/// height (which may overlap the base) instead of raising it.
enum class Anchor : uint8_t {
CenterRaised, ///< centered over the base, lifted above its top (default)
CenterNative, ///< centered over the base at font-native height
RightNative, ///< right edges aligned, font-native height
LeftNative, ///< left edges aligned, font-native height
};
constexpr Anchor anchorFor(const uint32_t cp) {
switch (cp) {
case 0x05BC: // dagesh / mapiq / shuruk dot: inside the letter body
case 0x05BA: // holam haser for vav: straight above the vav stem
return Anchor::CenterNative;
case 0x05C1: // shin dot: over the letter's right arm
return Anchor::RightNative;
case 0x05B9: // holam: above the letter's left corner
case 0x05C2: // sin dot: over the letter's left arm
return Anchor::LeftNative;
default:
return Anchor::CenterRaised;
}
}
/// Rotated-90CW variant of centerOver. In the rotated coordinate system
/// Horizontal offset from the base bitmap's left edge to the mark bitmap's
/// left edge for a given anchor.
constexpr int anchorShift(const Anchor anchor, const int baseWidth, const int markWidth) {
switch (anchor) {
case Anchor::LeftNative:
return 0;
case Anchor::RightNative:
return baseWidth - markWidth;
default:
return baseWidth / 2 - markWidth / 2;
}
}
/// Compute the cursor-X at which to render a combining mark so its bitmap
/// lands at its anchor position over the base glyph's bitmap.
constexpr int anchorOver(const Anchor anchor, const int baseCursorPos, const int baseLeft, const int baseWidth,
const int markLeft, const int markWidth) {
return baseCursorPos + baseLeft + anchorShift(anchor, baseWidth, markWidth) - markLeft;
}
/// Rotated-90CW variant of anchorOver. In the rotated coordinate system
/// renderCharImpl uses (cursorY - left) instead of (cursorX + left), so
/// every left/width term inverts sign.
constexpr int centerOverRotated90CW(int baseCursorPos, int baseLeft, int baseWidth, int markLeft, int markWidth) {
return baseCursorPos - baseLeft - baseWidth / 2 + markWidth / 2 + markLeft;
constexpr int anchorOverRotated90CW(const Anchor anchor, const int baseCursorPos, const int baseLeft,
const int baseWidth, const int markLeft, const int markWidth) {
return baseCursorPos - baseLeft - anchorShift(anchor, baseWidth, markWidth) + markLeft;
}
/// For combining marks that sit entirely above the baseline, compute how many
/// pixels to raise the mark so there is at least MIN_GAP_PX between its bottom
/// edge and the top of the base glyph. Returns 0 for marks that extend to or
/// below the baseline (e.g. cedilla, dot-below, ogonek).
constexpr int raiseAboveBase(int markTop, int markHeight, int baseTop) {
/// below the baseline (e.g. cedilla, dot-below, ogonek) and for anchors that
/// keep the font-native height (dagesh must stay inside the letter, the
/// shin/sin dots touch its arms).
constexpr int raiseAboveBase(const Anchor anchor, const int markTop, const int markHeight, const int baseTop) {
if (anchor != Anchor::CenterRaised) return 0;
if (markTop - markHeight <= 0) return 0;
const int gap = markTop - markHeight - baseTop;
return (gap < MIN_GAP_PX) ? (MIN_GAP_PX - gap) : 0;
@@ -61,6 +108,20 @@ constexpr int raiseAboveBase(int markTop, int markHeight, int baseTop) {
} // namespace combiningMark
/// GCC/Clang (the ESP32 firmware toolchain) pack structs with __attribute__((packed)).
/// MSVC (host unit tests) has no equivalent attribute and instead needs a #pragma pack
/// region achieving the same 1-byte alignment. These macros keep the on-disk font layout
/// identical across both toolchains.
#if defined(_MSC_VER)
#define EPD_PACKED_BEGIN __pragma(pack(push, 1))
#define EPD_PACKED_END __pragma(pack(pop))
#define EPD_PACKED_ATTR
#else
#define EPD_PACKED_BEGIN
#define EPD_PACKED_END
#define EPD_PACKED_ATTR __attribute__((packed))
#endif
/// Fixed-point conventions used by EpdGlyph and EpdFontData:
/// advanceX: 12.4 unsigned fixed-point in uint16_t (use fp4::toPixel)
/// kernMatrix: 4.4 signed fixed-point in int8_t (use fp4::toPixel)
@@ -95,17 +156,21 @@ typedef struct {
/// Maps a codepoint to a kerning class ID, sorted by codepoint for binary search.
/// Class IDs are 1-based; codepoints not in the table have implicit class 0 (no kerning).
EPD_PACKED_BEGIN
typedef struct {
uint16_t codepoint; ///< Unicode codepoint
uint8_t classId; ///< 1-based kerning class ID
} __attribute__((packed)) EpdKernClassEntry;
} EPD_PACKED_ATTR EpdKernClassEntry;
EPD_PACKED_END
/// Ligature substitution for a specific glyph pair, sorted by `pair` for binary search.
/// `pair` encodes (leftCodepoint << 16 | rightCodepoint) for single-key lookup.
EPD_PACKED_BEGIN
typedef struct {
uint32_t pair; ///< Packed codepoint pair (left << 16 | right)
uint32_t ligatureCp; ///< Codepoint of the replacement ligature glyph
} __attribute__((packed)) EpdLigaturePair;
} EPD_PACKED_ATTR EpdLigaturePair;
EPD_PACKED_END
/// Data stored for FONT AS A WHOLE
typedef struct {
+1 -1
View File
@@ -1,7 +1,7 @@
#include "EpdFontFamily.h"
const EpdFont* EpdFontFamily::getFont(const Style style) const {
// Extract font style bits (ignore UNDERLINE bit for font selection)
// Extract font style bits; render-time overlay bits do not affect font selection.
const bool hasBold = (style & BOLD) != 0;
const bool hasItalic = (style & ITALIC) != 0;
+18 -1
View File
@@ -3,7 +3,21 @@
class EpdFontFamily {
public:
enum Style : uint8_t { REGULAR = 0, BOLD = 1, ITALIC = 2, BOLD_ITALIC = 3, UNDERLINE = 4 };
// Bitmask of text style flags carried per-word through layout and serialized in page cache.
// Bits 0-1 select the font variant (BOLD/ITALIC); bits 2-5 are decoration/positioning overlays
// applied at render time without changing the underlying font. getFont() ignores all bits
// above bit 1 so decorations compose freely with bold/italic (e.g. BOLD | UNDERLINE | SUP).
enum Style : uint8_t {
REGULAR = 0,
BOLD = 1,
ITALIC = 2,
BOLD_ITALIC = 3,
UNDERLINE = 4, // drawn as a line below baseline by TextBlock::render()
STRIKETHROUGH = 8, // drawn as a line through midline by TextBlock::render()
SUP = 16, // superscript: glyph scaled 50%, raised ~40% of ascender
SUB = 32, // subscript: glyph scaled 50%, lowered ~25% of ascender
};
static constexpr uint8_t TEXT_DECORATION_MASK = static_cast<uint8_t>(UNDERLINE | STRIKETHROUGH);
explicit EpdFontFamily(const EpdFont* regular, const EpdFont* bold = nullptr, const EpdFont* italic = nullptr,
const EpdFont* boldItalic = nullptr)
@@ -14,6 +28,9 @@ class EpdFontFamily {
const EpdGlyph* getGlyph(uint32_t cp, Style style = REGULAR) const;
int8_t getKerning(uint32_t leftCp, uint32_t rightCp, Style style = REGULAR) const;
uint32_t applyLigatures(uint32_t cp, const char*& text, Style style = REGULAR) const;
static constexpr bool hasTextDecoration(const Style style) {
return (static_cast<uint8_t>(style) & TEXT_DECORATION_MASK) != 0;
}
private:
const EpdFont* regular;
+26 -20
View File
@@ -33,12 +33,24 @@ void FontDecompressor::freePageBuffer() {
}
void FontDecompressor::freeHotGroup() {
hotGroup.clear();
hotGroup.shrink_to_fit();
free(hotGroup);
hotGroup = nullptr;
hotGroupCapacity = 0;
hotGroupFont = nullptr;
hotGroupIndex = UINT16_MAX;
hotGlyphBuf.clear();
hotGlyphBuf.shrink_to_fit();
free(hotGlyphBuf);
hotGlyphBuf = nullptr;
hotGlyphBufCapacity = 0;
}
bool FontDecompressor::ensureCapacity(uint8_t*& buf, uint32_t& capacity, uint32_t needed) {
if (capacity >= needed) return true;
// Grow-only, free-then-malloc: every caller fully rewrites the buffer after a grow, so the
// old contents are dead -- freeing first gives the allocator its best shot on a tight heap.
free(buf);
buf = static_cast<uint8_t*>(malloc(needed)); // owned by FontDecompressor, freed in freeHotGroup()
capacity = buf ? needed : 0;
return buf != nullptr;
}
uint16_t FontDecompressor::getGroupIndex(const EpdFontData* fontData, uint32_t glyphIndex) {
@@ -170,24 +182,20 @@ const uint8_t* FontDecompressor::getBitmap(const EpdFontData* fontData, const Ep
}
// Check if hot group already has this group decompressed — if not, decompress it
if (!(!hotGroup.empty() && hotGroupFont == fontData && hotGroupIndex == groupIndex)) {
if (!(hotGroup != nullptr && hotGroupFont == fontData && hotGroupIndex == groupIndex)) {
stats.cacheMisses++;
const EpdFontGroup& group = fontData->groups[groupIndex];
hotGroup.resize(group.uncompressedSize);
if (hotGroup.empty()) {
// ensureCapacity may free the buffer, so the cached-group identity dies with it either way.
hotGroupFont = nullptr;
hotGroupIndex = UINT16_MAX;
if (!ensureCapacity(hotGroup, hotGroupCapacity, group.uncompressedSize)) {
LOG_ERR("FDC", "Failed to allocate %u bytes for hot group %u", group.uncompressedSize, groupIndex);
hotGroupFont = nullptr;
hotGroupIndex = UINT16_MAX;
stats.getBitmapTimeUs += micros() - tStart;
return nullptr;
}
if (!decompressGroup(fontData, groupIndex, hotGroup.data(), group.uncompressedSize)) {
hotGroup.clear();
hotGroup.shrink_to_fit();
hotGroupFont = nullptr;
hotGroupIndex = UINT16_MAX;
if (!decompressGroup(fontData, groupIndex, hotGroup, group.uncompressedSize)) {
stats.getBitmapTimeUs += micros() - tStart;
return nullptr;
}
@@ -200,18 +208,16 @@ const uint8_t* FontDecompressor::getBitmap(const EpdFontData* fontData, const Ep
}
// Compact just the requested glyph from byte-aligned data into scratch buffer
if (glyph->dataLength > hotGlyphBuf.size()) {
hotGlyphBuf.resize(glyph->dataLength);
}
if (hotGlyphBuf.empty()) {
if (!ensureCapacity(hotGlyphBuf, hotGlyphBufCapacity, glyph->dataLength)) {
LOG_ERR("FDC", "Failed to allocate %u bytes for glyph scratch", (unsigned)glyph->dataLength);
stats.getBitmapTimeUs += micros() - tStart;
return nullptr;
}
uint32_t alignedOff = getAlignedOffset(fontData, groupIndex, glyphIndex);
compactSingleGlyph(&hotGroup[alignedOff], hotGlyphBuf.data(), glyph->width, glyph->height);
compactSingleGlyph(&hotGroup[alignedOff], hotGlyphBuf, glyph->width, glyph->height);
stats.getBitmapTimeUs += micros() - tStart;
return hotGlyphBuf.data();
return hotGlyphBuf;
}
// --- Prewarm: pre-decompress glyph bitmaps for a page of text ---
+12 -5
View File
@@ -2,8 +2,6 @@
#include <InflateReader.h>
#include <vector>
#include "EpdFontData.h"
class FontDecompressor {
@@ -67,13 +65,22 @@ class FontDecompressor {
// Hot group: last decompressed group (byte-aligned) for non-prewarmed fallback path.
// Kept in byte-aligned format; individual glyphs are compacted on demand into hotGlyphBuf.
// Nothrow high-water malloc buffers, NOT std::vector: getBitmap() runs on the render path,
// and under -fno-exceptions a vector resize that hits OOM abort()s the firmware instead of
// failing (field crash: hotGroup.resize() -> std::bad_alloc -> abort with ~11 KB free).
// ensureCapacity() returns false on OOM so the caller can skip the glyph gracefully.
const EpdFontData* hotGroupFont = nullptr;
uint16_t hotGroupIndex = UINT16_MAX;
std::vector<uint8_t> hotGroup;
uint8_t* hotGroup = nullptr; // owned; freed in freeHotGroup()/dtor
uint32_t hotGroupCapacity = 0;
// Scratch buffer for compacting a single glyph from the hot group.
// Valid until the next getBitmap() call.
std::vector<uint8_t> hotGlyphBuf;
// Valid until the next getBitmap() call. Same ownership/OOM contract as hotGroup.
uint8_t* hotGlyphBuf = nullptr;
uint32_t hotGlyphBufCapacity = 0;
// Grow (never shrink) an owned buffer to at least `needed` bytes; false on OOM, buffer freed.
static bool ensureCapacity(uint8_t*& buf, uint32_t& capacity, uint32_t needed);
void freePageBuffer();
void freeHotGroup();
+219 -116
View File
@@ -9,16 +9,20 @@
#include <cstring>
#include <memory>
#include "EpdFontFamily.h"
static_assert(sizeof(EpdGlyph) == 16, "EpdGlyph must be 16 bytes to match .cpfont file layout");
static_assert(sizeof(EpdUnicodeInterval) == 12, "EpdUnicodeInterval must be 12 bytes to match .cpfont file layout");
static_assert(sizeof(EpdKernClassEntry) == 3, "EpdKernClassEntry must be 3 bytes to match .cpfont file layout");
static_assert(sizeof(EpdLigaturePair) == 8, "EpdLigaturePair must be 8 bytes to match .cpfont file layout");
// FNV-1a hash for content-based font ID generation
static constexpr uint32_t FNV_OFFSET = 2166136261u;
static constexpr uint32_t FNV_PRIME = 16777619u;
namespace {
static uint32_t fnv1a(const uint8_t* data, size_t len, uint32_t hash = FNV_OFFSET) {
// FNV-1a hash for content-based font ID generation
constexpr uint32_t FNV_OFFSET = 2166136261u;
constexpr uint32_t FNV_PRIME = 16777619u;
uint32_t fnv1a(const uint8_t* data, size_t len, uint32_t hash = FNV_OFFSET) {
for (size_t i = 0; i < len; i++) {
hash ^= data[i];
hash *= FNV_PRIME;
@@ -27,16 +31,44 @@ static uint32_t fnv1a(const uint8_t* data, size_t len, uint32_t hash = FNV_OFFSE
}
// .cpfont magic bytes
static constexpr char CPFONT_MAGIC[8] = {'C', 'P', 'F', 'O', 'N', 'T', '\0', '\0'};
constexpr char CPFONT_MAGIC[8] = {'C', 'P', 'F', 'O', 'N', 'T', '\0', '\0'};
// CPFONT_VERSION is defined as a #define in SdCardFont.h so it can be
// stringified into FONT_MANIFEST_URL.
static constexpr uint32_t HEADER_SIZE = 32;
static constexpr uint32_t STYLE_TOC_ENTRY_SIZE = 32;
constexpr uint32_t HEADER_SIZE = 32;
constexpr uint32_t STYLE_TOC_ENTRY_SIZE = 32;
// Helper to read little-endian values from byte buffer
static inline uint16_t readU16(const uint8_t* p) { return p[0] | (p[1] << 8); }
static inline int16_t readI16(const uint8_t* p) { return static_cast<int16_t>(p[0] | (p[1] << 8)); }
static inline uint32_t readU32(const uint8_t* p) { return p[0] | (p[1] << 8) | (p[2] << 16) | (p[3] << 24); }
inline uint16_t readU16(const uint8_t* p) { return p[0] | (p[1] << 8); }
inline int16_t readI16(const uint8_t* p) { return static_cast<int16_t>(p[0] | (p[1] << 8)); }
inline uint32_t readU32(const uint8_t* p) { return p[0] | (p[1] << 8) | (p[2] << 16) | (p[3] << 24); }
// Walks a null-terminated UTF-8 string and appends each unique codepoint to
// codepoints[0..cpCount-1] via O(n²) dedup. Returns true if the buffer
// reached maxCount (cap hit), false if all codepoints fit.
bool collectUniqueCodepoints(const char* text, uint32_t* codepoints, uint32_t& cpCount, uint32_t maxCount) {
const unsigned char* p = reinterpret_cast<const unsigned char*>(text);
while (*p) {
uint32_t cp = utf8NextCodepoint(&p);
if (cp == 0) break;
bool found = false;
for (uint32_t i = 0; i < cpCount; i++) {
if (codepoints[i] == cp) {
found = true;
break;
}
}
if (!found) {
if (cpCount >= maxCount) return true;
codepoints[cpCount++] = cp;
}
}
return false;
}
const char* asCStr(const std::string& s) { return s.c_str(); }
const char* asCStr(const char* s) { return s; }
} // namespace
SdCardFont::~SdCardFont() { freeAll(); }
@@ -83,6 +115,9 @@ void SdCardFont::freeStyleAll(PerStyle& s) {
freeStyleMiniData(s);
delete[] s.fullIntervals;
s.fullIntervals = nullptr;
delete[] s.bmpIntervals;
s.bmpIntervals = nullptr;
s.intervalsAreBmp16 = false;
freeStyleKernLigatureData(s);
s.present = false;
}
@@ -136,7 +171,7 @@ bool SdCardFont::loadStyleKernLigatureData(PerStyle& s) {
return true;
}
FsFile file;
HalFile file;
if (!Storage.openFileForRead("SDCF", filePath_, file)) {
LOG_ERR("SDCF", "Failed to open .cpfont for kern/lig: %s", filePath_);
return false;
@@ -313,7 +348,7 @@ bool SdCardFont::buildMiniKernMatrix(PerStyle& s, const uint32_t* codepoints, ui
// Step 6: read the full matrix's rows for each used left class, keep only
// columns for used right classes. One SD seek + one read per used left class;
// a row is kernRightClassCount bytes (~200 for Literata).
FsFile file;
HalFile file;
if (!Storage.openFileForRead("SDCF", filePath_, file)) {
LOG_ERR("SDCF", "Failed to open .cpfont for mini kern: %s", filePath_);
freeStyleMiniKern(s);
@@ -393,7 +428,7 @@ bool SdCardFont::load(const char* path) {
strncpy(filePath_, path, sizeof(filePath_) - 1);
filePath_[sizeof(filePath_) - 1] = '\0';
FsFile file;
HalFile file;
if (!Storage.openFileForRead("SDCF", path, file)) {
LOG_ERR("SDCF", "Failed to open .cpfont: %s", path);
return false;
@@ -484,59 +519,94 @@ bool SdCardFont::load(const char* path) {
styleCount_ = styleCount;
contentHash_ = hash;
// Load full intervals into RAM for each present style
// Load full intervals into RAM for each present style. BMP-only fonts with
// fewer than 65536 glyphs use a compact 6-byte interval table instead of the
// on-disk 12-byte table; large sparse CJK subsets otherwise keep tens of KB
// of always-resident heap just for lookup metadata.
for (uint8_t i = 0; i < MAX_STYLES; i++) {
auto& s = styles_[i];
if (!s.present) continue;
s.fullIntervals = new (std::nothrow) EpdUnicodeInterval[s.header.intervalCount];
if (!s.fullIntervals) {
LOG_ERR("SDCF", "Failed to allocate %u intervals for style %u", s.header.intervalCount, i);
freeAll();
return false;
}
if (!file.seekSet(s.intervalsFileOffset)) {
LOG_ERR("SDCF", "Failed to seek to intervals for style %u", i);
freeAll();
return false;
}
size_t intervalsBytes = s.header.intervalCount * sizeof(EpdUnicodeInterval);
if (file.read(reinterpret_cast<uint8_t*>(s.fullIntervals), intervalsBytes) != static_cast<int>(intervalsBytes)) {
LOG_ERR("SDCF", "Failed to read intervals for style %u", i);
freeAll();
return false;
}
// Validate interval contents before any later code (findGlobalGlyphIndex,
// glyph reads) trusts them. A malformed file could otherwise drive
// out-of-range glyph indices into bogus on-disk reads.
{
uint32_t expectedOffset = 0;
uint32_t prevLast = 0;
bool canUseBmp16 = s.header.glyphCount <= UINT16_MAX;
uint32_t expectedOffset = 0;
uint32_t prevLast = 0;
EpdUnicodeInterval iv{};
for (uint32_t j = 0; j < s.header.intervalCount; ++j) {
if (file.read(reinterpret_cast<uint8_t*>(&iv), sizeof(iv)) != sizeof(iv)) {
LOG_ERR("SDCF", "Failed to read interval %u for style %u", j, i);
freeAll();
return false;
}
if (iv.first > iv.last) {
LOG_ERR("SDCF", "Style %u: invalid interval %u (first 0x%lX > last 0x%lX)", i, j,
static_cast<unsigned long>(iv.first), static_cast<unsigned long>(iv.last));
file.close();
freeAll();
return false;
}
const uint32_t span = iv.last - iv.first + 1;
const bool overlapsPrev = (j > 0 && iv.first <= prevLast);
const bool spanTooBig = (span > s.header.glyphCount);
const bool offsetMismatch = (iv.offset != expectedOffset);
const bool offsetOverruns = (iv.offset > s.header.glyphCount - span);
if (overlapsPrev || spanTooBig || offsetMismatch || offsetOverruns) {
LOG_ERR("SDCF", "Style %u: invalid interval layout at %u (overlap=%d span=%u offMis=%d offOver=%d)", i, j,
overlapsPrev, span, offsetMismatch, offsetOverruns);
file.close();
freeAll();
return false;
}
if (iv.first > UINT16_MAX || iv.last > UINT16_MAX || iv.offset > UINT16_MAX) {
canUseBmp16 = false;
}
expectedOffset += span;
prevLast = iv.last;
}
if (!file.seekSet(s.intervalsFileOffset)) {
LOG_ERR("SDCF", "Failed to seek back to intervals for style %u", i);
freeAll();
return false;
}
if (canUseBmp16) {
s.bmpIntervals = new (std::nothrow) PerStyle::BmpInterval16[s.header.intervalCount];
if (!s.bmpIntervals) {
LOG_ERR("SDCF", "Failed to allocate compact intervals for style %u", i);
freeAll();
return false;
}
for (uint32_t j = 0; j < s.header.intervalCount; ++j) {
const auto& iv = s.fullIntervals[j];
if (iv.first > iv.last) {
LOG_ERR("SDCF", "Style %u: invalid interval %u (first 0x%lX > last 0x%lX)", i, j,
static_cast<unsigned long>(iv.first), static_cast<unsigned long>(iv.last));
file.close();
if (file.read(reinterpret_cast<uint8_t*>(&iv), sizeof(iv)) != sizeof(iv)) {
LOG_ERR("SDCF", "Failed to read compact interval %u for style %u", j, i);
freeAll();
return false;
}
const uint32_t span = iv.last - iv.first + 1;
const bool overlapsPrev = (j > 0 && iv.first <= prevLast);
const bool spanTooBig = (span > s.header.glyphCount);
const bool offsetMismatch = (iv.offset != expectedOffset);
const bool offsetOverruns = (iv.offset > s.header.glyphCount - span);
if (overlapsPrev || spanTooBig || offsetMismatch || offsetOverruns) {
LOG_ERR("SDCF", "Style %u: invalid interval layout at %u (overlap=%d span=%u offMis=%d offOver=%d)", i, j,
overlapsPrev, span, offsetMismatch, offsetOverruns);
file.close();
freeAll();
return false;
}
expectedOffset += span;
prevLast = iv.last;
s.bmpIntervals[j] = {static_cast<uint16_t>(iv.first), static_cast<uint16_t>(iv.last),
static_cast<uint16_t>(iv.offset)};
}
s.intervalsAreBmp16 = true;
} else {
s.fullIntervals = new (std::nothrow) EpdUnicodeInterval[s.header.intervalCount];
if (!s.fullIntervals) {
LOG_ERR("SDCF", "Failed to allocate %u intervals for style %u", s.header.intervalCount, i);
freeAll();
return false;
}
size_t intervalsBytes = s.header.intervalCount * sizeof(EpdUnicodeInterval);
if (file.read(reinterpret_cast<uint8_t*>(s.fullIntervals), intervalsBytes) != static_cast<int>(intervalsBytes)) {
LOG_ERR("SDCF", "Failed to read intervals for style %u", i);
freeAll();
return false;
}
}
@@ -571,13 +641,15 @@ int32_t SdCardFont::findGlobalGlyphIndex(const PerStyle& s, uint32_t codepoint)
int right = static_cast<int>(s.header.intervalCount) - 1;
while (left <= right) {
int mid = left + (right - left) / 2;
const auto& interval = s.fullIntervals[mid];
if (codepoint < interval.first) {
const uint32_t first = s.intervalsAreBmp16 ? s.bmpIntervals[mid].first : s.fullIntervals[mid].first;
const uint32_t last = s.intervalsAreBmp16 ? s.bmpIntervals[mid].last : s.fullIntervals[mid].last;
if (codepoint < first) {
right = mid - 1;
} else if (codepoint > interval.last) {
} else if (codepoint > last) {
left = mid + 1;
} else {
return static_cast<int32_t>(interval.offset + (codepoint - interval.first));
const uint32_t offset = s.intervalsAreBmp16 ? s.bmpIntervals[mid].offset : s.fullIntervals[mid].offset;
return static_cast<int32_t>(offset + (codepoint - first));
}
}
return -1;
@@ -587,6 +659,8 @@ int32_t SdCardFont::findGlobalGlyphIndex(const PerStyle& s, uint32_t codepoint)
int SdCardFont::prewarm(const char* utf8Text, uint8_t styleMask, bool metadataOnly) {
if (!loaded_) return -1;
styleMask = resolveStyleMask(styleMask);
if (styleMask == 0) return 0;
unsigned long startMs = millis();
@@ -764,7 +838,7 @@ int SdCardFont::prewarmStyle(uint8_t styleIdx, const uint32_t* codepoints, uint3
std::sort(readOrder, readOrder + validCount,
[&](uint32_t a, uint32_t b) { return mappings[a].globalIndex < mappings[b].globalIndex; });
FsFile file;
HalFile file;
if (!Storage.openFileForRead("SDCF", filePath_, file)) {
LOG_ERR("SDCF", "Failed to reopen .cpfont for prewarm (style %u)", styleIdx);
delete[] readOrder;
@@ -1015,62 +1089,10 @@ uint16_t SdCardFont::getAdvance(uint32_t codepoint, uint8_t style) const {
return 0;
}
int SdCardFont::buildAdvanceTable(const char* utf8Text, uint8_t styleMask) {
if (!loaded_) return -1;
// Note: advance table is preserved across calls. We only fetch codepoints
// not already present, then merge them in. Use clearPersistentCache() to
// wipe the table when the font/size/family changes.
unsigned long startMs = millis();
// Step 1: Extract unique codepoints, capped at MAX_UNIQUE_CODEPOINTS.
// The dedup buffer is sized to the cap, not total chars — a large EPUB section
// may contain 50K+ characters but real text has far fewer unique codepoints.
// 4096 × 4 bytes = 16KB temporary; bounded regardless of input size.
static constexpr uint32_t MAX_UNIQUE_CODEPOINTS = 4096;
uint32_t* codepoints = new (std::nothrow) uint32_t[MAX_UNIQUE_CODEPOINTS];
if (!codepoints) {
LOG_ERR("SDCF", "buildAdvanceTable: failed to allocate codepoint buffer (%u bytes)", MAX_UNIQUE_CODEPOINTS * 4);
return -1;
}
uint32_t cpCount = 0;
bool hitCap = false;
// Second pass: collect unique codepoints via O(n²) dedup.
// Bounded by uniqueCount × totalChars comparisons. For 2000 unique from 2291 total,
// worst case ~4.6M comparisons of uint32_t — ~30ms on 160MHz RISC-V, acceptable
// for one-time section indexing.
const unsigned char* p = reinterpret_cast<const unsigned char*>(utf8Text);
while (*p) {
uint32_t cp = utf8NextCodepoint(&p);
if (cp == 0) break;
bool found = false;
for (uint32_t i = 0; i < cpCount; i++) {
if (codepoints[i] == cp) {
found = true;
break;
}
}
if (!found) {
if (cpCount >= MAX_UNIQUE_CODEPOINTS) {
hitCap = true;
break;
}
codepoints[cpCount++] = cp;
}
}
if (hitCap) {
LOG_ERR("SDCF", "buildAdvanceTable: unique codepoint cap (%u) hit, layout may be approximate",
MAX_UNIQUE_CODEPOINTS);
}
// Sort for ordered glyph index mapping and final table output
std::sort(codepoints, codepoints + cpCount);
// Step 2: For each requested style, fetch any codepoints not yet cached and
// merge them into the persistent advance table.
// Given a sorted array of unique codepoints, resolve glyph indices per style,
// batch-read advanceX from SD, and merge into the persistent advance table.
// Caller owns the codepoints buffer.
int SdCardFont::fetchAdvancesForCodepoints(uint32_t* codepoints, uint32_t cpCount, uint8_t styleMask) {
int totalMissed = 0;
for (uint8_t si = 0; si < MAX_STYLES; si++) {
if (!(styleMask & (1 << si)) || !styles_[si].present) continue;
@@ -1097,13 +1119,17 @@ int SdCardFont::buildAdvanceTable(const char* utf8Text, uint8_t styleMask) {
uint32_t needCount = 0;
uint32_t missedThisStyle = 0;
const int32_t replacementIdx = findGlobalGlyphIndex(s, REPLACEMENT_GLYPH);
for (uint32_t i = 0; i < cpCount; i++) {
const uint32_t cp = codepoints[i];
if (advanceTableLookup(si, cp, nullptr)) continue; // already cached
int32_t idx = findGlobalGlyphIndex(s, cp);
if (idx < 0) {
missedThisStyle++;
continue;
if (replacementIdx < 0) {
missedThisStyle++;
continue;
}
idx = replacementIdx;
}
mappings[needCount].codepoint = cp;
mappings[needCount].glyphIndex = idx;
@@ -1118,7 +1144,7 @@ int SdCardFont::buildAdvanceTable(const char* utf8Text, uint8_t styleMask) {
[](const CpIdx& a, const CpIdx& b) { return a.glyphIndex < b.glyphIndex; });
// Open file once and read advanceX for each needed glyph.
FsFile file;
HalFile file;
if (!Storage.openFileForRead("SDCF", filePath_, file)) {
LOG_ERR("SDCF", "buildAdvanceTable: failed to open .cpfont for style %u", si);
continue;
@@ -1165,12 +1191,60 @@ int SdCardFont::buildAdvanceTable(const char* utf8Text, uint8_t styleMask) {
ADVANCE_CACHE_LIMIT);
}
delete[] codepoints;
return totalMissed;
}
template <typename Iter>
int SdCardFont::buildAdvanceTableRange(Iter begin, Iter end, bool includeSpace, bool includeHyphen, uint8_t styleMask,
const char* extraText) {
if (!loaded_) return -1;
styleMask = resolveStyleMask(styleMask);
if (styleMask == 0) return 0;
unsigned long startMs = millis();
// +2 reserved slots for space and hyphen injected after the main scan.
static constexpr uint32_t MAX_UNIQUE_CODEPOINTS = 4096;
uint32_t* codepoints = new (std::nothrow) uint32_t[MAX_UNIQUE_CODEPOINTS + 2];
if (!codepoints) {
LOG_ERR("SDCF", "buildAdvanceTable: failed to allocate codepoint buffer (%u bytes)", MAX_UNIQUE_CODEPOINTS * 4);
return -1;
}
uint32_t cpCount = 0;
bool hitCap = false;
for (auto it = begin; it != end && !hitCap; ++it) {
hitCap = collectUniqueCodepoints(asCStr(*it), codepoints, cpCount, MAX_UNIQUE_CODEPOINTS);
}
if (extraText && !hitCap) {
hitCap = collectUniqueCodepoints(extraText, codepoints, cpCount, MAX_UNIQUE_CODEPOINTS);
}
if (includeSpace && std::none_of(codepoints, codepoints + cpCount, [](uint32_t c) { return c == ' '; }))
codepoints[cpCount++] = ' ';
if (includeHyphen && std::none_of(codepoints, codepoints + cpCount, [](uint32_t c) { return c == '-'; }))
codepoints[cpCount++] = '-';
if (hitCap) {
LOG_ERR("SDCF", "buildAdvanceTable: unique codepoint cap (%u) hit, layout may be approximate",
MAX_UNIQUE_CODEPOINTS);
}
std::sort(codepoints, codepoints + cpCount);
int totalMissed = fetchAdvancesForCodepoints(codepoints, cpCount, styleMask);
delete[] codepoints;
stats_.prewarmTotalMs = millis() - startMs;
return totalMissed;
}
int SdCardFont::buildAdvanceTable(const char* utf8Text, uint8_t styleMask, const char* extraText) {
return buildAdvanceTableRange(&utf8Text, &utf8Text + 1, false, false, styleMask, extraText);
}
int SdCardFont::buildAdvanceTable(const std::vector<std::string>& words, bool includeHyphen, uint8_t styleMask,
const char* extraText) {
return buildAdvanceTableRange(words.begin(), words.end(), words.size() > 1, includeHyphen, styleMask, extraText);
}
// --- Stats ---
void SdCardFont::logStats(const char* label) {
@@ -1190,6 +1264,35 @@ EpdFont* SdCardFont::getEpdFont(uint8_t style) {
bool SdCardFont::hasStyle(uint8_t style) const { return styles_[style & (MAX_STYLES - 1)].present; }
uint8_t SdCardFont::resolveStyle(uint8_t style) const {
static const uint8_t kFallbacks[MAX_STYLES][MAX_STYLES] = {
// REGULAR: REGULAR -> BOLD -> ITALIC -> BOLD_ITALIC
{EpdFontFamily::REGULAR, EpdFontFamily::BOLD, EpdFontFamily::ITALIC, EpdFontFamily::BOLD_ITALIC},
// BOLD: BOLD -> REGULAR -> BOLD_ITALIC -> ITALIC
{EpdFontFamily::BOLD, EpdFontFamily::REGULAR, EpdFontFamily::BOLD_ITALIC, EpdFontFamily::ITALIC},
// ITALIC: ITALIC -> REGULAR -> BOLD_ITALIC -> BOLD
{EpdFontFamily::ITALIC, EpdFontFamily::REGULAR, EpdFontFamily::BOLD_ITALIC, EpdFontFamily::BOLD},
// BOLD_ITALIC: BOLD_ITALIC -> BOLD -> ITALIC -> REGULAR
{EpdFontFamily::BOLD_ITALIC, EpdFontFamily::BOLD, EpdFontFamily::ITALIC, EpdFontFamily::REGULAR},
};
const uint8_t styleBits = style & (MAX_STYLES - 1);
for (uint8_t candidate : kFallbacks[styleBits]) {
if (styles_[candidate].present) return candidate;
}
return EpdFontFamily::REGULAR;
}
uint8_t SdCardFont::resolveStyleMask(uint8_t styleMask) const {
uint8_t resolvedMask = 0;
for (uint8_t si = 0; si < MAX_STYLES; si++) {
if (styleMask & (1 << si)) {
resolvedMask |= static_cast<uint8_t>(1u << resolveStyle(si));
}
}
return resolvedMask;
}
// --- On-demand glyph loading (overflow buffer) ---
const EpdGlyph* SdCardFont::onGlyphMiss(void* ctx, uint32_t codepoint) {
@@ -1199,7 +1302,7 @@ const EpdGlyph* SdCardFont::onGlyphMiss(void* ctx, uint32_t codepoint) {
if (!self->loaded_ || styleIdx >= MAX_STYLES || !self->styles_[styleIdx].present) return nullptr;
const auto& s = self->styles_[styleIdx];
if (!s.fullIntervals) return nullptr;
if (!s.fullIntervals && !s.bmpIntervals) return nullptr;
// Check overflow cache first (matching both codepoint and style)
for (uint32_t i = 0; i < self->overflowCount_; i++) {
@@ -1219,7 +1322,7 @@ const EpdGlyph* SdCardFont::onGlyphMiss(void* ctx, uint32_t codepoint) {
bool wasAtCapacity = (self->overflowCount_ == OVERFLOW_CAPACITY);
// Read glyph metadata into temporary
FsFile file;
HalFile file;
if (!Storage.openFileForRead("SDCF", self->filePath_, file)) {
LOG_ERR("SDCF", "Overflow: failed to open .cpfont");
return nullptr;
+32 -5
View File
@@ -1,6 +1,8 @@
#pragma once
#include <cstdint>
#include <string>
#include <vector>
#include "EpdFont.h"
#include "EpdFontData.h"
@@ -43,10 +45,14 @@ class SdCardFont {
int prewarm(const char* utf8Text, uint8_t styleMask = 0x0F, bool metadataOnly = false);
// Build a compact advance-only table for layout measurement.
// Extracts ALL unique codepoints from utf8Text (no MAX_PAGE_GLYPHS cap),
// Extracts ALL unique codepoints from words (no MAX_PAGE_GLYPHS cap),
// batch-reads advanceX from SD, stores in a sorted per-style table.
// extraText: optional additional codepoints to warm in the same SD pass
// (e.g. shaped Arabic presentation forms the measurement path will look up).
// Returns number of codepoints not found in font coverage.
int buildAdvanceTable(const char* utf8Text, uint8_t styleMask = 0x0F);
int buildAdvanceTable(const char* utf8Text, uint8_t styleMask = 0x0F, const char* extraText = nullptr);
int buildAdvanceTable(const std::vector<std::string>& words, bool includeHyphen, uint8_t styleMask = 0x0F,
const char* extraText = nullptr);
// Look up advanceX for a codepoint from the advance table.
// Returns the 12.4 fixed-point advance, or 0 if not found.
@@ -55,9 +61,9 @@ class SdCardFont {
// Returns true if advance table is populated for at least one style.
bool hasAdvanceTable() const;
// Free mini data for all styles, restore stub EpdFontData.
// Also clears the temporary advance table (built per layout pass) but
// preserves the persistent advance cache (reused across passes).
// Free mini data for all styles and restore stub EpdFontData.
// Preserves the persistent advance cache so repeated layout passes can reuse
// previously fetched metrics.
void clearCache();
// Drop the persistent advance cache. Call when unloading the SD font or
@@ -71,6 +77,13 @@ class SdCardFont {
// Returns true if the given style is present in this font file.
bool hasStyle(uint8_t style) const;
// Resolve requested style bits to the closest present style.
uint8_t resolveStyle(uint8_t style) const;
// Resolve every requested style bit through fallback and return the actual
// styles that need cache/advance preparation.
uint8_t resolveStyleMask(uint8_t styleMask) const;
// Number of styles present in this font file.
uint8_t styleCount() const { return styleCount_; }
@@ -130,6 +143,16 @@ class SdCardFont {
// Full intervals loaded from file (kept in RAM for codepoint lookup)
EpdUnicodeInterval* fullIntervals = nullptr;
EPD_PACKED_BEGIN
struct BmpInterval16 {
uint16_t first;
uint16_t last;
uint16_t offset;
} EPD_PACKED_ATTR;
EPD_PACKED_END
static_assert(sizeof(BmpInterval16) == 6, "BmpInterval16 must remain compact");
BmpInterval16* bmpIntervals = nullptr;
bool intervalsAreBmp16 = false;
// Persistent kern-class + ligature tables (lazy-loaded on first prewarm).
// The full kern MATRIX is NOT resident — on Literata-class fonts a single
@@ -229,6 +252,10 @@ class SdCardFont {
void applyKernLigaturePointers(PerStyle& s, EpdFontData& data) const;
void applyGlyphMissCallback(uint8_t styleIdx);
int32_t findGlobalGlyphIndex(const PerStyle& s, uint32_t codepoint) const;
int fetchAdvancesForCodepoints(uint32_t* codepoints, uint32_t cpCount, uint8_t styleMask);
template <typename Iter>
int buildAdvanceTableRange(Iter begin, Iter end, bool includeSpace, bool includeHyphen, uint8_t styleMask,
const char* extraText = nullptr);
int prewarmStyle(uint8_t styleIdx, const uint32_t* codepoints, uint32_t cpCount, bool metadataOnly);
// Global helpers
+5 -9
View File
@@ -34,19 +34,15 @@ bool SdCardFontManager::loadFamily(const SdCardFontFamilyInfo& family, GfxRender
unloadAll(renderer);
}
// Select by ordinal position: sort available sizes, then map the font size
// enum (SMALL=0 .. EXTRA_LARGE=3) to the corresponding slot. When the
// family has fewer sizes than 4, clamp to the last available size.
auto sizes = family.availableSizes();
if (sizes.empty()) {
// Select the physical point size closest to the built-in reader sizes. Some
// CJK font packs only ship larger sizes, so ordinal selection can make
// MEDIUM load 18pt+ and produce oversized pages on small devices.
const SdCardFontFileInfo* selected = family.findClosestReaderSize(fontSizeEnum);
if (!selected) {
LOG_ERR("SDMGR", "Family %s has no files to load", family.name.c_str());
return false;
}
uint8_t idx = fontSizeEnum;
if (idx >= sizes.size()) idx = sizes.size() - 1;
const SdCardFontFileInfo* selected = family.findFile(sizes[idx]);
auto* font = new (std::nothrow) SdCardFont();
if (!font) {
LOG_ERR("SDMGR", "Failed to allocate SdCardFont for %s", selected->path.c_str());
+5 -5
View File
@@ -15,10 +15,10 @@ class SdCardFontManager {
SdCardFontManager(const SdCardFontManager&) = delete;
SdCardFontManager& operator=(const SdCardFontManager&) = delete;
// Load the font file matching fontSizeEnum (SMALL=0 .. EXTRA_LARGE=3) by
// ordinal position in the family's sorted size list. Only one .cpfont file
// is loaded; other sizes remain on disk. This keeps resident interval +
// kern/ligature tables to one size's worth of memory.
// Load the font file whose physical point size is closest to the reader
// fontSizeEnum (SMALL=12, MEDIUM=14, LARGE=16, EXTRA_LARGE=18). Only one
// .cpfont file is loaded; other sizes remain on disk. This keeps resident
// interval + kern/ligature tables to one size's worth of memory.
// Returns true on success.
bool loadFamily(const SdCardFontFamilyInfo& family, GfxRenderer& renderer, uint8_t fontSizeEnum);
@@ -32,7 +32,7 @@ class SdCardFontManager {
// Get name of currently loaded family (empty if none).
const std::string& currentFamilyName() const { return loadedFamilyName_; };
// Point size that was actually loaded (closest match to targetPtSize).
// Point size that was actually loaded.
// 0 if nothing loaded.
uint8_t currentPointSize() const { return loadedPointSize_; };
+58 -4
View File
@@ -15,6 +15,60 @@ const SdCardFontFileInfo* SdCardFontFamilyInfo::findFile(uint8_t size, uint8_t s
return nullptr;
}
const SdCardFontFileInfo* SdCardFontFamilyInfo::findClosestReaderSize(const uint8_t fontSizeEnum,
const uint8_t style) const {
if (files.empty()) return nullptr;
// Collect sizes matching the requested style, sorted ascending.
std::vector<uint8_t> sizes;
for (const auto& f : files) {
if (f.style != style) continue;
sizes.push_back(f.pointSize);
}
if (sizes.empty()) return nullptr;
std::sort(sizes.begin(), sizes.end());
// When the family provides at least 4 sizes, use ordinal (index-based)
// selection so custom-built font sets (e.g. 10/12/14/16) map SMALL to
// the smallest file, not to a hardcoded 12pt target.
if (sizes.size() >= 4) {
uint8_t idx = fontSizeEnum;
if (idx >= sizes.size()) idx = sizes.size() - 1;
return findFile(sizes[idx], style);
}
// Fewer sizes than enum slots (e.g. CJK packs with only 2-3 sizes):
// fall back to closest-match against the built-in reader targets.
uint8_t target = 14;
switch (fontSizeEnum) {
case 0:
target = 12;
break;
case 2:
target = 16;
break;
case 3:
target = 18;
break;
case 1:
default:
target = 14;
break;
}
const SdCardFontFileInfo* best = nullptr;
uint8_t bestDelta = 255;
for (const auto& f : files) {
if (f.style != style) continue;
const uint8_t delta = f.pointSize > target ? f.pointSize - target : target - f.pointSize;
if (!best || delta < bestDelta || (delta == bestDelta && f.pointSize < best->pointSize)) {
best = &f;
bestDelta = delta;
}
}
return best;
}
bool SdCardFontFamilyInfo::hasSize(uint8_t size) const {
for (const auto& f : files) {
if (f.pointSize == size) return true;
@@ -77,12 +131,12 @@ bool SdCardFontRegistry::parseFilename(const char* filename, uint8_t& size, uint
}
void SdCardFontRegistry::scanDirectory(const char* dirPath, SdCardFontFamilyInfo& family) {
FsFile dir = Storage.open(dirPath);
HalFile dir = Storage.open(dirPath);
if (!dir || !dir.isDirectory()) return;
char nameBuffer[128];
while (true) {
FsFile entry = dir.openNextFile();
HalFile entry = dir.openNextFile();
if (!entry) break;
if (entry.isDirectory()) {
entry.close();
@@ -126,7 +180,7 @@ void SdCardFontRegistry::scanDirectory(const char* dirPath, SdCardFontFamilyInfo
// Skips families whose names already exist in `out` (de-duplicates between
// the hidden and visible roots — first scan wins).
void SdCardFontRegistry::scanRoot(const char* rootPath, std::vector<SdCardFontFamilyInfo>& out) {
FsFile root = Storage.open(rootPath);
HalFile root = Storage.open(rootPath);
if (!root) {
LOG_DBG("SDREG", "Fonts directory not found: %s", rootPath);
return;
@@ -138,7 +192,7 @@ void SdCardFontRegistry::scanRoot(const char* rootPath, std::vector<SdCardFontFa
char nameBuffer[128];
while (true) {
FsFile entry = root.openNextFile();
HalFile entry = root.openNextFile();
if (!entry) break;
if (entry.isDirectory()) {
entry.getName(nameBuffer, sizeof(nameBuffer));
+1
View File
@@ -18,6 +18,7 @@ struct SdCardFontFamilyInfo {
std::vector<SdCardFontFileInfo> files;
const SdCardFontFileInfo* findFile(uint8_t size, uint8_t style = 0) const;
const SdCardFontFileInfo* findClosestReaderSize(uint8_t fontSizeEnum, uint8_t style = 0) const;
bool hasSize(uint8_t size) const;
std::vector<uint8_t> availableSizes() const;
};
-16
View File
@@ -33,22 +33,6 @@
#include <builtinFonts/notosans_18_bolditalic.h>
#include <builtinFonts/notosans_18_italic.h>
#include <builtinFonts/notosans_18_regular.h>
#include <builtinFonts/opendyslexic_10_bold.h>
#include <builtinFonts/opendyslexic_10_bolditalic.h>
#include <builtinFonts/opendyslexic_10_italic.h>
#include <builtinFonts/opendyslexic_10_regular.h>
#include <builtinFonts/opendyslexic_12_bold.h>
#include <builtinFonts/opendyslexic_12_bolditalic.h>
#include <builtinFonts/opendyslexic_12_italic.h>
#include <builtinFonts/opendyslexic_12_regular.h>
#include <builtinFonts/opendyslexic_14_bold.h>
#include <builtinFonts/opendyslexic_14_bolditalic.h>
#include <builtinFonts/opendyslexic_14_italic.h>
#include <builtinFonts/opendyslexic_14_regular.h>
#include <builtinFonts/opendyslexic_8_bold.h>
#include <builtinFonts/opendyslexic_8_bolditalic.h>
#include <builtinFonts/opendyslexic_8_italic.h>
#include <builtinFonts/opendyslexic_8_regular.h>
#include <builtinFonts/ubuntu_10_bold.h>
#include <builtinFonts/ubuntu_10_regular.h>
#include <builtinFonts/ubuntu_12_bold.h>
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -6,6 +6,10 @@
!NotoSerif/**
!NotoSans/
!NotoSans/**
!NotoSansArabic/
!NotoSansArabic/**
!NotoSansHebrew/
!NotoSansHebrew/**
!OpenDyslexic/
!OpenDyslexic/**
!Ubuntu/
@@ -0,0 +1,93 @@
Copyright 2022 The Noto Project Authors (https://github.com/notofonts/arabic)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
https://openfontlicense.org
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
@@ -0,0 +1,93 @@
Copyright 2022 The Noto Project Authors (https://github.com/notofonts/hebrew)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
https://openfontlicense.org
-----------------------------------------------------------
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
-----------------------------------------------------------
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
-36
View File
@@ -80,42 +80,6 @@ ruby -rdigest -e 'puts [
].map{|f| Digest::SHA256.hexdigest(File.read(f)).to_i(16) }.sum % (2 ** 32) - (2 ** 31)'
))"
echo "#define OPENDYSLEXIC_8_FONT_ID ($(
ruby -rdigest -e 'puts [
"./opendyslexic_8_regular.h",
"./opendyslexic_8_bold.h",
"./opendyslexic_8_bolditalic.h",
"./opendyslexic_8_italic.h",
].map{|f| Digest::SHA256.hexdigest(File.read(f)).to_i(16) }.sum % (2 ** 32) - (2 ** 31)'
))"
echo "#define OPENDYSLEXIC_10_FONT_ID ($(
ruby -rdigest -e 'puts [
"./opendyslexic_10_regular.h",
"./opendyslexic_10_bold.h",
"./opendyslexic_10_bolditalic.h",
"./opendyslexic_10_italic.h",
].map{|f| Digest::SHA256.hexdigest(File.read(f)).to_i(16) }.sum % (2 ** 32) - (2 ** 31)'
))"
echo "#define OPENDYSLEXIC_12_FONT_ID ($(
ruby -rdigest -e 'puts [
"./opendyslexic_12_regular.h",
"./opendyslexic_12_bold.h",
"./opendyslexic_12_bolditalic.h",
"./opendyslexic_12_italic.h",
].map{|f| Digest::SHA256.hexdigest(File.read(f)).to_i(16) }.sum % (2 ** 32) - (2 ** 31)'
))"
echo "#define OPENDYSLEXIC_14_FONT_ID ($(
ruby -rdigest -e 'puts [
"./opendyslexic_14_regular.h",
"./opendyslexic_14_bold.h",
"./opendyslexic_14_bolditalic.h",
"./opendyslexic_14_italic.h",
].map{|f| Digest::SHA256.hexdigest(File.read(f)).to_i(16) }.sum % (2 ** 32) - (2 ** 31)'
))"
echo "#define UI_10_FONT_ID ($(
ruby -rdigest -e 'puts [
"./ubuntu_10_regular.h",
+47 -7
View File
@@ -33,6 +33,7 @@ import sys
import tempfile
import threading
import time
import socket
import urllib.request
from concurrent.futures import ProcessPoolExecutor, as_completed
from pathlib import Path
@@ -46,19 +47,47 @@ DEFAULT_CONFIG = SCRIPT_DIR / "sd-fonts.yaml"
DEFAULT_OUTPUT = SCRIPT_DIR / "output"
DOWNLOAD_DIR = SCRIPT_DIR / "downloaded_fonts"
INSTANCE_DIR = SCRIPT_DIR / "instanced_fonts"
DEFAULT_FALLBACK_FONT = EPDFONTS_DIR / "builtinFonts/source/NotoSans/NotoSans-Regular.ttf"
def download_font(url: str, dest: Path) -> Path:
"""Download a font file if not already cached. Returns the local path."""
_orig_getaddrinfo = socket.getaddrinfo
def _ipv4_only_getaddrinfo(*args, **kwargs):
"""getaddrinfo variant that drops AAAA records (IPv4 only)."""
return [ai for ai in _orig_getaddrinfo(*args, **kwargs) if ai[0] == socket.AF_INET]
def download_font(url: str, dest: Path, retries: int = 3) -> Path:
"""Download a font file if not already cached. Returns the local path.
Some sources (e.g. mirrors.ctan.org) are round-robin redirectors that land
on a different mirror each request; a mirror may advertise an IPv6 address a
host without an IPv6 route cannot reach ([Errno 101] Network is unreachable).
Retry on failure, forcing IPv4 resolution after the first attempt.
"""
if dest.exists():
return dest
dest.parent.mkdir(parents=True, exist_ok=True)
print(f" Downloading {dest.name}...")
try:
urllib.request.urlretrieve(url, dest)
except Exception as e:
dest.unlink(missing_ok=True)
raise RuntimeError(f"Failed to download {url}: {e}") from e
last_err = None
for attempt in range(1, retries + 1):
force_ipv4 = attempt > 1
if force_ipv4:
socket.getaddrinfo = _ipv4_only_getaddrinfo
try:
urllib.request.urlretrieve(url, dest)
break
except Exception as e: # noqa: BLE001 - reported via RuntimeError below
last_err = e
dest.unlink(missing_ok=True)
if attempt < retries:
print(f" Attempt {attempt} failed ({e}); retrying (IPv4-only)...")
finally:
if force_ipv4:
socket.getaddrinfo = _orig_getaddrinfo
else:
raise RuntimeError(f"Failed to download {url}: {last_err}") from last_err
size_kb = dest.stat().st_size / 1024
print(f" Downloaded {dest.name} ({size_kb:.0f} KB)")
return dest
@@ -186,12 +215,14 @@ def build_family(
# Multi-style mode
for style_name, font_path in resolved_styles.items():
cmd.extend([f"--{style_name}", str(font_path)])
cmd.extend([f"--fallback-{style_name}", str(DEFAULT_FALLBACK_FONT)])
else:
# Single-style mode
style_name = next(iter(resolved_styles))
font_path = resolved_styles[style_name]
cmd.append(str(font_path))
cmd.extend(["--style", style_name])
cmd.extend([f"--fallback-{style_name}", str(DEFAULT_FALLBACK_FONT)])
cmd.extend(["--intervals", intervals])
cmd.extend(["--sizes", sizes])
@@ -330,6 +361,15 @@ def main():
print("ERROR: No families defined in config", file=sys.stderr)
sys.exit(1)
if not DEFAULT_FALLBACK_FONT.exists() or not DEFAULT_FALLBACK_FONT.is_file():
print(
"ERROR: Missing default fallback font: "
f"{DEFAULT_FALLBACK_FONT}\n"
"This font is required for fallback glyphs in SD font builds.",
file=sys.stderr,
)
sys.exit(1)
# Filter if --only specified
if args.only:
only_names = set(args.only.split(","))
+42 -13
View File
@@ -7,7 +7,6 @@ cd "$(dirname "$0")"
READER_FONT_STYLES=("Regular" "Italic" "Bold" "BoldItalic")
NOTOSERIF_FONT_SIZES=(12 14 16 18)
NOTOSANS_FONT_SIZES=(12 14 16 18)
OPENDYSLEXIC_FONT_SIZES=(8 10 12 14)
for size in ${NOTOSERIF_FONT_SIZES[@]}; do
for style in ${READER_FONT_STYLES[@]}; do
@@ -29,30 +28,60 @@ for size in ${NOTOSANS_FONT_SIZES[@]}; do
done
done
for size in ${OPENDYSLEXIC_FONT_SIZES[@]}; do
for style in ${READER_FONT_STYLES[@]}; do
font_name="opendyslexic_${size}_$(echo $style | tr '[:upper:]' '[:lower:]')"
font_path="../builtinFonts/source/OpenDyslexic/OpenDyslexic-${style}.otf"
output_path="../builtinFonts/${font_name}.h"
python fontconvert.py $font_name $size $font_path --2bit --compress > $output_path
echo "Generated $output_path"
done
done
UI_FONT_SIZES=(10 12)
UI_FONT_STYLES=("Regular" "Bold")
# Arabic glyphs for UI text (menus, file browser titles). The built-in fonts
# must cover the *output* of MiniBidi's do_shape() — contextual presentation
# forms — not base letters, or shaped UI text silently drops glyphs.
# Curated for firmware-size budget: core Arabic (Presentation Forms-B,
# incl. the Lam-Alef ligature forms) plus the Farsi/Urdu extra letters'
# Presentation Forms-A blocks, the few characters shaping leaves at their
# base codepoint, Arabic punctuation, and both digit sets. No harakat and
# no Sindhi/Pashto/Kurdish forms — book text gets those from SD-card fonts.
ARABIC_INTERVALS=(
--additional-intervals 0x060C,0x060C # Arabic comma
--additional-intervals 0x061B,0x061B # Arabic semicolon
--additional-intervals 0x061F,0x061F # Arabic question mark
--additional-intervals 0x0621,0x0621 # hamza (non-joining, never shaped)
--additional-intervals 0x0640,0x0640 # tatweel
--additional-intervals 0x0660,0x0669 # Arabic-Indic digits
--additional-intervals 0x06BA,0x06BA # noon ghunna base (initial/medial keep base cp)
--additional-intervals 0x06D4,0x06D4 # Urdu full stop
--additional-intervals 0x06F0,0x06F9 # extended Arabic-Indic digits (Farsi/Urdu)
--additional-intervals 0xFB56,0xFB59 # peh (Farsi)
--additional-intervals 0xFB66,0xFB69 # tteh (Urdu)
--additional-intervals 0xFB7A,0xFB7D # tcheh (Farsi)
--additional-intervals 0xFB88,0xFB95 # ddal, jeh, rreh (Urdu), keheh, gaf (Farsi/Urdu)
--additional-intervals 0xFB9E,0xFB9F # noon ghunna isolated/final (Urdu)
--additional-intervals 0xFBA6,0xFBB1 # heh goal, heh doachashmee, yeh barree(+hamza) (Urdu)
--additional-intervals 0xFBFC,0xFBFF # farsi yeh (Farsi/Urdu)
--additional-intervals 0xFE80,0xFEFC # Presentation Forms-B: core Arabic + Lam-Alef
)
for size in ${UI_FONT_SIZES[@]}; do
for style in ${UI_FONT_STYLES[@]}; do
font_name="ubuntu_${size}_$(echo $style | tr '[:upper:]' '[:lower:]')"
font_path="../builtinFonts/source/Ubuntu/Ubuntu-${style}.ttf"
hebrew_path="../builtinFonts/source/NotoSansHebrew/NotoSansHebrew-${style}.ttf"
arabic_path="../builtinFonts/source/NotoSansArabic/NotoSansArabic-${style}.ttf"
# Ubuntu lacks the Latin Extended Additional block (U+1EA0-U+1EF9) used for
# Vietnamese tone marks. Append a Vietnamese-only Ubuntu cut so those glyphs
# are filled from it while every glyph Ubuntu already has stays unchanged
# (fontstack is ordered by descending priority).
viet_path="../builtinFonts/source/Ubuntu/Ubuntu-Vietnamese-${style}.ttf"
output_path="../builtinFonts/${font_name}.h"
python fontconvert.py $font_name $size $font_path > $output_path
python fontconvert.py $font_name $size $font_path $hebrew_path $arabic_path $viet_path \
--additional-intervals 0x05D0,0x05EA "${ARABIC_INTERVALS[@]}" > $output_path
echo "Generated $output_path"
done
done
python fontconvert.py notosans_8_regular 8 ../builtinFonts/source/NotoSans/NotoSans-Regular.ttf > ../builtinFonts/notosans_8_regular.h
python fontconvert.py notosans_8_regular 8 \
../builtinFonts/source/NotoSans/NotoSans-Regular.ttf \
../builtinFonts/source/NotoSansHebrew/NotoSansHebrew-Regular.ttf \
../builtinFonts/source/NotoSansArabic/NotoSansArabic-Regular.ttf \
--additional-intervals 0x05D0,0x05EA "${ARABIC_INTERVALS[@]}" > ../builtinFonts/notosans_8_regular.h
echo ""
echo "Running compression verification..."
+1 -1
View File
@@ -248,7 +248,7 @@ unmerged_intervals = sorted(intervals + add_ints)
intervals = []
unvalidated_intervals = []
for i_start, i_end in unmerged_intervals:
if len(unvalidated_intervals) > 0 and i_start + 1 <= unvalidated_intervals[-1][1]:
if len(unvalidated_intervals) > 0 and i_start <= unvalidated_intervals[-1][1] + 1:
unvalidated_intervals[-1] = (unvalidated_intervals[-1][0], max(unvalidated_intervals[-1][1], i_end))
continue
unvalidated_intervals.append((i_start, i_end))
+47 -8
View File
@@ -40,9 +40,11 @@ INTERVAL_PRESETS = {
"ascii": [(0x0020, 0x007E)],
"latin1": [(0x0080, 0x00FF)],
"latin-ext": [(0x0020, 0x007E), (0x0080, 0x00FF), (0x0100, 0x024F),
(0x1E00, 0x1EFF), (0x2000, 0x206F), (0xFB00, 0xFB06)],
(0x02B0, 0x02FF), (0x1E00, 0x1EFF), (0x2000, 0x206F),
(0xFB00, 0xFB06)],
"greek": [(0x0370, 0x03FF), (0x1F00, 0x1FFF)],
"cyrillic": [(0x0400, 0x04FF), (0x0500, 0x052F)],
"hebrew": [(0x0590, 0x05FF), (0xFB1D, 0xFB4F)],
"georgian": [(0x10A0, 0x10FF), (0x2D00, 0x2D2F)],
"armenian": [(0x0530, 0x058F)],
"ethiopic": [(0x1200, 0x137F), (0x1380, 0x139F), (0x2D80, 0x2DDF)],
@@ -59,12 +61,14 @@ INTERVAL_PRESETS = {
(0x2190, 0x21FF), (0x2200, 0x22FF), (0x2500, 0x257F),
(0x25A0, 0x25FF), (0x2600, 0x26FF), (0x2700, 0x27BF)],
# Composite preset for English-language literary fiction including scifi/popsci.
# Greek for physics terms, math operators, miscellaneous symbols (♪♫♬), dingbats.
"reading": [(0x0020, 0x024F), (0x0300, 0x036F), (0x0370, 0x03FF),
# Greek for physics terms, math operators, geometric shapes, uncommon
# dialogue punctuation, CJK quote marks, miscellaneous symbols (♪♫♬), dingbats.
"reading": [(0x0020, 0x024F), (0x02B0, 0x02FF), (0x0300, 0x036F), (0x0370, 0x03FF),
(0x0400, 0x04FF), (0x1E00, 0x1EFF), (0x2000, 0x206F),
(0x2070, 0x209F), (0x20A0, 0x20CF), (0x2150, 0x218F),
(0x2190, 0x21FF), (0x2200, 0x22FF), (0x2500, 0x257F),
(0x25A0, 0x25FF), (0x2600, 0x26FF), (0x2700, 0x27BF),
(0x2900, 0x29FF), (0x2E00, 0x2E7F), (0x3000, 0x303F),
(0xFB00, 0xFB06)],
# Matches the built-in font intervals from fontconvert.py exactly
"builtin": [(0x0000, 0x007F), (0x0080, 0x00FF), (0x0100, 0x017F),
@@ -515,7 +519,8 @@ def extract_ligatures_fonttools(font_path, codepoints):
return pairs
def rasterize_font_style(fontfile, size, intervals, style_id=0, force_autohint=False):
def rasterize_font_style(fontfile, size, intervals, style_id=0, force_autohint=False,
fallback_fontfile=None):
"""Rasterize all glyphs for one font style. Returns StyleRasterData."""
import freetype
@@ -528,6 +533,10 @@ def rasterize_font_style(fontfile, size, intervals, style_id=0, force_autohint=F
# it before set_char_size() would waste work at the default size and risk
# Invalid_Size_Handle on some fonts.
face.set_char_size(size << 6, size << 6, 150, 150)
fallback_face = None
if fallback_fontfile:
fallback_face = freetype.Face(fallback_fontfile)
fallback_face.set_char_size(size << 6, size << 6, 150, 150)
load_flags = freetype.FT_LOAD_RENDER
if force_autohint:
@@ -538,6 +547,11 @@ def rasterize_font_style(fontfile, size, intervals, style_id=0, force_autohint=F
if glyph_index > 0:
face.load_glyph(glyph_index, load_flags)
return face
if fallback_face:
fallback_glyph_index = fallback_face.get_char_index(code_point)
if fallback_glyph_index > 0:
fallback_face.load_glyph(fallback_glyph_index, load_flags)
return fallback_face
return None
# Validate intervals: remove codepoints not present in the font.
@@ -549,7 +563,9 @@ def rasterize_font_style(fontfile, size, intervals, style_id=0, force_autohint=F
for i_start, i_end in intervals:
start = i_start
for code_point in range(i_start, i_end + 1):
if face.get_char_index(code_point) == 0:
has_primary = face.get_char_index(code_point) != 0
has_fallback = fallback_face and fallback_face.get_char_index(code_point) != 0
if not has_primary and not has_fallback:
if start < code_point:
validated_intervals.append((start, code_point - 1))
start = code_point + 1
@@ -760,10 +776,11 @@ def style_sections_total_size(sections):
# --- File writers ---
def generate_cpfont_multistyle(style_fonts, size, intervals, output_path,
force_autohint=False):
force_autohint=False, fallback_style_fonts=None):
"""Generate a multi-style v4 .cpfont file.
style_fonts: dict of {style_id: fontfile_path} e.g. {0: "Regular.ttf", 2: "Italic.ttf"}
fallback_style_fonts: optional dict of {style_id: fallback_fontfile_path}
"""
MAGIC = b"CPFONT\x00\x00"
HEADER_SIZE = 32
@@ -773,12 +790,15 @@ def generate_cpfont_multistyle(style_fonts, size, intervals, output_path,
# Rasterize each style
raster_data = {} # style_id -> StyleRasterData
fallback_style_fonts = fallback_style_fonts or {}
for style_id in sorted(style_fonts.keys()):
fontfile = style_fonts[style_id]
fallback_fontfile = fallback_style_fonts.get(style_id)
print(f" Rasterizing style {style_id}...", file=sys.stderr)
raster_data[style_id] = rasterize_font_style(
fontfile, size, intervals, style_id=style_id,
force_autohint=force_autohint)
force_autohint=force_autohint,
fallback_fontfile=fallback_fontfile)
# Pack binary sections for each style
packed_sections = {} # style_id -> tuple of section bytearrays
@@ -889,6 +909,14 @@ def main():
help="Font file for italic style.")
parser.add_argument("--bolditalic", dest="font_bolditalic",
help="Font file for bold-italic style.")
parser.add_argument("--fallback-regular", dest="fallback_regular",
help="Fallback font file for regular style.")
parser.add_argument("--fallback-bold", dest="fallback_bold",
help="Fallback font file for bold style.")
parser.add_argument("--fallback-italic", dest="fallback_italic",
help="Fallback font file for italic style.")
parser.add_argument("--fallback-bolditalic", dest="fallback_bolditalic",
help="Fallback font file for bold-italic style.")
args = parser.parse_args()
@@ -910,6 +938,16 @@ def main():
if args.font_bolditalic:
style_fonts[3] = args.font_bolditalic
fallback_style_fonts = {}
if args.fallback_regular:
fallback_style_fonts[0] = args.fallback_regular
if args.fallback_bold:
fallback_style_fonts[1] = args.fallback_bold
if args.fallback_italic:
fallback_style_fonts[2] = args.fallback_italic
if args.fallback_bolditalic:
fallback_style_fonts[3] = args.fallback_bolditalic
is_multistyle = len(style_fonts) > 0
fontfile = args.fontfile
@@ -977,7 +1015,8 @@ def main():
print(f"Generating {output_path} (size {sz}, {len(style_fonts)} style(s), v4)...", file=sys.stderr)
total_size += generate_cpfont_multistyle(
style_fonts, sz, intervals, output_path,
force_autohint=args.force_autohint)
force_autohint=args.force_autohint,
fallback_style_fonts=fallback_style_fonts)
print(f"\nTotal: {len(sizes)} files, {total_size / 1024 / 1024:.2f} MB", file=sys.stderr)
+42
View File
@@ -113,6 +113,36 @@ families:
italic: {url: "https://raw.githubusercontent.com/google/fonts/main/ofl/bitter/Bitter-Italic%5Bwght%5D.ttf", variable: {wght: 500}}
bolditalic: {url: "https://raw.githubusercontent.com/google/fonts/main/ofl/bitter/Bitter-Italic%5Bwght%5D.ttf", variable: {wght: 700}}
- name: Domitian
description: "A humanist serif for literary reading (Latin, Greek, Cyrillic)"
intervals: latin-ext,greek,cyrillic
sizes: [12, 14, 16, 18]
styles:
regular: {url: "https://mirrors.ctan.org/fonts/domitian/opentype/Domitian-Roman.otf"}
bold: {url: "https://mirrors.ctan.org/fonts/domitian/opentype/Domitian-Bold.otf"}
italic: {url: "https://mirrors.ctan.org/fonts/domitian/opentype/Domitian-Italic.otf"}
bolditalic: {url: "https://mirrors.ctan.org/fonts/domitian/opentype/Domitian-BoldItalic.otf"}
- name: LibreBaskerville
description: "A serif reimplementing the classic Baskerville (Latin)"
intervals: latin-ext
sizes: [12, 14, 16, 18]
styles:
regular: {url: "https://raw.githubusercontent.com/impallari/Libre-Baskerville/master/fonts/ttf/LibreBaskerville-Regular.ttf"}
bold: {url: "https://raw.githubusercontent.com/impallari/Libre-Baskerville/master/fonts/ttf/LibreBaskerville-Bold.ttf"}
italic: {url: "https://raw.githubusercontent.com/impallari/Libre-Baskerville/master/fonts/ttf/LibreBaskerville-Italic.ttf"}
bolditalic: {url: "https://raw.githubusercontent.com/impallari/Libre-Baskerville/master/fonts/ttf/LibreBaskerville-BoldItalic.ttf"}
- name: Vollkorn
description: "A serif for bread and butter use by Friedrich Althausen (Latin, Greek, Cyrillic)"
intervals: latin-ext,greek,cyrillic
sizes: [12, 14, 16, 18]
styles:
regular: {url: "https://raw.githubusercontent.com/google/fonts/main/ofl/vollkorn/Vollkorn%5Bwght%5D.ttf", variable: {wght: 400}}
bold: {url: "https://raw.githubusercontent.com/google/fonts/main/ofl/vollkorn/Vollkorn%5Bwght%5D.ttf", variable: {wght: 700}}
italic: {url: "https://raw.githubusercontent.com/google/fonts/main/ofl/vollkorn/Vollkorn-Italic%5Bwght%5D.ttf", variable: {wght: 400}}
bolditalic: {url: "https://raw.githubusercontent.com/google/fonts/main/ofl/vollkorn/Vollkorn-Italic%5Bwght%5D.ttf", variable: {wght: 700}}
# ── Sans-serif ─────────────────────────────────────────────────────────
- name: NotoSansExtended
@@ -187,6 +217,18 @@ families:
italic: {url: "https://raw.githubusercontent.com/adobe-fonts/source-code-pro/release/TTF/SourceCodePro-It.ttf"}
bolditalic: {url: "https://raw.githubusercontent.com/adobe-fonts/source-code-pro/release/TTF/SourceCodePro-BoldIt.ttf"}
# ── Dyslexia ───────────────────────────────────────────────────────────────
- name: OpenDyslexic
description: "Dyslexia-friendly font (Latin)"
intervals: latin-ext
sizes: [8, 10, 12, 14]
styles:
regular: {path: "builtinFonts/source/OpenDyslexic/OpenDyslexic-Regular.otf"}
bold: {path: "builtinFonts/source/OpenDyslexic/OpenDyslexic-Bold.otf"}
italic: {path: "builtinFonts/source/OpenDyslexic/OpenDyslexic-Italic.otf"}
bolditalic: {path: "builtinFonts/source/OpenDyslexic/OpenDyslexic-BoldItalic.otf"}
# ── Accessibility ──────────────────────────────────────────────────────
- name: AtkinsonHyperlegibleNext
+87 -88
View File
@@ -5,6 +5,7 @@
#include <JpegToBmpConverter.h>
#include <Logging.h>
#include <PngToBmpConverter.h>
#include <Utf8.h>
#include <ZipFile.h>
#include "Epub/parsers/ContainerParser.h"
@@ -44,7 +45,7 @@ bool Epub::findContentOpfFile(std::string* contentOpfFile) const {
return true;
}
bool Epub::parseContentOpf(BookMetadataCache::BookMetadata& bookMetadata) {
bool Epub::parseContentOpf(BookMetadataCache::BookMetadata& bookMetadata, const bool writeSpineEntries) {
std::string contentOpfFilePath;
if (!findContentOpfFile(&contentOpfFilePath)) {
LOG_ERR("EBP", "Could not find content.opf in zip");
@@ -61,7 +62,8 @@ bool Epub::parseContentOpf(BookMetadataCache::BookMetadata& bookMetadata) {
return false;
}
ContentOpfParser opfParser(getCachePath(), getBasePath(), contentOpfSize, bookMetadataCache.get());
ContentOpfParser opfParser(getCachePath(), getBasePath(), contentOpfSize,
writeSpineEntries ? bookMetadataCache.get() : nullptr);
if (!opfParser.setup()) {
LOG_ERR("EBP", "Could not setup content.opf parser");
return false;
@@ -72,8 +74,9 @@ bool Epub::parseContentOpf(BookMetadataCache::BookMetadata& bookMetadata) {
return false;
}
// Grab data from opfParser into epub
bookMetadata.title = opfParser.title;
// Grab data from opfParser into epub. Normalize titles to NFC so NFD (combining
// mark) text renders correctly — the device fonts have no mark positioning.
bookMetadata.title = utf8ComposeNfc(opfParser.title);
bookMetadata.author = opfParser.author;
bookMetadata.language = opfParser.language;
bookMetadata.coverItemHref = opfParser.coverItemHref;
@@ -104,8 +107,9 @@ bool Epub::parseContentOpf(BookMetadataCache::BookMetadata& bookMetadata) {
const auto endPos = coverPageHtml.find('"', pos);
if (endPos != std::string::npos) {
const auto ref = std::string_view{coverPageHtml}.substr(pos, endPos - pos);
// Check if it's an image file
if (FsHelpers::hasPngExtension(ref) || FsHelpers::hasJpgExtension(ref) || FsHelpers::hasGifExtension(ref)) {
// Cover BMP generation supports JPG/PNG only; skip GIF so an unsupported wrapper image
// does not block a later supported cover reference.
if (FsHelpers::hasPngExtension(ref) || FsHelpers::hasJpgExtension(ref)) {
imageRef = ref;
break;
}
@@ -116,7 +120,7 @@ bool Epub::parseContentOpf(BookMetadataCache::BookMetadata& bookMetadata) {
}
if (!imageRef.empty()) {
bookMetadata.coverItemHref = FsHelpers::normalisePath(coverPageBase + imageRef);
bookMetadata.coverItemHref = FsHelpers::normalisePath(FsHelpers::decodeUriEscapes(coverPageBase + imageRef));
LOG_DBG("EBP", "Found cover image from guide: %s", bookMetadata.coverItemHref.c_str());
}
}
@@ -149,18 +153,11 @@ bool Epub::parseTocNcxFile() const {
LOG_DBG("EBP", "Parsing toc ncx file: %s", tocNcxItem.c_str());
const auto tmpNcxPath = getCachePath() + "/toc.ncx";
FsFile tempNcxFile;
if (!Storage.openFileForWrite("EBP", tmpNcxPath, tempNcxFile)) {
size_t ncxSize;
if (!getItemSize(tocNcxItem, &ncxSize)) {
LOG_ERR("EBP", "Could not get size of toc ncx file");
return false;
}
readItemContentsToStream(tocNcxItem, tempNcxFile, 1024);
// Explicitly close() file before reopening for reading
tempNcxFile.close();
if (!Storage.openFileForRead("EBP", tmpNcxPath, tempNcxFile)) {
return false;
}
const auto ncxSize = tempNcxFile.size();
TocNcxParser ncxParser(contentBasePath, ncxSize, bookMetadataCache.get());
@@ -169,29 +166,13 @@ bool Epub::parseTocNcxFile() const {
return false;
}
const auto ncxBuffer = static_cast<uint8_t*>(malloc(1024));
if (!ncxBuffer) {
LOG_ERR("EBP", "Could not allocate memory for toc ncx parser");
// Stream the decompressed NCX straight into the parser instead of round-tripping
// through a temp file on the SD card (decompress -> write -> reopen -> reread -> delete).
if (!readItemContentsToStream(tocNcxItem, ncxParser, 1024)) {
LOG_ERR("EBP", "Could not read toc ncx file");
return false;
}
while (tempNcxFile.available()) {
const auto readSize = tempNcxFile.read(ncxBuffer, 1024);
if (readSize == 0) break;
const auto processedSize = ncxParser.write(ncxBuffer, readSize);
if (processedSize != readSize) {
LOG_ERR("EBP", "Could not process all toc ncx data");
free(ncxBuffer);
return false;
}
}
free(ncxBuffer);
// Explicitly close() file before calling Storage.remove()
tempNcxFile.close();
Storage.remove(tmpNcxPath.c_str());
LOG_DBG("EBP", "Parsed TOC items");
return true;
}
@@ -205,18 +186,11 @@ bool Epub::parseTocNavFile() const {
LOG_DBG("EBP", "Parsing toc nav file: %s", tocNavItem.c_str());
const auto tmpNavPath = getCachePath() + "/toc.nav";
FsFile tempNavFile;
if (!Storage.openFileForWrite("EBP", tmpNavPath, tempNavFile)) {
size_t navSize;
if (!getItemSize(tocNavItem, &navSize)) {
LOG_ERR("EBP", "Could not get size of toc nav file");
return false;
}
readItemContentsToStream(tocNavItem, tempNavFile, 1024);
// Explicitly close() file before reopening for reading
tempNavFile.close();
if (!Storage.openFileForRead("EBP", tmpNavPath, tempNavFile)) {
return false;
}
const auto navSize = tempNavFile.size();
// Note: We can't use `contentBasePath` here as the nav file may be in a different folder to the content.opf
// and the HTMLX nav file will have hrefs relative to itself
@@ -228,32 +202,41 @@ bool Epub::parseTocNavFile() const {
return false;
}
const auto navBuffer = static_cast<uint8_t*>(malloc(1024));
if (!navBuffer) {
LOG_ERR("EBP", "Could not allocate memory for toc nav parser");
// Stream the decompressed nav document straight into the parser instead of round-tripping
// through a temp file on the SD card (decompress -> write -> reopen -> reread -> delete).
if (!readItemContentsToStream(tocNavItem, navParser, 1024)) {
LOG_ERR("EBP", "Could not read toc nav file");
return false;
}
while (tempNavFile.available()) {
const auto readSize = tempNavFile.read(navBuffer, 1024);
const auto processedSize = navParser.write(navBuffer, readSize);
if (processedSize != readSize) {
LOG_ERR("EBP", "Could not process all toc nav data");
free(navBuffer);
return false;
}
}
free(navBuffer);
// Explicitly close() file before calling Storage.remove()
tempNavFile.close();
Storage.remove(tmpNavPath.c_str());
LOG_DBG("EBP", "Parsed TOC nav items");
return true;
}
void Epub::discoverCssFilesFromZip() {
const std::string& opfDir = contentBasePath;
ZipFile zf(filepath);
if (!zf.enumerateFilePaths([&](std::string_view filePath) {
if (!opfDir.empty() && filePath.find(opfDir) != 0) {
return;
}
if (!FsHelpers::hasCssExtension(filePath)) {
return;
}
if (std::find(cssFiles.begin(), cssFiles.end(), filePath) != cssFiles.end()) {
return;
}
LOG_DBG("EBP", "Discovered CSS file via ZIP enumeration: %.*s", (int)filePath.size(), filePath.data());
cssFiles.push_back(std::string{filePath});
})) {
LOG_ERR("EBP", "Failed to enumerate ZIP file paths for CSS discovery");
}
}
void Epub::parseCssFiles() const {
// Maximum CSS file size we'll attempt to parse (uncompressed)
// Larger files risk memory exhaustion on ESP32
@@ -297,7 +280,7 @@ void Epub::parseCssFiles() const {
// Extract CSS file to temp location
const auto tmpCssPath = getCachePath() + "/.tmp.css";
FsFile tempCssFile;
HalFile tempCssFile;
if (!Storage.openFileForWrite("EBP", tmpCssPath, tempCssFile)) {
LOG_ERR("EBP", "Could not create temp CSS file");
continue;
@@ -328,9 +311,9 @@ void Epub::parseCssFiles() const {
if (!cssParser->saveToCache()) {
LOG_ERR("EBP", "Failed to save CSS rules to cache");
}
cssParser->clear();
LOG_DBG("EBP", "Loaded %zu CSS style rules from %zu files", cssParser->ruleCount(), cssFiles.size());
cssParser->clear();
}
// load in the meta data for the epub file
@@ -350,15 +333,29 @@ bool Epub::load(const bool buildIfMissing, const bool skipLoadingCss) {
LOG_DBG("EBP", "CSS rules cache missing or stale, attempting to parse CSS files");
cssParser->deleteCache();
if (!parseContentOpf(bookMetadataCache->coreMetadata)) {
BookMetadataCache::BookMetadata cachedMetadata = bookMetadataCache->coreMetadata;
if (!parseContentOpf(cachedMetadata, /*writeSpineEntries=*/false)) {
LOG_ERR("EBP", "Could not parse content.opf from cached bookMetadata for CSS files");
// continue anyway - book will work without CSS and we'll still load any inline style CSS
} else {
discoverCssFilesFromZip();
}
bookMetadataCache.reset();
parseCssFiles();
bookMetadataCache.reset(new BookMetadataCache(cachePath));
if (!bookMetadataCache->load()) {
LOG_ERR("EBP", "Failed to reload cache after CSS rebuild");
return false;
}
// Invalidate section caches so they are rebuilt with the new CSS
Storage.removeDir((cachePath + "/sections").c_str());
}
}
// Release the resolved CSS rule map: it is only needed transiently while building
// section caches, and createSectionFile reloads it from cache on demand. Holding it
// resident pins tens of KB for the whole reading session (more on warm resume into
// an already-cached chapter, where createSectionFile never runs to clear it).
cssParser->clear();
LOG_DBG("EBP", "Loaded ePub: %s", filepath.c_str());
return true;
}
@@ -391,6 +388,7 @@ bool Epub::load(const bool buildIfMissing, const bool skipLoadingCss) {
LOG_ERR("EBP", "Could not parse content.opf");
return false;
}
discoverCssFilesFromZip();
if (!bookMetadataCache->endContentOpfPass()) {
LOG_ERR("EBP", "Could not end writing content.opf pass");
return false;
@@ -448,6 +446,13 @@ bool Epub::load(const bool buildIfMissing, const bool skipLoadingCss) {
LOG_DBG("EBP", "Could not cleanup tmp files - ignoring");
}
if (!skipLoadingCss) {
// Parse CSS before reloading book.bin to leave more heap for CSS rule-table growth.
bookMetadataCache.reset();
parseCssFiles();
Storage.removeDir((cachePath + "/sections").c_str());
}
// Reload the cache from disk so it's in the correct state
bookMetadataCache.reset(new BookMetadataCache(cachePath));
if (!bookMetadataCache->load()) {
@@ -455,12 +460,6 @@ bool Epub::load(const bool buildIfMissing, const bool skipLoadingCss) {
return false;
}
if (!skipLoadingCss) {
// Parse CSS files after cache reload
parseCssFiles();
Storage.removeDir((cachePath + "/sections").c_str());
}
LOG_DBG("EBP", "Loaded ePub: %s", filepath.c_str());
return true;
}
@@ -545,7 +544,7 @@ bool Epub::generateCoverBmp(bool cropped) const {
LOG_DBG("EBP", "Generating BMP from JPG cover image (%s mode)", cropped ? "cropped" : "fit");
const auto coverJpgTempPath = getCachePath() + "/.cover.jpg";
FsFile coverJpg;
HalFile coverJpg;
if (!Storage.openFileForWrite("EBP", coverJpgTempPath, coverJpg)) {
return false;
}
@@ -557,7 +556,7 @@ bool Epub::generateCoverBmp(bool cropped) const {
return false;
}
FsFile coverBmp;
HalFile coverBmp;
if (!Storage.openFileForWrite("EBP", getCoverBmpPath(cropped), coverBmp)) {
return false;
}
@@ -579,7 +578,7 @@ bool Epub::generateCoverBmp(bool cropped) const {
LOG_DBG("EBP", "Generating BMP from PNG cover image (%s mode)", cropped ? "cropped" : "fit");
const auto coverPngTempPath = getCachePath() + "/.cover.png";
FsFile coverPng;
HalFile coverPng;
if (!Storage.openFileForWrite("EBP", coverPngTempPath, coverPng)) {
return false;
}
@@ -591,7 +590,7 @@ bool Epub::generateCoverBmp(bool cropped) const {
return false;
}
FsFile coverBmp;
HalFile coverBmp;
if (!Storage.openFileForWrite("EBP", getCoverBmpPath(cropped), coverBmp)) {
return false;
}
@@ -634,7 +633,7 @@ bool Epub::generateThumbBmp(int height) const {
LOG_DBG("EBP", "Generating thumb BMP from JPG cover image");
const auto coverJpgTempPath = getCachePath() + "/.cover.jpg";
FsFile coverJpg;
HalFile coverJpg;
if (!Storage.openFileForWrite("EBP", coverJpgTempPath, coverJpg)) {
return false;
}
@@ -646,7 +645,7 @@ bool Epub::generateThumbBmp(int height) const {
return false;
}
FsFile thumbBmp;
HalFile thumbBmp;
if (!Storage.openFileForWrite("EBP", getThumbBmpPath(height), thumbBmp)) {
return false;
}
@@ -671,7 +670,7 @@ bool Epub::generateThumbBmp(int height) const {
LOG_DBG("EBP", "Generating thumb BMP from PNG cover image");
const auto coverPngTempPath = getCachePath() + "/.cover.png";
FsFile coverPng;
HalFile coverPng;
if (!Storage.openFileForWrite("EBP", coverPngTempPath, coverPng)) {
return false;
}
@@ -683,7 +682,7 @@ bool Epub::generateThumbBmp(int height) const {
return false;
}
FsFile thumbBmp;
HalFile thumbBmp;
if (!Storage.openFileForWrite("EBP", getThumbBmpPath(height), thumbBmp)) {
return false;
}
@@ -707,7 +706,7 @@ bool Epub::generateThumbBmp(int height) const {
}
// Write an empty bmp file to avoid generation attempts in the future
FsFile thumbBmp;
HalFile thumbBmp;
Storage.openFileForWrite("EBP", getThumbBmpPath(height), thumbBmp);
return false;
}
@@ -863,10 +862,10 @@ float Epub::calculateProgress(const int currentSpineIndex, const float currentSp
int Epub::resolveHrefToSpineIndex(const std::string& href) const {
if (!bookMetadataCache || !bookMetadataCache->isLoaded()) return -1;
// Extract filename (remove #anchor)
std::string target = href;
size_t hashPos = target.find('#');
if (hashPos != std::string::npos) target = target.substr(0, hashPos);
// Split before decoding so escaped '#' characters in filenames stay part of the path.
const size_t hashPos = href.find('#');
const std::string rawTarget = hashPos != std::string::npos ? href.substr(0, hashPos) : href;
const std::string target = FsHelpers::normalisePath(FsHelpers::decodeUriEscapes(rawTarget));
// Same-file reference (anchor-only)
if (target.empty()) return -1;
+2 -1
View File
@@ -31,9 +31,10 @@ class Epub {
std::vector<std::string> cssFiles;
bool findContentOpfFile(std::string* contentOpfFile) const;
bool parseContentOpf(BookMetadataCache::BookMetadata& bookMetadata);
bool parseContentOpf(BookMetadataCache::BookMetadata& bookMetadata, bool writeSpineEntries = true);
bool parseTocNcxFile() const;
bool parseTocNavFile() const;
void discoverCssFilesFromZip();
void parseCssFiles() const;
public:
+137 -63
View File
@@ -1,7 +1,9 @@
#include "BookMetadataCache.h"
#include <BufferedFile.h>
#include <Logging.h>
#include <Serialization.h>
#include <Utf8.h>
#include <ZipFile.h>
#include <deque>
@@ -9,10 +11,56 @@
#include "FsHelpers.h"
namespace {
constexpr uint8_t BOOK_CACHE_VERSION = 5;
constexpr uint8_t BOOK_CACHE_VERSION = 8; // v8: TOC/book titles stored NFC-composed
constexpr char bookBinFile[] = "/book.bin";
constexpr char tmpSpineBinFile[] = "/spine.bin.tmp";
constexpr char tmpTocBinFile[] = "/toc.bin.tmp";
// Buffer size for the buildBookBin streams. 3 buffers x 4KB, transient (freed on
// return); 4KB = 8 SD sectors per transfer, enough to stop the sector-cache thrash.
constexpr size_t BUILD_IO_BUFFER_SIZE = 4096;
// Entry (de)serializers, templated so they run over HalFile and the Buffered*
// wrappers alike (two instantiations each -- a few hundred bytes of flash, in
// exchange for the build path streaming at SD speed instead of per-pod).
template <typename F>
uint32_t writeSpineEntryTo(F& file, const BookMetadataCache::SpineEntry& entry) {
const uint32_t pos = file.position();
serialization::writeString(file, entry.href);
serialization::writePod(file, entry.cumulativeSize);
serialization::writePod(file, entry.tocIndex);
return pos;
}
template <typename F>
uint32_t writeTocEntryTo(F& file, const BookMetadataCache::TocEntry& entry) {
const uint32_t pos = file.position();
serialization::writeString(file, entry.title);
serialization::writeString(file, entry.href);
serialization::writeString(file, entry.anchor);
serialization::writePod(file, entry.level);
serialization::writePod(file, entry.spineIndex);
return pos;
}
template <typename F>
BookMetadataCache::SpineEntry readSpineEntryFrom(F& file) {
BookMetadataCache::SpineEntry entry;
serialization::readString(file, entry.href);
serialization::readPod(file, entry.cumulativeSize);
serialization::readPod(file, entry.tocIndex);
return entry;
}
template <typename F>
BookMetadataCache::TocEntry readTocEntryFrom(F& file) {
BookMetadataCache::TocEntry entry;
serialization::readString(file, entry.title);
serialization::readString(file, entry.href);
serialization::readString(file, entry.anchor);
serialization::readPod(file, entry.level);
serialization::readPod(file, entry.spineIndex);
return entry;
}
} // namespace
/* ============= WRITING / BUILDING FUNCTIONS ================ */
@@ -29,13 +77,23 @@ bool BookMetadataCache::beginContentOpfPass() {
LOG_DBG("BMC", "Beginning content opf pass");
// Open spine file for writing
return Storage.openFileForWrite("BMC", cachePath + tmpSpineBinFile, spineFile);
if (!Storage.openFileForWrite("BMC", cachePath + tmpSpineBinFile, spineFile)) {
return false;
}
// Wrapper OOM is fine: createSpineEntry falls back to unbuffered writes.
passOut = makeUniqueNoThrow<serialization::BufferedFileWriter>(spineFile, BUILD_IO_BUFFER_SIZE);
return true;
}
bool BookMetadataCache::endContentOpfPass() {
const bool flushed = !passOut || passOut->flush();
passOut.reset();
// Explicit close() required: member variable persists beyond function scope
spineFile.close();
return true;
if (!flushed) {
LOG_ERR("BMC", "Failed writing spine tmp file");
}
return flushed;
}
bool BookMetadataCache::beginTocPass() {
@@ -73,10 +131,17 @@ bool BookMetadataCache::beginTocPass() {
useSpineHrefIndex = false;
}
// Wrapper OOM is fine: createTocEntry falls back to unbuffered writes.
passOut = makeUniqueNoThrow<serialization::BufferedFileWriter>(tocFile, BUILD_IO_BUFFER_SIZE);
return true;
}
bool BookMetadataCache::endTocPass() {
const bool flushed = !passOut || passOut->flush();
passOut.reset();
if (!flushed) {
LOG_ERR("BMC", "Failed writing toc tmp file");
}
// Explicit close() required: member variables persist beyond function scope
tocFile.close();
spineFile.close();
@@ -85,7 +150,7 @@ bool BookMetadataCache::endTocPass() {
spineHrefIndex.shrink_to_fit();
useSpineHrefIndex = false;
return true;
return flushed;
}
bool BookMetadataCache::endWrite() {
@@ -118,6 +183,14 @@ bool BookMetadataCache::buildBookBin(const std::string& epubPath, const BookMeta
return false;
}
// Buffered streams for the whole build: every access below is sequential per
// file, but interleaved ACROSS files, which thrashes SdFat's single shared
// sector cache when unbuffered (one 512B SD transaction per 4-byte pod --
// measured 31s for a 1,732-spine omnibus). Three 4KB buffers, freed on return.
serialization::BufferedFileWriter bookOut(bookFile, BUILD_IO_BUFFER_SIZE);
serialization::BufferedFileReader spineIn(spineFile, BUILD_IO_BUFFER_SIZE);
serialization::BufferedFileReader tocIn(tocFile, BUILD_IO_BUFFER_SIZE);
constexpr uint32_t headerASize =
sizeof(BOOK_CACHE_VERSION) + /* LUT Offset */ sizeof(uint32_t) + sizeof(spineCount) + sizeof(tocCount);
const uint32_t metadataSize = metadata.title.size() + metadata.author.size() + metadata.language.size() +
@@ -127,31 +200,34 @@ bool BookMetadataCache::buildBookBin(const std::string& epubPath, const BookMeta
const uint32_t lutOffset = headerASize + metadataSize;
// Header A
serialization::writePod(bookFile, BOOK_CACHE_VERSION);
serialization::writePod(bookFile, lutOffset);
serialization::writePod(bookFile, spineCount);
serialization::writePod(bookFile, tocCount);
serialization::writePod(bookOut, BOOK_CACHE_VERSION);
serialization::writePod(bookOut, lutOffset);
serialization::writePod(bookOut, spineCount);
serialization::writePod(bookOut, tocCount);
// Metadata
serialization::writeString(bookFile, metadata.title);
serialization::writeString(bookFile, metadata.author);
serialization::writeString(bookFile, metadata.language);
serialization::writeString(bookFile, metadata.coverItemHref);
serialization::writeString(bookFile, metadata.textReferenceHref);
serialization::writeString(bookOut, metadata.title);
serialization::writeString(bookOut, metadata.author);
serialization::writeString(bookOut, metadata.language);
serialization::writeString(bookOut, metadata.coverItemHref);
serialization::writeString(bookOut, metadata.textReferenceHref);
// Loop through spine entries, writing LUT positions
spineFile.seek(0);
spineIn.seek(0);
for (int i = 0; i < spineCount; i++) {
uint32_t pos = spineFile.position();
auto spineEntry = readSpineEntry(spineFile);
serialization::writePod(bookFile, pos + lutOffset + lutSize);
const uint32_t pos = spineIn.position();
readSpineEntryFrom(spineIn);
serialization::writePod(bookOut, pos + lutOffset + lutSize);
}
// Total size of the spine tmp file: entries land in book.bin after the toc LUT
// and the full spine block, so toc LUT positions are offset by it.
const auto spineBytes = static_cast<uint32_t>(spineIn.position());
// Loop through toc entries, writing LUT positions
tocFile.seek(0);
tocIn.seek(0);
for (int i = 0; i < tocCount; i++) {
uint32_t pos = tocFile.position();
auto tocEntry = readTocEntry(tocFile);
serialization::writePod(bookFile, pos + lutOffset + lutSize + static_cast<uint32_t>(spineFile.position()));
const uint32_t pos = tocIn.position();
readTocEntryFrom(tocIn);
serialization::writePod(bookOut, pos + lutOffset + lutSize + spineBytes);
}
// LUTs complete
@@ -159,9 +235,9 @@ bool BookMetadataCache::buildBookBin(const std::string& epubPath, const BookMeta
// Build spineIndex->tocIndex mapping in one pass (O(n) instead of O(n*m))
std::deque<int16_t> spineToTocIndex(spineCount, -1);
tocFile.seek(0);
tocIn.seek(0);
for (int j = 0; j < tocCount; j++) {
auto tocEntry = readTocEntry(tocFile);
auto tocEntry = readTocEntryFrom(tocIn);
if (tocEntry.spineIndex >= 0 && tocEntry.spineIndex < spineCount) {
if (spineToTocIndex[tocEntry.spineIndex] == -1) {
spineToTocIndex[tocEntry.spineIndex] = static_cast<int16_t>(j);
@@ -196,9 +272,9 @@ bool BookMetadataCache::buildBookBin(const std::string& epubPath, const BookMeta
std::deque<ZipFile::SizeTarget> targets;
targets.resize(spineCount);
spineFile.seek(0);
spineIn.seek(0);
for (int i = 0; i < spineCount; i++) {
auto entry = readSpineEntry(spineFile);
auto entry = readSpineEntryFrom(spineIn);
std::string path = FsHelpers::normalisePath(entry.href);
ZipFile::SizeTarget t;
@@ -223,10 +299,10 @@ bool BookMetadataCache::buildBookBin(const std::string& epubPath, const BookMeta
}
uint32_t cumSize = 0;
spineFile.seek(0);
spineIn.seek(0);
int lastSpineTocIndex = -1;
for (int i = 0; i < spineCount; i++) {
auto spineEntry = readSpineEntry(spineFile);
auto spineEntry = readSpineEntryFrom(spineIn);
spineEntry.tocIndex = spineToTocIndex[i];
@@ -259,23 +335,33 @@ bool BookMetadataCache::buildBookBin(const std::string& epubPath, const BookMeta
spineEntry.cumulativeSize = cumSize;
// Write out spine data to book.bin
writeSpineEntry(bookFile, spineEntry);
writeSpineEntryTo(bookOut, spineEntry);
}
// Close opened zip file
zip.close();
// Loop through toc entries from toc file writing to book.bin
tocFile.seek(0);
tocIn.seek(0);
for (int i = 0; i < tocCount; i++) {
auto tocEntry = readTocEntry(tocFile);
writeTocEntry(bookFile, tocEntry);
auto tocEntry = readTocEntryFrom(tocIn);
writeTocEntryTo(bookOut, tocEntry);
}
const bool written = bookOut.flush();
// Explicit close() required: member variables persist beyond function scope
bookFile.close();
spineFile.close();
tocFile.close();
if (!written) {
// A short write (card full/removed) would leave a truncated book.bin that
// still passes the version check on load; remove it so the next open rebuilds.
LOG_ERR("BMC", "Failed writing book.bin, removing truncated file");
Storage.remove((cachePath + bookBinFile).c_str());
return false;
}
LOG_DBG("BMC", "Successfully built book.bin");
return true;
}
@@ -292,22 +378,12 @@ bool BookMetadataCache::cleanupTmpFiles() const {
return true;
}
uint32_t BookMetadataCache::writeSpineEntry(FsFile& file, const SpineEntry& entry) const {
const uint32_t pos = file.position();
serialization::writeString(file, entry.href);
serialization::writePod(file, entry.cumulativeSize);
serialization::writePod(file, entry.tocIndex);
return pos;
uint32_t BookMetadataCache::writeSpineEntry(HalFile& file, const SpineEntry& entry) const {
return writeSpineEntryTo(file, entry);
}
uint32_t BookMetadataCache::writeTocEntry(FsFile& file, const TocEntry& entry) const {
const uint32_t pos = file.position();
serialization::writeString(file, entry.title);
serialization::writeString(file, entry.href);
serialization::writeString(file, entry.anchor);
serialization::writePod(file, entry.level);
serialization::writePod(file, entry.spineIndex);
return pos;
uint32_t BookMetadataCache::writeTocEntry(HalFile& file, const TocEntry& entry) const {
return writeTocEntryTo(file, entry);
}
// Note: for the LUT to be accurate, this **MUST** be called for all spine items before `addTocEntry` is ever called
@@ -319,7 +395,11 @@ void BookMetadataCache::createSpineEntry(const std::string& href) {
}
const SpineEntry entry(href, 0, -1);
writeSpineEntry(spineFile, entry);
if (passOut) {
writeSpineEntryTo(*passOut, entry);
} else {
writeSpineEntry(spineFile, entry);
}
spineCount++;
}
@@ -364,8 +444,14 @@ void BookMetadataCache::createTocEntry(const std::string& title, const std::stri
}
}
const TocEntry entry(title, href, anchor, level, spineIndex);
writeTocEntry(tocFile, entry);
// Compose the title to NFC at index time so the cache stores precomposed glyphs;
// device fonts have no combining-mark positioning, so NFD titles render broken.
const TocEntry entry(utf8ComposeNfc(title), href, anchor, level, spineIndex);
if (passOut) {
writeTocEntryTo(*passOut, entry);
} else {
writeTocEntry(tocFile, entry);
}
tocCount++;
}
@@ -438,20 +524,8 @@ BookMetadataCache::TocEntry BookMetadataCache::getTocEntry(const int index) {
return readTocEntry(bookFile);
}
BookMetadataCache::SpineEntry BookMetadataCache::readSpineEntry(FsFile& file) const {
SpineEntry entry;
serialization::readString(file, entry.href);
serialization::readPod(file, entry.cumulativeSize);
serialization::readPod(file, entry.tocIndex);
return entry;
BookMetadataCache::SpineEntry BookMetadataCache::readSpineEntry(HalFile& file) const {
return readSpineEntryFrom(file);
}
BookMetadataCache::TocEntry BookMetadataCache::readTocEntry(FsFile& file) const {
TocEntry entry;
serialization::readString(file, entry.title);
serialization::readString(file, entry.href);
serialization::readString(file, entry.anchor);
serialization::readPod(file, entry.level);
serialization::readPod(file, entry.spineIndex);
return entry;
}
BookMetadataCache::TocEntry BookMetadataCache::readTocEntry(HalFile& file) const { return readTocEntryFrom(file); }
+14 -7
View File
@@ -1,9 +1,11 @@
#pragma once
#include <BufferedFile.h>
#include <HalStorage.h>
#include <algorithm>
#include <deque>
#include <memory>
#include <string>
class BookMetadataCache {
@@ -50,10 +52,15 @@ class BookMetadataCache {
bool loaded;
bool buildMode;
FsFile bookFile;
HalFile bookFile;
// Temp file handles during build
FsFile spineFile;
FsFile tocFile;
HalFile spineFile;
HalFile tocFile;
// Buffers the per-entry tmp-file writes during the OPF/TOC passes: those
// writes interleave with zip-inflate SD reads, and unbuffered they thrash
// SdFat's shared sector cache (one 512B transaction per 4-byte pod). One
// wrapper serves whichever pass is active (spine, then toc).
std::unique_ptr<serialization::BufferedFileWriter> passOut;
// Index for fast href→spineIndex lookup (used only for large EPUBs)
struct SpineHrefIndexEntry {
@@ -76,10 +83,10 @@ class BookMetadataCache {
return hash;
}
uint32_t writeSpineEntry(FsFile& file, const SpineEntry& entry) const;
uint32_t writeTocEntry(FsFile& file, const TocEntry& entry) const;
SpineEntry readSpineEntry(FsFile& file) const;
TocEntry readTocEntry(FsFile& file) const;
uint32_t writeSpineEntry(HalFile& file, const SpineEntry& entry) const;
uint32_t writeTocEntry(HalFile& file, const TocEntry& entry) const;
SpineEntry readSpineEntry(HalFile& file) const;
TocEntry readTocEntry(HalFile& file) const;
public:
BookMetadata coreMetadata;
+107 -9
View File
@@ -1,13 +1,30 @@
#include "Page.h"
#include <GfxRenderer.h>
#include <Logging.h>
#include <Serialization.h>
#include <new>
namespace {
template <typename Predicate>
void renderFilteredPageElements(const std::vector<std::shared_ptr<PageElement>>& elements, GfxRenderer& renderer,
const int fontId, const int xOffset, const int yOffset, Predicate&& predicate) {
for (const auto& element : elements) {
if (predicate(*element)) {
element->render(renderer, fontId, xOffset, yOffset);
}
}
}
} // namespace
void PageLine::render(GfxRenderer& renderer, const int fontId, const int xOffset, const int yOffset) {
block->render(renderer, fontId, xPos + xOffset, yPos + yOffset);
}
bool PageLine::serialize(FsFile& file) {
bool PageLine::serialize(HalFile& file) {
serialization::writePod(file, xPos);
serialization::writePod(file, yPos);
@@ -15,14 +32,24 @@ bool PageLine::serialize(FsFile& file) {
return block->serialize(file);
}
std::unique_ptr<PageLine> PageLine::deserialize(FsFile& file) {
std::unique_ptr<PageLine> PageLine::deserialize(HalFile& file) {
int16_t xPos;
int16_t yPos;
serialization::readPod(file, xPos);
serialization::readPod(file, yPos);
auto tb = TextBlock::deserialize(file);
return std::unique_ptr<PageLine>(new PageLine(std::move(tb), xPos, yPos));
if (!tb) {
LOG_ERR("PGE", "Deserialization failed: null TextBlock");
return nullptr;
}
auto* line = new (std::nothrow) PageLine(std::move(tb), xPos, yPos);
if (!line) {
LOG_ERR("PGE", "Deserialization failed: could not allocate PageLine");
return nullptr;
}
return std::unique_ptr<PageLine>(line);
}
void PageImage::render(GfxRenderer& renderer, const int fontId, const int xOffset, const int yOffset) {
@@ -30,7 +57,11 @@ void PageImage::render(GfxRenderer& renderer, const int fontId, const int xOffse
imageBlock->render(renderer, xPos + xOffset, yPos + yOffset);
}
bool PageImage::serialize(FsFile& file) {
void PageImage::renderPlaceholder(GfxRenderer& renderer, const int xOffset, const int yOffset) const {
imageBlock->renderPlaceholder(renderer, xPos + xOffset, yPos + yOffset);
}
bool PageImage::serialize(HalFile& file) {
serialization::writePod(file, xPos);
serialization::writePod(file, yPos);
@@ -38,7 +69,7 @@ bool PageImage::serialize(FsFile& file) {
return imageBlock->serialize(file);
}
std::unique_ptr<PageImage> PageImage::deserialize(FsFile& file) {
std::unique_ptr<PageImage> PageImage::deserialize(HalFile& file) {
int16_t xPos;
int16_t yPos;
serialization::readPod(file, xPos);
@@ -48,13 +79,68 @@ std::unique_ptr<PageImage> PageImage::deserialize(FsFile& file) {
return std::unique_ptr<PageImage>(new PageImage(std::move(ib), xPos, yPos));
}
void PageHorizontalRule::render(GfxRenderer& renderer, const int fontId, const int xOffset, const int yOffset) {
(void)fontId;
if (width == 0 || thickness == 0) {
return;
}
renderer.drawLine(xPos + xOffset, yPos + yOffset, xPos + xOffset + width - 1, yPos + yOffset, thickness, true);
}
bool PageHorizontalRule::serialize(HalFile& file) {
serialization::writePod(file, xPos);
serialization::writePod(file, yPos);
serialization::writePod(file, width);
serialization::writePod(file, thickness);
return true;
}
std::unique_ptr<PageHorizontalRule> PageHorizontalRule::deserialize(HalFile& file) {
int16_t xPos = 0;
int16_t yPos = 0;
uint16_t width = 0;
uint8_t thickness = 0;
serialization::readPod(file, xPos);
serialization::readPod(file, yPos);
serialization::readPod(file, width);
serialization::readPod(file, thickness);
if (width == 0 || thickness == 0) {
LOG_ERR("PGE", "Deserialization failed: invalid horizontal rule metadata (width=%u thickness=%u)", width,
thickness);
return nullptr;
}
auto* rule = new (std::nothrow) PageHorizontalRule(width, thickness, xPos, yPos);
if (!rule) {
LOG_ERR("PGE", "Deserialization failed: could not allocate PageHorizontalRule");
return nullptr;
}
return std::unique_ptr<PageHorizontalRule>(rule);
}
void Page::render(GfxRenderer& renderer, const int fontId, const int xOffset, const int yOffset) const {
for (auto& element : elements) {
element->render(renderer, fontId, xOffset, yOffset);
renderFilteredPageElements(elements, renderer, fontId, xOffset, yOffset, [](const PageElement&) { return true; });
}
void Page::renderImages(GfxRenderer& renderer, const int fontId, const int xOffset, const int yOffset) const {
renderFilteredPageElements(elements, renderer, fontId, xOffset, yOffset,
[](const PageElement& element) { return element.getTag() == TAG_PageImage; });
}
void Page::renderWithImagePlaceholders(GfxRenderer& renderer, const int fontId, const int xOffset,
const int yOffset) const {
for (const auto& element : elements) {
if (element->getTag() == TAG_PageImage) {
static_cast<const PageImage&>(*element).renderPlaceholder(renderer, xOffset, yOffset);
} else {
element->render(renderer, fontId, xOffset, yOffset);
}
}
}
bool Page::serialize(FsFile& file) const {
bool Page::serialize(HalFile& file) const {
const uint16_t count = elements.size();
serialization::writePod(file, count);
@@ -82,7 +168,7 @@ bool Page::serialize(FsFile& file) const {
return true;
}
std::unique_ptr<Page> Page::deserialize(FsFile& file) {
std::unique_ptr<Page> Page::deserialize(HalFile& file) {
auto page = std::unique_ptr<Page>(new Page());
uint16_t count;
@@ -94,10 +180,22 @@ std::unique_ptr<Page> Page::deserialize(FsFile& file) {
if (tag == TAG_PageLine) {
auto pl = PageLine::deserialize(file);
if (!pl) {
return nullptr;
}
page->elements.push_back(std::move(pl));
} else if (tag == TAG_PageImage) {
auto pi = PageImage::deserialize(file);
if (!pi) {
return nullptr;
}
page->elements.push_back(std::move(pi));
} else if (tag == TAG_PageHorizontalRule) {
auto rule = PageHorizontalRule::deserialize(file);
if (!rule) {
return nullptr;
}
page->elements.push_back(std::move(rule));
} else {
LOG_ERR("PGE", "Deserialization failed: Unknown tag %u", tag);
return nullptr;
+33 -8
View File
@@ -12,7 +12,8 @@
enum PageElementTag : uint8_t {
TAG_PageLine = 1,
TAG_PageImage = 2, // New tag
TAG_PageImage = 2,
TAG_PageHorizontalRule = 3,
};
// represents something that has been added to a page
@@ -23,7 +24,7 @@ class PageElement {
explicit PageElement(const int16_t xPos, const int16_t yPos) : xPos(xPos), yPos(yPos) {}
virtual ~PageElement() = default;
virtual void render(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) = 0;
virtual bool serialize(FsFile& file) = 0;
virtual bool serialize(HalFile& file) = 0;
virtual PageElementTag getTag() const = 0; // Add type identification
};
@@ -36,9 +37,9 @@ class PageLine final : public PageElement {
: PageElement(xPos, yPos), block(std::move(block)) {}
const std::shared_ptr<TextBlock>& getBlock() const { return block; }
void render(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) override;
bool serialize(FsFile& file) override;
bool serialize(HalFile& file) override;
PageElementTag getTag() const override { return TAG_PageLine; }
static std::unique_ptr<PageLine> deserialize(FsFile& file);
static std::unique_ptr<PageLine> deserialize(HalFile& file);
};
// New PageImage class
@@ -49,12 +50,27 @@ class PageImage final : public PageElement {
PageImage(std::shared_ptr<ImageBlock> block, const int16_t xPos, const int16_t yPos)
: PageElement(xPos, yPos), imageBlock(std::move(block)) {}
void render(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) override;
bool serialize(FsFile& file) override;
void renderPlaceholder(GfxRenderer& renderer, int xOffset, int yOffset) const;
bool serialize(HalFile& file) override;
PageElementTag getTag() const override { return TAG_PageImage; }
static std::unique_ptr<PageImage> deserialize(FsFile& file);
static std::unique_ptr<PageImage> deserialize(HalFile& file);
const ImageBlock& getImageBlock() const { return *imageBlock; }
};
class PageHorizontalRule final : public PageElement {
uint16_t width;
uint8_t thickness;
public:
PageHorizontalRule(uint16_t width, uint8_t thickness, const int16_t xPos, const int16_t yPos)
: PageElement(xPos, yPos), width(width), thickness(thickness) {}
void render(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) override;
bool serialize(HalFile& file) override;
PageElementTag getTag() const override { return TAG_PageHorizontalRule; }
static std::unique_ptr<PageHorizontalRule> deserialize(HalFile& file);
};
class Page {
public:
// the list of block index and line numbers on this page
@@ -73,8 +89,10 @@ class Page {
}
void render(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) const;
bool serialize(FsFile& file) const;
static std::unique_ptr<Page> deserialize(FsFile& file);
void renderImages(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) const;
void renderWithImagePlaceholders(GfxRenderer& renderer, int fontId, int xOffset, int yOffset) const;
bool serialize(HalFile& file) const;
static std::unique_ptr<Page> deserialize(HalFile& file);
// Check if page contains any images (used to force full refresh)
bool hasImages() const {
@@ -82,6 +100,13 @@ class Page {
[](const std::shared_ptr<PageElement>& el) { return el->getTag() == TAG_PageImage; });
}
bool hasImagesNeedingDecode() const {
return std::any_of(elements.begin(), elements.end(), [](const std::shared_ptr<PageElement>& element) {
return element->getTag() == TAG_PageImage &&
static_cast<const PageImage&>(*element).getImageBlock().needsDecode();
});
}
// Get bounding box of all images on the page (union of image rects)
// Returns false if no images. Coordinates are relative to page origin.
bool getImageBoundingBox(int16_t& outX, int16_t& outY, int16_t& outW, int16_t& outH) const {
+556 -150
View File
@@ -1,6 +1,8 @@
#include "ParsedText.h"
#include <BidiUtils.h>
#include <GfxRenderer.h>
#include <Logging.h>
#include <Utf8.h>
#include <algorithm>
@@ -18,6 +20,20 @@ namespace {
// Soft hyphen byte pattern used throughout EPUBs (UTF-8 for U+00AD).
constexpr char SOFT_HYPHEN_UTF8[] = "\xC2\xAD";
constexpr size_t SOFT_HYPHEN_BYTES = 2;
// Paragraph-level direction: scan the first N words to find base direction.
constexpr size_t RTL_PARAGRAPH_PROBE_WORDS = 3;
// Per-word: scan enough chars to see through leading neutrals (quotes, numbers)
// before giving up. 64 is a hedge for pathological cases like long numeric tokens.
constexpr int RTL_PER_WORD_PROBE_DEPTH = 64;
constexpr size_t MIN_JUSTIFY_GAPS = 1;
// Byte-level pre-check: Hebrew UTF-8 lead bytes 0xD6-0xD7, Arabic/Syriac 0xD8-0xDB.
bool mayContainRtlBytes(const char* str) {
for (const auto* p = reinterpret_cast<const unsigned char*>(str); *p; ++p) {
if (*p >= 0xD6 && *p <= 0xDB) return true;
}
return false;
}
// Returns the first rendered codepoint of a word (skipping leading soft hyphens).
uint32_t firstCodepoint(const std::string& word) {
@@ -43,6 +59,134 @@ uint32_t lastCodepoint(const std::string& word) {
bool containsSoftHyphen(const std::string& word) { return word.find(SOFT_HYPHEN_UTF8) != std::string::npos; }
bool isNoBreakBeforeCjkPunctuation(const uint32_t cp) {
switch (cp) {
case '.':
case ',':
case ':':
case ';':
case '!':
case '?':
case ')':
case ']':
case '}':
case 0x00BB: // »
case 0x2019: //
case 0x201D: // ”
case 0x3001: // 、
case 0x3002: // 。
case 0x3009: // 〉
case 0x300B: // 》
case 0x300D: // 」
case 0x300F: // 』
case 0x3011: // 】
case 0x3015: //
case 0x3017: // 〗
case 0x3019: // 〙
case 0x301B: // 〛
case 0xFF01: //
case 0xFF09: //
case 0xFF0C: //
case 0xFF0E: //
case 0xFF1A: //
case 0xFF1B: //
case 0xFF1F: //
case 0xFF3D: //
case 0xFF5D: //
return true;
default:
return false;
}
}
bool isNoBreakAfterCjkPunctuation(const uint32_t cp) {
switch (cp) {
case '(':
case '[':
case '{':
case 0x00AB: // «
case 0x2018: //
case 0x201C: // “
case 0x3008: // 〈
case 0x300A: // 《
case 0x300C: // 「
case 0x300E: // 『
case 0x3010: // 【
case 0x3014: //
case 0x3016: // 〖
case 0x3018: // 〘
case 0x301A: // 〚
case 0xFF08: //
case 0xFF3B: //
case 0xFF5B: //
return true;
default:
return false;
}
}
bool containsCjkBreakableCodepoint(const std::string& text) {
const auto* ptr = reinterpret_cast<const unsigned char*>(text.c_str());
while (*ptr) {
const uint32_t cp = utf8NextCodepoint(&ptr);
if (utf8IsCjkBreakable(cp)) {
return true;
}
}
return false;
}
bool hasCjkBreakOpportunityBetween(const uint32_t leftCp, const uint32_t rightCp) {
if (!utf8IsCjkBreakable(leftCp) && !utf8IsCjkBreakable(rightCp)) return false;
if (isNoBreakAfterCjkPunctuation(leftCp) || isNoBreakBeforeCjkPunctuation(rightCp)) return false;
if (utf8IsCombiningMark(rightCp)) return false;
return true;
}
std::vector<size_t> cjkCharacterBreakByteOffsets(const std::string& text) {
struct CodepointBoundary {
uint32_t cp;
size_t endOffset;
};
std::vector<CodepointBoundary> codepoints;
codepoints.reserve(text.size());
bool hasCjkBreakable = false;
const auto* ptr = reinterpret_cast<const unsigned char*>(text.c_str());
const auto* const start = ptr;
while (*ptr) {
const uint32_t cp = utf8NextCodepoint(&ptr);
if (cp == 0) break;
if (utf8IsCjkBreakable(cp)) {
hasCjkBreakable = true;
}
codepoints.push_back({cp, static_cast<size_t>(ptr - start)});
}
if (!hasCjkBreakable || codepoints.size() < 2) return {};
std::vector<size_t> allowedOffsets;
allowedOffsets.reserve(codepoints.size() - 1);
for (size_t i = 0; i + 1 < codepoints.size(); ++i) {
const uint32_t current = codepoints[i].cp;
const uint32_t next = codepoints[i + 1].cp;
if (!hasCjkBreakOpportunityBetween(current, next)) continue;
allowedOffsets.push_back(codepoints[i].endOffset);
}
return allowedOffsets;
}
int computeJustifyExtra(const int spareSpace, const size_t gapCount) {
if (gapCount < MIN_JUSTIFY_GAPS || spareSpace <= 0) return 0;
// Distribute the spare space evenly across gaps. Do NOT bail out to 0 when the
// per-gap stretch is large: a sparse line (few words on a wide page) legitimately
// needs big gaps to reach the margin. Returning 0 there disables justification for
// that line, leaving it right-aligned (RTL) / left-aligned (LTR) — the mismatched
// alignment bug. Match the un-capped behavior of the old code.
return spareSpace / static_cast<int>(gapCount);
}
// Removes every soft hyphen in-place so rendered glyphs match measured widths.
void stripSoftHyphensInPlace(std::string& word) {
size_t pos = 0;
@@ -111,53 +255,112 @@ void ParsedText::addWord(std::string word, const EpdFontFamily::Style fontStyle,
const bool attachToPrevious) {
if (word.empty()) return;
// The device fonts carry no combining-mark positioning, so EPUB text stored in NFD
// (a base letter followed by separate combining accents -- common for Vietnamese,
// and used for many EPUB <h1> chapter headings) renders with the marks detached or
// misplaced. Compose to NFC here, the single funnel every word passes through, so a
// precomposed glyph is used instead. This runs once per word at layout time (the
// result is cached in the section file) and is a cheap no-op for mark-free text.
word = utf8ComposeNfc(word);
EpdFontFamily::Style baseStyle = fontStyle;
if (underline) {
baseStyle = static_cast<EpdFontFamily::Style>(baseStyle | EpdFontFamily::UNDERLINE);
}
const bool wordStartsRtl = !hasRtlWord && mayContainRtlBytes(word.c_str()) &&
BidiUtils::startsWithRtl(word.c_str(), RTL_PER_WORD_PROBE_DEPTH);
// Already-bold text should stay fully bold; focus splitting would make its suffix regular later.
if (!this->focusReadingEnabled || (baseStyle & EpdFontFamily::BOLD) != 0) {
words.push_back(std::move(word));
const auto pushToken = [&](std::string token, const bool continues, const bool noSpaceBefore,
const bool isFocusSuffix) {
words.push_back(std::move(token));
wordStyles.push_back(baseStyle);
wordContinues.push_back(attachToPrevious);
wordIsFocusSuffix.push_back(false);
return;
wordContinues.push_back(continues);
wordNoSpaceBefore.push_back(noSpaceBefore);
wordIsFocusSuffix.push_back(isFocusSuffix);
};
bool effectiveAttachToPrevious = attachToPrevious;
bool effectiveNoSpaceBefore = false;
if (attachToPrevious && !words.empty() &&
hasCjkBreakOpportunityBetween(lastCodepoint(words.back()), firstCodepoint(word))) {
effectiveAttachToPrevious = false;
effectiveNoSpaceBefore = true;
}
// --- FOCUS READING LOGIC BELOW ---
const auto ensureTokenCapacity = [&](const size_t additionalTokens) {
if (additionalTokens == 0) return;
const size_t requiredSize = words.size() + additionalTokens;
if (words.capacity() >= requiredSize) return;
// Pre-reserve capacity to prevent mid-word heap reallocations.
size_t maxPossibleNewTokens = word.length();
size_t requiredSize = words.size() + maxPossibleNewTokens;
if (words.capacity() < requiredSize) {
// Emulate standard geometric growth (doubling) to ensure we don't reallocate on every word.
size_t newCapacity = words.capacity() * 2;
// Ensure the doubled capacity is actually enough for this specific word
if (newCapacity < requiredSize) {
newCapacity = requiredSize;
}
// Set a sensible minimum starting size so the first few words don't trigger tiny reallocations
size_t newCapacity = words.capacity();
if (newCapacity < 16) {
newCapacity = 16;
}
while (newCapacity < requiredSize) {
newCapacity *= 2;
}
words.reserve(newCapacity);
wordStyles.reserve(newCapacity);
wordContinues.reserve(newCapacity);
wordNoSpaceBefore.reserve(newCapacity);
wordIsFocusSuffix.reserve(newCapacity);
};
if (auto breakOffsets = cjkCharacterBreakByteOffsets(word); !breakOffsets.empty()) {
// CJK-heavy paragraphs can push hundreds of tiny tokens quickly when CSS toggles
// inline styles. Reserve once up front to avoid repeated vector growth reallocations.
ensureTokenCapacity(breakOffsets.size() + 1);
bool firstToken = true;
size_t tokenStart = 0;
for (const size_t breakOffset : breakOffsets) {
if (breakOffset <= tokenStart || breakOffset > word.size()) continue;
pushToken(word.substr(tokenStart, breakOffset - tokenStart), firstToken ? effectiveAttachToPrevious : false,
firstToken ? effectiveNoSpaceBefore : true, false);
firstToken = false;
tokenStart = breakOffset;
}
if (tokenStart < word.size()) {
pushToken(word.substr(tokenStart), firstToken ? effectiveAttachToPrevious : false,
firstToken ? effectiveNoSpaceBefore : true, false);
}
if (wordStartsRtl) {
hasRtlWord = true;
}
return;
}
if (containsCjkBreakableCodepoint(word)) {
pushToken(std::move(word), effectiveAttachToPrevious, effectiveNoSpaceBefore, false);
if (wordStartsRtl) {
hasRtlWord = true;
}
return;
}
// Already-bold text should stay fully bold; focus splitting would make its suffix regular later.
if (!this->focusReadingEnabled || (baseStyle & EpdFontFamily::BOLD) != 0) {
pushToken(std::move(word), effectiveAttachToPrevious, effectiveNoSpaceBefore, false);
if (wordStartsRtl) {
hasRtlWord = true;
}
return;
}
// --- FOCUS READING LOGIC BELOW ---
// Worst case: a segment boundary on each byte (highly punctuated UTF-8 text).
ensureTokenCapacity(word.length());
// Lambda helper to process and push individual sub-segments of the string
// Use std::string_view to avoid heap allocations when slicing
auto processSegment = [&](std::string_view segment, bool isWord, bool attach) {
auto processSegment = [&](std::string_view segment, bool isWord, bool attach, bool noSpaceBefore) {
if (!isWord) {
// Punctuation and Numbers stay regular
words.emplace_back(segment);
wordStyles.push_back(baseStyle);
wordContinues.push_back(attach);
wordNoSpaceBefore.push_back(noSpaceBefore);
wordIsFocusSuffix.push_back(false);
} else {
size_t charCount = 0;
@@ -179,6 +382,7 @@ void ParsedText::addWord(std::string word, const EpdFontFamily::Style fontStyle,
words.emplace_back(segment);
wordStyles.push_back(static_cast<EpdFontFamily::Style>(baseStyle | EpdFontFamily::BOLD));
wordContinues.push_back(attach);
wordNoSpaceBefore.push_back(noSpaceBefore);
wordIsFocusSuffix.push_back(false);
} else {
countPtr = reinterpret_cast<const unsigned char*>(segment.data());
@@ -191,12 +395,14 @@ void ParsedText::addWord(std::string word, const EpdFontFamily::Style fontStyle,
words.emplace_back(segment.substr(0, splitByteOffset));
wordStyles.push_back(static_cast<EpdFontFamily::Style>(baseStyle | EpdFontFamily::BOLD));
wordContinues.push_back(attach);
wordNoSpaceBefore.push_back(noSpaceBefore);
wordIsFocusSuffix.push_back(false);
// Regular suffix - marked so extractLine can merge it back into single TextBlock entry
words.emplace_back(segment.substr(splitByteOffset));
wordStyles.push_back(baseStyle);
wordContinues.push_back(true);
wordNoSpaceBefore.push_back(false);
wordIsFocusSuffix.push_back(true);
}
}
@@ -224,7 +430,8 @@ void ParsedText::addWord(std::string word, const EpdFontFamily::Style fontStyle,
// Only the very first segment inherits the original attachToPrevious flag.
// Every subsequent segment MUST attach=true so it glues seamlessly to the prefix.
processSegment(segment, inWordSegment, isFirstSegment ? attachToPrevious : true);
processSegment(segment, inWordSegment, isFirstSegment ? effectiveAttachToPrevious : true,
isFirstSegment ? effectiveNoSpaceBefore : false);
// Setup for the next segment
segmentStart = currentCpStart;
@@ -236,7 +443,27 @@ void ParsedText::addWord(std::string word, const EpdFontFamily::Style fontStyle,
// Process the final remaining segment
size_t segmentLen = end - segmentStart;
std::string_view segment(reinterpret_cast<const char*>(segmentStart), segmentLen);
processSegment(segment, inWordSegment, isFirstSegment ? attachToPrevious : true);
processSegment(segment, inWordSegment, isFirstSegment ? effectiveAttachToPrevious : true,
isFirstSegment ? effectiveNoSpaceBefore : false);
if (wordStartsRtl) {
hasRtlWord = true;
}
}
int ParsedText::resolveFirstLineIndent(const bool isFirstLine, const GfxRenderer& renderer, const int fontId) const {
if (!isFirstLine || !isNaturalAlign) {
return 0;
}
if (blockStyle.textIndentDefined) {
if (blockStyle.textIndent < 0 || !extraParagraphSpacing) {
return blockStyle.textIndent;
}
return 0;
}
if (!extraParagraphSpacing) {
return renderer.getSpaceWidth(fontId, EpdFontFamily::REGULAR) * 3;
}
return 0;
}
// Consumes data to minimize memory usage
void ParsedText::layoutAndExtractLines(const GfxRenderer& renderer, const int fontId, const uint16_t viewportWidth,
@@ -246,8 +473,22 @@ void ParsedText::layoutAndExtractLines(const GfxRenderer& renderer, const int fo
return;
}
// Apply fixed transforms before any per-line layout work.
applyParagraphIndent();
// Per-paragraph RTL auto-detection: only when CSS/HTML didn't explicitly set direction.
// Explicit dir="ltr" must be respected and not overridden by content heuristic.
if (!blockStyle.directionDefined && hasRtlWord) {
// Check the first few words for RTL letter codepoints (no heap allocation).
const size_t wordsToScan = std::min(words.size(), RTL_PARAGRAPH_PROBE_WORDS);
for (size_t i = 0; i < wordsToScan; ++i) {
if (BidiUtils::startsWithRtl(words[i].c_str(), BidiUtils::RTL_PARAGRAPH_PROBE_DEPTH)) {
blockStyle.isRtl = true;
break;
}
}
}
isNaturalAlign =
blockStyle.alignment == CssTextAlign::Justify ||
(blockStyle.isRtl ? blockStyle.alignment == CssTextAlign::Right : blockStyle.alignment == CssTextAlign::Left);
// Ensure SD card font glyph metrics are loaded before measuring word widths.
// For flash-based fonts isSdCardFont() returns false and this block is skipped
@@ -255,20 +496,6 @@ void ParsedText::layoutAndExtractLines(const GfxRenderer& renderer, const int fo
// (advanceX only, no bitmaps) for all unique codepoints in this paragraph so
// that calculateWordWidths() can measure text without on-demand SD I/O.
if (renderer.isSdCardFont(fontId)) {
// Reserve upfront so the joined text allocates exactly once. Without this,
// paragraphs with many words trigger a chain of vector-like reallocations
// inside std::string during layout — visible in prewarm timings for SD fonts.
size_t totalSize = hyphenationEnabled ? 1 : 0;
if (!words.empty()) totalSize += words.size() - 1; // inter-word spaces
for (const auto& w : words) totalSize += w.size();
std::string allText;
allText.reserve(totalSize);
for (size_t i = 0; i < words.size(); i++) {
if (i > 0) allText += ' ';
allText += words[i];
}
if (hyphenationEnabled) allText += '-';
// Style mask: only ask the SD font to load advances for styles actually
// used in this paragraph. Style index is the low two bits (regular/bold/
// italic/bold-italic); the underline bit is irrelevant to advance metrics.
@@ -277,7 +504,7 @@ void ParsedText::layoutAndExtractLines(const GfxRenderer& renderer, const int fo
styleMask |= static_cast<uint8_t>(1u << (static_cast<uint8_t>(s) & 0x03));
}
if (styleMask == 0) styleMask = 0x01; // defensive: regular only
renderer.ensureSdCardFontReady(fontId, allText.c_str(), styleMask);
renderer.ensureSdCardFontReady(fontId, words, hyphenationEnabled, styleMask);
}
const int pageWidth = viewportWidth;
@@ -286,14 +513,16 @@ void ParsedText::layoutAndExtractLines(const GfxRenderer& renderer, const int fo
std::vector<size_t> lineBreakIndices;
if (hyphenationEnabled) {
// Use greedy layout that can split words mid-loop when a hyphenated prefix fits.
lineBreakIndices = computeHyphenatedLineBreaks(renderer, fontId, pageWidth, wordWidths, wordContinues);
lineBreakIndices =
computeHyphenatedLineBreaks(renderer, fontId, pageWidth, wordWidths, wordContinues, wordNoSpaceBefore);
} else {
lineBreakIndices = computeLineBreaks(renderer, fontId, pageWidth, wordWidths, wordContinues);
lineBreakIndices = computeLineBreaks(renderer, fontId, pageWidth, wordWidths, wordContinues, wordNoSpaceBefore);
}
const size_t lineCount = includeLastLine ? lineBreakIndices.size() : lineBreakIndices.size() - 1;
for (size_t i = 0; i < lineCount; ++i) {
extractLine(i, pageWidth, wordWidths, wordContinues, lineBreakIndices, processLine, renderer, fontId);
extractLine(i, pageWidth, wordWidths, wordContinues, wordNoSpaceBefore, lineBreakIndices, processLine, renderer,
fontId);
}
// Remove consumed words so size() reflects only remaining words
@@ -302,6 +531,7 @@ void ParsedText::layoutAndExtractLines(const GfxRenderer& renderer, const int fo
words.erase(words.begin(), words.begin() + consumed);
wordStyles.erase(wordStyles.begin(), wordStyles.begin() + consumed);
wordContinues.erase(wordContinues.begin(), wordContinues.begin() + consumed);
wordNoSpaceBefore.erase(wordNoSpaceBefore.begin(), wordNoSpaceBefore.begin() + consumed);
wordIsFocusSuffix.erase(wordIsFocusSuffix.begin(), wordIsFocusSuffix.begin() + consumed);
}
}
@@ -318,20 +548,13 @@ std::vector<uint16_t> ParsedText::calculateWordWidths(const GfxRenderer& rendere
}
std::vector<size_t> ParsedText::computeLineBreaks(const GfxRenderer& renderer, const int fontId, const int pageWidth,
std::vector<uint16_t>& wordWidths, std::vector<bool>& continuesVec) {
std::vector<uint16_t>& wordWidths, std::vector<bool>& continuesVec,
std::vector<bool>& noSpaceBeforeVec) {
if (words.empty()) {
return {};
}
// Calculate first line indent (only for left/justified text).
// Positive text-indent (paragraph indent) is suppressed when extraParagraphSpacing is on.
// Negative text-indent (hanging indent, e.g. margin-left:3em; text-indent:-1em) always applies —
// it is structural (positions the bullet/marker), not decorative.
const int firstLineIndent =
blockStyle.textIndentDefined && (blockStyle.textIndent < 0 || !extraParagraphSpacing) &&
(blockStyle.alignment == CssTextAlign::Justify || blockStyle.alignment == CssTextAlign::Left)
? blockStyle.textIndent
: 0;
const int firstLineIndent = resolveFirstLineIndent(true, renderer, fontId);
// Ensure any word that would overflow even as the first entry on a line is split using fallback hyphenation.
for (size_t i = 0; i < wordWidths.size(); ++i) {
@@ -365,7 +588,9 @@ std::vector<size_t> ParsedText::computeLineBreaks(const GfxRenderer& renderer, c
for (size_t j = i; j < totalWordCount; ++j) {
// Add space before word j, unless it's the first word on the line or a continuation
int gap = 0;
if (j > static_cast<size_t>(i) && !continuesVec[j]) {
if (j > static_cast<size_t>(i) && noSpaceBeforeVec[j]) {
gap = 0;
} else if (j > static_cast<size_t>(i) && !continuesVec[j]) {
gap =
renderer.getSpaceAdvance(fontId, lastCodepoint(words[j - 1]), firstCodepoint(words[j]), wordStyles[j - 1]);
} else if (j > static_cast<size_t>(i) && continuesVec[j]) {
@@ -437,33 +662,12 @@ std::vector<size_t> ParsedText::computeLineBreaks(const GfxRenderer& renderer, c
return lineBreakIndices;
}
void ParsedText::applyParagraphIndent() {
if (extraParagraphSpacing || words.empty()) {
return;
}
if (blockStyle.textIndentDefined) {
// CSS text-indent is explicitly set (even if 0) - don't use fallback EmSpace
// The actual indent positioning is handled in extractLine()
} else if (blockStyle.alignment == CssTextAlign::Justify || blockStyle.alignment == CssTextAlign::Left) {
// No CSS text-indent defined - use EmSpace fallback for visual indent
words.front().insert(0, "\xe2\x80\x83");
}
}
// Builds break indices while opportunistically splitting the word that would overflow the current line.
std::vector<size_t> ParsedText::computeHyphenatedLineBreaks(const GfxRenderer& renderer, const int fontId,
const int pageWidth, std::vector<uint16_t>& wordWidths,
std::vector<bool>& continuesVec) {
// Calculate first line indent (only for left/justified text).
// Positive text-indent (paragraph indent) is suppressed when extraParagraphSpacing is on.
// Negative text-indent (hanging indent, e.g. margin-left:3em; text-indent:-1em) always applies —
// it is structural (positions the bullet/marker), not decorative.
const int firstLineIndent =
blockStyle.textIndentDefined && (blockStyle.textIndent < 0 || !extraParagraphSpacing) &&
(blockStyle.alignment == CssTextAlign::Justify || blockStyle.alignment == CssTextAlign::Left)
? blockStyle.textIndent
: 0;
std::vector<bool>& continuesVec,
std::vector<bool>& noSpaceBeforeVec) {
const int firstLineIndent = resolveFirstLineIndent(true, renderer, fontId);
std::vector<size_t> lineBreakIndices;
size_t currentIndex = 0;
@@ -480,7 +684,9 @@ std::vector<size_t> ParsedText::computeHyphenatedLineBreaks(const GfxRenderer& r
while (currentIndex < wordWidths.size()) {
const bool isFirstWord = currentIndex == lineStart;
int spacing = 0;
if (!isFirstWord && !continuesVec[currentIndex]) {
if (!isFirstWord && noSpaceBeforeVec[currentIndex]) {
spacing = 0;
} else if (!isFirstWord && !continuesVec[currentIndex]) {
spacing = renderer.getSpaceAdvance(fontId, lastCodepoint(words[currentIndex - 1]),
firstCodepoint(words[currentIndex]), wordStyles[currentIndex - 1]);
} else if (!isFirstWord && continuesVec[currentIndex]) {
@@ -610,6 +816,7 @@ bool ParsedText::hyphenateWordAtIndex(const size_t wordIndex, const int availabl
// line, while "kilometer" moves to the next line.
// wordContinues[wordIndex] is intentionally left unchanged — the prefix keeps its original attachment.
wordContinues.insert(wordContinues.begin() + wordIndex + 1, false);
wordNoSpaceBefore.insert(wordNoSpaceBefore.begin() + wordIndex + 1, false);
// Update cached widths to reflect the new prefix/remainder pairing.
wordWidths[wordIndex] = static_cast<uint16_t>(chosenWidth);
@@ -619,23 +826,30 @@ bool ParsedText::hyphenateWordAtIndex(const size_t wordIndex, const int availabl
}
void ParsedText::extractLine(const size_t breakIndex, const int pageWidth, const std::vector<uint16_t>& wordWidths,
const std::vector<bool>& continuesVec, const std::vector<size_t>& lineBreakIndices,
const std::vector<bool>& continuesVec, const std::vector<bool>& noSpaceBeforeVec,
const std::vector<size_t>& lineBreakIndices,
const std::function<void(std::shared_ptr<TextBlock>)>& processLine,
const GfxRenderer& renderer, const int fontId) {
const size_t lineBreak = lineBreakIndices[breakIndex];
const size_t lastBreakAt = breakIndex > 0 ? lineBreakIndices[breakIndex - 1] : 0;
const size_t lineWordCount = lineBreak - lastBreakAt;
// Calculate first line indent (only for left/justified text).
// Positive text-indent (paragraph indent) is suppressed when extraParagraphSpacing is on.
// Negative text-indent (hanging indent, e.g. margin-left:3em; text-indent:-1em) always applies —
// it is structural (positions the bullet/marker), not decorative.
const bool isFirstLine = breakIndex == 0;
const int firstLineIndent =
isFirstLine && blockStyle.textIndentDefined && (blockStyle.textIndent < 0 || !extraParagraphSpacing) &&
(blockStyle.alignment == CssTextAlign::Justify || blockStyle.alignment == CssTextAlign::Left)
? blockStyle.textIndent
: 0;
const int firstLineIndent = resolveFirstLineIndent(breakIndex == 0, renderer, fontId);
// Build line data by moving from the original vectors using index range
std::vector<std::string> lineWords;
lineWords.reserve(lineWordCount);
std::vector<EpdFontFamily::Style> lineWordStyles;
lineWordStyles.reserve(lineWordCount);
for (size_t i = 0; i < lineWordCount; ++i) {
std::string word = std::move(words[lastBreakAt + i]);
if (containsSoftHyphen(word)) {
stripSoftHyphensInPlace(word);
}
lineWords.push_back(std::move(word));
lineWordStyles.push_back(wordStyles[lastBreakAt + i]);
}
// Calculate total word width for this line, count actual word gaps,
// and accumulate total natural gap widths (including space kerning adjustments).
@@ -646,21 +860,23 @@ void ParsedText::extractLine(const size_t breakIndex, const int pageWidth, const
for (size_t wordIdx = 0; wordIdx < lineWordCount; wordIdx++) {
lineWordWidthSum += wordWidths[lastBreakAt + wordIdx];
// Count gaps: each word after the first creates a gap, unless it's a continuation
if (wordIdx > 0 && !continuesVec[lastBreakAt + wordIdx]) {
if (wordIdx > 0 && noSpaceBeforeVec[lastBreakAt + wordIdx]) {
// Unicode break opportunity with no inserted Latin-style space. It is still
// a stretchable gap for justified CJK/Korean text.
actualGapCount++;
totalNaturalGaps +=
renderer.getSpaceAdvance(fontId, lastCodepoint(words[lastBreakAt + wordIdx - 1]),
firstCodepoint(words[lastBreakAt + wordIdx]), wordStyles[lastBreakAt + wordIdx - 1]);
} else if (wordIdx > 0 && !continuesVec[lastBreakAt + wordIdx]) {
actualGapCount++;
totalNaturalGaps += renderer.getSpaceAdvance(fontId, lastCodepoint(lineWords[wordIdx - 1]),
firstCodepoint(lineWords[wordIdx]), lineWordStyles[wordIdx - 1]);
} else if (wordIdx > 0 && continuesVec[lastBreakAt + wordIdx]) {
// Non-breaking space tokens (" " with continues=true) are visible, stretchable spaces —
// count them as justifiable gaps so justifyExtra is distributed to them too.
if (words[lastBreakAt + wordIdx] == " ") {
if (lineWords[wordIdx] == " ") {
actualGapCount++;
}
// Cross-boundary kerning for continuation words (e.g. nonbreaking spaces, attached punctuation)
totalNaturalGaps +=
renderer.getKerning(fontId, lastCodepoint(words[lastBreakAt + wordIdx - 1]),
firstCodepoint(words[lastBreakAt + wordIdx]), wordStyles[lastBreakAt + wordIdx - 1]);
totalNaturalGaps += renderer.getKerning(fontId, lastCodepoint(lineWords[wordIdx - 1]),
firstCodepoint(lineWords[wordIdx]), lineWordStyles[wordIdx - 1]);
}
}
@@ -668,81 +884,266 @@ void ParsedText::extractLine(const size_t breakIndex, const int pageWidth, const
const int effectivePageWidth = pageWidth - firstLineIndent;
const bool isLastLine = breakIndex == lineBreakIndices.size() - 1;
// For RTL, implicit/default Left alignment becomes Right alignment.
// Explicit text-align:left must remain left for CSS correctness.
const CssTextAlign effectiveAlignment =
(blockStyle.isRtl && !blockStyle.textAlignDefined && blockStyle.alignment == CssTextAlign::Left)
? CssTextAlign::Right
: blockStyle.alignment;
// For justified text, compute per-gap extra to distribute remaining space evenly
const int spareSpace = effectivePageWidth - lineWordWidthSum - totalNaturalGaps;
const int justifyExtra = (blockStyle.alignment == CssTextAlign::Justify && !isLastLine && actualGapCount >= 1)
? spareSpace / static_cast<int>(actualGapCount)
const int justifyExtra = (effectiveAlignment == CssTextAlign::Justify && !isLastLine)
? computeJustifyExtra(spareSpace, actualGapCount)
: 0;
// Calculate initial x position (first line starts at indent for left/justified text;
// may be negative for hanging indents, e.g. margin-left:3em; text-indent:-1em).
auto xpos = static_cast<int16_t>(firstLineIndent);
if (blockStyle.alignment == CssTextAlign::Right) {
xpos = effectivePageWidth - lineWordWidthSum - totalNaturalGaps;
} else if (blockStyle.alignment == CssTextAlign::Center) {
xpos = (effectivePageWidth - lineWordWidthSum - totalNaturalGaps) / 2;
}
// BiDi processing: reorder words with UAX#9 in full-line context.
visualOrderScratch.clear();
visualOrderScratch.reserve(lineWordCount);
// Skip expensive visual-order resolution for pure LTR paragraphs that have no RTL words.
const bool shouldResolveVisualOrder = blockStyle.isRtl || hasRtlWord;
const bool willReorder =
shouldResolveVisualOrder && BidiUtils::computeVisualWordOrder(lineWords, blockStyle.isRtl, visualOrderScratch);
// Pre-calculate X positions for words
// Continuation words attach to the previous word with no space before them
std::vector<int16_t> lineXPos;
lineXPos.reserve(lineWordCount);
for (size_t wordIdx = 0; wordIdx < lineWordCount; wordIdx++) {
lineXPos.push_back(xpos);
if (willReorder) {
reorderedWordsScratch.clear();
reorderedStylesScratch.clear();
reorderedWidthsScratch.clear();
reorderedContinuesScratch.clear();
reorderedNoSpaceBeforeScratch.clear();
reorderedFocusSuffixScratch.clear();
reorderedWordsScratch.reserve(visualOrderScratch.size());
reorderedStylesScratch.reserve(visualOrderScratch.size());
reorderedWidthsScratch.reserve(visualOrderScratch.size());
reorderedContinuesScratch.reserve(visualOrderScratch.size());
reorderedNoSpaceBeforeScratch.reserve(visualOrderScratch.size());
reorderedFocusSuffixScratch.reserve(visualOrderScratch.size());
const bool nextIsContinuation = wordIdx + 1 < lineWordCount && continuesVec[lastBreakAt + wordIdx + 1];
if (nextIsContinuation) {
int advance = wordWidths[lastBreakAt + wordIdx];
// Cross-boundary kerning for continuation words (e.g. nonbreaking spaces, attached punctuation)
advance +=
renderer.getKerning(fontId, lastCodepoint(words[lastBreakAt + wordIdx]),
firstCodepoint(words[lastBreakAt + wordIdx + 1]), wordStyles[lastBreakAt + wordIdx]);
// Non-breaking space tokens are stretchable — expand them during justification like normal spaces.
if (words[lastBreakAt + wordIdx] == " " && continuesVec[lastBreakAt + wordIdx] &&
blockStyle.alignment == CssTextAlign::Justify && !isLastLine) {
advance += justifyExtra;
for (size_t i = 0; i < visualOrderScratch.size(); ++i) {
const uint16_t src = visualOrderScratch[i];
reorderedWordsScratch.push_back(std::move(lineWords[src]));
reorderedStylesScratch.push_back(lineWordStyles[src]);
reorderedWidthsScratch.push_back(wordWidths[lastBreakAt + src]);
reorderedFocusSuffixScratch.push_back(wordIsFocusSuffix[lastBreakAt + src]);
// Continuation means "no break/gap between two adjacent logical tokens".
// After visual reordering (common in RTL), an adjacent logical pair can appear
// as either (prev -> curr) or (curr -> prev) in visual order; preserve both.
bool continues = false;
if (i > 0) {
const size_t prevSrc = visualOrderScratch[i - 1];
const size_t currSrc = src;
const bool forwardAdjacent = currSrc == prevSrc + 1;
const bool reverseAdjacent = prevSrc == currSrc + 1;
if (forwardAdjacent && continuesVec[lastBreakAt + currSrc]) {
continues = true;
} else if (reverseAdjacent && continuesVec[lastBreakAt + prevSrc]) {
continues = true;
}
}
reorderedContinuesScratch.push_back(continues);
reorderedNoSpaceBeforeScratch.push_back(!continues && noSpaceBeforeVec[lastBreakAt + src]);
}
int reorderedWordWidthSum = 0;
size_t reorderedGapCount = 0;
int reorderedNaturalGaps = 0;
for (size_t wordIdx = 0; wordIdx < reorderedWidthsScratch.size(); wordIdx++) {
reorderedWordWidthSum += reorderedWidthsScratch[wordIdx];
if (wordIdx > 0 && reorderedNoSpaceBeforeScratch[wordIdx]) {
// Unicode break opportunity with no inserted Latin-style space. It is still
// a stretchable gap for justified CJK/Korean text.
reorderedGapCount++;
} else if (wordIdx > 0 && !reorderedContinuesScratch[wordIdx]) {
reorderedGapCount++;
reorderedNaturalGaps += renderer.getSpaceAdvance(fontId, lastCodepoint(reorderedWordsScratch[wordIdx - 1]),
firstCodepoint(reorderedWordsScratch[wordIdx]),
reorderedStylesScratch[wordIdx - 1]);
} else if (wordIdx > 0 && reorderedContinuesScratch[wordIdx]) {
if (reorderedWordsScratch[wordIdx] == " ") {
reorderedGapCount++;
}
reorderedNaturalGaps +=
renderer.getKerning(fontId, lastCodepoint(reorderedWordsScratch[wordIdx - 1]),
firstCodepoint(reorderedWordsScratch[wordIdx]), reorderedStylesScratch[wordIdx - 1]);
}
}
const int reorderedSpare = effectivePageWidth - reorderedWordWidthSum - reorderedNaturalGaps;
const int reorderedJustifyExtra = (effectiveAlignment == CssTextAlign::Justify && !isLastLine)
? computeJustifyExtra(reorderedSpare, reorderedGapCount)
: 0;
const int justifyContribution = (effectiveAlignment == CssTextAlign::Justify && !isLastLine)
? reorderedJustifyExtra * static_cast<int>(reorderedGapCount)
: 0;
const int contentWidth = reorderedWordWidthSum + reorderedNaturalGaps + justifyContribution;
int xpos = 0;
if (blockStyle.isRtl) {
if (effectiveAlignment == CssTextAlign::Right || effectiveAlignment == CssTextAlign::Justify) {
xpos = effectivePageWidth - contentWidth;
} else if (effectiveAlignment == CssTextAlign::Center) {
xpos = (effectivePageWidth - contentWidth) / 2;
}
xpos += advance;
} else {
int gap = 0;
if (wordIdx + 1 < lineWordCount) {
gap = renderer.getSpaceAdvance(fontId, lastCodepoint(words[lastBreakAt + wordIdx]),
firstCodepoint(words[lastBreakAt + wordIdx + 1]),
wordStyles[lastBreakAt + wordIdx]);
xpos = firstLineIndent;
if (effectiveAlignment == CssTextAlign::Right) {
xpos = effectivePageWidth - contentWidth;
} else if (effectiveAlignment == CssTextAlign::Center) {
xpos = (effectivePageWidth - contentWidth) / 2;
}
if (blockStyle.alignment == CssTextAlign::Justify && !isLastLine) {
gap += justifyExtra;
}
for (size_t wordIdx = 0; wordIdx < reorderedWidthsScratch.size(); wordIdx++) {
lineXPos.push_back(static_cast<int16_t>(xpos));
xpos += reorderedWidthsScratch[wordIdx];
const bool nextIsContinuation =
wordIdx + 1 < reorderedWidthsScratch.size() && reorderedContinuesScratch[wordIdx + 1];
if (nextIsContinuation) {
int advance =
renderer.getKerning(fontId, lastCodepoint(reorderedWordsScratch[wordIdx]),
firstCodepoint(reorderedWordsScratch[wordIdx + 1]), reorderedStylesScratch[wordIdx]);
// wordIdx > 0 mirrors the gap accounting above (which skips index 0): a leading
// no-break space must not receive justifyExtra, or the line over-stretches by one
// gap and the last word is pushed past the right margin (issue #2185).
if (wordIdx > 0 && reorderedWordsScratch[wordIdx] == " " && reorderedContinuesScratch[wordIdx] &&
effectiveAlignment == CssTextAlign::Justify && !isLastLine) {
advance += reorderedJustifyExtra;
}
xpos += advance;
} else if (wordIdx + 1 < reorderedWidthsScratch.size()) {
const bool nextNoSpace = reorderedNoSpaceBeforeScratch[wordIdx + 1];
int gap = nextNoSpace ? 0
: renderer.getSpaceAdvance(fontId, lastCodepoint(reorderedWordsScratch[wordIdx]),
firstCodepoint(reorderedWordsScratch[wordIdx + 1]),
reorderedStylesScratch[wordIdx]);
if (effectiveAlignment == CssTextAlign::Justify && !isLastLine) {
gap += reorderedJustifyExtra;
}
xpos += gap;
}
}
lineWords.swap(reorderedWordsScratch);
lineWordStyles.swap(reorderedStylesScratch);
} else {
// Standard LTR/RTL positioning loop when no visual reordering is needed
if (blockStyle.isRtl) {
// RTL: position words from right to left
int xpos = effectivePageWidth;
if (effectiveAlignment == CssTextAlign::Left) {
// Explicit left alignment in RTL context
xpos = lineWordWidthSum + totalNaturalGaps;
} else if (effectiveAlignment == CssTextAlign::Center) {
xpos = (effectivePageWidth + lineWordWidthSum + totalNaturalGaps) / 2;
}
// For Right and Justify, start from right edge (xpos = effectivePageWidth)
for (size_t wordIdx = 0; wordIdx < lineWordCount; wordIdx++) {
xpos -= wordWidths[lastBreakAt + wordIdx];
lineXPos.push_back(static_cast<int16_t>(xpos));
const bool nextIsContinuation = wordIdx + 1 < lineWordCount && continuesVec[lastBreakAt + wordIdx + 1];
if (nextIsContinuation) {
// Cross-boundary kerning for continuation words
int advance = renderer.getKerning(fontId, lastCodepoint(lineWords[wordIdx]),
firstCodepoint(lineWords[wordIdx + 1]), lineWordStyles[wordIdx]);
// wordIdx > 0: see the LTR branch — a leading no-break space is not a justifiable gap.
if (wordIdx > 0 && lineWords[wordIdx] == " " && continuesVec[lastBreakAt + wordIdx] &&
effectiveAlignment == CssTextAlign::Justify && !isLastLine) {
advance += justifyExtra;
}
xpos -= advance;
} else {
int gap = 0;
bool nextNoSpace = false;
if (wordIdx + 1 < lineWordCount) {
nextNoSpace = noSpaceBeforeVec[lastBreakAt + wordIdx + 1];
gap = nextNoSpace
? 0
: renderer.getSpaceAdvance(fontId, lastCodepoint(lineWords[wordIdx]),
firstCodepoint(lineWords[wordIdx + 1]), lineWordStyles[wordIdx]);
}
if (wordIdx + 1 < lineWordCount && effectiveAlignment == CssTextAlign::Justify && !isLastLine) {
gap += justifyExtra;
}
xpos -= gap;
}
}
} else {
// LTR: position words from left to right
int xpos = firstLineIndent;
if (effectiveAlignment == CssTextAlign::Right) {
xpos = effectivePageWidth - lineWordWidthSum - totalNaturalGaps;
} else if (effectiveAlignment == CssTextAlign::Center) {
xpos = (effectivePageWidth - lineWordWidthSum - totalNaturalGaps) / 2;
}
for (size_t wordIdx = 0; wordIdx < lineWordCount; wordIdx++) {
lineXPos.push_back(static_cast<int16_t>(xpos));
const bool nextIsContinuation = wordIdx + 1 < lineWordCount && continuesVec[lastBreakAt + wordIdx + 1];
if (nextIsContinuation) {
int advance = wordWidths[lastBreakAt + wordIdx];
advance += renderer.getKerning(fontId, lastCodepoint(lineWords[wordIdx]),
firstCodepoint(lineWords[wordIdx + 1]), lineWordStyles[wordIdx]);
// wordIdx > 0 mirrors the gap accounting above (which skips index 0): a leading
// no-break space must not receive justifyExtra, or the line over-stretches by one
// gap and the last word is pushed past the right margin (issue #2185).
if (wordIdx > 0 && lineWords[wordIdx] == " " && continuesVec[lastBreakAt + wordIdx] &&
effectiveAlignment == CssTextAlign::Justify && !isLastLine) {
advance += justifyExtra;
}
xpos += advance;
} else {
int gap = 0;
bool nextNoSpace = false;
if (wordIdx + 1 < lineWordCount) {
nextNoSpace = noSpaceBeforeVec[lastBreakAt + wordIdx + 1];
gap = nextNoSpace
? 0
: renderer.getSpaceAdvance(fontId, lastCodepoint(lineWords[wordIdx]),
firstCodepoint(lineWords[wordIdx + 1]), lineWordStyles[wordIdx]);
}
if (wordIdx + 1 < lineWordCount && effectiveAlignment == CssTextAlign::Justify && !isLastLine) {
gap += justifyExtra;
}
xpos += wordWidths[lastBreakAt + wordIdx] + gap;
}
}
xpos += wordWidths[lastBreakAt + wordIdx] + gap;
}
}
// Build line data by moving from the original vectors using index range
std::vector<std::string> lineWords(std::make_move_iterator(words.begin() + lastBreakAt),
std::make_move_iterator(words.begin() + lineBreak));
std::vector<EpdFontFamily::Style> lineWordStyles(wordStyles.begin() + lastBreakAt, wordStyles.begin() + lineBreak);
for (auto& word : lineWords) {
if (containsSoftHyphen(word)) {
stripSoftHyphensInPlace(word);
}
}
const auto isFocusSuffixAt = [&](const size_t idx) {
return willReorder ? reorderedFocusSuffixScratch[idx] : wordIsFocusSuffix[lastBreakAt + idx];
};
// Fast path: when no word on this line was split for focus reading, skip the merge work
// entirely and pass empty boundary/suffixX vectors. TextBlock pays zero per-word RAM cost
// for these annotations when the vectors are empty.
bool lineHasFocusSplit = false;
for (size_t i = 0; i < lineWordCount; i++) {
if (wordIsFocusSuffix[lastBreakAt + i]) {
if (isFocusSuffixAt(i)) {
lineHasFocusSplit = true;
break;
}
}
if (!lineHasFocusSplit) {
processLine(std::make_shared<TextBlock>(std::move(lineWords), std::move(lineXPos), std::move(lineWordStyles),
std::vector<uint8_t>{}, std::vector<uint16_t>{}, blockStyle));
// TextBlock flattens the vectors into its arena; they stay owned here and die at return.
auto block = std::make_shared<TextBlock>(lineWords, lineXPos, lineWordStyles, std::vector<uint8_t>{},
std::vector<uint16_t>{}, blockStyle);
if (!block->valid()) {
LOG_ERR("PTX", "Dropping line: TextBlock arena allocation failed");
return;
}
processLine(std::move(block));
return;
}
@@ -761,17 +1162,18 @@ void ParsedText::extractLine(const size_t breakIndex, const int pageWidth, const
outSuffixX.reserve(lineWordCount);
for (size_t i = 0; i < lineWordCount; i++) {
if (wordIsFocusSuffix[lastBreakAt + i] && !outWords.empty()) {
if (isFocusSuffixAt(i) && !outWords.empty()) {
// Focus suffix: merge string into the preceding bold-prefix entry.
outWords.back() += lineWords[i];
} else {
// Normal word: check for a following focus suffix to record the byte boundary.
uint8_t boundary = 0;
uint16_t suffixX = 0;
if (i + 1 < lineWordCount && wordIsFocusSuffix[lastBreakAt + i + 1]) {
if (i + 1 < lineWordCount && isFocusSuffixAt(i + 1)) {
boundary = static_cast<uint8_t>(std::min(lineWords[i].size(), size_t{255}));
// Suffix x offset = layout-time advance of the bold prefix, already known from xpos table.
suffixX = static_cast<uint16_t>(lineXPos[i + 1] - lineXPos[i]);
const int suffixDelta = static_cast<int>(lineXPos[i + 1]) - static_cast<int>(lineXPos[i]);
suffixX = static_cast<uint16_t>(suffixDelta > 0 ? suffixDelta : 0);
}
outWords.push_back(std::move(lineWords[i]));
outXPos.push_back(lineXPos[i]);
@@ -786,6 +1188,10 @@ void ParsedText::extractLine(const size_t breakIndex, const int pageWidth, const
}
}
processLine(std::make_shared<TextBlock>(std::move(outWords), std::move(outXPos), std::move(outStyles),
std::move(outBoundaries), std::move(outSuffixX), blockStyle));
auto block = std::make_shared<TextBlock>(outWords, outXPos, outStyles, outBoundaries, outSuffixX, blockStyle);
if (!block->valid()) {
LOG_ERR("PTX", "Dropping line: TextBlock arena allocation failed");
return;
}
processLine(std::move(block));
}
+21 -6
View File
@@ -15,22 +15,35 @@ class GfxRenderer;
class ParsedText {
std::vector<std::string> words;
std::vector<EpdFontFamily::Style> wordStyles;
std::vector<bool> wordContinues; // true = word attaches to previous (no space before it)
std::vector<bool> wordContinues; // true = word attaches to previous with no break
std::vector<bool> wordNoSpaceBefore; // true = may break before token, but no synthetic space when joined
std::vector<bool> wordIsFocusSuffix; // true = token is the regular tail of a focus bold-prefix split
BlockStyle blockStyle;
bool extraParagraphSpacing;
bool hyphenationEnabled;
bool focusReadingEnabled;
bool isNaturalAlign;
bool hasRtlWord;
std::vector<std::string> reorderedWordsScratch;
std::vector<EpdFontFamily::Style> reorderedStylesScratch;
std::vector<uint16_t> reorderedWidthsScratch;
std::vector<bool> reorderedContinuesScratch;
std::vector<bool> reorderedNoSpaceBeforeScratch;
std::vector<bool> reorderedFocusSuffixScratch;
std::vector<uint16_t> visualOrderScratch;
void applyParagraphIndent();
int resolveFirstLineIndent(bool isFirstLine, const GfxRenderer& renderer, int fontId) const;
std::vector<size_t> computeLineBreaks(const GfxRenderer& renderer, int fontId, int pageWidth,
std::vector<uint16_t>& wordWidths, std::vector<bool>& continuesVec);
std::vector<uint16_t>& wordWidths, std::vector<bool>& continuesVec,
std::vector<bool>& noSpaceBeforeVec);
std::vector<size_t> computeHyphenatedLineBreaks(const GfxRenderer& renderer, int fontId, int pageWidth,
std::vector<uint16_t>& wordWidths, std::vector<bool>& continuesVec);
std::vector<uint16_t>& wordWidths, std::vector<bool>& continuesVec,
std::vector<bool>& noSpaceBeforeVec);
bool hyphenateWordAtIndex(size_t wordIndex, int availableWidth, const GfxRenderer& renderer, int fontId,
std::vector<uint16_t>& wordWidths, bool allowFallbackBreaks);
void extractLine(size_t breakIndex, int pageWidth, const std::vector<uint16_t>& wordWidths,
const std::vector<bool>& continuesVec, const std::vector<size_t>& lineBreakIndices,
const std::vector<bool>& continuesVec, const std::vector<bool>& noSpaceBeforeVec,
const std::vector<size_t>& lineBreakIndices,
const std::function<void(std::shared_ptr<TextBlock>)>& processLine, const GfxRenderer& renderer,
int fontId);
std::vector<uint16_t> calculateWordWidths(const GfxRenderer& renderer, int fontId);
@@ -41,7 +54,9 @@ class ParsedText {
: blockStyle(blockStyle),
extraParagraphSpacing(extraParagraphSpacing),
hyphenationEnabled(hyphenationEnabled),
focusReadingEnabled(focusReadingEnabled) {}
focusReadingEnabled(focusReadingEnabled),
isNaturalAlign(false),
hasRtlWord(false) {}
~ParsedText() = default;
void addWord(std::string word, EpdFontFamily::Style fontStyle, bool underline = false, bool attachToPrevious = false);
+574 -110
View File
@@ -2,6 +2,7 @@
#include <HalStorage.h>
#include <Logging.h>
#include <Memory.h>
#include <Serialization.h>
#include "Epub/css/CssParser.h"
@@ -10,33 +11,68 @@
#include "parsers/ChapterHtmlSlimParser.h"
namespace {
constexpr uint8_t SECTION_FILE_VERSION = 23;
// v28: text decoration bits now include line-through in serialized wordStyles.
// v29: TextBlock word data stored as one flat arena (offset table + NUL-terminated
// text blob) instead of length-prefixed strings and per-field arrays.
// v30: Arabic shaping changed both drawing and measurement (getTextAdvanceX now
// measures the shaped visual text); cached word positions from v29 no longer
// match what drawText renders.
constexpr uint8_t SECTION_FILE_VERSION = 31;
// Written into the version field while a build is in progress; patched to
// SECTION_FILE_VERSION only when the build is finalized. An abandoned /
// crash-interrupted .bin therefore carries version 0, which loadSectionFile rejects
// as unknown and clears -- so an incomplete file is never mistaken for a valid one.
constexpr uint8_t SECTION_FILE_INCOMPLETE_VERSION = 0;
// Written when a build is suspended partway (reader exited or device slept mid-build).
// The file carries valid pages 0..pageCount-1, all LUTs, and a trailer with the parse
// watermark (bytesConsumed, totalBytes) appended after the li LUT. loadSectionFile
// accepts it so a resume shows those pages instantly; the reader extends it by
// rebuilding in the background. Uses the same header layout as SECTION_FILE_VERSION,
// so finalized files are untouched by this feature; older firmware treats the sentinel
// as an unknown version and rebuilds, which is a safe downgrade.
// MUST change in lockstep with SECTION_FILE_VERSION: the sentinel IS the partial's
// format version, so a stale-format partial otherwise passes the header check and
// only fails (noisily, via the block-decode error path) when a page is loaded.
// Derived so the pairing can't be forgotten: 0xFE for v28, 0xFD for v29, ...
constexpr uint8_t SECTION_FILE_PARTIAL_VERSION = 0xFE - (SECTION_FILE_VERSION - 28);
constexpr uint32_t HEADER_SIZE = sizeof(uint8_t) + sizeof(int) + sizeof(float) + sizeof(bool) + sizeof(uint8_t) +
sizeof(uint16_t) + sizeof(uint16_t) + sizeof(uint16_t) + sizeof(bool) + sizeof(bool) +
sizeof(uint8_t) + sizeof(bool) + sizeof(uint32_t) + sizeof(uint32_t) +
sizeof(uint32_t) + sizeof(uint32_t);
struct PageLutEntry {
uint32_t fileOffset;
uint16_t paragraphIndex;
uint16_t listItemIndex;
};
} // namespace
// Out-of-line so the unique_ptr<ChapterHtmlSlimParser> in BuildContext can be
// constructed/destroyed where the parser's full definition is visible.
Section::Section(const std::shared_ptr<Epub>& epub, const int spineIndex, GfxRenderer& renderer)
: epub(epub),
spineIndex(spineIndex),
renderer(renderer),
filePath(epub->getCachePath() + "/sections/" + std::to_string(spineIndex) + ".bin") {}
// Suspend any in-progress build so every section.reset() / navigation / sleep path
// persists the pages already laid out as a partial .bin instead of discarding them
// (no-op once a build has completed or never started).
Section::~Section() { suspendBuild(); }
uint32_t Section::onPageComplete(std::unique_ptr<Page> page) {
if (!file) {
LOG_ERR("SCT", "File not open for writing page %d", pageCount);
LOG_ERR("SCT", "File not open for writing page %d", builtPageCount_);
return 0;
}
const uint32_t position = file.position();
if (!page->serialize(file)) {
LOG_ERR("SCT", "Failed to serialize page %d", pageCount);
LOG_ERR("SCT", "Failed to serialize page %d", builtPageCount_);
return 0;
}
LOG_DBG("SCT", "Page %d processed", pageCount);
LOG_DBG("SCT", "Page %d processed", builtPageCount_);
pageCount++;
builtPageCount_++;
// pageCount is the pages available to read: a rebuild over a partial only raises it
// once it has laid out more pages than the partial already covers.
if (builtPageCount_ > pageCount) {
pageCount = builtPageCount_;
}
return position;
}
@@ -55,7 +91,9 @@ void Section::writeSectionFileHeader(const int fontId, const float lineCompressi
sizeof(embeddedStyle) + sizeof(imageRendering) + sizeof(focusReadingEnabled) +
sizeof(uint32_t) + sizeof(uint32_t) + sizeof(uint32_t) + sizeof(uint32_t),
"Header size mismatch");
serialization::writePod(file, SECTION_FILE_VERSION);
// Written as the incomplete sentinel; finalizeBuild() patches it to
// SECTION_FILE_VERSION as the last step, committing the file.
serialization::writePod(file, SECTION_FILE_INCOMPLETE_VERSION);
serialization::writePod(file, fontId);
serialization::writePod(file, lineCompression);
serialization::writePod(file, extraParagraphSpacing);
@@ -82,16 +120,18 @@ bool Section::loadSectionFile(const int fontId, const float lineCompression, con
}
// Match parameters
bool filePartial = false;
{
uint8_t version;
serialization::readPod(file, version);
if (version != SECTION_FILE_VERSION) {
if (version != SECTION_FILE_VERSION && version != SECTION_FILE_PARTIAL_VERSION) {
// Explicit close() required: member variable persists beyond function scope
file.close();
LOG_ERR("SCT", "Deserialization failed: Unknown version %u", version);
clearCache();
return false;
}
filePartial = (version == SECTION_FILE_PARTIAL_VERSION);
int fileFontId;
uint16_t fileViewportWidth, fileViewportHeight;
@@ -126,14 +166,42 @@ bool Section::loadSectionFile(const int fontId, const float lineCompression, con
}
serialization::readPod(file, pageCount);
if (filePartial) {
// A partial's pageCount is the watermark of a suspended build. Read the watermark
// trailer (appended after the li LUT) so estimatedTotalPages can extrapolate.
uint32_t liLutOffset = 0;
file.seek(HEADER_SIZE - sizeof(uint32_t));
serialization::readPod(file, liLutOffset);
const uint32_t trailerOffset = liLutOffset + static_cast<uint32_t>(pageCount) * sizeof(uint16_t);
const bool trailerValid =
pageCount > 0 && liLutOffset >= HEADER_SIZE && trailerOffset + 2 * sizeof(uint32_t) <= file.size();
if (!trailerValid) {
file.close();
LOG_ERR("SCT", "Deserialization failed: malformed partial section");
clearCache();
pageCount = 0;
return false;
}
file.seek(trailerOffset);
serialization::readPod(file, partialBytesConsumed_);
serialization::readPod(file, partialTotalBytes_);
partial_ = true;
partialPageCount_ = pageCount;
}
// Explicit close() required: member variable persists beyond function scope
file.close();
LOG_DBG("SCT", "Deserialization succeeded: %d pages", pageCount);
LOG_DBG("SCT", "Deserialization succeeded: %d pages%s", pageCount, filePartial ? " (partial)" : "");
return true;
}
// Your updated class method (assuming you are using the 'SD' object, which is a wrapper for a specific filesystem)
bool Section::clearCache() const {
const std::string tmpBin = binTmpPath();
if (Storage.exists(tmpBin.c_str())) {
Storage.remove(tmpBin.c_str());
}
if (!Storage.exists(filePath.c_str())) {
LOG_DBG("SCT", "Cache does not exist, no action needed");
return true;
@@ -153,8 +221,43 @@ bool Section::createSectionFile(const int fontId, const float lineCompression, c
const uint16_t viewportHeight, const bool hyphenationEnabled, const bool embeddedStyle,
const uint8_t imageRendering, const bool focusReadingEnabled,
const std::function<void()>& popupFn) {
// One-shot build: start, then lay out the whole section in a single pass.
if (!startBuild(fontId, lineCompression, extraParagraphSpacing, paragraphAlignment, viewportWidth, viewportHeight,
hyphenationEnabled, embeddedStyle, imageRendering, focusReadingEnabled, popupFn)) {
return false;
}
if (!buildSomeMore(0)) { // 0 = build to completion
return false;
}
return buildComplete_;
}
bool Section::startBuild(const int fontId, const float lineCompression, const bool extraParagraphSpacing,
const uint8_t paragraphAlignment, const uint16_t viewportWidth, const uint16_t viewportHeight,
const bool hyphenationEnabled, const bool embeddedStyle, const uint8_t imageRendering,
const bool focusReadingEnabled, const std::function<void()>& popupFn) {
if (build_) {
LOG_ERR("SCT", "startBuild called while a build is already active");
return false;
}
buildComplete_ = false;
builtPageCount_ = 0;
// Pages from a loaded partial stay readable (from filePath) while this build writes
// to the tmp .bin, so availability never drops below the partial's watermark.
pageCount = partial_ ? partialPageCount_ : 0;
// Remove a stale tmp .bin from a crash-interrupted build; this build recreates it.
{
const std::string staleTmp = binTmpPath();
if (Storage.exists(staleTmp.c_str())) {
Storage.remove(staleTmp.c_str());
}
}
const auto localPath = epub->getSpineItem(spineIndex).href;
const auto tmpHtmlPath = epub->getCachePath() + "/.tmp_" + std::to_string(spineIndex) + ".html";
const auto htmlDir = epub->getCachePath() + "/html";
const auto htmlPath = htmlDir + "/" + std::to_string(spineIndex) + ".html";
const auto tmpHtmlPath = htmlDir + "/.tmp_" + std::to_string(spineIndex) + ".html";
// Create cache directory if it doesn't exist
{
@@ -162,162 +265,523 @@ bool Section::createSectionFile(const int fontId, const float lineCompression, c
Storage.mkdir(sectionsDir.c_str());
}
// Retry logic for SD card timing issues
bool success = false;
uint32_t fileSize = 0;
for (int attempt = 0; attempt < 3 && !success; attempt++) {
if (attempt > 0) {
LOG_DBG("SCT", "Retrying stream (attempt %d)...", attempt + 1);
delay(50); // Brief delay before retry
// Reuse the previously unzipped HTML if we already have it. The unzipped HTML is keyed only on the
// book (it lives in the per-book cache dir), not on render settings, so it survives the invalidation
// that wipes the layout (.bin) caches when font/margin/orientation change -- rebuilds then skip zip
// inflation entirely. It's promoted by an atomic rename as soon as the inflate succeeds (below), so
// even a window-only giant spine -- whose .bin never finalizes -- still caches its HTML, letting a
// reopen skip the multi-second inflate. If htmlPath exists it is known-complete.
const bool reusedHtml = Storage.exists(htmlPath.c_str());
bool htmlCached = reusedHtml;
if (reusedHtml) {
LOG_DBG("SCT", "Reusing cached HTML %s", htmlPath.c_str());
} else {
Storage.mkdir(htmlDir.c_str());
// Retry logic for SD card timing issues
bool streamed = false;
uint32_t fileSize = 0;
for (int attempt = 0; attempt < 3 && !streamed; attempt++) {
if (attempt > 0) {
LOG_DBG("SCT", "Retrying stream (attempt %d)...", attempt + 1);
delay(50); // Brief delay before retry
}
// Remove any incomplete file from previous attempt before retrying
if (Storage.exists(tmpHtmlPath.c_str())) {
Storage.remove(tmpHtmlPath.c_str());
}
HalFile tmpHtml;
if (!Storage.openFileForWrite("SCT", tmpHtmlPath, tmpHtml)) {
continue;
}
// Larger chunks mean far fewer SD writes inflating the HTML; a 1KB chunk turned a 584KB
// single-spine novel into ~570 tiny writes (multi-second). 8KB keeps the transient buffers
// small while cutting the write count 8x.
streamed = epub->readItemContentsToStream(localPath, tmpHtml, 8192);
fileSize = tmpHtml.size();
// Explicitly close() file before calling Storage.remove()
tmpHtml.close();
// If streaming failed, remove the incomplete file immediately
if (!streamed && Storage.exists(tmpHtmlPath.c_str())) {
Storage.remove(tmpHtmlPath.c_str());
LOG_DBG("SCT", "Removed incomplete temp file after failed attempt");
}
}
// Remove any incomplete file from previous attempt before retrying
if (Storage.exists(tmpHtmlPath.c_str())) {
Storage.remove(tmpHtmlPath.c_str());
if (!streamed) {
LOG_ERR("SCT", "Failed to stream item contents to temp file after retries");
return false;
}
FsFile tmpHtml;
if (!Storage.openFileForWrite("SCT", tmpHtmlPath, tmpHtml)) {
continue;
}
success = epub->readItemContentsToStream(localPath, tmpHtml, 1024);
fileSize = tmpHtml.size();
// Explicitly close() file before calling Storage.remove()
tmpHtml.close();
LOG_DBG("SCT", "Streamed temp HTML to %s (%d bytes)", tmpHtmlPath.c_str(), fileSize);
// If streaming failed, remove the incomplete file immediately
if (!success && Storage.exists(tmpHtmlPath.c_str())) {
Storage.remove(tmpHtmlPath.c_str());
LOG_DBG("SCT", "Removed incomplete temp file after failed attempt");
// Promote to the persistent HTML cache immediately -- the inflate is complete and the bytes are
// valid regardless of whether the layout build finishes, so reopening (even a window-only spine
// that never finalizes its .bin) skips re-inflation. If the rename fails we just parse the temp.
if (Storage.rename(tmpHtmlPath.c_str(), htmlPath.c_str())) {
htmlCached = true;
} else {
LOG_DBG("SCT", "Failed to promote HTML cache; parsing from temp");
}
}
if (!success) {
LOG_ERR("SCT", "Failed to stream item contents to temp file after retries");
return false;
}
LOG_DBG("SCT", "Streamed temp HTML to %s (%d bytes)", tmpHtmlPath.c_str(), fileSize);
if (!Storage.openFileForWrite("SCT", filePath, file)) {
if (!Storage.openFileForWrite("SCT", binTmpPath(), file)) {
if (!reusedHtml) Storage.remove(tmpHtmlPath.c_str());
return false;
}
// Header is written with the incomplete-version sentinel; finalizeBuild() commits it.
writeSectionFileHeader(fontId, lineCompression, extraParagraphSpacing, paragraphAlignment, viewportWidth,
viewportHeight, hyphenationEnabled, embeddedStyle, imageRendering, focusReadingEnabled);
std::vector<PageLutEntry> lut = {};
auto ctx = makeUniqueNoThrow<BuildContext>();
if (!ctx) {
LOG_ERR("SCT", "OOM: BuildContext");
file.close();
Storage.remove(binTmpPath().c_str());
if (!reusedHtml) Storage.remove(tmpHtmlPath.c_str());
return false;
}
// htmlCached == "htmlPath is the live cache" (reused, or just promoted). finalizeBuild/abandonBuild
// then leave the cached HTML alone; only an un-promoted temp (rename failed) is theirs to clean up.
ctx->reusedHtml = htmlCached;
ctx->htmlPath = htmlPath;
ctx->tmpHtmlPath = tmpHtmlPath;
ctx->parsePath = htmlCached ? htmlPath : tmpHtmlPath;
// Derive the content base directory and image cache path prefix for the parser
size_t lastSlash = localPath.find_last_of('/');
std::string contentBase = (lastSlash != std::string::npos) ? localPath.substr(0, lastSlash + 1) : "";
std::string imageBasePath = epub->getCachePath() + "/img_" + std::to_string(spineIndex) + "_";
const size_t lastSlash = localPath.find_last_of('/');
ctx->contentBase = (lastSlash != std::string::npos) ? localPath.substr(0, lastSlash + 1) : "";
ctx->imageBasePath = epub->getCachePath() + "/img_" + std::to_string(spineIndex) + "_";
CssParser* cssParser = nullptr;
if (embeddedStyle) {
cssParser = epub->getCssParser();
if (cssParser) {
if (!cssParser->loadFromCache()) {
LOG_ERR("SCT", "Failed to load CSS from cache");
ctx->cssParser = epub->getCssParser();
if (ctx->cssParser && !ctx->cssParser->loadFromCache()) {
LOG_ERR("SCT", "Failed to load CSS from cache");
}
}
// Collect TOC anchors for this spine so the parser can insert page breaks at chapter boundaries
std::vector<std::string> tocAnchors;
const int startTocIndex = epub->getTocIndexForSpineIndex(spineIndex);
if (startTocIndex >= 0) {
for (int i = startTocIndex; i < epub->getTocItemsCount(); i++) {
auto entry = epub->getTocItem(i);
if (entry.spineIndex != spineIndex) break;
if (!entry.anchor.empty()) {
tocAnchors.push_back(std::move(entry.anchor));
}
}
}
ChapterHtmlSlimParser visitor(
epub, tmpHtmlPath, renderer, fontId, lineCompression, extraParagraphSpacing, paragraphAlignment, viewportWidth,
viewportHeight, hyphenationEnabled, focusReadingEnabled,
[this, &lut](std::unique_ptr<Page> page, const uint16_t paragraphIndex, const uint16_t listItemIndex) {
lut.push_back({this->onPageComplete(std::move(page)), paragraphIndex, listItemIndex});
// The parser stores the path/contentBase/imageBasePath by reference, so they must
// live in the BuildContext (which outlives the parser). The page-complete callback
// captures the BuildContext pointer to append to its in-RAM LUT; build_ owns the
// context for the parser's whole lifetime.
BuildContext* ctxPtr = ctx.get();
ctx->parser = makeUniqueNoThrow<ChapterHtmlSlimParser>(
epub, ctxPtr->parsePath, renderer, fontId, lineCompression, extraParagraphSpacing, paragraphAlignment,
viewportWidth, viewportHeight, hyphenationEnabled, focusReadingEnabled,
[this, ctxPtr](std::unique_ptr<Page> page, const uint16_t paragraphIndex, const uint16_t listItemIndex) {
ctxPtr->lut.push_back({this->onPageComplete(std::move(page)), paragraphIndex, listItemIndex});
},
embeddedStyle, contentBase, imageBasePath, imageRendering, popupFn, cssParser);
Hyphenator::setPreferredLanguage(epub->getLanguage());
success = visitor.parseAndBuildPages();
Storage.remove(tmpHtmlPath.c_str());
if (!success) {
LOG_ERR("SCT", "Failed to parse XML and build pages");
// Explicitly close() file before calling Storage.remove()
embeddedStyle, ctxPtr->contentBase, ctxPtr->imageBasePath, imageRendering, std::move(tocAnchors), popupFn,
ctxPtr->cssParser);
if (!ctx->parser) {
LOG_ERR("SCT", "OOM: ChapterHtmlSlimParser");
if (ctx->cssParser) ctx->cssParser->clear();
file.close();
Storage.remove(filePath.c_str());
if (cssParser) {
cssParser->clear();
}
Storage.remove(binTmpPath().c_str());
if (!reusedHtml) Storage.remove(tmpHtmlPath.c_str());
return false;
}
Hyphenator::setPreferredLanguage(epub->getLanguage());
build_ = std::move(ctx);
if (!build_->parser->beginParse()) {
LOG_ERR("SCT", "Failed to begin parse");
abandonBuild();
return false;
}
build_->totalBytes = build_->parser->parseTotalBytes();
return true;
}
bool Section::buildSomeMore(const int maxPages) {
if (!build_ || !build_->parser) {
LOG_ERR("SCT", "buildSomeMore with no active build");
return false;
}
// Pace on pages laid out by THIS build, not pageCount: during a rebuild over a partial,
// pageCount stays pinned at the partial's watermark until the build passes it, which
// would otherwise turn one "small" chunk into a blocking rebuild of the whole watermark.
const int startCount = builtPageCount_;
for (;;) {
const auto status = build_->parser->parseStep();
if (status == ChapterHtmlSlimParser::ParseStatus::Error) {
LOG_ERR("SCT", "Parse error during incremental build");
abandonBuild();
return false;
}
if (status == ChapterHtmlSlimParser::ParseStatus::Done) {
return finalizeBuild();
}
// ParseStatus::More: yield once we've laid out the requested number of pages.
if (maxPages > 0 && (builtPageCount_ - startCount) >= maxPages) {
build_->bytesConsumed = build_->parser->parseBytesConsumed();
return true;
}
}
}
bool Section::hasHtmlCache() const {
const std::string htmlPath = epub->getCachePath() + "/html/" + std::to_string(spineIndex) + ".html";
return Storage.exists(htmlPath.c_str());
}
std::optional<uint16_t> Section::findAnchorDuringBuild(const std::string& anchor) const {
if (!build_ || !build_->parser) return std::nullopt;
for (const auto& [key, page] : build_->parser->getAnchors()) {
if (key == anchor) return page;
}
return std::nullopt;
}
std::optional<uint16_t> Section::findAnchor(const std::string& anchor) const {
if (const auto page = findAnchorDuringBuild(anchor)) {
return page;
}
// Fall back to the on-disk anchor map: a finalized section, or a partial whose map
// covers everything up to its watermark (nullopt past it -- build further and retry).
return getPageForAnchor(anchor);
}
uint16_t Section::estimatedTotalPages() const {
// Extrapolation from a suspended session's watermark trailer. A static snapshot, so no EMA
// damping is needed. Also the best guess while a rebuild is running but hasn't laid out
// enough pages yet to extrapolate from its own progress.
const auto partialEstimate = [this]() -> uint16_t {
if (!partial_ || partialBytesConsumed_ == 0 || partialTotalBytes_ <= partialBytesConsumed_) {
return pageCount;
}
const uint64_t est = static_cast<uint64_t>(partialPageCount_) * partialTotalBytes_ / partialBytesConsumed_;
if (est <= pageCount) return pageCount;
return est > 60000 ? 60000 : static_cast<uint16_t>(est);
};
if (!build_) {
return partial_ ? partialEstimate() : pageCount; // partial -> extrapolate, finalized -> exact
}
const uint32_t consumed = build_->bytesConsumed;
const uint32_t total = build_->totalBytes;
if (builtPageCount_ == 0 || consumed == 0 || total <= consumed) return partialEstimate();
// Raw extrapolation: scale the pages built so far by the fraction of HTML still unparsed. This
// re-derives from a growing, non-uniform sample, so it jitters up and down as the build crosses
// dense vs sparse regions of the chapter.
const uint64_t raw = static_cast<uint64_t>(builtPageCount_) * total / consumed;
// Damp that jitter with an exponential moving average. Step it once per build advance (keyed on
// bytesConsumed) rather than per status-bar redraw, so the smoothing rate doesn't depend on how
// often we repaint. As the build nears the end, consumed -> total and raw -> the built count, so
// the average settles onto the true count (and finalizeBuild then returns the exact pageCount).
constexpr float ALPHA = 0.25f; // weight of each new sample; lower = steadier but slower to settle
if (build_->smoothedEstimate <= 0) {
build_->smoothedEstimate = static_cast<float>(raw); // seed on the first estimate
} else if (consumed != build_->smoothedAtConsumed) {
build_->smoothedEstimate += ALPHA * (static_cast<float>(raw) - build_->smoothedEstimate);
}
build_->smoothedAtConsumed = consumed;
const uint64_t est = static_cast<uint64_t>(build_->smoothedEstimate + 0.5f);
if (est <= pageCount) return pageCount; // never fewer than the pages already available
return est > 60000 ? 60000 : static_cast<uint16_t>(est);
}
// Write the LUTs and anchor map into the open tmp .bin, patch the header with the built
// page count and table offsets, stamp `version` as the commit point, then swap the tmp
// file over filePath. For SECTION_FILE_PARTIAL_VERSION a watermark trailer
// (bytesConsumed, totalBytes) is appended after the li LUT so a later open can estimate
// the total page count. The parser must still be alive (anchors are read from it).
// On failure the tmp is removed and any pre-existing file at filePath is left intact.
bool Section::commitBuildFile(const uint8_t version, const uint32_t bytesConsumed, const uint32_t totalBytes) {
const bool asPartial = (version == SECTION_FILE_PARTIAL_VERSION);
const auto failCommit = [this]() {
// Explicit close() required before remove (member variable, O_RDWR handle).
file.close();
Storage.remove(binTmpPath().c_str());
return false;
};
const uint32_t lutOffset = file.position();
bool hasFailedLutRecords = false;
// Write LUT
for (const auto& entry : lut) {
for (const auto& entry : build_->lut) {
if (entry.fileOffset == 0) {
hasFailedLutRecords = true;
break;
LOG_ERR("SCT", "Failed to write LUT due to invalid page positions");
return failCommit();
}
serialization::writePod(file, entry.fileOffset);
}
if (hasFailedLutRecords) {
LOG_ERR("SCT", "Failed to write LUT due to invalid page positions");
// Explicitly close() file before calling Storage.remove()
file.close();
Storage.remove(filePath.c_str());
return false;
}
// Write anchor-to-page map for fragment navigation (e.g. footnote targets)
// Write anchor-to-page map for fragment navigation (e.g. footnote targets). For a
// partial, skip anchors that landed on the incomplete trailing page the suspend drops.
const uint32_t anchorMapOffset = file.position();
const auto& anchors = visitor.getAnchors();
serialization::writePod(file, static_cast<uint16_t>(anchors.size()));
const auto& anchors = build_->parser->getAnchors();
uint16_t anchorCount = 0;
for (const auto& [anchor, page] : anchors) {
if (!asPartial || page < builtPageCount_) anchorCount++;
}
serialization::writePod(file, anchorCount);
for (const auto& [anchor, page] : anchors) {
if (asPartial && page >= builtPageCount_) continue;
serialization::writeString(file, anchor);
serialization::writePod(file, page);
}
const uint32_t paragraphLutOffset = file.position();
serialization::writePod(file, static_cast<uint16_t>(lut.size()));
for (const auto& entry : lut) {
serialization::writePod(file, static_cast<uint16_t>(build_->lut.size()));
for (const auto& entry : build_->lut) {
serialization::writePod(file, entry.paragraphIndex);
}
const uint32_t liLutFileOffset = static_cast<uint32_t>(file.position());
for (const auto& entry : lut) {
for (const auto& entry : build_->lut) {
serialization::writePod(file, entry.listItemIndex);
}
// Patch header with final pageCount, lutOffset, anchorMapOffset, paragraphLutOffset, and liLutOffset
file.seek(HEADER_SIZE - sizeof(uint32_t) * 4 - sizeof(pageCount));
serialization::writePod(file, pageCount);
if (asPartial) {
// Watermark trailer, located on load as liLutOffset + pageCount * sizeof(uint16_t).
serialization::writePod(file, bytesConsumed);
serialization::writePod(file, totalBytes);
}
// Patch header with the built page count and section offsets...
file.seek(HEADER_SIZE - sizeof(uint32_t) * 4 - sizeof(builtPageCount_));
serialization::writePod(file, builtPageCount_);
serialization::writePod(file, lutOffset);
serialization::writePod(file, anchorMapOffset);
serialization::writePod(file, paragraphLutOffset);
serialization::writePod(file, liLutFileOffset);
// ...then commit by overwriting the sentinel version with the real one. Writing the
// version last makes it the commit point: a crash before here leaves version 0.
file.seek(0);
serialization::writePod(file, version);
// Explicit close() required: member variable persists beyond function scope
file.close();
if (cssParser) {
cssParser->clear();
// Swap into place. A crash between remove and rename loses the old file but keeps a
// fully-committed tmp; the next build just removes it and rebuilds.
if (Storage.exists(filePath.c_str())) {
Storage.remove(filePath.c_str());
}
if (!Storage.rename(binTmpPath().c_str(), filePath.c_str())) {
LOG_ERR("SCT", "Failed to move built section into place");
Storage.remove(binTmpPath().c_str());
return false;
}
return true;
}
std::unique_ptr<Page> Section::loadPageFromSectionFile() {
if (!Storage.openFileForRead("SCT", filePath, file)) {
bool Section::finalizeBuild() {
// Flush the trailing page (emits the last page via the completePageFn into the LUT).
build_->parser->finishParse();
if (!build_->reusedHtml) {
// Parse succeeded: promote the freshly unzipped HTML to the persistent cache so future
// rebuilds skip zip inflation. If promotion fails, drop the temp -- the build still succeeded.
if (!Storage.rename(build_->tmpHtmlPath.c_str(), build_->htmlPath.c_str())) {
LOG_DBG("SCT", "Failed to promote HTML cache, removing temp");
Storage.remove(build_->tmpHtmlPath.c_str());
}
}
const bool committed = commitBuildFile(SECTION_FILE_VERSION, 0, 0);
if (build_->cssParser) build_->cssParser->clear();
build_.reset();
if (!committed) {
// commitBuildFile removed filePath before the failed swap, so nothing valid remains.
partial_ = false;
partialPageCount_ = 0;
pageCount = 0;
builtPageCount_ = 0;
return false;
}
buildComplete_ = true;
partial_ = false;
partialPageCount_ = 0;
pageCount = builtPageCount_;
return true;
}
void Section::suspendBuild() {
if (!build_) return;
// Only worth persisting if this build produced pages a pre-existing partial doesn't
// already cover; otherwise keep the older (bigger) partial and just drop the tmp.
const bool worthKeeping = builtPageCount_ > 0 && (!partial_ || builtPageCount_ > partialPageCount_);
bool committed = false;
if (worthKeeping) {
// Capture the parse watermark and commit before tearing the parser down (the anchor
// map is read from it). The incomplete trailing page is intentionally not flushed:
// only fully laid-out pages are persisted, and the rebuild re-derives the rest.
const uint32_t consumed = static_cast<uint32_t>(build_->parser->parseBytesConsumed());
committed = commitBuildFile(SECTION_FILE_PARTIAL_VERSION, consumed, build_->totalBytes);
if (committed) {
partial_ = true;
partialPageCount_ = builtPageCount_;
partialBytesConsumed_ = consumed;
partialTotalBytes_ = build_->totalBytes;
LOG_INF("SCT", "Suspended build: %u pages persisted", builtPageCount_);
}
}
if (build_->parser) build_->parser->abortParse();
if (build_->cssParser) build_->cssParser->clear();
if (!committed && file) {
// Explicit close() required before remove (member variable, O_RDWR handle).
file.close();
Storage.remove(binTmpPath().c_str());
}
if (!build_->reusedHtml && Storage.exists(build_->tmpHtmlPath.c_str())) {
Storage.remove(build_->tmpHtmlPath.c_str());
}
build_.reset();
buildComplete_ = false;
pageCount = partial_ ? partialPageCount_ : 0;
builtPageCount_ = 0;
}
void Section::abandonBuild() {
if (!build_) return;
if (build_->parser) build_->parser->abortParse();
if (build_->cssParser) build_->cssParser->clear();
if (file) {
// Explicit close() required before remove (member variable, O_RDWR handle).
file.close();
Storage.remove(binTmpPath().c_str());
}
// A parse error would recur against the same HTML, so drop any partial too -- resuming
// from it would just re-enter the failing build every open.
if (Storage.exists(filePath.c_str())) {
Storage.remove(filePath.c_str());
}
if (!build_->reusedHtml && Storage.exists(build_->tmpHtmlPath.c_str())) {
Storage.remove(build_->tmpHtmlPath.c_str());
}
build_.reset();
buildComplete_ = false;
partial_ = false;
partialPageCount_ = 0;
pageCount = 0;
builtPageCount_ = 0;
}
std::unique_ptr<Page> Section::loadPageDuringBuild(const int page) {
if (!build_ || page < 0 || page >= static_cast<int>(build_->lut.size()) || !file) {
return nullptr;
}
const uint32_t pos = build_->lut[page].fileOffset;
if (pos == 0) {
return nullptr;
}
// The .bin is open O_RDWR for the build. Read the already-written page, then restore
// the write cursor so the next onPageComplete keeps appending where it left off.
const uint32_t writePos = file.position();
file.seek(pos);
auto p = Page::deserialize(file);
file.seek(writePos);
return p;
}
// Read a page from the committed file at filePath (finalized section or partial from a
// previous session). Uses a local handle so it is safe while a build holds the member
// `file` open on the tmp .bin.
std::unique_ptr<Page> Section::loadPageAt(const int page) const {
HalFile f;
if (!Storage.openFileForRead("SCT", filePath, f)) {
return nullptr;
}
file.seek(HEADER_SIZE - sizeof(uint32_t) * 4);
f.seek(HEADER_SIZE - sizeof(uint32_t) * 4);
uint32_t lutOffset;
serialization::readPod(file, lutOffset);
file.seek(lutOffset + sizeof(uint32_t) * currentPage);
serialization::readPod(f, lutOffset);
f.seek(lutOffset + sizeof(uint32_t) * page);
uint32_t pagePos;
serialization::readPod(file, pagePos);
file.seek(pagePos);
serialization::readPod(f, pagePos);
f.seek(pagePos);
auto page = Page::deserialize(file);
// Explicit close() required: member variable persists beyond function scope
file.close();
return page;
return Page::deserialize(f);
// No f.close() needed -- DESTRUCTOR_CLOSES_FILE=1 handles it at scope exit
}
std::unique_ptr<Page> Section::loadPage(const int page) {
if (page < 0) {
return nullptr;
}
if (build_ && page < static_cast<int>(build_->lut.size())) {
return loadPageDuringBuild(page);
}
// Not (yet) in the active build: serve from the file on disk -- a finalized section,
// or a partial from a previous session whose pages the rebuild hasn't reached again.
const int onDisk = partial_ ? partialPageCount_ : (build_ ? 0 : pageCount);
if (page >= onDisk) {
return nullptr;
}
return loadPageAt(page);
}
std::string Section::getTextFromSectionFile() {
std::string fullText;
auto p = loadPage(currentPage);
if (p) {
for (const auto& el : p->elements) {
if (el->getTag() == TAG_PageLine) {
const auto& line = static_cast<const PageLine&>(*el);
if (line.getBlock()) {
const auto& block = *line.getBlock();
for (uint16_t i = 0; i < block.wordCount(); i++) {
if (!fullText.empty()) fullText += " ";
fullText += block.wordText(i);
}
}
}
}
}
return fullText;
}
std::optional<uint16_t> Section::getCachedPageCount() const {
HalFile f;
if (!Storage.openFileForRead("SCT", filePath, f)) {
return std::nullopt;
}
const uint32_t fileSize = f.size();
if (fileSize < HEADER_SIZE) {
return std::nullopt;
}
// Only a finalized section's count is the chapter total; a partial's count is just the
// suspended build's watermark, which would skew progress mapping. Callers fall back to
// their own estimates.
uint8_t version;
serialization::readPod(f, version);
if (version != SECTION_FILE_VERSION) {
return std::nullopt;
}
f.seek(HEADER_SIZE - sizeof(uint32_t) * 4 - sizeof(uint16_t));
uint16_t count;
serialization::readPod(f, count);
return count;
}
std::optional<uint16_t> Section::getPageForAnchor(const std::string& anchor) const {
FsFile f;
HalFile f;
if (!Storage.openFileForRead("SCT", filePath, f)) {
return std::nullopt;
}
@@ -347,7 +811,7 @@ std::optional<uint16_t> Section::getPageForAnchor(const std::string& anchor) con
}
std::optional<uint16_t> Section::getPageForParagraphIndex(const uint16_t pIndex) const {
FsFile f;
HalFile f;
if (!Storage.openFileForRead("SCT", filePath, f)) {
return std::nullopt;
}
@@ -386,7 +850,7 @@ std::optional<uint16_t> Section::getPageForParagraphIndex(const uint16_t pIndex)
}
std::optional<uint16_t> Section::getParagraphIndexForPage(const uint16_t page) const {
FsFile f;
HalFile f;
if (!Storage.openFileForRead("SCT", filePath, f)) {
return std::nullopt;
}
@@ -418,7 +882,7 @@ std::optional<uint16_t> Section::getParagraphIndexForPage(const uint16_t page) c
}
std::optional<uint16_t> Section::getPageForListItemIndex(const uint16_t liIndex) const {
FsFile f;
HalFile f;
if (!Storage.openFileForRead("SCT", filePath, f)) {
return std::nullopt;
}
+110 -8
View File
@@ -3,34 +3,88 @@
#include <memory>
#include <optional>
#include <string>
#include <vector>
#include "Epub.h"
class Page;
class GfxRenderer;
class ChapterHtmlSlimParser;
class CssParser;
class Section {
std::shared_ptr<Epub> epub;
const int spineIndex;
GfxRenderer& renderer;
std::string filePath;
FsFile file;
HalFile file;
void writeSectionFileHeader(int fontId, float lineCompression, bool extraParagraphSpacing, uint8_t paragraphAlignment,
uint16_t viewportWidth, uint16_t viewportHeight, bool hyphenationEnabled,
bool embeddedStyle, uint8_t imageRendering, bool focusReadingEnabled);
uint32_t onPageComplete(std::unique_ptr<Page> page);
// Page-offset table entry, kept in RAM while an incremental build is running so
// already-built pages can be located in the partially-written .bin.
struct PageLutEntry {
uint32_t fileOffset;
uint16_t paragraphIndex;
uint16_t listItemIndex;
};
// Held only while an incremental build is in progress (see startBuild). Carries the
// live parser plus the strings it references (the parser stores them by reference)
// and the in-RAM page-offset table.
struct BuildContext {
std::unique_ptr<ChapterHtmlSlimParser> parser;
std::vector<PageLutEntry> lut;
std::string parsePath;
std::string contentBase;
std::string imageBasePath;
std::string htmlPath;
std::string tmpHtmlPath;
bool reusedHtml = false;
CssParser* cssParser = nullptr;
// HTML byte progress, for estimating the section's total page count while it's still building.
uint32_t bytesConsumed = 0;
uint32_t totalBytes = 0;
// Exponentially-smoothed page-count estimate (0 = not yet seeded) and the bytesConsumed at its
// last update. The raw byte-ratio estimate jitters as the build crosses dense/sparse regions;
// the EMA is stepped once per build advance (not per redraw) to damp that wobble.
float smoothedEstimate = 0;
uint32_t smoothedAtConsumed = 0;
};
std::unique_ptr<BuildContext> build_;
bool buildComplete_ = false;
// Pages laid out by the active build (== build_->lut.size()). Distinct from pageCount,
// which is the pages *available to read* and also counts a loaded partial file's pages.
uint16_t builtPageCount_ = 0;
// A partial section file (suspended build from a previous session) is loaded at filePath.
// Its pages 0..partialPageCount_-1 are readable while a rebuild extends past them.
bool partial_ = false;
uint16_t partialPageCount_ = 0;
// Parse watermark from the partial's trailer, for estimating the total page count.
uint32_t partialBytesConsumed_ = 0;
uint32_t partialTotalBytes_ = 0;
bool finalizeBuild();
// Write the LUTs/anchor map (and, for a partial, the watermark trailer), patch the
// header, stamp the version byte, and swap the tmp .bin over filePath.
bool commitBuildFile(uint8_t version, uint32_t bytesConsumed, uint32_t totalBytes);
// Builds write here and are swapped over filePath only on commit, so a prior
// partial/finalized file stays readable while a rebuild is in progress.
std::string binTmpPath() const { return filePath + ".part"; }
std::unique_ptr<Page> loadPageAt(int page) const;
// Read a page already laid out by the in-progress build (page < build LUT size), from
// the partially-written tmp .bin without disturbing the build's write cursor.
std::unique_ptr<Page> loadPageDuringBuild(int page);
public:
uint16_t pageCount = 0;
int currentPage = 0;
explicit Section(const std::shared_ptr<Epub>& epub, const int spineIndex, GfxRenderer& renderer)
: epub(epub),
spineIndex(spineIndex),
renderer(renderer),
filePath(epub->getCachePath() + "/sections/" + std::to_string(spineIndex) + ".bin") {}
~Section() = default;
// Constructor and destructor are out-of-line: BuildContext holds a unique_ptr to the
// forward-declared ChapterHtmlSlimParser, whose full definition is only visible in the .cpp.
explicit Section(const std::shared_ptr<Epub>& epub, int spineIndex, GfxRenderer& renderer);
~Section();
bool loadSectionFile(int fontId, float lineCompression, bool extraParagraphSpacing, uint8_t paragraphAlignment,
uint16_t viewportWidth, uint16_t viewportHeight, bool hyphenationEnabled, bool embeddedStyle,
uint8_t imageRendering, bool focusReadingEnabled);
@@ -39,11 +93,59 @@ class Section {
uint16_t viewportWidth, uint16_t viewportHeight, bool hyphenationEnabled, bool embeddedStyle,
uint8_t imageRendering, bool focusReadingEnabled,
const std::function<void()>& popupFn = nullptr);
std::unique_ptr<Page> loadPageFromSectionFile();
// Incremental build: lay out the section a few pages at a time so a large chapter
// can show its first page immediately and keep the UI responsive while the rest
// builds. createSectionFile() above is the one-shot wrapper over these.
// if (!startBuild(...)) fail;
// each tick: buildSomeMore(N); render up to pageCount; when isBuildComplete() stop.
bool startBuild(int fontId, float lineCompression, bool extraParagraphSpacing, uint8_t paragraphAlignment,
uint16_t viewportWidth, uint16_t viewportHeight, bool hyphenationEnabled, bool embeddedStyle,
uint8_t imageRendering, bool focusReadingEnabled, const std::function<void()>& popupFn = nullptr);
// Lay out up to maxPages more pages (maxPages <= 0 = build to completion). Returns
// false on error (the build is abandoned). Sets isBuildComplete() when finished.
bool buildSomeMore(int maxPages);
bool isBuilding() const { return static_cast<bool>(build_); }
bool isBuildComplete() const { return buildComplete_; }
// Best-known total page count: the exact pageCount once finalized, or a smoothed byte-based
// estimate (pages so far scaled by totalBytes/bytesConsumed, damped by an EMA) while a giant spine
// is still building, so "page X of Y" / progress don't read off the small build watermark.
uint16_t estimatedTotalPages() const;
void abandonBuild();
// Persist an in-progress build as a partial section file (version sentinel + LUTs +
// watermark trailer) instead of discarding it, so the next open of this spine can show
// its pages instantly and only rebuild in the background. Called by the destructor, so
// any teardown path (exit, sleep, navigation) keeps the work already done. Keeps a
// pre-existing partial when it covers more pages than this build reached.
void suspendBuild();
// True when a partial file was loaded: pageCount is a watermark, not the chapter total.
bool isPartial() const { return partial_; }
// Unified page read: from the active build if it has reached the page, otherwise from
// the on-disk file (finalized section, or a partial the rebuild hasn't caught up to).
std::unique_ptr<Page> loadPage(int page);
std::string getTextFromSectionFile();
// Resolve an anchor from the in-progress build first, then the on-disk anchor map
// (covers finalized sections and partials from a previous session).
std::optional<uint16_t> findAnchor(const std::string& anchor) const;
// True if this spine's unzipped HTML is already cached, so a build won't pay the (multi-second on a
// giant spine) zip inflation. Lets the reader skip the indexing popup on a fast reopen/rebuild.
bool hasHtmlCache() const;
// Look up the page number for an anchor id from the section cache file.
std::optional<uint16_t> getPageForAnchor(const std::string& anchor) const;
// Look up an anchor among the pages built so far by the in-progress build, so an anchor jump
// (TOC / chapter select, usually the chapter top = page 0) can resolve without laying out the
// whole chapter. Returns nullopt if the anchor hasn't been reached yet (build more) or no build.
std::optional<uint16_t> findAnchorDuringBuild(const std::string& anchor) const;
// Get the page count from the section cache file without fully loading it.
std::optional<uint16_t> getCachedPageCount() const;
// Look up the page number for a synthetic paragraph index from XPath p[N].
std::optional<uint16_t> getPageForParagraphIndex(uint16_t pIndex) const;
+22 -1
View File
@@ -29,6 +29,13 @@ struct BlockStyle {
int16_t textIndent = 0;
bool textIndentDefined = false; // true if text-indent was explicitly set in CSS
bool textAlignDefined = false; // true if text-align was explicitly set in CSS
bool isRtl = false; // true if resolved direction is RTL
bool directionDefined = false; // true if direction was explicitly set in CSS/HTML
// Set when this block was created by a <br> element. Used by startNewTextBlock to inject
// a full line-height gap when the <br> block stays empty (section-break use case).
// NOT propagated through getCombinedBlockStyle so it can't leak into sibling blocks.
bool fromBrElement = false;
// Combined insets (margin + padding)
[[nodiscard]] int16_t leftInset() const { return marginLeft + paddingLeft; }
@@ -84,6 +91,15 @@ struct BlockStyle {
result.paddingBottom = static_cast<int16_t>(child.paddingBottom + paddingBottom);
}
// Direction is not axis-specific. Inherit from parent when child doesn't define it.
if (!child.directionDefined && directionDefined) {
result.isRtl = isRtl;
result.directionDefined = true;
}
// fromBrElement is consumed by startNewTextBlock when an empty <br> block
// is merged with the following paragraph; never propagate it further.
result.fromBrElement = false;
return result;
}
@@ -107,7 +123,7 @@ struct BlockStyle {
blockStyle.paddingRight = std::min(cssStyle.paddingRight.toPixelsInt16(emSize, vw), maxHorizontalInsetPx);
// For textIndent: if it's a percentage we can't resolve (no viewport width),
// leave textIndentDefined=false so the EmSpace fallback in applyParagraphIndent() is used
// leave textIndentDefined=false so the space-width fallback in resolveFirstLineIndent() is used
if (cssStyle.hasTextIndent() && cssStyle.textIndent.isResolvable(vw)) {
blockStyle.textIndent = cssStyle.textIndent.toPixelsInt16(emSize, vw);
blockStyle.textIndentDefined = true;
@@ -119,6 +135,11 @@ struct BlockStyle {
} else {
blockStyle.alignment = paragraphAlignment;
}
// RTL direction from CSS/HTML
if (cssStyle.hasDirection()) {
blockStyle.isRtl = (cssStyle.direction == CssTextDirection::Rtl);
blockStyle.directionDefined = true;
}
return blockStyle;
}
};
+148 -25
View File
@@ -1,11 +1,14 @@
#include "ImageBlock.h"
#include <FontCacheManager.h>
#include <GfxRenderer.h>
#include <Logging.h>
#include <Serialization.h>
#include "../converters/DirectPixelWriter.h"
#include "../converters/ImageDecoderFactory.h"
#include <cstdlib>
#include "Epub/converters/DirectPixelWriter.h"
#include "Epub/converters/ImageDecoderFactory.h"
// Cache file format:
// - uint16_t width
@@ -28,24 +31,64 @@ std::string getCachePath(const std::string& imagePath) {
return imagePath + ".pxc";
}
bool readValidCacheHeader(HalFile& cacheFile, const int expectedWidth, const int expectedHeight, uint16_t& cachedWidth,
uint16_t& cachedHeight) {
if (cacheFile.read(&cachedWidth, 2) != 2 || cacheFile.read(&cachedHeight, 2) != 2) {
return false;
}
const int widthDiff = abs(cachedWidth - expectedWidth);
const int heightDiff = abs(cachedHeight - expectedHeight);
if (widthDiff > 1 || heightDiff > 1) {
return false;
}
const size_t bytesPerRow = (cachedWidth + 3) / 4;
const size_t expectedSize = 4 + bytesPerRow * cachedHeight;
return cacheFile.size() >= expectedSize;
}
// Pages are deserialized afresh on each visit. Keep a bounded, allocation-free
// record so an image that failed renders its placeholder directly for the rest
// of the reader session instead of paying another placeholder refresh and
// decode. The reader clears this on entry so transient memory/storage failures
// are retried.
constexpr size_t MAX_SESSION_IMAGE_FAILURES = 16;
uint64_t failedImageHashes[MAX_SESSION_IMAGE_FAILURES];
size_t failedImageCount = 0;
uint64_t imagePathHash(const std::string& path) {
uint64_t hash = 14695981039346656037ull;
for (const char c : path) {
hash ^= static_cast<uint8_t>(c);
hash *= 1099511628211ull;
}
return hash;
}
bool imageFailedThisSession(const std::string& path) {
const uint64_t hash = imagePathHash(path);
for (size_t i = 0; i < failedImageCount; i++) {
if (failedImageHashes[i] == hash) return true;
}
return false;
}
void rememberImageFailure(const std::string& path) {
if (failedImageCount == MAX_SESSION_IMAGE_FAILURES || imageFailedThisSession(path)) return;
failedImageHashes[failedImageCount++] = imagePathHash(path);
}
bool renderFromCache(GfxRenderer& renderer, const std::string& cachePath, int x, int y, int expectedWidth,
int expectedHeight) {
FsFile cacheFile;
HalFile cacheFile;
if (!Storage.openFileForRead("IMG", cachePath, cacheFile)) {
return false;
}
uint16_t cachedWidth, cachedHeight;
if (cacheFile.read(&cachedWidth, 2) != 2 || cacheFile.read(&cachedHeight, 2) != 2) {
return false;
}
// Verify dimensions are close (allow 1 pixel tolerance for rounding differences)
int widthDiff = abs(cachedWidth - expectedWidth);
int heightDiff = abs(cachedHeight - expectedHeight);
if (widthDiff > 1 || heightDiff > 1) {
LOG_ERR("IMG", "Cache dimension mismatch: %dx%d vs %dx%d", cachedWidth, cachedHeight, expectedWidth,
expectedHeight);
if (!readValidCacheHeader(cacheFile, expectedWidth, expectedHeight, cachedWidth, cachedHeight)) {
LOG_ERR("IMG", "Invalid image cache: %s", cachePath.c_str());
return false;
}
@@ -55,10 +98,22 @@ bool renderFromCache(GfxRenderer& renderer, const std::string& cachePath, int x,
LOG_DBG("IMG", "Loading from cache: %s (%dx%d)", cachePath.c_str(), cachedWidth, cachedHeight);
// Read and render row by row to minimize memory usage
// Read several rows per SD access. A full-page image is re-rendered on every
// grayscale strip pass (~14x per page), and a one-row-per-read loop here means
// cachedHeight (~728) tiny reads through the storage mutex + SdFat each time —
// the dominant cost of displaying an image page. Batching rows into a ~4KB
// buffer cuts that to ~20 reads per pass without holding the whole image.
const int bytesPerRow = (cachedWidth + 3) / 4; // 2 bits per pixel, 4 pixels per byte
uint8_t* rowBuffer = (uint8_t*)malloc(bytesPerRow);
if (!rowBuffer) {
int rowsPerRead = 4096 / bytesPerRow;
if (rowsPerRead < 1) rowsPerRead = 1;
if (rowsPerRead > cachedHeight) rowsPerRead = cachedHeight;
uint8_t* readBuffer = (uint8_t*)malloc((size_t)rowsPerRead * bytesPerRow);
if (!readBuffer) {
// Fall back to a single-row buffer under memory pressure.
rowsPerRead = 1;
readBuffer = (uint8_t*)malloc(bytesPerRow);
}
if (!readBuffer) {
LOG_ERR("IMG", "Failed to allocate row buffer");
return false;
}
@@ -66,16 +121,31 @@ bool renderFromCache(GfxRenderer& renderer, const std::string& cachePath, int x,
DirectPixelWriter pw;
pw.init(renderer);
int rowsInBuffer = 0;
int bufferRow = 0;
for (int row = 0; row < cachedHeight; row++) {
if (cacheFile.read(rowBuffer, bytesPerRow) != bytesPerRow) {
LOG_ERR("IMG", "Cache read error at row %d", row);
free(rowBuffer);
return false;
if (bufferRow >= rowsInBuffer) {
const int toRead = (cachedHeight - row < rowsPerRead) ? (cachedHeight - row) : rowsPerRead;
const size_t bytes = (size_t)toRead * bytesPerRow;
if (cacheFile.read(readBuffer, bytes) != static_cast<int>(bytes)) {
LOG_ERR("IMG", "Cache read error at row %d", row);
free(readBuffer);
return false;
}
rowsInBuffer = toRead;
bufferRow = 0;
}
const uint8_t* rowBuffer = readBuffer + (size_t)bufferRow * bytesPerRow;
bufferRow++;
const int destY = y + row;
pw.beginRow(destY);
for (int col = 0; col < cachedWidth; col++) {
// On a grayscale strip pass only a narrow column window of the image is in
// the active band; skip the rest instead of unpacking+clipping every pixel.
int colStart, colEnd;
pw.bandColRange(x, cachedWidth, colStart, colEnd);
for (int col = colStart; col < colEnd; col++) {
const int byteIdx = col >> 2; // col / 4
const int bitShift = 6 - (col & 3) * 2; // MSB first within byte
uint8_t pixelValue = (rowBuffer[byteIdx] >> bitShift) & 0x03;
@@ -84,14 +154,44 @@ bool renderFromCache(GfxRenderer& renderer, const std::string& cachePath, int x,
}
}
free(rowBuffer);
free(readBuffer);
LOG_DBG("IMG", "Cache render complete");
return true;
}
} // namespace
bool ImageBlock::hasValidCache() const {
const auto cachePath = getCachePath(imagePath);
HalFile cacheFile;
if (!Storage.openFileForRead("IMG", cachePath, cacheFile)) {
return false;
}
uint16_t cachedWidth, cachedHeight;
return readValidCacheHeader(cacheFile, width, height, cachedWidth, cachedHeight);
}
bool ImageBlock::needsDecode() const { return !imageFailedThisSession(imagePath) && !hasValidCache(); }
void ImageBlock::clearSessionRenderFailures() { failedImageCount = 0; }
void ImageBlock::renderPlaceholder(GfxRenderer& renderer, const int x, const int y) const {
renderer.fillRect(x, y, width, height, true);
if (width > 2 && height > 2) {
renderer.fillRect(x + 1, y + 1, width - 2, height - 2, false);
}
}
void ImageBlock::render(GfxRenderer& renderer, const int x, const int y) {
// The font-prewarm scan pass only accumulates glyphs; an image contributes
// none, and its DirectPixelWriter output bypasses the renderer's scan-mode
// suppression, so it would otherwise do a full (discarded) cache render every
// page view. Skip it here. The image still draws in the real BW/grayscale
// passes; on first view this just moves the one-time decode to the BW pass.
FontCacheManager* fcm = renderer.getFontCacheManager();
if (fcm && fcm->isScanning()) return;
LOG_DBG("IMG", "Rendering image at %d,%d: %s (%dx%d)", x, y, imagePath.c_str(), width, height);
const int screenWidth = renderer.getScreenWidth();
@@ -104,6 +204,21 @@ void ImageBlock::render(GfxRenderer& renderer, const int x, const int y) {
return;
}
// Tiled grayscale (#2190): skip the whole image when it doesn't touch the
// active band. The per-pixel writer already clips off-band pixels, but without
// this each of the ~7 bands per plane re-ran the full cache load / pixel walk
// and discarded the result — the dominant cost of AA on image pages. The check
// is orientation-aware and returns true when no strip is active, so the BW
// pass and non-tiled controllers render the image exactly as before.
if (!renderer.glyphIntersectsStrip(x, y, x + width - 1, y + height - 1)) {
return;
}
if (imageFailedThisSession(imagePath)) {
renderPlaceholder(renderer, x, y);
return;
}
// Try to render from cache first
std::string cachePath = getCachePath(imagePath);
if (renderFromCache(renderer, cachePath, x, y, width, height)) {
@@ -112,9 +227,11 @@ void ImageBlock::render(GfxRenderer& renderer, const int x, const int y) {
// No cache - need to decode the image
// Check if image file exists
FsFile file;
HalFile file;
if (!Storage.openFileForRead("IMG", imagePath, file)) {
LOG_ERR("IMG", "Image file not found: %s", imagePath.c_str());
rememberImageFailure(imagePath);
renderPlaceholder(renderer, x, y);
return;
}
size_t fileSize = file.size();
@@ -122,6 +239,8 @@ void ImageBlock::render(GfxRenderer& renderer, const int x, const int y) {
if (fileSize == 0) {
LOG_ERR("IMG", "Image file is empty: %s", imagePath.c_str());
rememberImageFailure(imagePath);
renderPlaceholder(renderer, x, y);
return;
}
@@ -141,6 +260,8 @@ void ImageBlock::render(GfxRenderer& renderer, const int x, const int y) {
ImageToFramebufferDecoder* decoder = ImageDecoderFactory::getDecoder(imagePath);
if (!decoder) {
LOG_ERR("IMG", "No decoder found for image: %s", imagePath.c_str());
rememberImageFailure(imagePath);
renderPlaceholder(renderer, x, y);
return;
}
@@ -149,20 +270,22 @@ void ImageBlock::render(GfxRenderer& renderer, const int x, const int y) {
bool success = decoder->decodeToFramebuffer(imagePath, renderer, config);
if (!success) {
LOG_ERR("IMG", "Failed to decode image: %s", imagePath.c_str());
rememberImageFailure(imagePath);
renderPlaceholder(renderer, x, y);
return;
}
LOG_DBG("IMG", "Decode successful");
}
bool ImageBlock::serialize(FsFile& file) {
bool ImageBlock::serialize(HalFile& file) {
serialization::writeString(file, imagePath);
serialization::writePod(file, width);
serialization::writePod(file, height);
return true;
}
std::unique_ptr<ImageBlock> ImageBlock::deserialize(FsFile& file) {
std::unique_ptr<ImageBlock> ImageBlock::deserialize(HalFile& file) {
std::string path;
serialization::readString(file, path);
int16_t w, h;
+6 -2
View File
@@ -16,13 +16,17 @@ class ImageBlock final : public Block {
int16_t getHeight() const { return height; }
bool imageExists() const;
bool hasValidCache() const;
bool needsDecode() const;
void renderPlaceholder(GfxRenderer& renderer, int x, int y) const;
static void clearSessionRenderFailures();
BlockType getType() override { return IMAGE_BLOCK; }
bool isEmpty() override { return false; }
void render(GfxRenderer& renderer, const int x, const int y);
bool serialize(FsFile& file);
static std::unique_ptr<ImageBlock> deserialize(FsFile& file);
bool serialize(HalFile& file);
static std::unique_ptr<ImageBlock> deserialize(HalFile& file);
private:
std::string imagePath;
+270 -83
View File
@@ -1,27 +1,172 @@
#include "TextBlock.h"
#include <BidiUtils.h>
#include <GfxRenderer.h>
#include <Logging.h>
#include <Memory.h>
#include <Serialization.h>
#include <cstring>
void TextBlock::render(const GfxRenderer& renderer, const int fontId, const int x, const int y) const {
size_t TextBlock::arenaSize(const uint16_t wordCount, const bool hasFocus, const uint16_t textBytes) {
// Layout documented in TextBlock.h: 16-bit arrays first, then 8-bit arrays, then text.
size_t size = static_cast<size_t>(wordCount) * (sizeof(uint16_t) + sizeof(int16_t) + sizeof(uint8_t));
if (hasFocus) {
size += static_cast<size_t>(wordCount) * (sizeof(uint16_t) + sizeof(uint8_t));
}
return size + textBytes;
}
void TextBlock::bindArenaPointers() {
uint8_t* base = arena.get();
const size_t wc = numWords;
textOffArr = reinterpret_cast<const uint16_t*>(base);
xposArr = reinterpret_cast<const int16_t*>(base + wc * 2);
size_t off = wc * 4;
if (focusPresent) {
focusSuffixXArr = reinterpret_cast<const uint16_t*>(base + off);
off += wc * 2;
}
stylesArr = base + off;
off += wc;
if (focusPresent) {
focusBoundaryArr = base + off;
off += wc;
}
textArr = reinterpret_cast<const char*>(base + off);
}
TextBlock::TextBlock(const std::vector<std::string>& words, const std::vector<int16_t>& wordXpos,
const std::vector<EpdFontFamily::Style>& wordStyles, const std::vector<uint8_t>& focusBoundary,
const std::vector<uint16_t>& focusSuffixX, const BlockStyle& blockStyle)
: blockStyle(blockStyle) {
// Focus annotations are optional: empty vectors mean no word in this block has a split.
// When present, they must be sized in lockstep with words[].
const bool hasFocus = !wordFocusBoundary.empty();
if (words.size() != wordXpos.size() || words.size() != wordStyles.size() ||
(hasFocus && (words.size() != wordFocusBoundary.size() || words.size() != wordFocusSuffixX.size()))) {
LOG_ERR("TXB", "Render skipped: size mismatch (words=%u, xpos=%u, styles=%u, boundary=%u, suffixX=%u)\n",
(uint32_t)words.size(), (uint32_t)wordXpos.size(), (uint32_t)wordStyles.size(),
(uint32_t)wordFocusBoundary.size(), (uint32_t)wordFocusSuffixX.size());
const bool hasFocus = !focusBoundary.empty();
if (words.size() != wordXpos.size() || words.size() != wordStyles.size() || words.size() > 10000 ||
(hasFocus && (words.size() != focusBoundary.size() || words.size() != focusSuffixX.size()))) {
LOG_ERR("TXB", "Construction failed: size mismatch (words=%u, xpos=%u, styles=%u, boundary=%u, suffixX=%u)",
static_cast<uint32_t>(words.size()), static_cast<uint32_t>(wordXpos.size()),
static_cast<uint32_t>(wordStyles.size()), static_cast<uint32_t>(focusBoundary.size()),
static_cast<uint32_t>(focusSuffixX.size()));
isValid = false;
return;
}
for (size_t i = 0; i < words.size(); i++) {
const int wordX = wordXpos[i] + x;
const EpdFontFamily::Style currentStyle = wordStyles[i];
const uint8_t boundary = hasFocus ? wordFocusBoundary[i] : 0;
numWords = static_cast<uint16_t>(words.size());
focusPresent = hasFocus;
if (numWords == 0) {
return; // valid empty block, no arena
}
// Pass 1: total text size, one NUL per word. A line is at most a physical
// row of the page, so uint16_t offsets are ample; reject anything larger.
size_t totalText = 0;
for (const auto& w : words) totalText += w.size() + 1;
if (totalText > UINT16_MAX) {
LOG_ERR("TXB", "Construction failed: text size %u exceeds arena limit", static_cast<uint32_t>(totalText));
numWords = 0;
focusPresent = false;
isValid = false;
return;
}
textBytes = static_cast<uint16_t>(totalText);
const size_t size = arenaSize(numWords, focusPresent, textBytes);
arena = makeUniqueNoThrow<uint8_t[]>(size);
if (!arena) {
LOG_ERR("TXB", "OOM: arena %u bytes", static_cast<uint32_t>(size));
numWords = 0;
textBytes = 0;
focusPresent = false;
isValid = false;
return;
}
bindArenaPointers();
// Pass 2: fill. Mutable aliases of the const views bound above.
auto* textOff = const_cast<uint16_t*>(textOffArr);
auto* xpos = const_cast<int16_t*>(xposArr);
auto* styles = const_cast<uint8_t*>(stylesArr);
auto* text = const_cast<char*>(textArr);
uint16_t off = 0;
for (uint16_t i = 0; i < numWords; i++) {
textOff[i] = off;
xpos[i] = wordXpos[i];
styles[i] = static_cast<uint8_t>(wordStyles[i]);
memcpy(text + off, words[i].data(), words[i].size());
off += static_cast<uint16_t>(words[i].size());
text[off++] = '\0';
}
if (focusPresent) {
auto* suffixX = const_cast<uint16_t*>(focusSuffixXArr);
auto* boundary = const_cast<uint8_t*>(focusBoundaryArr);
for (uint16_t i = 0; i < numWords; i++) {
suffixX[i] = focusSuffixX[i];
boundary[i] = focusBoundary[i];
}
}
}
void TextBlock::render(const GfxRenderer& renderer, const int fontId, const int x, const int y) const {
if (!isValid) {
LOG_ERR("TXB", "Render skipped: invalid block");
return;
}
const bool scanning = renderer.isFontCacheScanning();
const int ascender = renderer.getFontAscenderSize(fontId);
struct DecorationLineTracker {
EpdFontFamily::Style style;
int yOffset;
int startX = -1;
int endX = -1;
int yPos = 0;
bool active() const { return startX != -1; }
void reset() {
startX = -1;
endX = -1;
yPos = 0;
}
};
DecorationLineTracker decorationLines[] = {
{EpdFontFamily::UNDERLINE, ascender + 2},
{EpdFontFamily::STRIKETHROUGH, ascender * 4 / 5},
};
const auto flushDecoration = [&](DecorationLineTracker& line) {
if (line.active()) {
renderer.drawLine(line.startX, line.yPos, line.endX, line.yPos, 2, true);
line.reset();
}
};
const auto flushDecorations = [&]() {
for (auto& line : decorationLines) {
flushDecoration(line);
}
};
for (uint16_t i = 0; i < numWords; i++) {
const char* word = wordText(i);
const int wordX = xposArr[i] + x;
const EpdFontFamily::Style currentStyle = wordStyle(i);
const auto baseDir =
static_cast<BidiUtils::BidiBaseDir>(BidiUtils::detectParagraphLevel(word, blockStyle.isRtl ? 1 : 0));
const uint8_t boundary = focusBoundary(i);
// SUP/SUB shift the baseline passed to drawText; the glyph is also scaled 50% inside
// drawText, so these offsets are chosen relative to the full-size ascender:
// SUP: raise by 40% of ascender — sits clearly above the cap-height
// SUB: lower by 25% of ascender — descends below baseline without clashing with ascenders below
int wordY = y;
if ((currentStyle & EpdFontFamily::SUP) != 0) {
wordY -= ascender * 2 / 5;
} else if ((currentStyle & EpdFontFamily::SUB) != 0) {
wordY += ascender / 4;
}
if (boundary > 0) {
// Focus split: draw bold prefix, then the regular suffix at a pre-computed x offset.
@@ -33,64 +178,81 @@ void TextBlock::render(const GfxRenderer& renderer, const int fontId, const int
static_assert(sizeof(boldBuf) >= MAX_FOCUS_PREFIX_BYTES,
"boldBuf too small for max focus prefix (9 codepoints * 4 UTF-8 bytes + null)");
const auto boldStyle = static_cast<EpdFontFamily::Style>(currentStyle | EpdFontFamily::BOLD);
const size_t boldLen = std::min<size_t>({static_cast<size_t>(boundary), words[i].size(), sizeof(boldBuf) - 1});
memcpy(boldBuf, words[i].c_str(), boldLen);
const size_t boldLen =
std::min<size_t>({static_cast<size_t>(boundary), static_cast<size_t>(wordTextLen(i)), sizeof(boldBuf) - 1});
memcpy(boldBuf, word, boldLen);
boldBuf[boldLen] = '\0';
renderer.drawText(fontId, wordX, y, boldBuf, true, boldStyle);
const int suffixX = wordX + wordFocusSuffixX[i];
renderer.drawText(fontId, suffixX, y, words[i].c_str() + boldLen, true, currentStyle);
renderer.drawText(fontId, wordX, wordY, boldBuf, true, boldStyle, baseDir);
const int suffixX = wordX + focusSuffixXArr[i];
renderer.drawText(fontId, suffixX, wordY, word + boldLen, true, currentStyle, baseDir);
} else {
renderer.drawText(fontId, wordX, y, words[i].c_str(), true, currentStyle);
renderer.drawText(fontId, wordX, wordY, word, true, currentStyle, baseDir);
}
if ((currentStyle & EpdFontFamily::UNDERLINE) != 0) {
const std::string& w = words[i];
const int fullWordWidth = renderer.getTextWidth(fontId, w.c_str(), currentStyle);
// y is the top of the text line; add ascender to reach baseline, then offset 2px below
const int underlineY = y + renderer.getFontAscenderSize(fontId) + 2;
if (scanning) {
continue;
}
int startX = wordX;
int underlineWidth = fullWordWidth;
if (EpdFontFamily::hasTextDecoration(currentStyle)) {
int lineStartX = wordX;
int lineWidth = renderer.getTextWidth(fontId, word, currentStyle, baseDir);
// if word starts with em-space ("\xe2\x80\x83"), account for the additional indent before drawing the line
if (w.size() >= 3 && static_cast<uint8_t>(w[0]) == 0xE2 && static_cast<uint8_t>(w[1]) == 0x80 &&
static_cast<uint8_t>(w[2]) == 0x83) {
const char* visiblePtr = w.c_str() + 3;
const int prefixWidth = renderer.getTextAdvanceX(fontId, "\xe2\x80\x83", currentStyle);
const int visibleWidth = renderer.getTextWidth(fontId, visiblePtr, currentStyle);
startX = wordX + prefixWidth;
underlineWidth = visibleWidth;
if ((currentStyle & (EpdFontFamily::SUP | EpdFontFamily::SUB)) != 0) {
lineWidth = (lineWidth + 1) / 2;
}
renderer.drawLine(startX, underlineY, startX + underlineWidth, underlineY, true);
// Do not decorate the synthetic em-space used for paragraph indentation.
if (wordTextLen(i) >= 3 && static_cast<uint8_t>(word[0]) == 0xE2 && static_cast<uint8_t>(word[1]) == 0x80 &&
static_cast<uint8_t>(word[2]) == 0x83) {
const char* visibleText = word + 3;
lineStartX += renderer.getTextAdvanceX(fontId, "\xe2\x80\x83", currentStyle);
lineWidth = renderer.getTextWidth(fontId, visibleText, currentStyle, baseDir);
if ((currentStyle & (EpdFontFamily::SUP | EpdFontFamily::SUB)) != 0) {
lineWidth = (lineWidth + 1) / 2;
}
}
for (auto& line : decorationLines) {
if ((currentStyle & line.style) == 0) {
flushDecoration(line);
continue;
}
const int lineY = wordY + line.yOffset;
if (line.active() && line.yPos != lineY) {
flushDecoration(line);
}
if (!line.active()) {
line.startX = lineStartX;
line.yPos = lineY;
}
line.endX = lineStartX + lineWidth;
}
} else {
flushDecorations();
}
}
flushDecorations();
}
bool TextBlock::serialize(FsFile& file) const {
// Focus annotations are optional; vectors are either empty (no splits in this block)
// or sized in lockstep with words[].
const bool hasFocus = !wordFocusBoundary.empty();
if (words.size() != wordXpos.size() || words.size() != wordStyles.size() ||
(hasFocus && (words.size() != wordFocusBoundary.size() || words.size() != wordFocusSuffixX.size()))) {
LOG_ERR("TXB", "Serialization failed: size mismatch (words=%u, xpos=%u, styles=%u, boundary=%u, suffixX=%u)\n",
static_cast<uint32_t>(words.size()), static_cast<uint32_t>(wordXpos.size()),
static_cast<uint32_t>(wordStyles.size()), static_cast<uint32_t>(wordFocusBoundary.size()),
static_cast<uint32_t>(wordFocusSuffixX.size()));
bool TextBlock::serialize(HalFile& file) const {
if (!isValid) {
LOG_ERR("TXB", "Serialization failed: invalid block");
return false;
}
// Word data
serialization::writePod(file, static_cast<uint16_t>(words.size()));
for (const auto& w : words) serialization::writeString(file, w);
for (auto x : wordXpos) serialization::writePod(file, x);
for (auto s : wordStyles) serialization::writePod(file, s);
// Focus block: 1-byte presence flag, followed by per-word vectors only when present.
// Saves 3 bytes/word when focus reading is disabled or no word on this line was split.
serialization::writePod(file, static_cast<uint8_t>(hasFocus ? 1 : 0));
if (hasFocus) {
for (auto b : wordFocusBoundary) serialization::writePod(file, b);
for (auto sx : wordFocusSuffixX) serialization::writePod(file, sx);
// Word data: scalars, then the arena verbatim -- its in-memory layout is
// exactly the on-disk layout (see TextBlock.h), so one write covers all
// per-word arrays and the text blob.
serialization::writePod(file, numWords);
serialization::writePod(file, static_cast<uint8_t>(focusPresent ? 1 : 0));
serialization::writePod(file, textBytes);
if (numWords > 0) {
const size_t size = arenaSize(numWords, focusPresent, textBytes);
if (file.write(arena.get(), size) != size) {
LOG_ERR("TXB", "Serialization failed: arena write (%u bytes)", static_cast<uint32_t>(size));
return false;
}
}
// Style (alignment + margins/padding/indent)
@@ -106,47 +268,72 @@ bool TextBlock::serialize(FsFile& file) const {
serialization::writePod(file, blockStyle.paddingRight);
serialization::writePod(file, blockStyle.textIndent);
serialization::writePod(file, blockStyle.textIndentDefined);
serialization::writePod(file, blockStyle.isRtl);
serialization::writePod(file, blockStyle.directionDefined);
return true;
}
std::unique_ptr<TextBlock> TextBlock::deserialize(FsFile& file) {
std::unique_ptr<TextBlock> TextBlock::deserialize(HalFile& file) {
uint16_t wc;
std::vector<std::string> words;
std::vector<int16_t> wordXpos;
std::vector<EpdFontFamily::Style> wordStyles;
std::vector<uint8_t> wordFocusBoundary;
std::vector<uint16_t> wordFocusSuffixX;
BlockStyle blockStyle;
// Word count
uint8_t hasFocus;
uint16_t textBytes;
serialization::readPod(file, wc);
serialization::readPod(file, hasFocus);
serialization::readPod(file, textBytes);
// Sanity check: prevent allocation of unreasonably large vectors (max 10000 words per block)
// Sanity checks: cap the arena allocation and reject impossible geometry
// (every word carries at least its NUL terminator).
if (wc > 10000) {
LOG_ERR("TXB", "Deserialization failed: word count %u exceeds maximum", wc);
return nullptr;
}
if ((wc == 0 && textBytes != 0) || (wc > 0 && textBytes < wc)) {
LOG_ERR("TXB", "Deserialization failed: bad text size %u for %u words", textBytes, wc);
return nullptr;
}
// Word data
words.resize(wc);
wordXpos.resize(wc);
wordStyles.resize(wc);
for (auto& w : words) serialization::readString(file, w);
for (auto& x : wordXpos) serialization::readPod(file, x);
for (auto& s : wordStyles) serialization::readPod(file, s);
// Focus block: presence flag, then vectors only if present. Empty vectors when absent
// signal "no splits in this block" to render() (zero per-word RAM cost).
uint8_t hasFocus;
serialization::readPod(file, hasFocus);
if (hasFocus) {
wordFocusBoundary.resize(wc);
wordFocusSuffixX.resize(wc);
for (auto& b : wordFocusBoundary) serialization::readPod(file, b);
for (auto& sx : wordFocusSuffixX) serialization::readPod(file, sx);
std::unique_ptr<TextBlock> block(new (std::nothrow) TextBlock());
if (!block) {
LOG_ERR("TXB", "OOM: TextBlock");
return nullptr;
}
block->numWords = wc;
block->textBytes = textBytes;
block->focusPresent = hasFocus != 0;
if (wc > 0) {
const size_t size = arenaSize(wc, block->focusPresent, textBytes);
block->arena = makeUniqueNoThrow<uint8_t[]>(size);
if (!block->arena) {
LOG_ERR("TXB", "OOM: arena %u bytes", static_cast<uint32_t>(size));
return nullptr;
}
if (file.read(block->arena.get(), size) != size) {
LOG_ERR("TXB", "Deserialization failed: arena read (%u bytes)", static_cast<uint32_t>(size));
return nullptr;
}
block->bindArenaPointers();
// Validate offsets before anything dereferences wordText(): offset 0 first,
// strictly increasing, in bounds, and every word NUL-terminated (word i ends
// at the byte before offset i+1; the last word at the last text byte).
const uint16_t* textOff = block->textOffArr;
const char* text = block->textArr;
if (textOff[0] != 0 || text[textBytes - 1] != '\0') {
LOG_ERR("TXB", "Deserialization failed: corrupt text layout");
return nullptr;
}
for (uint16_t i = 1; i < wc; i++) {
if (textOff[i] <= textOff[i - 1] || textOff[i] >= textBytes || text[textOff[i] - 1] != '\0') {
LOG_ERR("TXB", "Deserialization failed: corrupt word offset %u", i);
return nullptr;
}
}
}
// Style (alignment + margins/padding/indent)
BlockStyle& blockStyle = block->blockStyle;
serialization::readPod(file, blockStyle.alignment);
serialization::readPod(file, blockStyle.textAlignDefined);
serialization::readPod(file, blockStyle.marginTop);
@@ -159,8 +346,8 @@ std::unique_ptr<TextBlock> TextBlock::deserialize(FsFile& file) {
serialization::readPod(file, blockStyle.paddingRight);
serialization::readPod(file, blockStyle.textIndent);
serialization::readPod(file, blockStyle.textIndentDefined);
serialization::readPod(file, blockStyle.isRtl);
serialization::readPod(file, blockStyle.directionDefined);
return std::unique_ptr<TextBlock>(new TextBlock(std::move(words), std::move(wordXpos), std::move(wordStyles),
std::move(wordFocusBoundary), std::move(wordFocusSuffixX),
blockStyle));
return block;
}
+71 -30
View File
@@ -9,44 +9,85 @@
#include "Block.h"
#include "BlockStyle.h"
// Represents a line of text on a page
// Represents a line of text on a page.
//
// All per-word data lives in ONE flat heap allocation (the arena) instead of
// six parallel vectors: a resident page holds ~25-30 of these blocks, and the
// vector-of-string layout cost ~250 throwing allocations per page load, which
// was the primary driver of heap fragmentation on the ESP32-C3.
//
// Arena layout, in order (2-byte alignment holds by construction: all 16-bit
// arrays come first and the arena base is allocator-aligned; RISC-V faults on
// unaligned multi-byte access):
// uint16_t textOff[wordCount] byte offset of word i's text in text[]
// int16_t xpos[wordCount]
// uint16_t focusSuffixX[wordCount] present only when focusPresent
// uint8_t styles[wordCount]
// uint8_t focusBoundary[wordCount] present only when focusPresent
// char text[textBytes] all words back to back, NUL-terminated
//
// Each word is stored NUL-terminated so render() can hand `text + textOff[i]`
// straight to C APIs (drawText) with no std::string materialization.
//
// Focus split semantics (unchanged from the vector layout): boundary N > 0
// means the first N bytes of word i render bold, the remainder in the base
// style. N is bounded to 9 codepoints (<= 36 UTF-8 bytes) by the clamp in
// ParsedText::addWord. focusSuffixX is the pre-computed pixel offset from the
// word start to the regular suffix. Both arrays are omitted from the arena
// entirely when no word on the line has a split (zero per-word RAM cost when
// focus reading is disabled).
class TextBlock final : public Block {
private:
std::vector<std::string> words;
std::vector<int16_t> wordXpos;
std::vector<EpdFontFamily::Style> wordStyles;
// Per-word focus boundary: N > 0 means the first N bytes of words[i] are rendered bold,
// the remainder in the base style. 0 means no split (whole word uses wordStyles[i]).
// N encodes the bold PREFIX length only — bounded to 9 codepoints (≤36 UTF-8 bytes) by
// FOCUS_READING_PERCENT's 1..9 clamp in ParsedText::addWord, so it always fits in uint8_t.
// Vector is empty when no focus splits exist anywhere in the block (zero per-word RAM cost
// when focus reading is disabled, or on lines that happen to contain no splittable words).
std::vector<uint8_t> wordFocusBoundary;
// Pre-computed pixel offset from word start to the regular suffix, stored when boundary > 0.
// Eliminates getTextAdvanceX from the render path. 0 when boundary == 0.
// Empty in lockstep with wordFocusBoundary.
std::vector<uint16_t> wordFocusSuffixX;
BlockStyle blockStyle;
uint16_t numWords = 0;
uint16_t textBytes = 0; // total size of the text region, including NULs
bool focusPresent = false;
bool isValid = true;
// The ONLY allocation: makeUniqueNoThrow, so OOM yields an invalid block
// instead of abort() (bare new is not nothrow with -fno-exceptions).
std::unique_ptr<uint8_t[]> arena;
// Typed views into the arena, bound once after the arena is filled. All
// 16-bit bases sit at even offsets, so direct dereference is alignment-safe.
const uint16_t* textOffArr = nullptr;
const int16_t* xposArr = nullptr;
const uint16_t* focusSuffixXArr = nullptr; // null when !focusPresent
const uint8_t* stylesArr = nullptr;
const uint8_t* focusBoundaryArr = nullptr; // null when !focusPresent
const char* textArr = nullptr;
TextBlock() = default; // deserialize() fills the fields directly
static size_t arenaSize(uint16_t wordCount, bool hasFocus, uint16_t textBytes);
void bindArenaPointers();
public:
explicit TextBlock(std::vector<std::string> words, std::vector<int16_t> word_xpos,
std::vector<EpdFontFamily::Style> word_styles, std::vector<uint8_t> focus_boundary,
std::vector<uint16_t> focus_suffix_x, const BlockStyle& blockStyle = BlockStyle())
: words(std::move(words)),
wordXpos(std::move(word_xpos)),
wordStyles(std::move(word_styles)),
wordFocusBoundary(std::move(focus_boundary)),
wordFocusSuffixX(std::move(focus_suffix_x)),
blockStyle(blockStyle) {}
// Flatten-on-construct: copies the layout-time vectors into the arena; the
// vectors die with the caller. On arena OOM the block is empty and valid()
// is false -- callers must check and fail the line instead of using it.
explicit TextBlock(const std::vector<std::string>& words, const std::vector<int16_t>& wordXpos,
const std::vector<EpdFontFamily::Style>& wordStyles, const std::vector<uint8_t>& focusBoundary,
const std::vector<uint16_t>& focusSuffixX, const BlockStyle& blockStyle = BlockStyle());
~TextBlock() override = default;
TextBlock(const TextBlock&) = delete;
TextBlock& operator=(const TextBlock&) = delete;
void setBlockStyle(const BlockStyle& blockStyle) { this->blockStyle = blockStyle; }
const BlockStyle& getBlockStyle() const { return blockStyle; }
const std::vector<std::string>& getWords() const { return words; }
bool isEmpty() override { return words.empty(); }
size_t wordCount() const { return words.size(); }
// given a renderer works out where to break the words into lines
bool isEmpty() override { return numWords == 0; }
bool valid() const { return isValid; }
uint16_t wordCount() const { return numWords; }
// NUL-terminated by construction; safe to pass to C APIs directly.
const char* wordText(const uint16_t i) const { return textArr + textOffArr[i]; }
uint16_t wordTextLen(const uint16_t i) const {
const uint16_t end = (i + 1 < numWords) ? textOffArr[i + 1] : textBytes;
return end - textOffArr[i] - 1; // exclude the NUL
}
int16_t wordXpos(const uint16_t i) const { return xposArr[i]; }
EpdFontFamily::Style wordStyle(const uint16_t i) const { return static_cast<EpdFontFamily::Style>(stylesArr[i]); }
uint8_t focusBoundary(const uint16_t i) const { return focusPresent ? focusBoundaryArr[i] : 0; }
uint16_t focusSuffixX(const uint16_t i) const { return focusPresent ? focusSuffixXArr[i] : 0; }
void render(const GfxRenderer& renderer, int fontId, int x, int y) const;
BlockType getType() override { return TEXT_BLOCK; }
bool serialize(FsFile& file) const;
static std::unique_ptr<TextBlock> deserialize(FsFile& file);
bool serialize(HalFile& file) const;
static std::unique_ptr<TextBlock> deserialize(HalFile& file);
};

Some files were not shown because too many files have changed in this diff Show More