## Problem
Many e-ink readers have limited image decoder support natively.
EPUBs with images in other formats than **baseline JPEG** frequently
cause:
- **Broken images**: pages render as blank, corrupted noise, or never
load
- **Slow rendering**: unoptimized images cause severe delays on e-ink
hardware, up to 7 seconds per page turn, with cover images taking up to
59 seconds to render
- **Broken covers**: the book thumbnail never generates
Fixing this today requires external tools before uploading.
---
## What this PR does
Adds an **optional, on-demand EPUB optimizer** to the file upload flow.
When enabled,
it converts all images to baseline JPEG directly in the browser — no
server, no internet,
no external tools needed.
**Conversion is opt-in. The standard upload flow is unchanged.**
---
## Real-world impact
The optimizer was applied in batch to **61 EPUBs**:
- 60 standard EPUBs: 198 MB → 55 MB (**−72.2%**, 143 MB saved)
- Text-dominant books: 8–46% smaller (covers and inline images
converted)
- Image-heavy / illustrated books: 65–93% smaller
- 1 Large manga volume (594 MB): 594 MB → 72 MB (**−87.8%**, 522 MB
saved)
- EPUB structural integrity fully maintained — zero new validation
issues introduced across all 61 books
*Size and integrity analysis:
[epub-comparator](https://github.com/pablohc/epub-comparator)*
From that set, **17 books were selected** as a representative sample
covering different content
types: image-heavy novels, pure manga, light novels with broken images,
and text-dominant books.
Each was benchmarked on two devices running in parallel, one on `master`
and one
on `PR#1224` — measuring render time across ~30 pages per book on
average.
### Rendering bugs fixed
| Book | Problem (original) | After optimization |
|------|--------------------|--------------------|
| Fairy Tale — Stephen King | Cover took **59.7 s** to render | 2.1 s
(−96%) |
| Cycle of the Werewolf — Stephen King | Cover took **23.3 s** to render
| 1.7 s (−93%) |
| Tomie: Complete Deluxe Ed. — Junji Ito | Cover took **18.3 s** to
render | 2.0 s (−89%) |
| Joel Dicker — El tigre (Ed. Ilustrada) | Cover took **14.5 s** to
render | 1.4 s (−90%) |
| Jackson, Holly — Asesinato para principiantes | Cover failed
completely (blank) | 2.0 s ✓ |
| Sentenced to Be a Hero — Yen Press | Cover failed, **8 images failed
to load** | All fixed ✓ |
| Flynn, Gillian — Perdida | Cover failed completely (blank) | 1.6 s ✓ |
| Chandler, Raymond — Asesino en la lluvia | Cover failed completely
(blank) | 2.0 s ✓ |
### Page render times — image-heavy EPUBs (avg per page)
| Book | Pages | Avg original | Avg optimized | Improvement | File size
|
|------|-------|-------------|---------------|-------------|-----------|
| Fairy Tale — Stephen King | 30 | 3,028 ms | 1,066 ms | **−64.8%** |
32.4 MB → 9.1 MB (−72%) |
| Cycle of the Werewolf — Stephen King | 33 | 3,026 ms | 1,558 ms |
**−48.5%** | 35.1 MB → 2.9 MB (−92%) |
| Joel Dicker — El tigre (Ed. Ilustrada) | 16 | 1,846 ms | 1,051 ms |
**−43.1%** | 5.3 MB → 0.4 MB (−93%) |
| Tomie: Complete Deluxe Ed. — Junji Ito | 30 | 4,817 ms | 2,802 ms |
**−41.8%** | 593.8 MB → 72.2 MB (−87.8%) |
| Sentenced to Be a Hero — Yen Press | 30 | 1,719 ms | 1,388 ms |
**−19.2%** | 15.2 MB → 1.6 MB (−90%) |
### Text-heavy EPUBs — no regression
| Book | Pages | Avg original | Avg optimized | Delta |
|------|-------|-------------|---------------|-------|
| Christie — Asesinato en el Orient Express | 30 | 1,672 ms | 1,646 ms |
−1.6% |
| Flynn — Perdida | 30 | 1,327 ms | 1,291 ms | −2.7% |
| Dicker — La verdad sobre el caso Harry Quebert | 30 | 1,132 ms | 1,084
ms | −4.2% |
| Hammett — El halcón maltés | 30 | 1,009 ms | 966 ms | −4.3% |
| Chandler — Asesino en la lluvia | 30 | 989 ms | 1,007 ms | +1.8% |
*Differences within ±5% — consistent with device measurement noise.*
*Render time benchmark:
[epub-optimization-benchmark](https://github.com/pablohc/epub-optimization-benchmark)*
---
## How to use it
**Single file:**
1. Click **Upload** (top of the page) — a modal opens. Use **Choose
files** to select one EPUB from your device.
2. Check **Optimize**.
- *(Optional)* Expand **Advanced Mode** — adjust quality, rotation, or
overlap; set individual images to H-Split / V-Split / Rotate.
3. Click **Optimize & Upload**.
**Batch (2+ files):**
1. Click **Upload** (top of the page) — a modal opens. Use **Choose
files** to select multiple EPUBs from your device.
2. Check **Optimize**.
- *(Optional)* Expand **Advanced Mode** — adjust quality.
3. Click **Upload** — all files are converted and uploaded sequentially.
Upload a batch of files, without optimization:
<img width="810" height="671" alt="image"
src="https://github.com/user-attachments/assets/d892ae13-0b87-4ea4-b6b8-340d56efc763"
/>
Batch file upload, with standard optimization:
<img width="809" height="707" alt="image"
src="https://github.com/user-attachments/assets/d32dbc88-1208-4555-bfcf-330ab91d2174"
/>
Optimization Phase (1/2):
<img width="807" height="1055" alt="image"
src="https://github.com/user-attachments/assets/fd4cd5f9-e56e-4ca1-9777-6926b9baf2bb"
/>
Upload Phase (2/2):
<img width="805" height="1065" alt="image"
src="https://github.com/user-attachments/assets/483294f0-02f0-4569-ae11-c10b3581d747"
/>
Batch upload successfully confirmed:
<img width="812" height="1043" alt="image"
src="https://github.com/user-attachments/assets/80c135bf-05c3-4c80-8755-2a04c68235bc"
/>
---
## Options
**Always active when the converter is enabled:**
- Converts PNG, WebP, BMP, GIF → baseline JPEG
- Smart downscaling to 480×800 px max (preserves aspect ratio)
- True grayscale for e-ink (BT.709 luminance, always on)
- SVG cover fix + OPF/NCX compliance repairs
**Advanced Mode (opt-in) — single file:**
- JPEG quality presets: 30% / 45% / 60% / 75% / **85%** (default) / 95%
- Rotation direction for split images: CW (default) / CCW
- Min overlap when splitting: 5% (default) / 10% / 15%
- Auto-download conversion log toggle (detailed stats per image)
- Per-image picker: set Normal / H-Split / V-Split / Rotate per image
individually,
with "Apply to all" for bulk assignment
**Advanced Mode (opt-in) — batch (2+ files):**
- JPEG quality presets: 30% / 45% / 60% / 75% / **85%** (default) / 95%
- Auto-download conversion log toggle (aggregated stats for all files)
---
## ⚠️ Known limitations
**KoReader hash-based sync will break** for converted files. The file
content changes,
so the hash no longer matches the original. Filename-based sync is
unaffected.
If you rely on KoReader hash sync, use the Calibre plugin or the web
tool instead.
---
## Build size impact
| Metric | master (a01bb29) | PR #1224 (a2ba5db) | Delta |
|---------------|------------------|--------------------|----------------|
| Flash used | 5,557 KB | 5,616 KB | +59 KB (+1.1%) |
| Flash free | 843 KB | 784 KB | −59 KB |
| Flash usage | 86.8% | 87.7% | +0.9 pp |
| RAM used | 95,156 B | 95,156 B | no change |
> Both builds compiled with `gh_release` environment in release mode
(ESP32-C3, 6,400 KB Flash).
> The +59 KB increase is entirely due to `jszip.min.js` embedded as a
> gzipped static asset served from Flash. RAM usage is identical,
> confirming no runtime overhead — the library runs in the browser,
> not on the ESP32. ~784 KB of Flash remain available.
---
## Alternatives considered
| Approach | Friction |
|----------|---------|
| **This PR** — integrated in upload flow | Zero: convert + upload in
one step, offline, any browser |
| Calibre plugin (in parallel development) | Requires a computer with
Calibre installed, same network |
| Web converters | Requires extra upload / download / transfer steps |
---
## Credits
Based on the converter algorithm developed by @zgredex.
Co-authored-by: @zgredex
---
### AI Usage
Did you use AI tools to help write this code? **PARTIALLY**
---------
Co-authored-by: zgredex <zgredex@users.noreply.github.com>
Store per-page paragraph indices in section cache to enable precise
XPath-to-page and page-to-XPath mapping without reparsing XHTML.
Forward path (upload): generates XPath directly from paragraph LUT
instead of byte-offset estimation, eliminating drift in chapters
with non-uniform content density.
Reverse path (download): resolves incoming KOReader XPath p[N] to
the exact page via paragraph LUT lookup.
Paragraph counter counts all <p> elements including display:none
to match ChapterXPathIndexer and crengine's standard XPath counting.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
## Summary
Fixes#1263
I spent half of my day(-off) reverse engineering the stock english
firmware V3.1.1, it's more or less like solving a sudoku with some known
pieces (like debug strings, known static addresses, known compiled
function, etc) and then the task is to guess the rest.
Long story short, this is the sleep routine that they use:
<img width="674" height="604" alt="image"
src="https://github.com/user-attachments/assets/6d53ce44-7bae-40c7-b4fb-24f898dbcc05"
/>
From the code above:
- They pull down GPIO13 (value = 0xd) before sleep
- They verify that power button is released by doing a delay loop of
50ms, similar to what we're doing
- `esp_sleep_config_gpio_isolate` is called but I'm not 100% sure why
- Pull up power button, note that it's likely redundant because power
button should already pulled up by `InputManager`
- `param1` and `param2` means enabling front/side buttons for wake up,
but it doesn't used in the code in reality. But I think it's physically
impossible, see the explanation below
- `param3` means "wake up from power button"
- `esp_sleep_start` is used; there is a logic to handle if it fails to
sleep, then retry recursively (no idea why!)
My observation is that they use GPIO13 so that it will be on HIGH state
when the chip is powered on, without any user space code to keep it on
that state. And once going to deep sleep, it goes into FLOATING by
default. That may explain why it need to be in LOW state before going to
sleep. (Nice trick btw)
Looking again at the circuit diagram provided
[here](https://github.com/sunwoods/Xteink-X4/blob/main/readme-img/sch.jpg)
(note: it's not official):
<img width="705" height="384" alt="image"
src="https://github.com/user-attachments/assets/b98d59fd-47ca-4d3d-a24a-94bf999e957b"
/>
It kinda make sense as the GPIO13 and VBUS (USB VCC) have the same role,
they are part of a simple "battery protection" cirtuit
Now, we may wonder, how the device wake up when there is no battery at
all?
<img width="440" height="323" alt="image"
src="https://github.com/user-attachments/assets/2981c411-239b-49a7-b9f7-9a75b6c1b6d3"
/>
It seems like power button is not just a simple switch between GPIO3 and
ground, but it also linked the POWER_CTRL, which leads to nowhere on the
diagram, but I suppose it connects the battery back for a short amount
of time, just enough for the MCU to wake up, and GPIO13 goes HIGH again.
It may also explain why power button becomes non-responsive for ~1
second after power on, as it's being pulled up by the current from
battery (remind: high = not pressed, low = pressed)
To test the theory above, I simply **comment out** the
`esp_deep_sleep_enable_gpio_wakeup`:
- On battery, power button works as nothing happen
- On USB, it doesn't wake up, I need to press RST
---
Important things about my analysis:
1. I had to name every function on the code above **manually**, but I'm
99% confident about it. The only function that I'm not sure is
`esp_wifi_bt_power_domain_off` ; Edit: it was indeed mislabeled, see
https://github.com/crosspoint-reader/crosspoint-reader/pull/1298#discussion_r2879670852
2. Some logic inside the stock firmware looks very strange, there is
almost no mention to "arduino" in the hardware, suggesting that they may
just call esp-idf functions directly, bypassing the arduino abstraction.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **NO**
---------
Co-authored-by: Zach Nelson <zach@zdnelson.com>
## Summary
**What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
Fix prewarm perf when a page contains many styles.
The prewarm page buffer was a single slot, so each `prewarmCache` call
for a new font style freed the previous style's glyphs. On pages with
multiple styles (regular + bold + italic), only the last style was
prewarmed. The others fell through to the hot-group compaction path at
~2-3ms per glyph.
This was most visible on rich formatting (e.g. this [Czech prayer
book](https://stahuj.kancional.cz/e-kniha/kancional.epub) with bold
headings, italic liturgical text, and regular body), where page renders
took 3-5 seconds instead of ~700ms.
Fix: use up to 4 page buffer slots (one per font style) so all styles
stay prewarmed simultaneously.
Fixes#1450.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? PARTIALLY: to diagnose and
brainstorm solutions.
## Summary
**What is the goal of this PR?**
This change deletes include/README, which is a PlatformIO boilerplate
placeholder file explaining what header files are. The include/
directory isn't used by this project (headers live in lib/ and src/), so
this is just cleanup.
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
* A simple tweak to pre-index the next chapter silently during normal
reading.
* Triggers silent pre-indexing of the next chapter when the penultimate
page of a chapter is rendered to reduce visible interruptions.
* Keeps existing indexing with popup when a reader jumps directly into
an unindexed chapter.
## Additional Context
* Reader input is temporarily blocked during silent indexing to avoid
navigation/index state conflicts.
* The penultimate page is used because readers typically spend longer
there than on the final page.
* This change optimizes linear reading flow while preserving reliable
indexing for non-linear navigation.
## Possible Improvements
* Add a setting for First Page Indexing vs Penultimate Page Pre-indexing
* Display an indexing icon in the status bar instead of using a popup
that overlaps book text.
Tested on device:
https://www.dropbox.com/scl/fi/29g5kjqgsi5e4hgujv38u/Silent-Indexing.MOV?rlkey=yemi4mosmev5vicaa7gpe49qw&dl=0
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
---------
Co-authored-by: Jake Kenneally <jakekenneally@gmail.com>
## Summary
**What is the goal of this PR?**
Update SKILL.md to stop instructing contributors to commit `I18nKeys.h`
and `I18nStrings.h`. All three generated i18n files have been gitignored
since 1b7dde07 and are regenerated at build time.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
* **What is the goal of this PR?** All praise goes to @didacta for his
PR #537. Just picked up the reviewer comments to contain the changes as
suggested (there was no response for more than 6 weeks, so I wanted to
reanimate this feature).
Just one addition: should recognize usb cable plug ins / retractions and
update the icon immediately
* **What changes are included?**
## Additional Context
see #537
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< NO >**_
jpirnay
