## Problem
Many e-ink readers have limited image decoder support natively.
EPUBs with images in other formats than **baseline JPEG** frequently
cause:
- **Broken images**: pages render as blank, corrupted noise, or never
load
- **Slow rendering**: unoptimized images cause severe delays on e-ink
hardware, up to 7 seconds per page turn, with cover images taking up to
59 seconds to render
- **Broken covers**: the book thumbnail never generates
Fixing this today requires external tools before uploading.
---
## What this PR does
Adds an **optional, on-demand EPUB optimizer** to the file upload flow.
When enabled,
it converts all images to baseline JPEG directly in the browser — no
server, no internet,
no external tools needed.
**Conversion is opt-in. The standard upload flow is unchanged.**
---
## Real-world impact
The optimizer was applied in batch to **61 EPUBs**:
- 60 standard EPUBs: 198 MB → 55 MB (**−72.2%**, 143 MB saved)
- Text-dominant books: 8–46% smaller (covers and inline images
converted)
- Image-heavy / illustrated books: 65–93% smaller
- 1 Large manga volume (594 MB): 594 MB → 72 MB (**−87.8%**, 522 MB
saved)
- EPUB structural integrity fully maintained — zero new validation
issues introduced across all 61 books
*Size and integrity analysis:
[epub-comparator](https://github.com/pablohc/epub-comparator)*
From that set, **17 books were selected** as a representative sample
covering different content
types: image-heavy novels, pure manga, light novels with broken images,
and text-dominant books.
Each was benchmarked on two devices running in parallel, one on `master`
and one
on `PR#1224` — measuring render time across ~30 pages per book on
average.
### Rendering bugs fixed
| Book | Problem (original) | After optimization |
|------|--------------------|--------------------|
| Fairy Tale — Stephen King | Cover took **59.7 s** to render | 2.1 s
(−96%) |
| Cycle of the Werewolf — Stephen King | Cover took **23.3 s** to render
| 1.7 s (−93%) |
| Tomie: Complete Deluxe Ed. — Junji Ito | Cover took **18.3 s** to
render | 2.0 s (−89%) |
| Joel Dicker — El tigre (Ed. Ilustrada) | Cover took **14.5 s** to
render | 1.4 s (−90%) |
| Jackson, Holly — Asesinato para principiantes | Cover failed
completely (blank) | 2.0 s ✓ |
| Sentenced to Be a Hero — Yen Press | Cover failed, **8 images failed
to load** | All fixed ✓ |
| Flynn, Gillian — Perdida | Cover failed completely (blank) | 1.6 s ✓ |
| Chandler, Raymond — Asesino en la lluvia | Cover failed completely
(blank) | 2.0 s ✓ |
### Page render times — image-heavy EPUBs (avg per page)
| Book | Pages | Avg original | Avg optimized | Improvement | File size
|
|------|-------|-------------|---------------|-------------|-----------|
| Fairy Tale — Stephen King | 30 | 3,028 ms | 1,066 ms | **−64.8%** |
32.4 MB → 9.1 MB (−72%) |
| Cycle of the Werewolf — Stephen King | 33 | 3,026 ms | 1,558 ms |
**−48.5%** | 35.1 MB → 2.9 MB (−92%) |
| Joel Dicker — El tigre (Ed. Ilustrada) | 16 | 1,846 ms | 1,051 ms |
**−43.1%** | 5.3 MB → 0.4 MB (−93%) |
| Tomie: Complete Deluxe Ed. — Junji Ito | 30 | 4,817 ms | 2,802 ms |
**−41.8%** | 593.8 MB → 72.2 MB (−87.8%) |
| Sentenced to Be a Hero — Yen Press | 30 | 1,719 ms | 1,388 ms |
**−19.2%** | 15.2 MB → 1.6 MB (−90%) |
### Text-heavy EPUBs — no regression
| Book | Pages | Avg original | Avg optimized | Delta |
|------|-------|-------------|---------------|-------|
| Christie — Asesinato en el Orient Express | 30 | 1,672 ms | 1,646 ms |
−1.6% |
| Flynn — Perdida | 30 | 1,327 ms | 1,291 ms | −2.7% |
| Dicker — La verdad sobre el caso Harry Quebert | 30 | 1,132 ms | 1,084
ms | −4.2% |
| Hammett — El halcón maltés | 30 | 1,009 ms | 966 ms | −4.3% |
| Chandler — Asesino en la lluvia | 30 | 989 ms | 1,007 ms | +1.8% |
*Differences within ±5% — consistent with device measurement noise.*
*Render time benchmark:
[epub-optimization-benchmark](https://github.com/pablohc/epub-optimization-benchmark)*
---
## How to use it
**Single file:**
1. Click **Upload** (top of the page) — a modal opens. Use **Choose
files** to select one EPUB from your device.
2. Check **Optimize**.
- *(Optional)* Expand **Advanced Mode** — adjust quality, rotation, or
overlap; set individual images to H-Split / V-Split / Rotate.
3. Click **Optimize & Upload**.
**Batch (2+ files):**
1. Click **Upload** (top of the page) — a modal opens. Use **Choose
files** to select multiple EPUBs from your device.
2. Check **Optimize**.
- *(Optional)* Expand **Advanced Mode** — adjust quality.
3. Click **Upload** — all files are converted and uploaded sequentially.
Upload a batch of files, without optimization:
<img width="810" height="671" alt="image"
src="https://github.com/user-attachments/assets/d892ae13-0b87-4ea4-b6b8-340d56efc763"
/>
Batch file upload, with standard optimization:
<img width="809" height="707" alt="image"
src="https://github.com/user-attachments/assets/d32dbc88-1208-4555-bfcf-330ab91d2174"
/>
Optimization Phase (1/2):
<img width="807" height="1055" alt="image"
src="https://github.com/user-attachments/assets/fd4cd5f9-e56e-4ca1-9777-6926b9baf2bb"
/>
Upload Phase (2/2):
<img width="805" height="1065" alt="image"
src="https://github.com/user-attachments/assets/483294f0-02f0-4569-ae11-c10b3581d747"
/>
Batch upload successfully confirmed:
<img width="812" height="1043" alt="image"
src="https://github.com/user-attachments/assets/80c135bf-05c3-4c80-8755-2a04c68235bc"
/>
---
## Options
**Always active when the converter is enabled:**
- Converts PNG, WebP, BMP, GIF → baseline JPEG
- Smart downscaling to 480×800 px max (preserves aspect ratio)
- True grayscale for e-ink (BT.709 luminance, always on)
- SVG cover fix + OPF/NCX compliance repairs
**Advanced Mode (opt-in) — single file:**
- JPEG quality presets: 30% / 45% / 60% / 75% / **85%** (default) / 95%
- Rotation direction for split images: CW (default) / CCW
- Min overlap when splitting: 5% (default) / 10% / 15%
- Auto-download conversion log toggle (detailed stats per image)
- Per-image picker: set Normal / H-Split / V-Split / Rotate per image
individually,
with "Apply to all" for bulk assignment
**Advanced Mode (opt-in) — batch (2+ files):**
- JPEG quality presets: 30% / 45% / 60% / 75% / **85%** (default) / 95%
- Auto-download conversion log toggle (aggregated stats for all files)
---
## ⚠️ Known limitations
**KoReader hash-based sync will break** for converted files. The file
content changes,
so the hash no longer matches the original. Filename-based sync is
unaffected.
If you rely on KoReader hash sync, use the Calibre plugin or the web
tool instead.
---
## Build size impact
| Metric | master (a01bb29) | PR #1224 (a2ba5db) | Delta |
|---------------|------------------|--------------------|----------------|
| Flash used | 5,557 KB | 5,616 KB | +59 KB (+1.1%) |
| Flash free | 843 KB | 784 KB | −59 KB |
| Flash usage | 86.8% | 87.7% | +0.9 pp |
| RAM used | 95,156 B | 95,156 B | no change |
> Both builds compiled with `gh_release` environment in release mode
(ESP32-C3, 6,400 KB Flash).
> The +59 KB increase is entirely due to `jszip.min.js` embedded as a
> gzipped static asset served from Flash. RAM usage is identical,
> confirming no runtime overhead — the library runs in the browser,
> not on the ESP32. ~784 KB of Flash remain available.
---
## Alternatives considered
| Approach | Friction |
|----------|---------|
| **This PR** — integrated in upload flow | Zero: convert + upload in
one step, offline, any browser |
| Calibre plugin (in parallel development) | Requires a computer with
Calibre installed, same network |
| Web converters | Requires extra upload / download / transfer steps |
---
## Credits
Based on the converter algorithm developed by @zgredex.
Co-authored-by: @zgredex
---
### AI Usage
Did you use AI tools to help write this code? **PARTIALLY**
---------
Co-authored-by: zgredex <zgredex@users.noreply.github.com>
## Summary
* **What is the goal of this PR?** Add a link to files to download on
web server
## Additional Context
* There was already an api to download files, i just added a link to it
for files.
---
### AI Usage
Did you use AI tools to help write this code? **NO**
Addresses #621
---------
Co-authored-by: Eliz Kilic <elizk@google.com>
## Summary
* **What is the goal of this PR?** Enhances the file manager with
multi-select deletion functionality and improved UI formatting.
* **What changes are included?**
* Added multi-select capability for file deletion in the web interface
* Fixed formatting issues in file table for folder rows
* Updated [.gitignore] to exclude additional build artifacts and cache
files
* Refactored CrossPointWebServer.cpp to support batch file deletion
* Enhanced FilesPage.html with improved UI for file selection and
deletion
## Additional Context
* The file deletion endpoint now handles multiple files in a single
request, improving efficiency when removing multiple files
* Changes are focused on the web file manager component only
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY**_
---------
Co-authored-by: Jessica Harrison <jessica.harrison@entelect.co.za>
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
Implement automatic dark theme on server files.
Instead of a big change proposed in
https://github.com/crosspoint-reader/crosspoint-reader/pull/837, this PR
introduces a simple implementation of light/dark themes.
* **What changes are included?**
- Choose `#6e9a82` as accent color (taken from
)
- Implement a very basic media query for dark themes (`@media
(prefers-color-scheme: dark)`)
- Update style using CSS variables
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
specific areas to focus on).
We can think of it as a incremental enhancement, this is the first phase
of a series of PRs (hopefully).
Next steps/Phases:
1. Light/Dark themes (this PR)
2. Load external CSS file to avoid duplication
3. HTML enhancement (for example, use dialog element instead of divs)
4. Use SVG instead of emojis
5. Use Vite + Typescript to improve DX and have better minification
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< YES | PARTIALLY | NO
>**_
---------
Co-authored-by: carlosbonadeo <carlosbonadeo@skyscanner.net>
## Summary
This is an updated version of @itsthisjustin's #346 that builds on
current master and also deduplicates the settings list so we don't have
two copies of the settings. In the Web UI, it should organize the
settings a little closer to what you see on device.
## Additional Context
I tested this live on device and it seems to play nicely for me. It's
re-based on master since master's settings stuff has moved somewhat
since the original PR and addresses the sole review comment #346 - it
also means that I don't need to manually key in the URL for my OPDS
server. :)
---
### AI Usage
My changes were implemented with Claude Opus 4.5 and Claude Code 2.1.25.
I don't know if @itsthisjustin's original work used AI assistance.
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
This adds renaming and moving files to the File Manager
* **What changes are included?**
New `/move` and `/rename` endpoints, and corresponding modals and icons
added. Uses the `file.rename()` function, after sanity checking.
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
specific areas to focus on).
Fixes#559, #661, #663. Only touches the File Manager, so low risk of
affecting other systems.
Simpler than #619, at the cost of not migrating the cache of renamed
books.
<img width="870" height="437" alt="image"
src="https://github.com/user-attachments/assets/73e0e750-dfc8-48e0-a7a6-9694470b7ded"
/>
<img width="575" height="318" alt="image"
src="https://github.com/user-attachments/assets/38c5fb19-c38a-436b-b3ad-75c1be7375ab"
/>
<img width="574" height="293" alt="image"
src="https://github.com/user-attachments/assets/1d2a2403-765d-473f-8c4f-c6968e9bbfeb"
/>
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
I used Codex for the implementation itself, and then carefully reviewed
the code myself. As this is a simple change and only to the webserver,
it is low risk.
## Summary
Resolves#562
Implements regex change to support valid characters discussed by
@daveallie in issue
[here](https://github.com/crosspoint-reader/crosspoint-reader/issues/562#issuecomment-3830809156).
Also rejects `.` and `..` as folder names which are invalid in FAT32 and
exFAT filesystems
## Additional Context
- Unsure on the wording for the alert, it feels overly explicit, but
that might be a good thing. Happy to change.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< PARTIALLY >**_
## Summary
* **What is the goal of this PR?** Fix WiFi file transfer stability
issues (especially crashes during uploads) and improve upload speed via
WebSocket binary protocol. File transfers now don't really crash as
much, if they do it recovers and speed has gone form 50KB/s to 300+KB/s.
* **What changes are included?**
- **WebSocket upload support** - Adds WebSocket binary protocol for file
uploads, achieving faster speeds 335 KB/s vs HTTP multipart)
- **Watchdog stability fixes** - Adds `esp_task_wdt_reset()` calls
throughout upload path to prevent watchdog timeouts during:
- File creation (FAT allocation can be slow)
- SD card write operations
- HTTP header parsing
- WebSocket chunk processing
- **4KB write buffering** - Batches SD card writes to reduce I/O
overhead
- **WiFi health monitoring** - Detects WiFi disconnection in STA mode
and exits gracefully
- **Improved handleClient loop** - 500 iterations with periodic watchdog
resets and button checks for responsiveness
- **Progress bar improvements** - Fixed jumping/inaccurate progress by
capping local progress at 95% until server confirms completion
- **Exit button responsiveness** - Button now checked inside the
handleClient loop every 64 iterations
- **Reduced exit delays** - Decreased shutdown delays from ~850ms to
~140ms
**Files changed:**
- `platformio.ini` - Added WebSockets library dependency
- `CrossPointWebServer.cpp/h` - WebSocket server, upload buffering,
watchdog resets
- `CrossPointWebServerActivity.cpp` - WiFi monitoring, improved loop,
button handling
- `FilesPage.html` - WebSocket upload JavaScript with HTTP fallback
## Additional Context
- WebSocket uses 4KB chunks with backpressure management to prevent
ESP32 buffer overflow
- Falls back to HTTP automatically if WebSocket connection fails
- The main bottleneck now is SD card write speed (~44% of transfer
time), not WiFi
- STA mode was more prone to crashes than AP mode due to external
network factors; WiFi health monitoring helps detect and handle
disconnections gracefully
---
### AI Usage
Did you use AI tools to help write this code? _**YES**_ Claude did it
ALL, I have no idea what I am doing, but my books transfer fast now.
---------
Co-authored-by: Claude <noreply@anthropic.com>
## Summary
* HTML files are now static, streamed directly to the client without
modification
* For any dynamic values, load via JSON APIs
* For files page, we stream the JSON content as we scan the directory to
avoid holding onto too much data
## Additional details
* We were previously building up a very large string all generated on
the X4 directly, we should be leveraging the browser
* Fixes https://github.com/daveallie/crosspoint-reader/issues/94
jpirnay