## Problem
Many e-ink readers have limited image decoder support natively.
EPUBs with images in other formats than **baseline JPEG** frequently
cause:
- **Broken images**: pages render as blank, corrupted noise, or never
load
- **Slow rendering**: unoptimized images cause severe delays on e-ink
hardware, up to 7 seconds per page turn, with cover images taking up to
59 seconds to render
- **Broken covers**: the book thumbnail never generates
Fixing this today requires external tools before uploading.
---
## What this PR does
Adds an **optional, on-demand EPUB optimizer** to the file upload flow.
When enabled,
it converts all images to baseline JPEG directly in the browser — no
server, no internet,
no external tools needed.
**Conversion is opt-in. The standard upload flow is unchanged.**
---
## Real-world impact
The optimizer was applied in batch to **61 EPUBs**:
- 60 standard EPUBs: 198 MB → 55 MB (**−72.2%**, 143 MB saved)
- Text-dominant books: 8–46% smaller (covers and inline images
converted)
- Image-heavy / illustrated books: 65–93% smaller
- 1 Large manga volume (594 MB): 594 MB → 72 MB (**−87.8%**, 522 MB
saved)
- EPUB structural integrity fully maintained — zero new validation
issues introduced across all 61 books
*Size and integrity analysis:
[epub-comparator](https://github.com/pablohc/epub-comparator)*
From that set, **17 books were selected** as a representative sample
covering different content
types: image-heavy novels, pure manga, light novels with broken images,
and text-dominant books.
Each was benchmarked on two devices running in parallel, one on `master`
and one
on `PR#1224` — measuring render time across ~30 pages per book on
average.
### Rendering bugs fixed
| Book | Problem (original) | After optimization |
|------|--------------------|--------------------|
| Fairy Tale — Stephen King | Cover took **59.7 s** to render | 2.1 s
(−96%) |
| Cycle of the Werewolf — Stephen King | Cover took **23.3 s** to render
| 1.7 s (−93%) |
| Tomie: Complete Deluxe Ed. — Junji Ito | Cover took **18.3 s** to
render | 2.0 s (−89%) |
| Joel Dicker — El tigre (Ed. Ilustrada) | Cover took **14.5 s** to
render | 1.4 s (−90%) |
| Jackson, Holly — Asesinato para principiantes | Cover failed
completely (blank) | 2.0 s ✓ |
| Sentenced to Be a Hero — Yen Press | Cover failed, **8 images failed
to load** | All fixed ✓ |
| Flynn, Gillian — Perdida | Cover failed completely (blank) | 1.6 s ✓ |
| Chandler, Raymond — Asesino en la lluvia | Cover failed completely
(blank) | 2.0 s ✓ |
### Page render times — image-heavy EPUBs (avg per page)
| Book | Pages | Avg original | Avg optimized | Improvement | File size
|
|------|-------|-------------|---------------|-------------|-----------|
| Fairy Tale — Stephen King | 30 | 3,028 ms | 1,066 ms | **−64.8%** |
32.4 MB → 9.1 MB (−72%) |
| Cycle of the Werewolf — Stephen King | 33 | 3,026 ms | 1,558 ms |
**−48.5%** | 35.1 MB → 2.9 MB (−92%) |
| Joel Dicker — El tigre (Ed. Ilustrada) | 16 | 1,846 ms | 1,051 ms |
**−43.1%** | 5.3 MB → 0.4 MB (−93%) |
| Tomie: Complete Deluxe Ed. — Junji Ito | 30 | 4,817 ms | 2,802 ms |
**−41.8%** | 593.8 MB → 72.2 MB (−87.8%) |
| Sentenced to Be a Hero — Yen Press | 30 | 1,719 ms | 1,388 ms |
**−19.2%** | 15.2 MB → 1.6 MB (−90%) |
### Text-heavy EPUBs — no regression
| Book | Pages | Avg original | Avg optimized | Delta |
|------|-------|-------------|---------------|-------|
| Christie — Asesinato en el Orient Express | 30 | 1,672 ms | 1,646 ms |
−1.6% |
| Flynn — Perdida | 30 | 1,327 ms | 1,291 ms | −2.7% |
| Dicker — La verdad sobre el caso Harry Quebert | 30 | 1,132 ms | 1,084
ms | −4.2% |
| Hammett — El halcón maltés | 30 | 1,009 ms | 966 ms | −4.3% |
| Chandler — Asesino en la lluvia | 30 | 989 ms | 1,007 ms | +1.8% |
*Differences within ±5% — consistent with device measurement noise.*
*Render time benchmark:
[epub-optimization-benchmark](https://github.com/pablohc/epub-optimization-benchmark)*
---
## How to use it
**Single file:**
1. Click **Upload** (top of the page) — a modal opens. Use **Choose
files** to select one EPUB from your device.
2. Check **Optimize**.
- *(Optional)* Expand **Advanced Mode** — adjust quality, rotation, or
overlap; set individual images to H-Split / V-Split / Rotate.
3. Click **Optimize & Upload**.
**Batch (2+ files):**
1. Click **Upload** (top of the page) — a modal opens. Use **Choose
files** to select multiple EPUBs from your device.
2. Check **Optimize**.
- *(Optional)* Expand **Advanced Mode** — adjust quality.
3. Click **Upload** — all files are converted and uploaded sequentially.
Upload a batch of files, without optimization:
<img width="810" height="671" alt="image"
src="https://github.com/user-attachments/assets/d892ae13-0b87-4ea4-b6b8-340d56efc763"
/>
Batch file upload, with standard optimization:
<img width="809" height="707" alt="image"
src="https://github.com/user-attachments/assets/d32dbc88-1208-4555-bfcf-330ab91d2174"
/>
Optimization Phase (1/2):
<img width="807" height="1055" alt="image"
src="https://github.com/user-attachments/assets/fd4cd5f9-e56e-4ca1-9777-6926b9baf2bb"
/>
Upload Phase (2/2):
<img width="805" height="1065" alt="image"
src="https://github.com/user-attachments/assets/483294f0-02f0-4569-ae11-c10b3581d747"
/>
Batch upload successfully confirmed:
<img width="812" height="1043" alt="image"
src="https://github.com/user-attachments/assets/80c135bf-05c3-4c80-8755-2a04c68235bc"
/>
---
## Options
**Always active when the converter is enabled:**
- Converts PNG, WebP, BMP, GIF → baseline JPEG
- Smart downscaling to 480×800 px max (preserves aspect ratio)
- True grayscale for e-ink (BT.709 luminance, always on)
- SVG cover fix + OPF/NCX compliance repairs
**Advanced Mode (opt-in) — single file:**
- JPEG quality presets: 30% / 45% / 60% / 75% / **85%** (default) / 95%
- Rotation direction for split images: CW (default) / CCW
- Min overlap when splitting: 5% (default) / 10% / 15%
- Auto-download conversion log toggle (detailed stats per image)
- Per-image picker: set Normal / H-Split / V-Split / Rotate per image
individually,
with "Apply to all" for bulk assignment
**Advanced Mode (opt-in) — batch (2+ files):**
- JPEG quality presets: 30% / 45% / 60% / 75% / **85%** (default) / 95%
- Auto-download conversion log toggle (aggregated stats for all files)
---
## ⚠️ Known limitations
**KoReader hash-based sync will break** for converted files. The file
content changes,
so the hash no longer matches the original. Filename-based sync is
unaffected.
If you rely on KoReader hash sync, use the Calibre plugin or the web
tool instead.
---
## Build size impact
| Metric | master (a01bb29) | PR #1224 (a2ba5db) | Delta |
|---------------|------------------|--------------------|----------------|
| Flash used | 5,557 KB | 5,616 KB | +59 KB (+1.1%) |
| Flash free | 843 KB | 784 KB | −59 KB |
| Flash usage | 86.8% | 87.7% | +0.9 pp |
| RAM used | 95,156 B | 95,156 B | no change |
> Both builds compiled with `gh_release` environment in release mode
(ESP32-C3, 6,400 KB Flash).
> The +59 KB increase is entirely due to `jszip.min.js` embedded as a
> gzipped static asset served from Flash. RAM usage is identical,
> confirming no runtime overhead — the library runs in the browser,
> not on the ESP32. ~784 KB of Flash remain available.
---
## Alternatives considered
| Approach | Friction |
|----------|---------|
| **This PR** — integrated in upload flow | Zero: convert + upload in
one step, offline, any browser |
| Calibre plugin (in parallel development) | Requires a computer with
Calibre installed, same network |
| Web converters | Requires extra upload / download / transfer steps |
---
## Credits
Based on the converter algorithm developed by @zgredex.
Co-authored-by: @zgredex
---
### AI Usage
Did you use AI tools to help write this code? **PARTIALLY**
---------
Co-authored-by: zgredex <zgredex@users.noreply.github.com>
## Summary
* **What is the goal of this PR?** Add setting to display hidden files /
directories in filebrowser / web file browser
* **What changes are included?**
## Additional Context
-
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< NO >**_
## Summary
**What is the goal of this PR?**
This change avoids the pattern of creating a `std::string` using
`.substr` in order to compare against a file extension literal.
```c++
std::string path;
if (path.length() >= 4 && path.substr(path.length() - 4) == ".ext")
```
The `checkFileExtension` utility has moved from StringUtils to
FsHelpers, to be available to code in lib/. The signature now accepts a
`std::string_view` instead of `std::string`, which makes the single
implementation reusable for Arduino `String`.
Added utility functions for commonly repeated extensions.
These changes **save about 2 KB of flash (5,999,427 to 5,997,343)**.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
* **What is the goal of this PR?** Eliminate the 3-file / 4-location
overhead for adding a new setting. Previously, every new setting
required manually editing JsonSettingsIO.cpp in two places (save +
load), duplicating knowledge already present in SettingsList.h. After
this PR, JsonSettingsIO.cpp never needs to be touched again for standard
settings.
* **What changes are included?**
* `SettingInfo` (in `SettingsActivity.h`) gains one new field: `bool
obfuscated` (base64 save/load for passwords), with a fluent builder
method `.withObfuscated()`. The previously proposed
`defaultValue`/`withDefault()` approach was dropped in favour of reading
the struct field's own initializer value as the fallback (see below).
* `SettingsList.h` entries are annotated with `.withObfuscated()` on the
OPDS password entry. The list is now returned as a `static const`
singleton (`const std::vector<SettingInfo>&`), so it is constructed
exactly once. A missing `key`/`category` on the
`statusBarProgressBarThickness` entry was also fixed — it was previously
skipped by the generic save loop, so changes were silently lost on
restart.
* `JsonSettingsIO::saveSettings` and `loadSettings` replace their ~90
lines of manual per-field code with a single generic loop over
`getSettingsList()`. The loop uses `info.key`,
`info.valuePtr`/`info.stringOffset`+`info.stringMaxLen` (for char-array
string fields), `info.enumValues.size()` (for enum clamping), and
`info.obfuscated`.
* **Default values**: instead of a duplicated `defaultValue` field in
`SettingInfo`, `loadSettings` reads `s.*(info.valuePtr)` *before*
overwriting it. Because `CrossPointSettings` is default-constructed
before `loadSettings` is called, this captures each field's struct
initializer value as the JSON-absent fallback. The single source of
truth for defaults is `CrossPointSettings.h`.
* One post-loop special case remains explicitly: the four `frontButton*`
remap fields (managed by the RemapFrontButtons sub-activity, not in
SettingsList) and `validateFrontButtonMapping()`.
* One pre-loop migration guard handles legacy settings files that
predate the status bar refactor: if `statusBarChapterPageCount` is
absent from the JSON, `applyLegacyStatusBarSettings()` is called first
so the generic loop picks up the migrated values as defaults and applies
its normal clamping.
* OPDS password backward-compat migration (plain `opdsPassword` →
obfuscated `opdsPassword_obf`) is preserved inside the generic
obfuscated-string path.
## Additional Context
Say we want to add a new `bookmarkStyle` enum setting with options
`DOT`, `LINE`, `NONE` and a default of `DOT`:
1. `src/CrossPointSettings.h` — add enum and member:
```cpp
enum BOOKMARK_STYLE { BOOKMARK_DOT = 0, BOOKMARK_LINE = 1, BOOKMARK_NONE = 2 };
uint8_t bookmarkStyle = BOOKMARK_DOT;
```
2. `lib/I18n/translations/english.yaml` — add display strings:
```yaml
STR_BOOKMARK_STYLE: "Bookmark Style"
STR_BOOKMARK_DOT: "Dot"
STR_BOOKMARK_LINE: "Line"
```
(Other language files will fall back to English if not translated. Run
`gen_i18n.py` to regenerate `I18nKeys.h`.)
3. `src/SettingsList.h` — add one entry in the appropriate category:
```cpp
SettingInfo::Enum(StrId::STR_BOOKMARK_STYLE, &CrossPointSettings::bookmarkStyle,
{StrId::STR_BOOKMARK_DOT, StrId::STR_BOOKMARK_LINE, StrId::STR_NONE_OPT},
"bookmarkStyle", StrId::STR_CAT_READER),
```
That's it — no default annotation needed anywhere, because
`bookmarkStyle = BOOKMARK_DOT` in the struct already provides the
fallback. The setting will automatically persist to JSON on save, load
with clamping on boot, appear in the device settings UI under the Reader
category, and be exposed via the web API — all with no further changes.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< PARTIALLY>**_
## Summary
* Refactored `HttpDownloader::downloadToFile` to use `FileWriteStream`
and `HTTPClient::writeToStream`, removing manual chunked downloading
logic, which was error-prone.
* Fixes
https://github.com/crosspoint-reader/crosspoint-reader/issues/632
## Additional Context
* Tested downloading files from OPDS with a size up to 10 mb.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< YES >**_
## Summary
* **What is the goal of this PR?** Add a link to files to download on
web server
## Additional Context
* There was already an api to download files, i just added a link to it
for files.
---
### AI Usage
Did you use AI tools to help write this code? **NO**
Addresses #621
---------
Co-authored-by: Eliz Kilic <elizk@google.com>
## Summary
Ref:
https://github.com/crosspoint-reader/crosspoint-reader/pull/1047#discussion_r2838439305
To reproduce:
1. Open file transfer
2. Join a network
3. Once it's connected, press (hold) back
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **NO**
jpirnay