Commit Graph
76 Commits
Author SHA1 Message Date
jpirnay 999cf74c58 Account for deduplication 2026-05-01 09:25:15 +02:00
jpirnay ad2e3765ba Some environment cleaning 2026-04-28 18:09:36 +02:00
Joel Goguen 0010b200ed feat(gha): Auto-publish releases
- GHA release workflow is updated to use an action for uploading all assets. The
	action docs say it updates a release if one already exists, and since this
	triggers on release and we pass a release tag we know the release already
	exists.
- GHA release candidate workflow is updated to auto-publish a pre-release
	release on push to a `release/**` branch. A script is added and used to
	generate a release message made up of the commits since the last release of
	any kind.
- ACT profiles are added to allow locally testing these workflows without
	needing to push and wait for GHA to run.
- Docs are updated to note how to test the workflows locally.
2026-04-23 22:48:41 -04:00
jpirnay 7da8b335ab Review comments 2026-04-21 22:16:23 +02:00
jpirnay ed01127bf2 More changes 2026-04-12 22:55:07 +02:00
jpirnay a811c96d74 Some comments 2026-04-12 22:34:28 +02:00
jpirnay bec30e3403 Review fixes 2026-04-12 22:25:12 +02:00
jpirnay c6e94d6966 Exclude commented out strings 2026-04-12 06:48:05 +02:00
jpirnay ce81cfee04 Fixing script 2026-04-11 09:47:18 +02:00
jpirnay bfeb806a0b Review changes 2026-04-11 09:43:48 +02:00
jpirnay 130dacbd38 fix: Wild pointer crash in JPEGDEC MCU_SKIP handling#1627 by itsthisjustin 2026-04-10 11:54:27 +02:00
martin brook 773238fdf5 chore: drop JPEGDEC patch in favour of upstream fix (#1465)
## Summary

The progressive JPEG fixes (AC table skip, MCU_SKIP guard) from PR #1136
have been fixed upstream in bitbank2/JPEGDEC@8628297. Pin to that commit
and remove the pre-build patch script.

## Additional Context

Tested on Strange Pictures epub

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< PARTIALLY >**_
2026-04-08 10:32:30 +02:00
jpirnay abbf69649e Revert "Merge pull request #25 from jpirnay/fix-pr1582" and refactor koreader 2026-04-07 20:24:51 +02:00
jpirnay 22267f038a Updated testscripts 2026-04-06 12:06:21 +02:00
jpirnay db7d5af56b Add test cases 2026-04-06 11:55:25 +02:00
jpirnay 2d6d14af1f Making screenshot command device dimensions aware 2026-04-05 11:29:23 +02:00
jpirnay 97d4400b55 Merge branch 'refactor-i18n_offset-table' of https://github.com/jpirnay/crosspoint-reader into mybuild 2026-04-03 11:42:01 +02:00
jpirnay 03217d871d Switch to language_code instead of language _order key 2026-04-03 11:41:34 +02:00
jpirnay 6de5ebc69c Merge branch 'feat-weather' of https://github.com/jpirnay/crosspoint-reader into mybuild 2026-04-03 11:06:17 +02:00
jpirnay 01d05e654d Bugfixing and refactoring - proper menu icons 2026-04-03 11:03:10 +02:00
jpirnay adc4efa9e6 Switch to defusedxml 2026-04-03 10:17:42 +02:00
jpirnay 783828adb0 Merge branch 'feat-weather' of https://github.com/jpirnay/crosspoint-reader into mybuild 2026-04-02 20:19:43 +02:00
jpirnay b3a80c162a Review changes 2026-04-02 20:19:19 +02:00
jpirnay a4a0e6b5dc Merge branch 'feat-weather' of https://github.com/jpirnay/crosspoint-reader into mybuild 2026-04-02 18:37:55 +02:00
jpirnay d7bd88bec7 clang-format agnostic 2026-04-02 18:30:31 +02:00
jpirnay 811b4807cd Increase icon size a bit 2026-04-02 18:28:24 +02:00
jpirnay 706eb0e8e7 Improve look and feel 2026-04-02 18:16:49 +02:00
jpirnay cd69b4da77 Slight reformat 2026-03-31 15:09:15 +02:00
jpirnay cf252dc20d Merge branch 'refactor-i18n_offset-table' of https://github.com/jpirnay/crosspoint-reader into mybuild 2026-03-31 14:52:24 +02:00
jpirnay 59466b0625 A bit of reformatting 2026-03-31 14:45:31 +02:00
jpirnay c8b166606e Stage scripts 2026-03-30 18:12:52 +02:00
jpirnay f764d9c6a0 Merge remote-tracking branch 'pablohc/refactor/pr-1258-cover-overlay' into mybuild
# Conflicts:
#	lib/I18n/translations/english.yaml
#	scripts/gen_i18n.py
#	src/CrossPointSettings.h
#	src/SettingsList.h
#	src/activities/boot_sleep/SleepActivity.cpp
#	src/activities/boot_sleep/SleepActivity.h
2026-03-30 16:12:00 +02:00
pablohc 4e4fcd278a fix: gen_i18n.py wraps comment lines to avoid clang-format reflow 2026-03-29 22:48:15 +02:00
pablohc 4699ff0b88 fix: wrap long language comment in gen_i18n.py output 2026-03-29 22:44:43 +02:00
jpirnay a2aee4f4ae Merge branch 'fix-css-pre' of https://github.com/jpirnay/crosspoint-reader into mybuild 2026-03-28 18:16:11 +01:00
jpirnay 7f6933ab22 Recognize css pre elements 2026-03-28 16:47:55 +01:00
jpirnay 1d3405318c Integrate PT 1455 2026-03-26 11:16:53 +01:00
jpirnay 1af3f81336 PR 1507 2026-03-26 11:05:13 +01:00
jpirnay ac42d5ef36 Merge branch 'refactor-i18n_offset-table' of https://github.com/jpirnay/crosspoint-reader into mybuild 2026-03-26 11:02:18 +01:00
jpirnay 5da520eb9e Clean up language codes 2026-03-26 10:42:15 +01:00
jpirnay 3f5a2663f6 Exit if non-unique identifiers found 2026-03-26 10:30:14 +01:00
jpirnay e30fa6eb3a Merge branch 'perf-ws-upload-speed' of https://github.com/jpirnay/crosspoint-reader into mybuild 2026-03-25 13:07:32 +01:00
jpirnay e87f8035be Increase upload speed 2026-03-25 12:10:00 +01:00
jpirnay 84e3850d5d Merge branch 'feat-kosync-onlongpress' of https://github.com/jpirnay/crosspoint-reader into mybuild 2026-03-23 12:02:25 +01:00
pablohcandzgredex 5458155e79 feat: integrated epub optimizer (#1224)
## Problem

Many e-ink readers have limited image decoder support natively.
EPUBs with images in other formats than **baseline JPEG** frequently
cause:

- **Broken images**: pages render as blank, corrupted noise, or never
load
- **Slow rendering**: unoptimized images cause severe delays on e-ink
hardware, up to 7 seconds per page turn, with cover images taking up to
59 seconds to render
- **Broken covers**: the book thumbnail never generates

Fixing this today requires external tools before uploading.

---

## What this PR does

Adds an **optional, on-demand EPUB optimizer** to the file upload flow.
When enabled,
it converts all images to baseline JPEG directly in the browser — no
server, no internet,
no external tools needed.

**Conversion is opt-in. The standard upload flow is unchanged.**

---

## Real-world impact

The optimizer was applied in batch to **61 EPUBs**:
- 60 standard EPUBs: 198 MB → 55 MB (**−72.2%**, 143 MB saved)
- Text-dominant books: 8–46% smaller (covers and inline images
converted)
  - Image-heavy / illustrated books: 65–93% smaller
- 1 Large manga volume (594 MB): 594 MB → 72 MB (**−87.8%**, 522 MB
saved)
- EPUB structural integrity fully maintained — zero new validation
issues introduced across all 61 books

*Size and integrity analysis:
[epub-comparator](https://github.com/pablohc/epub-comparator)*

From that set, **17 books were selected** as a representative sample
covering different content
types: image-heavy novels, pure manga, light novels with broken images,
and text-dominant books.
Each was benchmarked on two devices running in parallel, one on `master`
and one
on `PR#1224` — measuring render time across ~30 pages per book on
average.

### Rendering bugs fixed

| Book | Problem (original) | After optimization |
|------|--------------------|--------------------|
| Fairy Tale — Stephen King | Cover took **59.7 s** to render | 2.1 s
(−96%) |
| Cycle of the Werewolf — Stephen King | Cover took **23.3 s** to render
| 1.7 s (−93%) |
| Tomie: Complete Deluxe Ed. — Junji Ito | Cover took **18.3 s** to
render | 2.0 s (−89%) |
| Joel Dicker — El tigre (Ed. Ilustrada) | Cover took **14.5 s** to
render | 1.4 s (−90%) |
| Jackson, Holly — Asesinato para principiantes | Cover failed
completely (blank) | 2.0 s ✓ |
| Sentenced to Be a Hero — Yen Press | Cover failed, **8 images failed
to load** | All fixed ✓ |
| Flynn, Gillian — Perdida | Cover failed completely (blank) | 1.6 s ✓ |
| Chandler, Raymond — Asesino en la lluvia | Cover failed completely
(blank) | 2.0 s ✓ |

### Page render times — image-heavy EPUBs (avg per page)

| Book | Pages | Avg original | Avg optimized | Improvement | File size
|
|------|-------|-------------|---------------|-------------|-----------|
| Fairy Tale — Stephen King | 30 | 3,028 ms | 1,066 ms | **−64.8%** |
32.4 MB → 9.1 MB (−72%) |
| Cycle of the Werewolf — Stephen King | 33 | 3,026 ms | 1,558 ms |
**−48.5%** | 35.1 MB → 2.9 MB (−92%) |
| Joel Dicker — El tigre (Ed. Ilustrada) | 16 | 1,846 ms | 1,051 ms |
**−43.1%** | 5.3 MB → 0.4 MB (−93%) |
| Tomie: Complete Deluxe Ed. — Junji Ito | 30 | 4,817 ms | 2,802 ms |
**−41.8%** | 593.8 MB → 72.2 MB (−87.8%) |
| Sentenced to Be a Hero — Yen Press | 30 | 1,719 ms | 1,388 ms |
**−19.2%** | 15.2 MB → 1.6 MB (−90%) |

### Text-heavy EPUBs — no regression

| Book | Pages | Avg original | Avg optimized | Delta |
|------|-------|-------------|---------------|-------|
| Christie — Asesinato en el Orient Express | 30 | 1,672 ms | 1,646 ms |
−1.6% |
| Flynn — Perdida | 30 | 1,327 ms | 1,291 ms | −2.7% |
| Dicker — La verdad sobre el caso Harry Quebert | 30 | 1,132 ms | 1,084
ms | −4.2% |
| Hammett — El halcón maltés | 30 | 1,009 ms | 966 ms | −4.3% |
| Chandler — Asesino en la lluvia | 30 | 989 ms | 1,007 ms | +1.8% |

*Differences within ±5% — consistent with device measurement noise.*

*Render time benchmark:
[epub-optimization-benchmark](https://github.com/pablohc/epub-optimization-benchmark)*

---
## How to use it

**Single file:**
1. Click **Upload** (top of the page) — a modal opens. Use **Choose
files** to select one EPUB from your device.
2. Check **Optimize**.
- *(Optional)* Expand **Advanced Mode** — adjust quality, rotation, or
overlap; set individual images to H-Split / V-Split / Rotate.
3. Click **Optimize & Upload**.

**Batch (2+ files):**
1. Click **Upload** (top of the page) — a modal opens. Use **Choose
files** to select multiple EPUBs from your device.
2. Check **Optimize**.
   - *(Optional)* Expand **Advanced Mode** — adjust quality.
3. Click **Upload** — all files are converted and uploaded sequentially.

Upload a batch of files, without optimization:
<img width="810" height="671" alt="image"
src="https://github.com/user-attachments/assets/d892ae13-0b87-4ea4-b6b8-340d56efc763"
/>

Batch file upload, with standard optimization:
<img width="809" height="707" alt="image"
src="https://github.com/user-attachments/assets/d32dbc88-1208-4555-bfcf-330ab91d2174"
/>

Optimization Phase (1/2):
<img width="807" height="1055" alt="image"
src="https://github.com/user-attachments/assets/fd4cd5f9-e56e-4ca1-9777-6926b9baf2bb"
/>

Upload Phase (2/2):
<img width="805" height="1065" alt="image"
src="https://github.com/user-attachments/assets/483294f0-02f0-4569-ae11-c10b3581d747"
/>

Batch upload successfully confirmed:
<img width="812" height="1043" alt="image"
src="https://github.com/user-attachments/assets/80c135bf-05c3-4c80-8755-2a04c68235bc"
/>

---

## Options

**Always active when the converter is enabled:**
- Converts PNG, WebP, BMP, GIF → baseline JPEG
- Smart downscaling to 480×800 px max (preserves aspect ratio)
- True grayscale for e-ink (BT.709 luminance, always on)
- SVG cover fix + OPF/NCX compliance repairs

**Advanced Mode (opt-in) — single file:**
- JPEG quality presets: 30% / 45% / 60% / 75% / **85%** (default) / 95%
- Rotation direction for split images: CW (default) / CCW
- Min overlap when splitting: 5% (default) / 10% / 15%
- Auto-download conversion log toggle (detailed stats per image)
- Per-image picker: set Normal / H-Split / V-Split / Rotate per image
individually,
  with "Apply to all" for bulk assignment

**Advanced Mode (opt-in) — batch (2+ files):**
- JPEG quality presets: 30% / 45% / 60% / 75% / **85%** (default) / 95%
- Auto-download conversion log toggle (aggregated stats for all files)

---

## ⚠️ Known limitations

**KoReader hash-based sync will break** for converted files. The file
content changes,
so the hash no longer matches the original. Filename-based sync is
unaffected.

If you rely on KoReader hash sync, use the Calibre plugin or the web
tool instead.

---
## Build size impact

| Metric | master (a01bb29) | PR #1224 (a2ba5db) | Delta |

|---------------|------------------|--------------------|----------------|
| Flash used | 5,557 KB | 5,616 KB | +59 KB (+1.1%) |
| Flash free | 843 KB | 784 KB | −59 KB |
| Flash usage | 86.8% | 87.7% | +0.9 pp |
| RAM used | 95,156 B | 95,156 B | no change |

> Both builds compiled with `gh_release` environment in release mode
(ESP32-C3, 6,400 KB Flash).
> The +59 KB increase is entirely due to `jszip.min.js` embedded as a
> gzipped static asset served from Flash. RAM usage is identical,
> confirming no runtime overhead — the library runs in the browser,
> not on the ESP32. ~784 KB of Flash remain available.

---

## Alternatives considered

| Approach | Friction |
|----------|---------|
| **This PR** — integrated in upload flow | Zero: convert + upload in
one step, offline, any browser |
| Calibre plugin (in parallel development) | Requires a computer with
Calibre installed, same network |
| Web converters | Requires extra upload / download / transfer steps |

---

## Credits

Based on the converter algorithm developed by @zgredex.
Co-authored-by: @zgredex

---

### AI Usage

Did you use AI tools to help write this code? **PARTIALLY**

---------

Co-authored-by: zgredex <zgredex@users.noreply.github.com>
2026-03-22 19:53:15 +00:00
jpirnay 5d8dc56f33 Minor changes 2026-03-19 17:58:45 +01:00
jpirnay f23d01093e Amend regex pattern for tr identification 2026-03-17 07:07:47 +01:00
jpirnay 2419cc43bf fix missing header 2026-03-16 23:52:13 +01:00
jpirnay 7c0095e5c5 Remove no longer needed include 2026-03-16 20:03:46 +01:00
jpirnay 2e8ee680a7 perf: strip unused strings during PlatformIO build 2026-03-16 15:47:35 +01:00