Script-range boundaries alone don't bound group size — dense Unicode
blocks (CJK, user fonts) can produce groups of hundreds of KB. Add a
64 KB hard cap: when adding the next glyph would exceed it, close the
current group and start a new one with the same script ID.
64 KB is well above the largest current built-in group (~50 KB for
notosans_18 Cyrillic) and a safe transient malloc on the ESP32-C3.
The decompressor side already handles any number of groups per font.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The hot group buffer (formerly up to 50 KB BSS / heap) is removed.
During prewarm, each group is now decompressed into a transient malloc
that is freed immediately after its glyphs are extracted — only one
group buffer and the page buffer coexist at a time.
For getBitmap() cache misses (rare: only hit when a glyph wasn't
covered by prewarm), the group is also decompressed transiently and
freed after the single glyph is compacted into _hotGlyphBuf.
Peak heap during prewarm is now: page buffer + one group buffer.
Outside of prewarm, heap usage is only the page buffer itself.
Works correctly for large future fonts (Vietnamese, CJK) regardless
of group size, with no permanent allocation overhead.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace the 50 KB static BSS _hotGroupBuf array with a heap pointer
allocated once per font (ensureHotGroupBuf). The buffer is sized to the
largest group in the active font — 6 KB for small fonts, ~50 KB for 18pt
— so no memory is wasted when rendering body text at typical sizes.
The buffer persists across pages (freed only in freeHotGroup/deinit),
so there is exactly one malloc per font session rather than one per page.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- FontDecompressor: replace heap-allocated std::vector hot-group and glyph
scratch buffers with static BSS arrays, eliminating per-page malloc/free
and the heap fragmentation it causes during rendering
- FontDecompressor: add bounds checks on group/glyph buffer sizes; sort
prewarm group list by ascending index for sequential flash reads
- ParsedText: upgrade line-break cost function to Knuth-Plass cubic badness
(strongly penalises very loose lines, lenient on moderate looseness)
- ParsedText: exclude gaps before closing punctuation (. , ) » etc.) from
justification distribution so they stay at natural space width
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
**What is the goal of this PR?**
A tweak to the fixed-point x-advance and kerning calculations to ensure
that the spacing between any two glyphs is always calculated
consistently.
I noticed that sometimes I'd see common character pairs like "oo" more
than once on a page, and the distance between the two snapped to
different pixels depending on the running accumulated error for the line
of text.
This change uses a differential rounding approach where each glyph's
x-advance plus the kerning relative to the next glyph are combined in
fixed-point precision, then snapped to a pixel to draw the next glyph.
This results in a consistent inter-glyph spacing any time the same two
glyphs show up adjacent to each other, regardless of the accumulated
error across the line.
---
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY**_
## Summary
**What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
Fix prewarm perf when a page contains many styles.
The prewarm page buffer was a single slot, so each `prewarmCache` call
for a new font style freed the previous style's glyphs. On pages with
multiple styles (regular + bold + italic), only the last style was
prewarmed. The others fell through to the hot-group compaction path at
~2-3ms per glyph.
This was most visible on rich formatting (e.g. this [Czech prayer
book](https://stahuj.kancional.cz/e-kniha/kancional.epub) with bold
headings, italic liturgical text, and regular body), where page renders
took 3-5 seconds instead of ~700ms.
Fix: use up to 4 page buffer slots (one per font style) so all styles
stay prewarmed simultaneously.
Fixes#1450.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? PARTIALLY: to diagnose and
brainstorm solutions.
## Purpose
This PR includes some preparatory changes that are needed for an
upcoming performant CJK font feature. The changes have no impact on
render time and heap allocation for latin text. **Despite this, I think
these changes stand on their own as a better font
compression/decompression implementation.**
## Summary
- Font decompressor rewrite: Replaced the 4-slot LRU group cache with a
two-tier system — a page buffer (glyphs prewarmed before rendering
begins) and a hot-group fallback (last decompressed group retained for
non-prewarmed
glyphs).
- Byte-aligned compressed bitmap format: Glyph bitmaps within compressed
groups are now stored row-padded rather than tightly packed before
DEFLATE compression, improving compression ratios by making identical
pixel rows produce
identical byte patterns. Glyphs are compacted back to packed format on
demand at render time. Reduces flash size by 155 KB.
- Page prewarm system: Added `Page::collectText` and
`Page::getDominantStyle` to extract per-style glyph requirements before
rendering, and `GfxRenderer::prewarmFontCache` to pre-decompress only
the groups needed for the dominant style
— eliminating mid-render decompression for the common case.
- UTF-8 robustness fixes: `utf8NextCodepoint` now validates continuation
bytes and returns a replacement glyph on malformed input;
`ChapterHtmlSlimParser` correctly preserves incomplete multi-byte
sequences across word-buffer flush
boundaries rather than splitting them.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_ Architecture and
design was done by me, refined a bit by Claude. Code mostly by Claude,
but not entirely.
jpirnay