9b3885135f3afbb0539b68dbf0ffd99ee53febb5
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Adrian Wilkins-Caruana
|
0c9e8b3ece |
fix: Fix prewarm perf when a page contains many styles (#1451)
## Summary **What is the goal of this PR?** (e.g., Implements the new feature for file uploading.) Fix prewarm perf when a page contains many styles. The prewarm page buffer was a single slot, so each `prewarmCache` call for a new font style freed the previous style's glyphs. On pages with multiple styles (regular + bold + italic), only the last style was prewarmed. The others fell through to the hot-group compaction path at ~2-3ms per glyph. This was most visible on rich formatting (e.g. this [Czech prayer book](https://stahuj.kancional.cz/e-kniha/kancional.epub) with bold headings, italic liturgical text, and regular body), where page renders took 3-5 seconds instead of ~700ms. Fix: use up to 4 page buffer slots (one per font style) so all styles stay prewarmed simultaneously. Fixes #1450. --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? PARTIALLY: to diagnose and brainstorm solutions. |
||
|
Adrian Wilkins-Caruana
|
f1e9dc7f30 |
perf: font-compression improvements (#1056)
## Purpose This PR includes some preparatory changes that are needed for an upcoming performant CJK font feature. The changes have no impact on render time and heap allocation for latin text. **Despite this, I think these changes stand on their own as a better font compression/decompression implementation.** ## Summary - Font decompressor rewrite: Replaced the 4-slot LRU group cache with a two-tier system — a page buffer (glyphs prewarmed before rendering begins) and a hot-group fallback (last decompressed group retained for non-prewarmed glyphs). - Byte-aligned compressed bitmap format: Glyph bitmaps within compressed groups are now stored row-padded rather than tightly packed before DEFLATE compression, improving compression ratios by making identical pixel rows produce identical byte patterns. Glyphs are compacted back to packed format on demand at render time. Reduces flash size by 155 KB. - Page prewarm system: Added `Page::collectText` and `Page::getDominantStyle` to extract per-style glyph requirements before rendering, and `GfxRenderer::prewarmFontCache` to pre-decompress only the groups needed for the dominant style — eliminating mid-render decompression for the common case. - UTF-8 robustness fixes: `utf8NextCodepoint` now validates continuation bytes and returns a replacement glyph on malformed input; `ChapterHtmlSlimParser` correctly preserves incomplete multi-byte sequences across word-buffer flush boundaries rather than splitting them. --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**YES**_ Architecture and design was done by me, refined a bit by Claude. Code mostly by Claude, but not entirely. |
||
|
Adrian Wilkins-Caruana
|
052f497b9e |
fix: force auto-hinting for Bookerly to fix inconsistent stem widths (#1098)
## Summary Bookerly's native TrueType hinting is effectively a no-op at the sizes used here, causing FreeType to place stems at inconsistent sub-pixel positions. This results in the 'k' stem (8-bit fringe: 0x38=56) falling just below the 2-bit quantization threshold while 'l' and 'h' stems (fringes: 0x4C=76, 0x40=64) land above it --- making 'k' visibly narrower (2.00px vs 2.33px effective width). FreeType's auto-hinter snaps all stems to consistent grid positions, normalizing effective stem width to 2.67px across all glyphs. Adds --force-autohint flag to fontconvert.py and applies it to Bookerly only. NotoSans, OpenDyslexic, and Ubuntu fonts are unaffected. Here is an example of before/after. Take notice of the vertical stems on characters like `l`, `k`, `n`, `i`, etc. The font is Bookerly 12pt regular: **BEFORE**:  **AFTER**:  Claude generated this script to quantitatively determine that this change makes the vertical stems on a variety of characters more consistent for Bookerly _only_. <details> <summary>Python script</summary> ```python #!/usr/bin/env python3 """Compare stem consistency across all font families with and without auto-hinting. Run from repo root: python3 compare_all_fonts.py """ import freetype DPI = 150 CHARS = ["k", "l", "h", "i", "b", "d"] SIZES = [12, 14, 16, 18] FONTS = { "Bookerly": "lib/EpdFont/builtinFonts/source/Bookerly/Bookerly-Regular.ttf", "NotoSans": "lib/EpdFont/builtinFonts/source/NotoSans/NotoSans-Regular.ttf", "OpenDyslexic": "lib/EpdFont/builtinFonts/source/OpenDyslexic/OpenDyslexic-Regular.otf", "Ubuntu": "lib/EpdFont/builtinFonts/source/Ubuntu/Ubuntu-Regular.ttf", } MODES = { "default": freetype.FT_LOAD_RENDER, "autohint": freetype.FT_LOAD_RENDER | freetype.FT_LOAD_FORCE_AUTOHINT, } def q4to2(v): if v >= 12: return 3 elif v >= 8: return 2 elif v >= 4: return 1 else: return 0 def get_stem_eff(face, char, flags): gi = face.get_char_index(ord(char)) if gi == 0: return None face.load_glyph(gi, flags) bm = face.glyph.bitmap w, h = bm.width, bm.rows if w == 0 or h == 0: return None p2 = [] for y in range(h): row = [] for x in range(w): row.append(q4to2(bm.buffer[y * bm.pitch + x] >> 4)) p2.append(row) # Measure leftmost stem in stable middle rows mid_start, mid_end = h // 4, h - h // 4 widths = [] for y in range(mid_start, mid_end): first = next((x for x in range(w) if p2[y][x] > 0), -1) if first < 0: continue last = first for x in range(first, w): if p2[y][x] > 0: last = x else: break eff = sum(p2[y][x] for x in range(first, last + 1)) / 3.0 widths.append(eff) return round(sum(widths) / len(widths), 2) if widths else None def main(): for font_name, font_path in FONTS.items(): try: freetype.Face(font_path) except Exception: print(f"\n {font_name}: SKIPPED (file not found)") continue print(f"\n{'=' * 80}") print(f" {font_name}") print(f"{'=' * 80}") for size in SIZES: print(f"\n {size}pt:") print(f" {'':6s}", end="") for c in CHARS: print(f" '{c}' ", end="") print(" | spread") for mode_name, flags in MODES.items(): face = freetype.Face(font_path) face.set_char_size(size << 6, size << 6, DPI, DPI) vals = [] print(f" {mode_name:6s}", end="") for c in CHARS: v = get_stem_eff(face, c, flags) vals.append(v) print(f" {v:5.2f}" if v else " N/A", end="") valid = [v for v in vals if v is not None] spread = max(valid) - min(valid) if len(valid) >= 2 else 0 marker = " <-- inconsistent" if spread > 0.5 else "" print(f" | {spread:.2f}{marker}") if __name__ == "__main__": main() ``` </details> Here are the results. The table compares how the font-generation `autohint` flag affects the range of widths of various characters. Lower `spread` mean that glyph stroke widths should appear more consistent. ``` Spread = max stem width - min stem width across glyphs (lower = more consistent): ┌──────────────┬──────┬─────────┬──────────┬──────────┐ │ Font │ Size │ Default │ Autohint │ Winner │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ Bookerly │ 12pt │ 1.49 │ 1.12 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 14pt │ 1.39 │ 1.13 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 16pt │ 1.38 │ 1.16 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 18pt │ 1.90 │ 1.58 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ NotoSans │ 12pt │ 1.16 │ 0.94 │ mixed │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 14pt │ 0.83 │ 1.14 │ default │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 16pt │ 1.41 │ 1.51 │ default │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 18pt │ 1.74 │ 1.63 │ mixed │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ OpenDyslexic │ 12pt │ 2.22 │ 1.44 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 14pt │ 2.57 │ 3.29 │ default │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 16pt │ 3.13 │ 2.60 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 18pt │ 3.21 │ 3.23 │ ~tied │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ Ubuntu │ 12pt │ 1.25 │ 1.31 │ default │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 14pt │ 1.41 │ 1.64 │ default │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 16pt │ 2.21 │ 1.71 │ autohint │ ├──────────────┼──────┼─────────┼──────────┼──────────┤ │ │ 18pt │ 1.80 │ 1.71 │ autohint │ └──────────────┴──────┴─────────┴──────────┴──────────┘ ``` --- ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? I used AI to make sure I'm not doing something stupid, since I'm not a typography expert. I made the changes though. |
||
|
Adrian Wilkins-CaruanaandClaude Opus 4.6
|
47aa0dda76 |
perf: Reduce overall flash usage by 30.7% by compressing built-in fonts (#831)
## Summary **What is the goal of this PR?** Compress reader font bitmaps to reduce flash usage by 30.7%. **What changes are included?** - New `EpdFontGroup` struct and extended `EpdFontData` with `groups`/`groupCount` fields - `--compress` flag in `fontconvert.py`: groups glyphs (ASCII base group + groups of 8) and compresses each with raw DEFLATE - `FontDecompressor` class with 4-slot LRU cache for on-demand decompression during rendering - `GfxRenderer` transparently routes bitmap access through `getGlyphBitmap()` (compressed or direct flash) - Uses `uzlib` for decompression with minimal heap overhead. - 48 reader fonts (Bookerly, NotoSans 12-18pt, OpenDyslexic) regenerated with compression; 5 UI fonts unchanged - Round-trip verification script (`verify_compression.py`) runs as part of font generation ## Additional Context ## Flash & RAM | | baseline | font-compression | Difference | |--|--------|-----------------|------------| | Flash (ELF) | 6,302,476 B (96.2%) | 4,365,022 B (66.6%) | -1,937,454 B (-30.7%) | | firmware.bin | 6,468,192 B | 4,531,008 B | -1,937,184 B (-29.9%) | | RAM | 101,700 B (31.0%) | 103,076 B (31.5%) | +1,376 B (+0.5%) | ## Script-Based Grouping (Cold Cache) Comparison of uncompressed baseline vs script-based group compression (4-slot LRU cache, cleared each page). Glyphs are grouped by Unicode block (ASCII, Latin-1, Latin Extended-A, Combining Marks, Cyrillic, General Punctuation, etc.) instead of sequential groups of 8. ### Render Time | | Baseline | Compressed (cold cache) | Difference | |---|---|---|---| | **Median** | 414.9 ms | 431.6 ms | +16.7 ms (+4.0%) | | **Pages** | 37 | 37 | | ### Memory Usage | | Baseline | Compressed (cold cache) | Difference | |---|---|---|---| | **Heap free (median)** | 187.0 KB | 176.3 KB | -10.7 KB | | **Heap free (min)** | 186.0 KB | 166.5 KB | -19.5 KB | | **Largest block (median)** | 148.0 KB | 128.0 KB | -20.0 KB | | **Largest block (min)** | 148.0 KB | 120.0 KB | -28.0 KB | ### Cache Effectiveness | | Misses/page | Hit rate | |---|---|---| | **Compressed (cold cache)** | 2.1 | 99.85% | ------ ### AI Usage While CrossPoint doesn't have restrictions on AI tools in contributing, please be transparent about their usage as it helps set the right context for reviewers. Did you use AI tools to help write this code? _**YES**_ Implementation was done by Claude Code (Opus 4.6) based on a plan developed collaboratively. All generated font headers were verified with an automated round-trip decompression test. The firmware was compiled successfully but has not yet been tested on-device. --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |