Replace per-page buffer free+realloc pattern with capacity tracking that only reallocates when needed size exceeds current capacity. This prevents heap fragmentation from non-coalescing holes that occurred when each page's freed block rarely fit the next page's allocation. After a few page turns, capacities converge on the book's maximum and page turns stop touching the allocator entirely.
## Summary
This PR adds the [Libre Baskerville
](https://github.com/impallari/Libre-Baskerville) font family to the
fonts available for download in CrossPoint.
## Additional Context
Baskerville is a classic font family, widely available in some form or
the other in most ereaders and widely used in publishing and in the
academy.
Libre Baskerville is a libre (SIL Open Font License v1) implementation
of it, which makes it possible for CrossPoint to ship it without issues.
---
### AI Usage
Did you use AI tools to help write this code? _**NO**_
## Summary
SD card font fixes:
- `TxtReaderActivity` needs to call `renderer.ensureSdCardFontReady` to
build the advance lookup table to support rendering with SD card fonts.
This revealed that `TxtReaderActivity` was inconsistently performing
layout with `getTextWidth`, when the renderer actually uses
`getTextAdvanceX`, which can lead to minor inconsistencies in alignment.
- Avoid allocating one big `allText` string in
`ParsedText::layoutAndExtractLines`. Instead, pass the vector of word
strings directly to `SdCardFont::buildAdvanceTable`, where the algorithm
just needs to iterate codepoints anyway.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY**_
---------
Co-authored-by: Justin Mitchell <justin@jmitch.com>
## Summary
- Fixes#1928 by having the prewarm and advance table functions resolve
fallback styles
---
### AI Usage
Did you use AI tools to help write this code? YES - Codex
---------
Co-authored-by: Uri Tauber <uritaube@gmail.com>
## Summary
* **What is the goal of this PR?** fix#1956.
## Additional Context
[NixOS-and-Flakes-Book_ch3_p7_4pct_130044.bmp](https://github.com/user-attachments/files/27648392/NixOS-and-Flakes-Book_ch3_p7_4pct_130044.bmp)
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< YES >**_
Reverts crosspoint-reader/crosspoint-reader#1912
This was meant to be more robust with partial SD card fonts, but causes
trouble for folks using custom font sizes. We need a better approach to
decouple numeric font sizes from S/M/L/XL settings.
## Summary
Consolidate the script help/startup fixes so standalone helper commands
can show usage without requiring runtime-only dependencies or generating
files.
## Details
Several helper scripts imported optional packages, evaluated newer
annotations, or started generation before users could inspect CLI usage.
On a fresh checkout, this made common help paths fail on missing
packages such as `cairosvg`, `Pillow`, `freetype-py`, `fontTools`,
`pyphen`, `pyserial`, or on Python 3.9 annotation evaluation. A few
generators also treated `--help` as a normal output argument and wrote
files instead of printing usage.
This change keeps runtime dependencies on the code paths that need them,
handles lightweight help/list commands first, and adds explicit `--help`
handling for generator scripts that previously started work.
## Validation
- `python3 <script> --help` across tracked Python scripts, excluding
`scripts/patch_jpegdec.py` because it is a PlatformIO pre-build hook
rather than a standalone CLI
- `python3 scripts/convert_icon.py` still exits with usage for missing
required arguments
- `python3 lib/EpdFont/scripts/fontconvert_sdcard.py --list-presets`
- `python3 -m py_compile` for the changed scripts
## Summary
#1910 caused a massive performance degradation in the way rasterized
glyph buffers were handled during pixel iteration. This change restores
the original performance characteristics so the font generation job
finishes in a reasonable amount of time.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
## Summary
One of the recent font conversion script changes obliterated
performance. Add a verbose mode to build-sd-fonts.py to help diagnose
problems. Changed the CI process to use verbose mode.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
Add SdCardFontFamilyInfo::pickClosestSize(targetPtSize) and have the
manager and SdCardFontSystem drive size selection from a target point
size derived from the user's font-size enum (SMALL=12, MEDIUM=14,
LARGE=16, EXTRA_LARGE=18) rather than indexing the family's sorted size
list by enum ordinal.
The ordinal-slot approach mis-selects whenever a family doesn't ship the
canonical {12,14,16,18} set: a family with only [10,14,18] would map
SMALL/MEDIUM/LARGE/EXTRA_LARGE to 10/14/18/18 — fine for SMALL but
arbitrary for the rest. Closest-pt always picks the on-disk file nearest
to the user-intended point size, with a deterministic smaller-pt
tie-break.
No change for canonical-sized families.
## Summary
* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
* **What changes are included?**
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
specific areas to focus on).
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< YES | PARTIALLY | NO
>**_
## Summary
**What is the goal of this PR?**
* Provide a better default weight of Bitter font for improved rendering
**What changes are included?**
* Changes the default weight for Bitter font that is defined in the
custom font catalog to medium (500) from regular (400) for improved
rendering on e-ink, especially when font anti-aliasing is turned on
## Additional Context
* Bitter font at regular weight can become washed out when AA is on due
to some of the thinner stems and posts of the font. Bumping the weight
up to medium makes it sturdier and holds up better when AA is turned on.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< NO >**_
Fontconvert_sdcard.py:
* Move face.set_char_size() before any load_glyph() call. The validation
loop runs load_glyph() with FT_LOAD_RENDER, which renders at the
*active* size — calling it before set_char_size() wastes work at the
default size and triggers Invalid_Size_Handle on some fonts.
* Walk bitmap.buffer using bitmap.pitch with negative-pitch handling.
The previous linear iteration assumed pitch == width and a top-down
layout, which silently corrupts output for any padded or bottom-up
bitmap FreeType returns.
* Fix operator-precedence bug in 2-bit tail-padding: px << (4 - … % 4) *
2 evaluated as (px << (4 - … % 4)) * 2 due to << binding tighter than *.
Add the missing outer parens.
* Drop SMP codepoints (> U+FFFF) from kern and ligature tables before
packing — the binary format uses uint16/uint32 codepoint fields and
would raise struct.error otherwise.
* Skip GPOS kern subtables that aren't lookup type 2 (PairPos).
_extract_pairpos_subtable assumes Type-2 layout; cursive attachment and
other types reachable through the kern feature crash inside it.
Fontconvert.py:
* Force UTF-8 stdout so `python fontconvert.py … > foo.h` on Windows
doesn't emit UTF-16 / replacement characters in the generated header.
## Summary
* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
* **What changes are included?**
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
specific areas to focus on).
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< YES | PARTIALLY | NO
>**_
instantiateVariableFont() returns a *new* TTFont with the requested axes
pinned; the existing call discards that return and proceeds to
font.save() the original (still-variable) font. The "static instance"
written to disk is therefore identical to the source variable font and
the requested axis values are silently ignored.
For sd-fonts.yaml entries that use the `variable: {wght: 400}` form
(e.g. Bitter, Lora) this means every weight ends up rasterised from the
same default-weight glyphs — bold and regular look the same.
Capture the return value, and pass two diagnostically useful kwargs
while we're touching the call:
* updateFontNames=True — rewrite the name table so the saved TTF reports
its actual weight/style instead of retaining the source variable-font
names.
* optimize=False — skip the gvar interpolation optimisation;
fully pinning every axis drops gvar anyway,
so the work would be wasted.
The atomic-write scaffolding around the call is left untouched.
## Summary
* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
* **What changes are included?**
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
specific areas to focus on).
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< YES | PARTIALLY | NO
>**_
Add GROUP_MAX_UNCOMPRESSED_BYTES=65536 on top of the existing script-
based grouping. When adding the next glyph would push a group past the
cap, the group is closed and a new one started with the same script ID.
Without the cap, the size of a script group is bounded only by however
many glyphs the font supplies in that block. Dense scripts (CJK,
Vietnamese precomposed, user-supplied fonts with large Unicode blocks)
can produce a single group whose uncompressed size exceeds what fits in
the embedded decompressor's transient malloc on the ESP32-C3, which
manifests as a runtime allocation failure instead of a build error.
The 64 KB ceiling is large enough to hold any single built-in script
group with headroom and small enough to be a comfortable transient
allocation on-device.
The check uses byte-aligned size (4-pixel-aligned row stride × height),
which is what the decompressor actually consumes — not the packed
on-disk length. A defensive guard rejects any single glyph whose own
size exceeds the cap with a clear error pointing at the offending
codepoint.
## Summary
* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
* **What changes are included?**
## Additional Context
* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
specific areas to focus on).
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**< YES | PARTIALLY | NO
>**_
## Summary
* Previous `fontconvert_sdcard.py` requires that the intervals are named
(i.e. listed in `INTERVAL_PRESETS`, ex. "latin1"), this change will also
allow them to be unnamed (ex. "(0x2100-0x214F)") for a more flexible
usage.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
* Fix missing requirement (needed by
lib/EpdFont/scripts/build-sd-fonts.py)
* Add a root requirements.txt to simplify installation
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
Add a complete SD card font subsystem that enables users to install and
use custom fonts beyond the three built-in families. This combines the
back-end firmware support (#1327) with the font configuration, build
pipeline, CI distribution, and user-facing management UI (#1392).
Core font system:
- Custom .cpfont binary format (v4) with multi-style support (regular,
bold, italic, bold-italic) packed into a single file per size
- On-demand glyph loading from SD card with two-pass prewarm rendering
to bulk-read glyphs per page, achieving near-flash performance for
Latin text (~697ms vs ~681ms) and viable CJK rendering (~32% slower)
- Persistent advance cache for layout measurement without SD I/O
- Overflow ring buffer for glyph cache misses during rendering
- Memory-conscious design: only advance tables kept in RAM; glyph
bitmaps, kern tables, and ligatures loaded on demand from SD
Font management:
- On-device WiFi download from GitHub Releases with manifest-based
discovery, install/update detection, and progress UI
- Web interface font upload, listing, and deletion via /fonts page
- Manual SD card copy to /fonts/ or /.fonts/ directories
- Font selection integrated into Settings > Reader > Font Family
Build pipeline:
- Declarative YAML config (sd-fonts.yaml) as single source of truth
for the 17-family font library (serif, sans, mono, accessibility)
- Python converter (fontconvert_sdcard.py) for TTF/OTF to .cpfont with
FreeType rasterization, class-based kerning, and ligature extraction
- Parallel build orchestrator with variable font instance extraction
- CI workflow publishing versioned + stable releases to a dedicated
crosspoint-fonts repository with auto-incrementing revision tags
- Centralized version constants (cpfont_version.py) shared across
build tooling and CI, with firmware headers as manual sync points
Additional fixes:
- CJK characters no longer get hyphens inserted at line breaks
- Advance table eliminates 30+ second stalls during CJK section
indexing for paragraphs with >512 unique codepoints
Closes#930
Co-authored-by: Zach Nelson <zach@zdnelson.com>
Co-authored-by: Justin <itsthisjustin@users.noreply.github.com>
Co-authored-by: jpirnay <jens@pirnay.com>
Co-authored-by: mcrosson <kemonine@kemonine.info>
## Summary
The Python scripts in the current repo have many requirements that are
not mentioned in any requirements.txt files, so I have therefore added
them to the directories that need them.
Question: Should these changes maybe moved into a "root"
requirements.txt file?
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
Fixes#258. Bookerly is not licensed for use in CrossPoint. Switched to
Google's open Noto Serif font.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
**What is the goal of this PR?**
Reading a book with frequent numbers, I noticed that the spacing between
numeral glyphs was strangely large. This was because Bookerly and Noto
Sans default to tabular figures, where every digit gets an identical
advance width. This is designed for column alignment in spreadsheets,
but in rendering prose it produces visually wide gaps between digits.
This change adds a `--pnum` flag to fontconvert.py that applies the
font's OpenType `pnum` (proportional numerals) feature during
conversion. When active, the converter:
- Parses the GSUB table for pnum SingleSubst lookups
- Resolves substitute glyph indices via fonttools' glyph order
- Loads the proportional alternate glyphs instead of the tabular
defaults
- Includes substitute glyph names in kern pair extraction, so kerning
data that references proportional alternates is captured
Bookerly's proportional alternates also carry digit-digit and
digit-punctuation kerning that the tabular glyphs lack (e.g., at 16pt
7->4 at -1.69px, 7->. at -2.31px, 7->1 at +1.00px).
Noto Sans gains proportional advances but no new kerning (its
proportional glyphs have no kern class data in the font).
OpenDyslexic is unaffected. Its `cmap` already points to proportional
glyphs, so `--pnum` is a no-op. `--pnum` is intentionally omitted from
OpenDyslexic in the build script for deliberately uniform digit spacing
as an accessibility choice.
UI fonts (Ubuntu, notosans_8) also omit `--pnum` to preserve tabular
alignment for page numbers, battery percentages, etc.
| Before | After |
| -- | -- |
| <img
src="https://github.com/user-attachments/files/26042238/screenshot-31673.bmp"
width="300" /> | <img
src="https://github.com/user-attachments/files/26042241/screenshot-124075.bmp"
width="300" /> |
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_
## Summary
**What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
Combining diacritical marks (U+0300–U+036F) were positioned using a
heuristic that centered them at the midpoint of the base glyph's
**advance width**. This worked acceptably for Bookerly but produced
visibly off-center marks for Noto Sans due to a fundamental difference
in how the two fonts design their combining mark metrics.
Builds on the work of #1037.
## The problem
The two built-in body fonts encode combining mark `left` offsets with
very different conventions:
| Mark | Bookerly `left` | Noto Sans `left` |
|---|---|---|
| U+0301 (acute) | -2 | -10 |
| U+0300 (grave) | -5 | -15 |
| U+0302 (circumflex) | -5 | -5 |
| U+0323 (dot below) | -2 | -11 |
Noto Sans uses large negative `left` values because its marks are
designed for placement at the post-advance cursor position, with `left`
pulling the bitmap back over the base glyph. Bookerly uses small offsets
because its marks sit closer to the glyph origin. The old `advance/2`
centering split the difference poorly — it happened to land close to
correct for Bookerly but placed Noto Sans marks roughly 6px left of
center on a typical lowercase letter.
There was also a bug in the vertical gap heuristic. It unconditionally
computed a `raiseBy` value to prevent above-baseline marks from
colliding with tall base glyphs, but it applied the same logic to
**below-baseline** marks like cedilla (U+0327), dot below (U+0323), and
ogonek (U+0328). For those marks, the math produced a large positive
raise (e.g., 24px for dot-below on 'a'), launching them above the
x-height instead of keeping them below the baseline.
## The fix
**Horizontal positioning**: Instead of centering at `advance/2`, align
the mark bitmap's visual midpoint directly over the base glyph bitmap's
visual midpoint. This uses the base glyph's actual `left` and `width`
rather than its advance width, producing correct results regardless of
how the font encodes its mark offsets.
**Vertical positioning**: The raise heuristic now checks `markTop -
markHeight > 0` and skips below-baseline marks entirely, leaving them at
their font-designed position.
**Consolidation**: The shared math is extracted into two `constexpr`
helpers (`combiningMark::centerOver` and
`combiningMark::raiseAboveBase`) in `EpdFontData.h`, eliminating the
previously triplicated inline calculations across `drawText`,
`drawTextRotated90CW`, and `getTextBounds`. The `MIN_COMBINING_GAP_PX`
constant is also centralized as `combiningMark::MIN_GAP_PX`.
| Before | After |
| -- | -- |
| <img
src="https://github.com/user-attachments/files/25752257/before-noto.bmp"
width="250" /> | <img
src="https://github.com/user-attachments/files/25752258/after-noto.bmp"
width="250" /> |
| <img
src="https://github.com/user-attachments/files/25752259/before-bookerly.bmp"
width="250" /> | <img
src="https://github.com/user-attachments/files/25752260/after-bookerly.bmp"
width="250" /> |
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES to analyze
differences between Noto Sans and Bookerly font metrics**_
---------
Co-authored-by: Uri Tauber <142022451+Uri-Tauber@users.noreply.github.com>
## Summary
**What is the goal of this PR?**
A tweak to the fixed-point x-advance and kerning calculations to ensure
that the spacing between any two glyphs is always calculated
consistently.
I noticed that sometimes I'd see common character pairs like "oo" more
than once on a page, and the distance between the two snapped to
different pixels depending on the running accumulated error for the line
of text.
This change uses a differential rounding approach where each glyph's
x-advance plus the kerning relative to the next glyph are combined in
fixed-point precision, then snapped to a pixel to draw the next glyph.
This results in a consistent inter-glyph spacing any time the same two
glyphs show up adjacent to each other, regardless of the accumulated
error across the line.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY**_
## Summary
**What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
Fix prewarm perf when a page contains many styles.
The prewarm page buffer was a single slot, so each `prewarmCache` call
for a new font style freed the previous style's glyphs. On pages with
multiple styles (regular + bold + italic), only the last style was
prewarmed. The others fell through to the hot-group compaction path at
~2-3ms per glyph.
This was most visible on rich formatting (e.g. this [Czech prayer
book](https://stahuj.kancional.cz/e-kniha/kancional.epub) with bold
headings, italic liturgical text, and regular body), where page renders
took 3-5 seconds instead of ~700ms.
Fix: use up to 4 page buffer slots (one per font style) so all styles
stay prewarmed simultaneously.
Fixes#1450.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? PARTIALLY: to diagnose and
brainstorm solutions.
## Purpose
This PR includes some preparatory changes that are needed for an
upcoming performant CJK font feature. The changes have no impact on
render time and heap allocation for latin text. **Despite this, I think
these changes stand on their own as a better font
compression/decompression implementation.**
## Summary
- Font decompressor rewrite: Replaced the 4-slot LRU group cache with a
two-tier system — a page buffer (glyphs prewarmed before rendering
begins) and a hot-group fallback (last decompressed group retained for
non-prewarmed
glyphs).
- Byte-aligned compressed bitmap format: Glyph bitmaps within compressed
groups are now stored row-padded rather than tightly packed before
DEFLATE compression, improving compression ratios by making identical
pixel rows produce
identical byte patterns. Glyphs are compacted back to packed format on
demand at render time. Reduces flash size by 155 KB.
- Page prewarm system: Added `Page::collectText` and
`Page::getDominantStyle` to extract per-style glyph requirements before
rendering, and `GfxRenderer::prewarmFontCache` to pre-decompress only
the groups needed for the dominant style
— eliminating mid-render decompression for the common case.
- UTF-8 robustness fixes: `utf8NextCodepoint` now validates continuation
bytes and returns a replacement glyph on malformed input;
`ChapterHtmlSlimParser` correctly preserves incomplete multi-byte
sequences across word-buffer flush
boundaries rather than splitting them.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES**_ Architecture and
design was done by me, refined a bit by Claude. Code mostly by Claude,
but not entirely.
## Summary
**What is the goal of this PR?**
Hopefully fixes#1182.
_Note: I think letterforms got a "heavier" appearance after #1098, which
makes this more noticeable. The current version of this PR reverts the
change to add `--force-autohint` for Bookerly, which to me seems to
bring the font back to a more aesthetic and consistent weight._
#### Problem
Character spacing was uneven in certain words. The word "drew" in
Bookerly was the clearest example: a visible gap between `d` and `r`,
while `e` and `w` appeared tightly condensed. The root cause was
twofold:
1. **Integer-only glyph advances.** `advanceX` was stored as a `uint8_t`
of whole pixels, sourced from FreeType's hinted `advance.x` (which
grid-fits to integers). A glyph whose true advance is 15.56px was stored
as 16px -- an error of +0.44px per character that compounds across a
line.
2. **Floor-rounded kerning.** Kern adjustments were converted with
`math.floor()`, which systematically over-tightened negative kerns. A
kern of -0.3px became -1px -- a 0.7px over-correction that visibly
closed gaps.
Combined, these produced the classic symptom: some pairs too wide,
others too tight, with the imbalance varying per word.
#### Solution: fixed-point accumulation with 1/16-pixel resolution, for
sub-pixel precision during text layout
All font metrics now use a "fixed-point 4" format -- 4 fractional bits
giving 1/16-pixel (0.0625px) resolution. This is implemented with plain
integer arithmetic (shifts and adds), requiring no floating-point on the
ESP32.
**How it works:**
A value like 15.56px is stored as the integer `249`:
```
249 = 15 * 16 + 9 (where 9/16 = 0.5625, closest to 0.56)
```
Two storage widths share the same 4 fractional bits:
| Field | Type | Format | Range | Use |
|-------|------|--------|-------|-----|
| `advanceX` | `uint16_t` | 12.4 | 0 -- 4095.9375 px | Glyph advance
width |
| `kernMatrix` | `int8_t` | 4.4 | -8.0 -- +7.9375 px | Kerning
adjustment |
Because both have 4 fractional bits, they add directly into a single
`int32_t` accumulator during layout. The accumulator is only snapped to
the nearest whole pixel at the moment each glyph is rendered:
```cpp
int32_t xFP = fp4::fromPixel(startX); // pixel to 12.4: startX << 4
for each character:
xFP += kernFP; // add 4.4 kern (sign-extends into int32_t)
int xPx = fp4::toPixel(xFP); // snap to nearest pixel: (xFP + 8) >> 4
render glyph at xPx;
xFP += glyph->advanceX; // add 12.4 advance
```
Fractional remainders carry forward indefinitely. Rounding errors stay
below +/- 0.5px and never compound.
#### Concrete example: "drew" in Bookerly
**Before** (integer advances, floor-rounded kerning):
| Char | Advance | Kern | Cursor | Snap | Gap from prev |
|------|---------|------|--------|------|---------------|
| d | 16 px | -- | 33 | 33 | -- |
| r | 12 px | 0 | 49 | 49 | ~2px |
| e | 13 px | -1 | 60 | 60 | ~0px |
| w | 22 px | -1 | 72 | 72 | ~0px |
The d-to-r gap was visibly wider than the tightly packed `rew`.
**After** (12.4 advances, 4.4 kerning, fractional accumulation):
| Char | Advance (FP) | Kern (FP) | Accumulator | Snap | Ink start | Gap
from prev |
|------|-------------|-----------|-------------|------|-----------|---------------|
| d | 249 (15.56px) | -- | 528 | 33 | 34 | -- |
| r | 184 (11.50px) | 0 | 777 | 49 | 49 | 0px |
| e | 208 (13.00px) | -8 (-0.50px) | 953 | 60 | 61 | 1px |
| w | 356 (22.25px) | -4 (-0.25px) | 1157 | 72 | 72 | 0px |
Spacing is now `0, 1, 0` pixels -- nearly uniform. Verified on-device:
all 5 copies of "drew" in the test EPUB produce identical spacing,
confirming zero accumulator drift.
#### Changes
**Font conversion (`fontconvert.py`)**
- Use `linearHoriAdvance` (FreeType 16.16, unhinted) instead of
`advance.x` (26.6, grid-fitted to integers) for glyph advances
- Encode kern values as 4.4 fixed-point with `round()` instead of
`floor()`
- Add `fp4_from_ft16_16()` and `fp4_from_design_units()` helper
functions
- Add module-level documentation of fixed-point conventions
**Font data structures (`EpdFontData.h`)**
- `EpdGlyph::advanceX`: `uint8_t` to `uint16_t` (no memory cost due to
existing struct padding)
- Add `fp4` namespace with `constexpr` helpers: `fromPixel()`,
`toPixel()`, `toFloat()`
- Document fixed-point conventions
**Font API (`EpdFont.h/cpp`, `EpdFontFamily.h/cpp`)**
- `getKerning()` return type: `int8_t` to `int` (to avoid truncation of
the 4.4 value)
**Rendering (`GfxRenderer.cpp`)**
- `drawText()`: replace integer cursor with `int32_t` fixed-point
accumulator
- `drawTextRotated90CW()`: same accumulator treatment for vertical
layout
- `getTextAdvanceX()`, `getSpaceWidth()`, `getSpaceKernAdjust()`,
`getKerning()`: convert from fixed-point to pixel at API boundary
**Regenerated all built-in font headers** with new 12.4 advances and 4.4
kern values.
#### Memory impact
Zero additional RAM. The `advanceX` field grew from `uint8_t` to
`uint16_t`, but the `EpdGlyph` struct already had 1 byte of padding at
that position, so the struct size is unchanged. The fixed-point
accumulator is a single `int32_t` on the stack.
#### Test plan
- [ ] Verify "drew" spacing in Bookerly at small, medium, and large
sizes
- [ ] Verify uppercase kerning pairs: AVERY, WAVE, VALUE
- [ ] Verify ligature words: coffee, waffle, office
- [ ] Verify all built-in fonts render correctly at each size
- [ ] Verify rotated text (progress bar percentage) renders correctly
- [ ] Verify combining marks (accented characters) still position
correctly
- [ ] Spot-check a full-length book for any layout regressions
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES, Claude Opus 4.6
helped figure out a non-floating point approach for sub-pixel error
accumulation**_
## Summary
**What is the goal of this PR?**
Rewrite of font routines to use std binary search algorithms instead of
custom repeated implementations: `lookupKernClass`,
`EpdFont::getLigature`, and `EpdFont::getGlyph`.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
**What is the goal of this PR?**
Correction to #1157, which (embarrassingly) failed to actually include
the updated font header files. (Maybe we should generate these at build
time?)
Add Latin Extended-B glyphs for Croatian, Romanian, Pinyin, and European
diacritical variants. Fixes#921.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY, confirmed
codepoint ranges with Claude**_
## Summary
**What is the goal of this PR?**
Add Latin Extended-B glyphs for Croatian, Romanian, Pinyin, and European
diacritical variants. Fixes#921.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY, confirmed
codepoint ranges with Claude**_
## Summary
* **What is the goal of this PR?**
Add Vietnamese glyphs support for the reader's built-in fonts, enabling
proper rendering of Vietnamese text in EPUB content.
* **What changes are included?**
- Added 3 new Unicode intervals to `fontconvert.py` covering Vietnamese
characters:
- **Latin Extended-B** (Vietnamese subset only): `U+01A0–U+01B0` — Ơ/ơ,
Ư/ư
- **Vietnamese Extended**: `U+1EA0–U+1EF9` — All precomposed Vietnamese
characters with tone marks (Ả, Ấ, Ầ, Ẩ, Ẫ, Ậ, Ắ, …, Ỹ)
- Re-generated all 54 built-in font header files (Bookerly, Noto Sans,
OpenDyslexic, Ubuntu across all sizes and styles) to include the new
Vietnamese glyphs.
## Additional Context
* **Scope**: This PR only covers the **reader** fonts. The outer UI
still uses the Ubuntu font which does not fully support Vietnamese — UI
and i18n will be addressed in a follow-up PR (per discussion in PR
#1124).
* **Memory impact**:
| Metric | Before | After | Delta |
|---|---|---|---|
| Flash Data (`.rodata`) | 2,971,028 B | 3,290,748 B | **+319,720 B
(+10.8%)** |
| Total image size | 4,663,235 B | 4,982,955 B | **+319,720 B (+6.9%)**
|
| Flash usage | 69.1% | 74.0% | **+4.9 pp** |
| RAM usage | 29.0% | 29.0% | **No change** |
* **Risk**: Low — this is a data-only change (font glyph tables in
`.rodata`). No logic changes, no RAM impact. Flash headroom remains
comfortable at 74%.
---
### AI Usage
Did you use AI tools to help write this code? _**PARTIALLY**_
AI was used to identify the minimal set of Unicode ranges needed for
Vietnamese support and to assist with the PR description.
---------
Co-authored-by: danoooob <danoooob@example.com>
## Summary
**What is the goal of this PR?**
Improved typesetting, including
[kerning](https://en.wikipedia.org/wiki/Kerning) and
[ligatures](https://en.wikipedia.org/wiki/Ligature_(writing)#Latin_alphabet).
**What changes are included?**
- The script to convert built-in fonts now adds kerning and ligature
information to the generated font headers.
- Epub page layout calculates proper kerning spaces and makes ligature
substitutions according to the selected font.
## Additional Context
- I am not a typography expert.
- The implementation has been reworked from the earlier version, so it
is no longer necessary to omit Open Dyslexic, and kerning data now
covers all fonts, styles, and codepoints for which we include bitmap
data.
- Claude Opus 4.6 helped with a lot of this.
- There's an included test epub document with lots of kerning and
ligature examples, shown in the photos.
**_After some time to mature, I think this change is in decent shape to
merge and get people testing._**
After opening this PR I came across #660, which overlaps in adding
ligature support.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**YES, Claude Opus 4.6**_
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
## Summary
Bookerly's native TrueType hinting is effectively a no-op at the sizes
used here, causing FreeType to place stems at inconsistent sub-pixel
positions. This results in the 'k' stem (8-bit fringe: 0x38=56) falling
just below the 2-bit quantization threshold while 'l' and 'h' stems
(fringes: 0x4C=76, 0x40=64) land above it --- making 'k' visibly
narrower (2.00px vs 2.33px effective width).
FreeType's auto-hinter snaps all stems to consistent grid positions,
normalizing effective stem width to 2.67px across all glyphs.
Adds --force-autohint flag to fontconvert.py and applies it to Bookerly
only. NotoSans, OpenDyslexic, and Ubuntu fonts are unaffected.
Here is an example of before/after. Take notice of the vertical stems on
characters like `l`, `k`, `n`, `i`, etc. The font is Bookerly 12pt
regular:
**BEFORE**:
**AFTER**:
Claude generated this script to quantitatively determine that this
change makes the vertical stems on a variety of characters more
consistent for Bookerly _only_.
<details>
<summary>Python script</summary>
```python
#!/usr/bin/env python3
"""Compare stem consistency across all font families with and without
auto-hinting.
Run from repo root:
python3 compare_all_fonts.py
"""
import freetype
DPI = 150
CHARS = ["k", "l", "h", "i", "b", "d"]
SIZES = [12, 14, 16, 18]
FONTS = {
"Bookerly":
"lib/EpdFont/builtinFonts/source/Bookerly/Bookerly-Regular.ttf",
"NotoSans":
"lib/EpdFont/builtinFonts/source/NotoSans/NotoSans-Regular.ttf",
"OpenDyslexic":
"lib/EpdFont/builtinFonts/source/OpenDyslexic/OpenDyslexic-Regular.otf",
"Ubuntu": "lib/EpdFont/builtinFonts/source/Ubuntu/Ubuntu-Regular.ttf",
}
MODES = {
"default": freetype.FT_LOAD_RENDER,
"autohint": freetype.FT_LOAD_RENDER | freetype.FT_LOAD_FORCE_AUTOHINT,
}
def q4to2(v):
if v >= 12:
return 3
elif v >= 8:
return 2
elif v >= 4:
return 1
else:
return 0
def get_stem_eff(face, char, flags):
gi = face.get_char_index(ord(char))
if gi == 0:
return None
face.load_glyph(gi, flags)
bm = face.glyph.bitmap
w, h = bm.width, bm.rows
if w == 0 or h == 0:
return None
p2 = []
for y in range(h):
row = []
for x in range(w):
row.append(q4to2(bm.buffer[y * bm.pitch + x] >> 4))
p2.append(row)
# Measure leftmost stem in stable middle rows
mid_start, mid_end = h // 4, h - h // 4
widths = []
for y in range(mid_start, mid_end):
first = next((x for x in range(w) if p2[y][x] > 0), -1)
if first < 0:
continue
last = first
for x in range(first, w):
if p2[y][x] > 0:
last = x
else:
break
eff = sum(p2[y][x] for x in range(first, last + 1)) / 3.0
widths.append(eff)
return round(sum(widths) / len(widths), 2) if widths else None
def main():
for font_name, font_path in FONTS.items():
try:
freetype.Face(font_path)
except Exception:
print(f"\n {font_name}: SKIPPED (file not found)")
continue
print(f"\n{'=' * 80}")
print(f" {font_name}")
print(f"{'=' * 80}")
for size in SIZES:
print(f"\n {size}pt:")
print(f" {'':6s}", end="")
for c in CHARS:
print(f" '{c}' ", end="")
print(" | spread")
for mode_name, flags in MODES.items():
face = freetype.Face(font_path)
face.set_char_size(size << 6, size << 6, DPI, DPI)
vals = []
print(f" {mode_name:6s}", end="")
for c in CHARS:
v = get_stem_eff(face, c, flags)
vals.append(v)
print(f" {v:5.2f}" if v else " N/A", end="")
valid = [v for v in vals if v is not None]
spread = max(valid) - min(valid) if len(valid) >= 2 else 0
marker = " <-- inconsistent" if spread > 0.5 else ""
print(f" | {spread:.2f}{marker}")
if __name__ == "__main__":
main()
```
</details>
Here are the results. The table compares how the font-generation
`autohint` flag affects the range of widths of various characters. Lower
`spread` mean that glyph stroke widths should appear more consistent.
```
Spread = max stem width - min stem width across glyphs (lower = more consistent):
┌──────────────┬──────┬─────────┬──────────┬──────────┐
│ Font │ Size │ Default │ Autohint │ Winner │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ Bookerly │ 12pt │ 1.49 │ 1.12 │ autohint │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 14pt │ 1.39 │ 1.13 │ autohint │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 16pt │ 1.38 │ 1.16 │ autohint │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 18pt │ 1.90 │ 1.58 │ autohint │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ NotoSans │ 12pt │ 1.16 │ 0.94 │ mixed │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 14pt │ 0.83 │ 1.14 │ default │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 16pt │ 1.41 │ 1.51 │ default │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 18pt │ 1.74 │ 1.63 │ mixed │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ OpenDyslexic │ 12pt │ 2.22 │ 1.44 │ autohint │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 14pt │ 2.57 │ 3.29 │ default │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 16pt │ 3.13 │ 2.60 │ autohint │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 18pt │ 3.21 │ 3.23 │ ~tied │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ Ubuntu │ 12pt │ 1.25 │ 1.31 │ default │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 14pt │ 1.41 │ 1.64 │ default │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 16pt │ 2.21 │ 1.71 │ autohint │
├──────────────┼──────┼─────────┼──────────┼──────────┤
│ │ 18pt │ 1.80 │ 1.71 │ autohint │
└──────────────┴──────┴─────────┴──────────┴──────────┘
```
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? I used AI to make sure I'm
not doing something stupid, since I'm not a typography expert. I made
the changes though.
## Summary
**What is the goal of this PR?**
Consolidated repeated logic to fall back to REPLACEMENT_GLYPH.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
Justin Mitchell
