fontconvert.py: cap group uncompressed size at 64 KB

Script-range boundaries alone don't bound group size — dense Unicode
blocks (CJK, user fonts) can produce groups of hundreds of KB. Add a
64 KB hard cap: when adding the next glyph would exceed it, close the
current group and start a new one with the same script ID.

64 KB is well above the largest current built-in group (~50 KB for
notosans_18 Cyrillic) and a safe transient malloc on the ESP32-C3.
The decompressor side already handles any number of groups per font.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
jpirnay
2026-04-22 08:36:36 +02:00
co-authored by Claude Sonnet 4.6
parent e76118dff0
commit 23a77c6aa5
+17 -1
View File
@@ -726,6 +726,12 @@ if compress:
# are grouped together for efficient LRU caching on the embedded target.
# Since glyphs are in codepoint order, glyphs in the same Unicode block
# are contiguous in the array and form natural groups.
#
# A hard size cap (GROUP_MAX_UNCOMPRESSED_BYTES) is applied on top of script
# boundaries: if adding the next glyph would push the uncompressed group size
# over the cap, the group is closed and a new one started with the same script
# ID. This keeps the embedded decompressor's transient malloc bounded regardless
# of font density (CJK, Vietnamese, user-supplied fonts with large Unicode blocks).
SCRIPT_GROUP_RANGES = [
(0x0000, 0x007F), # ASCII
(0x0080, 0x00FF), # Latin-1 Supplement
@@ -743,6 +749,10 @@ if compress:
(0xFFFD, 0xFFFD), # Replacement Character
]
# 64 KB cap: large enough to hold any single built-in font group with headroom,
# small enough to be a comfortable transient malloc on the ESP32-C3.
GROUP_MAX_UNCOMPRESSED_BYTES = 65536
def get_script_group(code_point):
for i, (start, end) in enumerate(SCRIPT_GROUP_RANGES):
if start <= code_point <= end:
@@ -753,17 +763,23 @@ if compress:
current_group_id = None
group_start = 0
group_count = 0
group_uncompressed = 0
for i, (props, packed) in enumerate(all_glyphs):
sg = get_script_group(props.code_point)
if sg != current_group_id:
glyph_aligned_size = ((props.width + 3) // 4) * props.height if props.width > 0 and props.height > 0 else 0
size_overflow = group_uncompressed + glyph_aligned_size > GROUP_MAX_UNCOMPRESSED_BYTES
if sg != current_group_id or size_overflow:
if group_count > 0:
groups.append((group_start, group_count))
current_group_id = sg
group_start = i
group_count = 1
group_uncompressed = glyph_aligned_size
else:
group_count += 1
group_uncompressed += glyph_aligned_size
if group_count > 0:
groups.append((group_start, group_count))