feat: cap compressed group size at 64 KB (#1913)

Add GROUP_MAX_UNCOMPRESSED_BYTES=65536 on top of the existing script-
based grouping. When adding the next glyph would push a group past the
cap, the group is closed and a new one started with the same script ID.

Without the cap, the size of a script group is bounded only by however
many glyphs the font supplies in that block. Dense scripts (CJK,
Vietnamese precomposed, user-supplied fonts with large Unicode blocks)
can produce a single group whose uncompressed size exceeds what fits in
the embedded decompressor's transient malloc on the ESP32-C3, which
manifests as a runtime allocation failure instead of a build error.

The 64 KB ceiling is large enough to hold any single built-in script
group with headroom and small enough to be a comfortable transient
allocation on-device.

The check uses byte-aligned size (4-pixel-aligned row stride × height),
which is what the decompressor actually consumes — not the packed
on-disk length. A defensive guard rejects any single glyph whose own
size exceeds the cap with a clear error pointing at the offending
codepoint.

## Summary

* **What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
* **What changes are included?**

## Additional Context

* Add any other information that might be helpful for the reviewer
(e.g., performance implications, potential risks,
  specific areas to focus on).

---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**< YES | PARTIALLY | NO
>**_
This commit is contained in:
jpirnay
2026-05-10 10:04:59 -05:00
committed by GitHub
parent 91de6ac278
commit c5d2dc2e00
+33 -2
View File
@@ -792,6 +792,15 @@ if compress:
# are grouped together for efficient LRU caching on the embedded target.
# Since glyphs are in codepoint order, glyphs in the same Unicode block
# are contiguous in the array and form natural groups.
#
# On top of script boundaries, a hard size cap (GROUP_MAX_UNCOMPRESSED_BYTES)
# is applied: if adding the next glyph would push the uncompressed group
# size over the cap, the group is closed and a new one started with the
# same script ID. This bounds the embedded decompressor's transient
# malloc regardless of font density (CJK, Vietnamese, user-supplied
# fonts with large Unicode blocks). Without it, a single dense script
# group can balloon past what fits in a transient page-decompress
# allocation on the device.
SCRIPT_GROUP_RANGES = [
(0x0000, 0x007F), # ASCII
(0x0080, 0x00FF), # Latin-1 Supplement
@@ -809,6 +818,11 @@ if compress:
(0xFFFD, 0xFFFD), # Replacement Character
]
# 64 KB cap: large enough to hold any single built-in script group with
# headroom, small enough to be a comfortable transient malloc on the
# ESP32-C3.
GROUP_MAX_UNCOMPRESSED_BYTES = 65536
def get_script_group(code_point):
for i, (start, end) in enumerate(SCRIPT_GROUP_RANGES):
if start <= code_point <= end:
@@ -819,17 +833,34 @@ if compress:
current_group_id = None
group_start = 0
group_count = 0
group_uncompressed = 0
for i, (props, packed) in enumerate(all_glyphs):
for i, (props, _) in enumerate(all_glyphs):
sg = get_script_group(props.code_point)
if sg != current_group_id:
# Use the byte-aligned size (4-pixel-aligned row stride) rather than
# the packed length, since the decompressor consumes byte-aligned
# buffers. Empty glyphs contribute zero.
glyph_aligned_size = (((props.width + 3) // 4) * props.height
if props.width > 0 and props.height > 0 else 0)
if glyph_aligned_size > GROUP_MAX_UNCOMPRESSED_BYTES:
raise ValueError(
f"Glyph {i} (code point U+{props.code_point:04X}) byte-aligned size "
f"{glyph_aligned_size} exceeds GROUP_MAX_UNCOMPRESSED_BYTES="
f"{GROUP_MAX_UNCOMPRESSED_BYTES}. Consider: (1) increasing GROUP_MAX_UNCOMPRESSED_BYTES, "
f"(2) reducing font size, or (3) excluding this codepoint."
)
size_overflow = group_uncompressed + glyph_aligned_size > GROUP_MAX_UNCOMPRESSED_BYTES
if sg != current_group_id or size_overflow:
if group_count > 0:
groups.append((group_start, group_count))
current_group_id = sg
group_start = i
group_count = 1
group_uncompressed = glyph_aligned_size
else:
group_count += 1
group_uncompressed += glyph_aligned_size
if group_count > 0:
groups.append((group_start, group_count))