eric/Crosspoint
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: Fix hyphenation and rendering of decomposed characters (#1037)

Browse Source 
## Summary

* This PR fixes decomposed diacritic handling end-to-end:
- Hyphenation: normalize common Latin base+combining sequences to
precomposed codepoints before Liang pattern matching, so decomposed
words hyphenate correctly
- Rendering: correct combining-mark placement logic so non-spacing marks
are attached to the preceding base glyph in normal and rotated text
rendering paths, with corresponding text-bounds consistency updates.
- Hyphenation around non breaking space variants have been fixed (and
extended)
- Hyphenation of terms that already included of hyphens were fixed to
include Liang pattern application (eg "US-Satellitensystem" was
*exclusively* broken at the existing hyphen)

## Additional Context

* Before
<img width="800" height="480" alt="2"
src="https://github.com/user-attachments/assets/b9c515c4-ab75-45cc-8b52-f4d86bce519d"
/>


* After
<img width="480" height="800" alt="fix1"
src="https://github.com/user-attachments/assets/4999f6a8-f51c-4c0a-b144-f153f77ddb57"
/>
<img width="800" height="480" alt="fix2"
src="https://github.com/user-attachments/assets/7355126b-80c7-441f-b390-4e0897ee3fb6"
/>

* Note 1: the hyphenation fix is not a 100% bullet proof implementation.
It adds composition of *common* base+combining sequences (e.g. O +
U+0308 -> Ö) during codepoint collection. A complete solution would
require implementing proper Unicode normalization (at least NFC,
possibly NFKC in specific cases) before hyphenation and rendering,
instead of hand-mapping a few combining marks. That was beyond the scope
of this fix.

* Note 2: the render fix should be universal and not limited to the
constraints outlined above: it properly x-centers the compund glyph over
the previous one, and it uses at least 1pt of visual distance in y.

Before:
<img width="478" height="167" alt="Image"
src="https://github.com/user-attachments/assets/f8db60d5-35b1-4477-96d0-5003b4e4a2a1"
/>

After: 
<img width="479" height="180" alt="Image"
src="https://github.com/user-attachments/assets/1b48ef97-3a77-475a-8522-23f4aca8e904"
/>

* This should resolve the issues described in #998 
---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**PARTIALLY**_
This commit is contained in:
jpirnay
2026-02-22 13:11:07 +11:00
committed by GitHub
parent 472c56284e
commit c0ed60daa6
8 changed files with 459 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lib/Epub/Epub/parsers/ChapterHtmlSlimParser.cpp
+38 -6
View File
@@ -594,28 +594,60 @@ void XMLCALL ChapterHtmlSlimParser::characterData(void* userData, const XML_Char
continue;
}
// Detect U+00A0 (non-breaking space): UTF-8 encoding is 0xC2 0xA0
// Render a visible space without allowing a line break around it.
// Detect U+00A0 (non-breaking space, UTF-8: 0xC2 0xA0) or
// U+202F (narrow no-break space, UTF-8: 0xE2 0x80 0xAF).
//
// Both are rendered as a visible space but must never allow a line break around them.
// We split the no-break space into its own word token and link the surrounding words
// with continuation flags so the layout engine treats them as an indivisible group.
//
// Example: "200&#xA0;Quadratkilometer" or "200&#x202F;Quadratkilometer"
// Input bytes: "200\xC2\xA0Quadratkilometer" (or 0xE2 0x80 0xAF for U+202F)
// Tokens produced:
// [0] "200" continues=false
// [1] " " continues=true (attaches to "200", no gap)
// [2] "Quadratkilometer" continues=true (attaches to " ", no gap)
//
// The continuation flags prevent the line-breaker from inserting a line break
// between "200" and "Quadratkilometer". However, "Quadratkilometer" is now a
// standalone word for hyphenation purposes, so Liang patterns can produce
// "200 Quadrat-" / "kilometer" instead of the unusable "200" / "Quadratkilometer".
if (static_cast<uint8_t>(s[i]) == 0xC2 && i + 1 < len && static_cast<uint8_t>(s[i + 1]) == 0xA0) {
// Flush any pending text so style is applied correctly.
if (self->partWordBufferIndex > 0) {
self->flushPartWordBuffer();
}
// Add a standalone space that attaches to the previous word.
self->partWordBuffer[0] = ' ';
self->partWordBuffer[1] = '\0';
self->partWordBufferIndex = 1;
self->nextWordContinues = true; // Attach space to previous word (no break).
self->flushPartWordBuffer();
// Ensure the next real word attaches to this space (no break).
self->nextWordContinues = true;
self->nextWordContinues = true; // Next real word attaches to this space (no break).
i++; // Skip the second byte (0xA0)
continue;
}
// U+202F (narrow no-break space) — identical logic to U+00A0 above.
if (static_cast<uint8_t>(s[i]) == 0xE2 && i + 2 < len && static_cast<uint8_t>(s[i + 1]) == 0x80 &&
static_cast<uint8_t>(s[i + 2]) == 0xAF) {
if (self->partWordBufferIndex > 0) {
self->flushPartWordBuffer();
}
self->partWordBuffer[0] = ' ';
self->partWordBuffer[1] = '\0';
self->partWordBufferIndex = 1;
self->nextWordContinues = true;
self->flushPartWordBuffer();
self->nextWordContinues = true;
i += 2; // Skip the remaining two bytes (0x80 0xAF)
continue;
}
// Skip Zero Width No-Break Space / BOM (U+FEFF) = 0xEF 0xBB 0xBF
const XML_Char FEFF_BYTE_1 = static_cast<XML_Char>(0xEF);
const XML_Char FEFF_BYTE_2 = static_cast<XML_Char>(0xBB);