fix: Fix hyphenation and rendering of decomposed characters (#1037)

## Summary

* This PR fixes decomposed diacritic handling end-to-end:
- Hyphenation: normalize common Latin base+combining sequences to
precomposed codepoints before Liang pattern matching, so decomposed
words hyphenate correctly
- Rendering: correct combining-mark placement logic so non-spacing marks
are attached to the preceding base glyph in normal and rotated text
rendering paths, with corresponding text-bounds consistency updates.
- Hyphenation around non breaking space variants have been fixed (and
extended)
- Hyphenation of terms that already included of hyphens were fixed to
include Liang pattern application (eg "US-Satellitensystem" was
*exclusively* broken at the existing hyphen)

## Additional Context

* Before
<img width="800" height="480" alt="2"
src="https://github.com/user-attachments/assets/b9c515c4-ab75-45cc-8b52-f4d86bce519d"
/>


* After
<img width="480" height="800" alt="fix1"
src="https://github.com/user-attachments/assets/4999f6a8-f51c-4c0a-b144-f153f77ddb57"
/>
<img width="800" height="480" alt="fix2"
src="https://github.com/user-attachments/assets/7355126b-80c7-441f-b390-4e0897ee3fb6"
/>

* Note 1: the hyphenation fix is not a 100% bullet proof implementation.
It adds composition of *common* base+combining sequences (e.g. O +
U+0308 -> Ö) during codepoint collection. A complete solution would
require implementing proper Unicode normalization (at least NFC,
possibly NFKC in specific cases) before hyphenation and rendering,
instead of hand-mapping a few combining marks. That was beyond the scope
of this fix.

* Note 2: the render fix should be universal and not limited to the
constraints outlined above: it properly x-centers the compund glyph over
the previous one, and it uses at least 1pt of visual distance in y.

Before:
<img width="478" height="167" alt="Image"
src="https://github.com/user-attachments/assets/f8db60d5-35b1-4477-96d0-5003b4e4a2a1"
/>

After: 
<img width="479" height="180" alt="Image"
src="https://github.com/user-attachments/assets/1b48ef97-3a77-475a-8522-23f4aca8e904"
/>

* This should resolve the issues described in #998 
---

### AI Usage

While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.

Did you use AI tools to help write this code? _**PARTIALLY**_
This commit is contained in:
jpirnay
2026-02-22 13:11:07 +11:00
committed by GitHub
parent 472c56284e
commit c0ed60daa6
8 changed files with 459 additions and 29 deletions
+24 -9
View File
@@ -378,20 +378,35 @@ bool ParsedText::hyphenateWordAtIndex(const size_t wordIndex, const int availabl
words.insert(insertWordIt, remainder);
wordStyles.insert(insertStyleIt, style);
// The remainder inherits whatever continuation status the original word had with the word after it.
// Find the continues entry for the original word and insert the remainder's entry after it.
// Continuation flag handling after splitting a word into prefix + remainder.
//
// The prefix keeps the original word's continuation flag so that no-break-space groups
// stay linked. The remainder always gets continues=false because it starts on the next
// line and is not attached to the prefix.
//
// Example: "200&#xA0;Quadratkilometer" produces tokens:
// [0] "200" continues=false
// [1] " " continues=true
// [2] "Quadratkilometer" continues=true <-- the word being split
//
// After splitting "Quadratkilometer" at "Quadrat-" / "kilometer":
// [0] "200" continues=false
// [1] " " continues=true
// [2] "Quadrat-" continues=true (KEPT — still attached to the no-break group)
// [3] "kilometer" continues=false (NEW — starts fresh on the next line)
//
// This lets the backtracking loop keep the entire prefix group ("200 Quadrat-") on one
// line, while "kilometer" moves to the next line.
auto continuesIt = wordContinues.begin();
std::advance(continuesIt, wordIndex);
const bool originalContinuedToNext = *continuesIt;
// The original word (now prefix) does NOT continue to remainder (hyphen separates them)
*continuesIt = false;
// *continuesIt is intentionally left unchanged — the prefix keeps its original attachment.
const auto insertContinuesIt = std::next(continuesIt);
wordContinues.insert(insertContinuesIt, originalContinuedToNext);
wordContinues.insert(insertContinuesIt, false);
// Keep the indexed vector in sync if provided
// Keep the indexed vector in sync if provided.
if (continuesVec) {
(*continuesVec)[wordIndex] = false;
continuesVec->insert(continuesVec->begin() + wordIndex + 1, originalContinuedToNext);
// (*continuesVec)[wordIndex] stays unchanged — prefix keeps its attachment.
continuesVec->insert(continuesVec->begin() + wordIndex + 1, false);
}
// Update cached widths to reflect the new prefix/remainder pairing.