Crosspoint/docs/contributing/koreader-sync-xpath-mapping.md

KOReader Sync XPath Mapping

This note documents how CrossPoint maps reading positions to and from KOReader sync payloads.

Related architecture overview: koreader-synchronization.md

Problem

CrossPoint internally stores position as:

• spineIndex (chapter index, 0-based)
• pageNumber + totalPages

KOReader sync payload stores:

• progress (XPath-like location)
• percentage (overall progress)

A direct 1:1 mapping is not guaranteed because page layout differs between engines/devices.

DocFragment Index Convention

KOReader uses 1-based XPath predicates throughout, following standard XPath conventions. The first EPUB spine item is DocFragment[1], the second is DocFragment[2], and so on.

CrossPoint stores spine items as 0-based indices internally. The conversion is:

• Generating XPath (to KOReader): DocFragment[spineIndex + 1]
• Parsing XPath (from KOReader): spineIndex = DocFragment[N] - 1

Reference: koreader/koreader#11585 confirms this via a KOReader contributor mapping spine items to DocFragment numbers.

Current Strategy

CrossPoint -> KOReader

Implemented in ProgressMapper::toKOReader.

1. Compute overall percentage from chapter/page.
2. Generate XPath via byte-offset estimation (ChapterXPathIndexer::findXPathForProgress), producing a …/text()[K].M anchor proportional to intra-spine progress.
3. If XPath extraction fails, fallback to synthetic chapter path:
   • /body/DocFragment[spineIndex + 1]/body

The paragraph LUT (see below) is intentionally not used for upload: snapping to the start of p[N] when the user is mid-paragraph causes pulled positions to land at the start of the paragraph (and at the start of the chapter when an opening paragraph spans many pages). The LUT remains in use for the reverse direction.

KOReader -> CrossPoint

Implemented in ProgressMapper::toCrossPoint.

1. Attempt to parse DocFragment[N] from incoming XPath; convert N to 0-based spineIndex = N - 1.
2. If valid, attempt XPath-to-offset mapping via ChapterXPathIndexer::findProgressForXPath.
3. Extract paragraph index from XPath via ChapterXPathIndexer::tryExtractParagraphIndexFromXPath (e.g. /body/DocFragment[7]/body/p[685]/text().96 → paragraphIndex = 685).
4. Convert resolved intra-spine progress to page estimate.
5. If XPath path is invalid/unresolvable, fallback to percentage-based chapter/page estimation.

When a paragraph index is available, EpubReaderActivity refines the page estimate using the section cache's per-page paragraph LUT (Section::getPageForParagraphIndex). This finds the first page whose recorded paragraph index is >= the target, giving a more accurate landing position than byte-offset-based estimation alone.

ChapterXPathIndexer Design

The module reparses one spine XHTML on demand using Expat and builds temporary anchors:

Source-of-truth note: XPath anchors are built from the original EPUB spine XHTML bytes (zip item contents), not from CrossPoint's distilled section render cache. This is intentional to preserve KOReader XPath compatibility.

• anchor: <xpath, textOffset>
• textOffset counts non-whitespace bytes
• When multiple anchors exist for the same path, the one with the smallest textOffset is used (start of element), not the latest periodic anchor.

Forward lookup (CrossPoint → XPath): uses upper_bound to find the last anchor at or before the target text offset, ensuring the returned XPath corresponds to the element the user is currently inside rather than the next element.

Matching for reverse lookup:

1. exact path match — reported as exact=yes
2. index-insensitive path match (div[2] vs div[3] tolerated) — reported as exact=no
3. ancestor fallback — reported as exact=no

If no match is found, caller must fallback to percentage.

Memory / Safety Constraints (ESP32-C3)

The implementation intentionally avoids full DOM storage.

• Parse one chapter only.
• Keep anchors in transient vectors only for duration of call.
• Free XML parser and chapter byte buffer on all success/failure paths.
• No persistent cache structures are introduced by this module.

Paragraph Index LUT

The section cache stores a per-page paragraph index LUT built during page layout (ChapterHtmlSlimParser). Each entry records the 1-based <p> sibling index (direct children of <body>, matching XPath convention) at the time each page was completed.

This enables two lookups without reparsing:

• XPath → page (Section::getPageForParagraphIndex): finds the first page where the recorded paragraph index >= target. Used when applying remote KOReader progress.
• Page → XPath (Section::getParagraphIndexForPage): returns the paragraph index for a given page. Used when uploading local progress to KOReader.

The paragraph counter in ChapterHtmlSlimParser counts all <p> elements at body-child level, including display:none elements. This matches ChapterXPathIndexer and crengine's standard XPath same-name sibling counting.

Known Limitations

• Page number on reverse mapping is still an estimate (renderer differences). The paragraph LUT refines this but cannot guarantee exact page matching.
• XPath mapping intentionally uses original spine XHTML while pagination comes from distilled renderer output, so minor roundtrip page drift is expected.
• Image-only/low-text chapters may yield coarse anchors.
• Extremely malformed XHTML can force fallback behavior.

Operational Logging

ProgressMapper logs mapping source in reverse direction:

• xpath when XPath mapping path was used
• percentage when fallback path was used

It also logs exactness (exact=yes/no) for XPath matches. Note that exact=yes is only set for a full path match with correct indices; index-insensitive and ancestor matches always log exact=no.