fix: improve KOSync bidirectional position matching accuracy (#1897)
## Summary **Goal:** Fix bidirectional KOSync position matching between CrossPoint and KOReader so that syncing in either direction lands on the correct page with character-level accuracy. **Changes included:** **Download — `toCrossPoint` (server XPath → CrossPoint page)** - **XPath ancestry mode for structured elements**: The previous `ParagraphStreamer` only tracked `<p>` elements. Replaced with a full ancestor-walking mode that correctly resolves XPaths pointing into `<li>`, `<ul>`, and other structured elements. Char offset within the target element is bounded to the matched element's content only. - **Slash-in-attribute-value corrupts depth tracking**: `processByteInTag()` treated every `/` byte as a self-closing tag marker, including `/` inside quoted attribute values (e.g. `xmlns="http://..."`, `src="Links/image.jpg"`). This drove `htmlDepth` to 0 prematurely, causing the ancestry search to exit far short of the target paragraph. Fixed with `inAttrQuote` tracking. - **Off-by-one in page formula**: `intra * totalPages` rounds up incorrectly for last-page positions. Changed to `intra * (totalPages - 1)` to map the `[0, 1]` intra fraction correctly onto the `[0, totalPages-1]` page range. Example: page 14 of 17 was returned as 15. **Upload — `toKOReader` (CrossPoint page → server XPath)** - **Off-by-one in page-to-intra formula**: Symmetric fix — `pageNumber / totalPages` changed to `pageNumber / (totalPages - 1)`, with the guard updated from `> 0` to `> 1` to avoid division by zero. - **`<li>`-based XPath generation**: When the current page starts on a list item, `findXPathForProgress` now generates `ul[N]/li[M]` XPaths rather than falling back to the preceding `<p>`. Requires the new `listItemIndex` field in `PageLutEntry` (section cache version bumped to 23). - **Text-node precision with correct `text()[N].M` format**: KOReader expects `text()[N].M` where `N` is the 1-based index of the specific text node within the element. The previous attempt generated `text().M` (no brackets), which caused KOReader to jump to the front of the book. Implements a per-element text-node index stack in `XPathProgressResolver` — parallel to the existing element path stack — that correctly tracks text node indices relative to each element. Empty text nodes from bare anchor elements (`<a id="anchor"/>`) are intentionally skipped, matching KOReader's own text node counting behavior. **Reviewer-caught bugs** - **Double `onCloseTag()` on malformed `</br/>`**: Both the `tagIsClose` path and the self-closing `/` check were firing, double-decrementing `htmlDepth`. Fixed with a `!tagIsClose` guard. - **Dangling pointer in `LOG_DBG`**: `std::to_string(*nextParagraphPage).c_str()` passed a pointer to a temporary destroyed before the variadic call. Fixed with `snprintf` into a stack `char[8]` buffer. ## Additional Context - Section cache version bumped from 22 → 23 due to the new `listItemIndex` field in `PageLutEntry`. Users upgrading will see a one-time re-render of all cached sections on first load — no data loss. - The `textNodeIndexStack` in `XPathProgressResolver` is a `std::vector<int>` that mirrors the existing `path` and `parentStates` stacks — same depth, same lifetime. No additional heap pressure beyond what was already present. - All fixes verified on device with *Gentle and Lowly* by Dane C. Ortlund (spine 21, 17 pages). Download syncs land on the correct page; upload syncs land at the correct paragraph with character-level offset. ## Test plan - [ ] Download: sync from KOReader → CrossPoint lands on correct page for `text()[N].M` XPaths - [ ] Download: ancestry correctly resolves `<li>` positions inbound from KOReader - [ ] Upload: sync from CrossPoint → KOReader lands within one page for mid-paragraph positions - [ ] Upload: sync from CrossPoint → KOReader correctly targets `<li>` elements when page starts on a list item - [ ] Upload: `text()[N].M` format XPaths do not cause KOReader to jump to front of book - [ ] Section cache version 23: delete `.crosspoint/` and verify clean re-parse with no crashes --- ### AI Usage Did you use AI tools to help write this code? **YES** — developed with Claude Code (Anthropic). --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Wylan Swets
co-authored by
Claude Sonnet 4.6
Claude Sonnet 4.6
parent
628794f8b8
commit
bf894fd343
@@ -7,6 +7,7 @@
|
||||
#include <WiFi.h>
|
||||
#include <esp_sntp.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cassert>
|
||||
|
||||
#include "Epub/Section.h"
|
||||
@@ -183,15 +184,57 @@ void KOReaderSyncActivity::performSync() {
|
||||
KOReaderPosition koPos = {remoteProgress.progress, remoteProgress.percentage};
|
||||
remotePosition = ProgressMapper::toCrossPoint(epub, koPos, currentSpineIndex, totalPagesInSpine);
|
||||
|
||||
// If XPath carried a paragraph index, refine the page using the section cache's
|
||||
// per-page paragraph LUT instead of anchor matching.
|
||||
if (remotePosition.hasParagraphIndex) {
|
||||
// Refine page using section cache LUTs: li index, anchor, or paragraph index.
|
||||
if (remotePosition.hasLiIndex || remotePosition.xpathAnchorId[0] != '\0' || remotePosition.hasParagraphIndex) {
|
||||
Section tempSection(epub, remotePosition.spineIndex, renderer);
|
||||
const auto paragraphPage = tempSection.getPageForParagraphIndex(remotePosition.paragraphIndex);
|
||||
if (paragraphPage.has_value()) {
|
||||
LOG_DBG("KOSync", "Paragraph %u resolved to page %d (was %d)", remotePosition.paragraphIndex, *paragraphPage,
|
||||
remotePosition.pageNumber);
|
||||
remotePosition.pageNumber = *paragraphPage;
|
||||
bool refined = false;
|
||||
if (remotePosition.hasLiIndex) {
|
||||
const auto liPage = tempSection.getPageForListItemIndex(remotePosition.liIndex);
|
||||
if (liPage.has_value()) {
|
||||
LOG_DBG("KOSync", "Li index %u -> page %d (was %d)", remotePosition.liIndex, *liPage,
|
||||
remotePosition.pageNumber);
|
||||
remotePosition.pageNumber = *liPage;
|
||||
refined = true;
|
||||
} else {
|
||||
LOG_DBG("KOSync", "Li index %u not found in section LUT", remotePosition.liIndex);
|
||||
}
|
||||
}
|
||||
if (!refined && remotePosition.xpathAnchorId[0] != '\0') {
|
||||
const auto anchorPage = tempSection.getPageForAnchor(std::string(remotePosition.xpathAnchorId));
|
||||
if (anchorPage.has_value()) {
|
||||
LOG_DBG("KOSync", "Anchor '%s' -> page %d (was %d)", remotePosition.xpathAnchorId, *anchorPage,
|
||||
remotePosition.pageNumber);
|
||||
remotePosition.pageNumber = *anchorPage;
|
||||
refined = true;
|
||||
} else {
|
||||
LOG_DBG("KOSync", "Anchor '%s' not found in section cache", remotePosition.xpathAnchorId);
|
||||
}
|
||||
}
|
||||
if (!refined && remotePosition.hasParagraphIndex) {
|
||||
const auto paragraphPage = tempSection.getPageForParagraphIndex(remotePosition.paragraphIndex);
|
||||
const auto nextParagraphPage = tempSection.getPageForParagraphIndex(remotePosition.paragraphIndex + 1);
|
||||
if (paragraphPage.has_value()) {
|
||||
int refinedPage = std::max(remotePosition.pageNumber, static_cast<int>(*paragraphPage));
|
||||
if (nextParagraphPage.has_value()) {
|
||||
const int lutSpan = static_cast<int>(*nextParagraphPage) - static_cast<int>(*paragraphPage);
|
||||
// Only cap when the LUT span is >1. A span of 1 means the LUT granularity is too
|
||||
// coarse to trust over the intra-spine position (e.g. a stale cache where the paragraph
|
||||
// occupies different pages than at build time).
|
||||
if (lutSpan > 1 && refinedPage >= static_cast<int>(*nextParagraphPage)) {
|
||||
refinedPage = static_cast<int>(*nextParagraphPage) - 1;
|
||||
}
|
||||
}
|
||||
char nextParaBuf[8];
|
||||
if (nextParagraphPage.has_value())
|
||||
snprintf(nextParaBuf, sizeof(nextParaBuf), "%d", *nextParagraphPage);
|
||||
else
|
||||
snprintf(nextParaBuf, sizeof(nextParaBuf), "none");
|
||||
LOG_DBG("KOSync", "Paragraph %u -> LUT page %d, nextPara page %s, intra page %d, using %d",
|
||||
remotePosition.paragraphIndex, *paragraphPage, nextParaBuf, remotePosition.pageNumber, refinedPage);
|
||||
remotePosition.pageNumber = refinedPage;
|
||||
} else {
|
||||
LOG_DBG("KOSync", "Paragraph %u not found in section LUT", remotePosition.paragraphIndex);
|
||||
}
|
||||
}
|
||||
}
|
||||
// localProgress was pre-computed in EpubReaderActivity before the Epub was released.
|
||||
|
||||
Reference in New Issue
Block a user