Update docs

docs/contributing/architecture.md

@@ -125,6 +125,23 @@ Notes:
 - rendering favors reusing precomputed layout data to keep page turns responsive on constrained hardware
 - progress/session state is persisted so the reader can reopen at the last position after reboot/sleep
 
+## KOReader sync position mapping
+
+KOReader sync integration is implemented under `lib/KOReaderSync/` and is used by
+`src/activities/reader/KOReaderSyncActivity.*`.
+
+Position translation currently follows a dual-path strategy:
+
+- CrossPoint -> KOReader: prefer element-level XPath extracted from the current
+  spine XHTML; fallback to chapter-level `DocFragment` path when needed.
+- KOReader -> CrossPoint: prefer incoming XPath resolution; fallback to
+  percentage-based estimation if XPath is invalid or cannot be resolved.
+
+Detailed algorithm and constraints (including low-memory rationale for ESP32-C3)
+are documented in:
+
+- [KOReader Sync XPath Mapping](koreader-sync-xpath-mapping.md)
+
 ## State and persistence
 
 Two singletons are central:

docs/contributing/koreader-sync-xpath-mapping.md

@@ -0,0 +1,83 @@
+# KOReader Sync XPath Mapping
+
+This note documents how CrossPoint maps reading positions to and from KOReader sync payloads.
+
+## Problem
+
+CrossPoint internally stores position as:
+
+- `spineIndex` (chapter index)
+- `pageNumber` + `totalPages`
+
+KOReader sync payload stores:
+
+- `progress` (XPath-like location)
+- `percentage` (overall progress)
+
+A direct 1:1 mapping is not guaranteed because page layout differs between engines/devices.
+
+## Current Strategy
+
+### CrossPoint -> KOReader
+
+Implemented in `ProgressMapper::toKOReader`.
+
+1. Compute overall `percentage` from chapter/page.
+2. Attempt to compute a real element-level XPath via `ChapterXPathIndexer::findXPathForProgress`.
+3. If XPath extraction fails, fallback to synthetic chapter path:
+   - `/body/DocFragment[N]/body`
+
+### KOReader -> CrossPoint
+
+Implemented in `ProgressMapper::toCrossPoint`.
+
+1. Attempt to parse `DocFragment[N]` from incoming XPath.
+2. If valid, attempt XPath-to-offset mapping via `ChapterXPathIndexer::findProgressForXPath`.
+3. Convert resolved intra-spine progress to page estimate.
+4. If XPath path is invalid/unresolvable, fallback to percentage-based chapter/page estimation.
+
+## ChapterXPathIndexer Design
+
+The module reparses **one spine XHTML** on demand using Expat and builds temporary anchors:
+
+- anchor: `<xpath, textOffset>`
+- `textOffset` counts non-whitespace bytes
+
+Matching for reverse lookup:
+
+1. exact path match
+2. index-insensitive path match (`div[2]` vs `div[3]` tolerated)
+3. ancestor fallback
+
+If no match is found, caller must fallback to percentage.
+
+## Memory / Safety Constraints (ESP32-C3)
+
+The implementation intentionally avoids full DOM storage.
+
+- Parse one chapter only.
+- Keep anchors in transient vectors only for duration of call.
+- Free XML parser and chapter byte buffer on all success/failure paths.
+- No persistent cache structures are introduced by this module.
+
+## Known Limitations
+
+- Page number on reverse mapping is still an estimate (renderer differences).
+- Image-only/low-text chapters may yield coarse anchors.
+- Extremely malformed XHTML can force fallback behavior.
+
+## Operational Logging
+
+`ProgressMapper` logs mapping source in reverse direction:
+
+- `xpath` when XPath mapping path was used
+- `percentage` when fallback path was used
+
+It also logs exactness (`exact=yes/no`) for XPath matches.
+
+## Validation
+
+Use test vectors in:
+
+- `test/koreader_sync/roundtrip_vectors.md`
+- `test/koreader_sync/memory_resource_qa.md`

lib/KOReaderSync/ChapterXPathIndexer.cpp

@@ -13,6 +13,10 @@
 
 namespace {
 
+// Anchor used for both mapping directions.
+// textOffset is counted as visible (non-whitespace) bytes from chapter start.
+// xpath points to the nearest element path at/near that offset.
+
 struct XPathAnchor {
   size_t textOffset = 0;
   std::string xpath;
@@ -24,6 +28,9 @@ struct StackNode {
   bool hasTextAnchor = false;
 };
 
+// ParserState is intentionally ephemeral and created per lookup call.
+// It holds only one spine parse worth of data to avoid retaining structures
+// that would increase long-lived heap usage on the ESP32-C3.
 struct ParserState {
   explicit ParserState(const int spineIndex) : spineIndex(spineIndex) { siblingCounters.emplace_back(); }
 
@@ -37,6 +44,11 @@ struct ParserState {
 
   std::string baseXPath() const { return "/body/DocFragment[" + std::to_string(spineIndex) + "]/body"; }
 
+  // Canonicalize incoming KOReader XPath before matching:
+  // - remove all whitespace
+  // - lowercase tags
+  // - strip optional trailing /text()
+  // - strip trailing slash
   static std::string normalizeXPath(const std::string& input) {
     if (input.empty()) {
       return "";
@@ -65,6 +77,8 @@ struct ParserState {
     return out;
   }
 
+  // Remove bracketed numeric predicates so paths can be compared even when
+  // index counters differ between parser implementations.
   static std::string removeIndices(const std::string& xpath) {
     std::string out;
     out.reserve(xpath.size());
@@ -96,6 +110,9 @@ struct ParserState {
     return depth;
   }
 
+  // Resolve a path to the best anchor offset.
+  // If exact node path is not found, progressively trim trailing segments and
+  // match ancestors to obtain a stable approximate location.
   bool pickBestAnchorByPath(const std::string& targetPath, const bool ignoreIndices, size_t& outTextOffset,
                             bool& outExact) const {
     if (targetPath.empty() || anchors.empty()) {
@@ -147,6 +164,7 @@ struct ParserState {
     return value;
   }
 
+  // Elements that should not contribute text position anchors.
   static bool isSkippableTag(const std::string& tag) { return tag == "head" || tag == "script" || tag == "style"; }
 
   static bool isWhitespaceOnly(const XML_Char* text, const int len) {
@@ -158,6 +176,8 @@ struct ParserState {
     return true;
   }
 
+  // Count non-whitespace bytes to keep offsets stable against formatting-only
+  // differences and indentation in source XHTML.
   static size_t countVisibleBytes(const XML_Char* text, const int len) {
     size_t count = 0;
     for (int i = 0; i < len; i++) {
@@ -192,6 +212,8 @@ struct ParserState {
     return xpath;
   }
 
+  // Adds first anchor for an element when text begins and periodic anchors in
+  // longer runs so matching has sufficient granularity without exploding memory.
   void addAnchorIfNeeded() {
     if (!insideBody() || stack.empty()) {
       return;
@@ -269,6 +291,7 @@ struct ParserState {
     return it->xpath;
   }
 
+  // Convert path -> progress ratio by matching to nearest available anchor.
   bool chooseProgressForXPath(const std::string& xpath, float& outIntraSpineProgress, bool& outExactMatch) const {
     if (anchors.empty()) {
       return false;

lib/KOReaderSync/ChapterXPathIndexer.h

@@ -6,38 +6,58 @@
 #include <string>
 
 /**
- * Builds element-level XPath anchors for a spine item and picks the best match
+ * Lightweight XPath/progress bridge for KOReader sync.
- * for an intra-spine progress value.
+ *
+ * Why this exists:
+ * - CrossPoint stores reading position as chapter/page.
+ * - KOReader sync uses XPath + percentage.
+ *
+ * This utility reparses exactly one spine XHTML item with Expat and builds
+ * transient text anchors (<xpath, textOffset>) so we can translate in both
+ * directions without keeping a full DOM in memory.
+ *
+ * Design constraints (ESP32-C3):
+ * - No persistent full-book structures.
+ * - Parse-on-demand and free memory immediately.
+ * - Keep fallback behavior deterministic if parsing/matching fails.
  */
 class ChapterXPathIndexer {
  public:
   /**
+   * Convert an intra-spine progress ratio to the nearest element-level XPath.
+   *
    * @param epub Loaded EPUB instance
    * @param spineIndex Current spine item index
    * @param intraSpineProgress Position within the spine item [0.0, 1.0]
-   * @return Best matching XPath, or empty string on failure
+   * @return Best matching XPath for KOReader, or empty string on failure
    */
   static std::string findXPathForProgress(const std::shared_ptr<Epub>& epub, int spineIndex, float intraSpineProgress);
 
   /**
-   * Resolve a KOReader XPath to an intra-spine progress value.
+   * Resolve a KOReader XPath to an intra-spine progress ratio.
+   *
+   * Matching strategy:
+   * 1) exact anchor path match,
+   * 2) index-insensitive path match,
+   * 3) ancestor fallback.
    *
    * @param epub Loaded EPUB instance
    * @param spineIndex Spine item index to parse
    * @param xpath Incoming KOReader XPath
    * @param outIntraSpineProgress Resolved position within spine [0.0, 1.0]
-   * @param outExactMatch True when an exact anchor match was found
+   * @param outExactMatch True only for full exact path match
-   * @return true if an exact or ancestor match was resolved
+   * @return true if any match was resolved; false means caller should fallback
    */
   static bool findProgressForXPath(const std::shared_ptr<Epub>& epub, int spineIndex, const std::string& xpath,
                                    float& outIntraSpineProgress, bool& outExactMatch);
 
   /**
-   * Parse the DocFragment index from a KOReader-style XPath.
+   * Parse DocFragment index from KOReader-style path segment:
+   * /body/DocFragment[N]/body/...
    *
    * @param xpath KOReader XPath
    * @param outSpineIndex Parsed DocFragment index (0-based)
-   * @return true when DocFragment[...] is present and valid
+   * @return true when DocFragment[N] exists and N is valid integer >= 0
    */
   static bool tryExtractSpineIndexFromXPath(const std::string& xpath, int& outSpineIndex);
 };

lib/KOReaderSync/ProgressMapper.cpp

@@ -6,7 +6,6 @@
 
 #include "ChapterXPathIndexer.h"
 
-
 KOReaderPosition ProgressMapper::toKOReader(const std::shared_ptr<Epub>& epub, const CrossPointPosition& pos) {
   KOReaderPosition result;
 

lib/KOReaderSync/ProgressMapper.h

@@ -27,9 +27,16 @@ struct KOReaderPosition {
  * CrossPoint tracks position as (spineIndex, pageNumber).
  * KOReader uses XPath-like strings + percentage.
  *
- * CrossPoint first tries to extract an element-level XPath by reparsing the
+ * Forward mapping (CrossPoint -> KOReader):
- * current spine XHTML and mapping intra-spine progress to text anchors.
+ * - Prefer element-level XPath extracted from current spine XHTML.
- * If extraction fails, it falls back to a synthetic chapter-level XPath.
+ * - Fallback to synthetic chapter XPath if extraction fails.
+ *
+ * Reverse mapping (KOReader -> CrossPoint):
+ * - Prefer incoming XPath (DocFragment + element path) when resolvable.
+ * - Fallback to percentage-based approximation when XPath is missing/invalid.
+ *
+ * This keeps behavior stable on low-memory devices while improving round-trip
+ * sync precision when KOReader provides detailed paths.
  */
 class ProgressMapper {
  public:
@@ -45,8 +52,9 @@ class ProgressMapper {
   /**
    * Convert KOReader position to CrossPoint format.
    *
-   * Note: The returned pageNumber may be approximate since different
+   * Uses XPath-first resolution when possible and percentage fallback otherwise.
-   * rendering settings produce different page counts.
+   * Returned pageNumber can still be approximate because page counts differ
+   * across renderer/font/layout settings.
    *
    * @param epub The EPUB book
    * @param koPos KOReader position