Add paragraph index LUT for accurate KOReader position sync
Store per-page paragraph indices in section cache to enable precise XPath-to-page and page-to-XPath mapping without reparsing XHTML. Forward path (upload): generates XPath directly from paragraph LUT instead of byte-offset estimation, eliminating drift in chapters with non-uniform content density. Reverse path (download): resolves incoming KOReader XPath p[N] to the exact page via paragraph LUT lookup. Paragraph counter counts all <p> elements including display:none to match ChapterXPathIndexer and crengine's standard XPath counting. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
jpirnay
co-authored by
Claude Opus 4.6
Claude Opus 4.6
parent
dccb82642d
commit
b625b8bd26
+2
-2
@@ -848,7 +848,7 @@ rm -rf /path/to/sd/.crosspoint/epub_<hash>/sections/
|
||||
|
||||
**Current Versions** (as of docs/file-formats.md):
|
||||
- `book.bin`: **Version 5** (metadata structure)
|
||||
- `section.bin`: **Version 12** (layout structure)
|
||||
- `section.bin`: **Version 20** (layout structure, includes paragraph LUT)
|
||||
|
||||
**Version Increment Rules**:
|
||||
1. **ALWAYS increment version** BEFORE changing binary structure
|
||||
@@ -858,7 +858,7 @@ rm -rf /path/to/sd/.crosspoint/epub_<hash>/sections/
|
||||
**Example** (incrementing section format version):
|
||||
```cpp
|
||||
// lib/Epub/Epub/Section.cpp
|
||||
static constexpr uint8_t SECTION_FILE_VERSION = 13; // Was 12, now 13
|
||||
static constexpr uint8_t SECTION_FILE_VERSION = 20; // Was 19, now 20
|
||||
|
||||
// Add new field to structure
|
||||
struct PageLine {
|
||||
|
||||
@@ -36,8 +36,10 @@ via a KOReader contributor mapping spine items to DocFragment numbers.
|
||||
Implemented in `ProgressMapper::toKOReader`.
|
||||
|
||||
1. Compute overall `percentage` from chapter/page.
|
||||
2. Attempt to compute a real element-level XPath via `ChapterXPathIndexer::findXPathForProgress`.
|
||||
3. If XPath extraction fails, fallback to synthetic chapter path:
|
||||
2. If a paragraph index is available from the section cache LUT (`CrossPointPosition::hasParagraphIndex`),
|
||||
generate an XPath directly: `/body/DocFragment[spineIndex + 1]/body/p[paragraphIndex]`.
|
||||
3. Otherwise, attempt byte-offset estimation via `ChapterXPathIndexer::findXPathForProgress`.
|
||||
4. If XPath extraction fails, fallback to synthetic chapter path:
|
||||
- `/body/DocFragment[spineIndex + 1]/body`
|
||||
|
||||
### KOReader -> CrossPoint
|
||||
@@ -46,8 +48,15 @@ Implemented in `ProgressMapper::toCrossPoint`.
|
||||
|
||||
1. Attempt to parse `DocFragment[N]` from incoming XPath; convert N to 0-based `spineIndex = N - 1`.
|
||||
2. If valid, attempt XPath-to-offset mapping via `ChapterXPathIndexer::findProgressForXPath`.
|
||||
3. Convert resolved intra-spine progress to page estimate.
|
||||
4. If XPath path is invalid/unresolvable, fallback to percentage-based chapter/page estimation.
|
||||
3. Extract paragraph index from XPath via `ChapterXPathIndexer::tryExtractParagraphIndexFromXPath`
|
||||
(e.g. `/body/DocFragment[7]/body/p[685]/text().96` → `paragraphIndex = 685`).
|
||||
4. Convert resolved intra-spine progress to page estimate.
|
||||
5. If XPath path is invalid/unresolvable, fallback to percentage-based chapter/page estimation.
|
||||
|
||||
When a paragraph index is available, `EpubReaderActivity` refines the page estimate using
|
||||
the section cache's per-page paragraph LUT (`Section::getPageForParagraphIndex`). This finds
|
||||
the first page whose recorded paragraph index is >= the target, giving a more accurate
|
||||
landing position than byte-offset-based estimation alone.
|
||||
|
||||
## ChapterXPathIndexer Design
|
||||
|
||||
@@ -81,9 +90,27 @@ The implementation intentionally avoids full DOM storage.
|
||||
- Free XML parser and chapter byte buffer on all success/failure paths.
|
||||
- No persistent cache structures are introduced by this module.
|
||||
|
||||
## Paragraph Index LUT
|
||||
|
||||
The section cache stores a per-page paragraph index LUT built during page layout
|
||||
(`ChapterHtmlSlimParser`). Each entry records the 1-based `<p>` sibling index
|
||||
(direct children of `<body>`, matching XPath convention) at the time each page was completed.
|
||||
|
||||
This enables two lookups without reparsing:
|
||||
|
||||
- **XPath → page** (`Section::getPageForParagraphIndex`): finds the first page where the
|
||||
recorded paragraph index >= target. Used when applying remote KOReader progress.
|
||||
- **Page → XPath** (`Section::getParagraphIndexForPage`): returns the paragraph index for
|
||||
a given page. Used when uploading local progress to KOReader.
|
||||
|
||||
The paragraph counter in `ChapterHtmlSlimParser` counts **all** `<p>` elements at body-child
|
||||
level, including `display:none` elements. This matches `ChapterXPathIndexer` and crengine's
|
||||
standard XPath same-name sibling counting.
|
||||
|
||||
## Known Limitations
|
||||
|
||||
- Page number on reverse mapping is still an estimate (renderer differences).
|
||||
The paragraph LUT refines this but cannot guarantee exact page matching.
|
||||
- XPath mapping intentionally uses original spine XHTML while pagination comes from distilled renderer output, so minor roundtrip page drift is expected.
|
||||
- Image-only/low-text chapters may yield coarse anchors.
|
||||
- Extremely malformed XHTML can force fallback behavior.
|
||||
|
||||
+37
-13
@@ -104,7 +104,7 @@ if (parsedSize != fileSize) {
|
||||
|
||||
## `section.bin`
|
||||
|
||||
### Version 8
|
||||
### Version 20
|
||||
|
||||
ImHex Pattern:
|
||||
|
||||
@@ -114,7 +114,7 @@ import std.string;
|
||||
import std.core;
|
||||
|
||||
// === Configuration ===
|
||||
#define EXPECTED_VERSION 8
|
||||
#define EXPECTED_VERSION 20
|
||||
#define MAX_STRING_LENGTH 65535
|
||||
|
||||
// === String Structure ===
|
||||
@@ -175,36 +175,60 @@ struct Page {
|
||||
PageElement elements[elementCount] [[inline]];
|
||||
};
|
||||
|
||||
// === Anchor Map Entry ===
|
||||
|
||||
struct AnchorEntry {
|
||||
String anchorId [[comment("HTML id attribute value")]];
|
||||
u16 pageNumber [[comment("Page where the anchor appears")]];
|
||||
};
|
||||
|
||||
// === Section Bin Structure ===
|
||||
|
||||
struct SectionBin {
|
||||
// Header
|
||||
u8 version [[comment("Format version"), color("FFD93D")]];
|
||||
|
||||
|
||||
// Version validation
|
||||
if (version != EXPECTED_VERSION) {
|
||||
std::error(std::format("Unsupported version: {} (expected {})", version, EXPECTED_VERSION));
|
||||
}
|
||||
|
||||
|
||||
// Cache busting parameters
|
||||
s32 fontId;
|
||||
float lineCompression;
|
||||
bool extraParagraphSpacing;
|
||||
u8 paragraphAlignment;
|
||||
u16 viewportWidth;
|
||||
u16 vieportHeight;
|
||||
u16 viewportHeight;
|
||||
u16 pageCount;
|
||||
u32 lutOffset;
|
||||
|
||||
bool hyphenationEnabled;
|
||||
bool embeddedStyle;
|
||||
u8 imageRendering;
|
||||
u32 pageLutOffset [[comment("Offset to page offset LUT")]];
|
||||
u32 anchorMapOffset [[comment("Offset to anchor map")]];
|
||||
u32 paragraphLutOffset [[comment("Offset to per-page paragraph index LUT")]];
|
||||
|
||||
Page page[pageCount];
|
||||
|
||||
|
||||
// === Page Offset LUT ===
|
||||
// Validate LUT offset alignment
|
||||
u32 currentOffset = $;
|
||||
if (currentOffset != lutOffset) {
|
||||
std::warning(std::format("LUT offset mismatch: expected 0x{:X}, got 0x{:X}", lutOffset, currentOffset));
|
||||
if (currentOffset != pageLutOffset) {
|
||||
std::warning(std::format("Page LUT offset mismatch: expected 0x{:X}, got 0x{:X}", pageLutOffset, currentOffset));
|
||||
}
|
||||
|
||||
// Lookup Tables
|
||||
u32 lut[pageCount];
|
||||
|
||||
u32 pageOffsets[pageCount] [[comment("File offsets to serialized pages")]];
|
||||
|
||||
// === Anchor Map ===
|
||||
u16 anchorCount;
|
||||
AnchorEntry anchors[anchorCount];
|
||||
|
||||
// === Paragraph Index LUT ===
|
||||
// One entry per page: the 1-based <p> sibling index (XPath convention)
|
||||
// at the time each page was completed during parsing.
|
||||
// Used to resolve KOReader XPath p[N] positions to page numbers.
|
||||
u16 paragraphEntryCount;
|
||||
u16 paragraphIndexPerPage[paragraphEntryCount] [[comment("1-based <p> index at page completion")]];
|
||||
};
|
||||
|
||||
// === File Parsing ===
|
||||
|
||||
+119
-9
@@ -10,10 +10,21 @@
|
||||
#include "parsers/ChapterHtmlSlimParser.h"
|
||||
|
||||
namespace {
|
||||
constexpr uint8_t SECTION_FILE_VERSION = 18;
|
||||
constexpr uint32_t HEADER_SIZE = sizeof(uint8_t) + sizeof(int) + sizeof(float) + sizeof(bool) + sizeof(uint8_t) +
|
||||
sizeof(uint16_t) + sizeof(uint16_t) + sizeof(uint16_t) + sizeof(bool) + sizeof(bool) +
|
||||
sizeof(uint8_t) + sizeof(uint32_t) + sizeof(uint32_t);
|
||||
constexpr uint8_t SECTION_FILE_VERSION = 20;
|
||||
constexpr uint32_t HEADER_SIZE = sizeof(uint8_t) + // SECTION_FILE_VERSION
|
||||
sizeof(int) + // fontId
|
||||
sizeof(float) + // lineCompression
|
||||
sizeof(bool) + // extraParagraphSpacing
|
||||
sizeof(uint8_t) + // paragraphAlignment
|
||||
sizeof(uint16_t) + // viewportWidth
|
||||
sizeof(uint16_t) + // viewportHeight
|
||||
sizeof(uint16_t) + // pageCount (stored as 16-bit in header)
|
||||
sizeof(bool) + // hyphenationEnabled
|
||||
sizeof(bool) + // embeddedStyle
|
||||
sizeof(uint8_t) + // imageRendering
|
||||
sizeof(uint32_t) + // page LUT offset
|
||||
sizeof(uint32_t) + // anchor map offset
|
||||
sizeof(uint32_t); // paragraph LUT offset
|
||||
} // namespace
|
||||
|
||||
uint32_t Section::onPageComplete(std::unique_ptr<Page> page) {
|
||||
@@ -44,7 +55,8 @@ void Section::writeSectionFileHeader(const int fontId, const float lineCompressi
|
||||
static_assert(HEADER_SIZE == sizeof(SECTION_FILE_VERSION) + sizeof(fontId) + sizeof(lineCompression) +
|
||||
sizeof(extraParagraphSpacing) + sizeof(paragraphAlignment) + sizeof(viewportWidth) +
|
||||
sizeof(viewportHeight) + sizeof(pageCount) + sizeof(hyphenationEnabled) +
|
||||
sizeof(embeddedStyle) + sizeof(imageRendering) + sizeof(uint32_t) + sizeof(uint32_t),
|
||||
sizeof(embeddedStyle) + sizeof(imageRendering) + sizeof(uint32_t) +
|
||||
sizeof(uint32_t) + sizeof(uint32_t),
|
||||
"Header size mismatch");
|
||||
serialization::writePod(file, SECTION_FILE_VERSION);
|
||||
serialization::writePod(file, fontId);
|
||||
@@ -59,6 +71,7 @@ void Section::writeSectionFileHeader(const int fontId, const float lineCompressi
|
||||
serialization::writePod(file, pageCount); // Placeholder for page count (will be initially 0, patched later)
|
||||
serialization::writePod(file, static_cast<uint32_t>(0)); // Placeholder for LUT offset (patched later)
|
||||
serialization::writePod(file, static_cast<uint32_t>(0)); // Placeholder for anchor map offset (patched later)
|
||||
serialization::writePod(file, static_cast<uint32_t>(0)); // Placeholder for paragraph LUT offset (patched later)
|
||||
}
|
||||
|
||||
bool Section::loadSectionFile(const int fontId, const float lineCompression, const bool extraParagraphSpacing,
|
||||
@@ -249,11 +262,20 @@ bool Section::createSectionFile(const int fontId, const float lineCompression, c
|
||||
serialization::writePod(file, page);
|
||||
}
|
||||
|
||||
// Patch header with final pageCount, lutOffset, and anchorMapOffset
|
||||
file.seek(HEADER_SIZE - sizeof(uint32_t) * 2 - sizeof(pageCount));
|
||||
// Write per-page paragraph index LUT for XPath-to-page resolution
|
||||
const uint32_t paragraphLutOffset = file.position();
|
||||
const auto& paragraphPerPage = visitor.getParagraphIndexPerPage();
|
||||
serialization::writePod(file, static_cast<uint16_t>(paragraphPerPage.size()));
|
||||
for (const uint16_t& pIdx : paragraphPerPage) {
|
||||
serialization::writePod(file, pIdx);
|
||||
}
|
||||
|
||||
// Patch header with final pageCount, lutOffset, anchorMapOffset, and paragraphLutOffset
|
||||
file.seek(HEADER_SIZE - sizeof(uint32_t) * 3 - sizeof(pageCount));
|
||||
serialization::writePod(file, pageCount);
|
||||
serialization::writePod(file, lutOffset);
|
||||
serialization::writePod(file, anchorMapOffset);
|
||||
serialization::writePod(file, paragraphLutOffset);
|
||||
file.close();
|
||||
if (cssParser) {
|
||||
cssParser->clear();
|
||||
@@ -266,7 +288,7 @@ std::unique_ptr<Page> Section::loadPageFromSectionFile() {
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
file.seek(HEADER_SIZE - sizeof(uint32_t) * 2);
|
||||
file.seek(HEADER_SIZE - sizeof(uint32_t) * 3);
|
||||
uint32_t lutOffset;
|
||||
serialization::readPod(file, lutOffset);
|
||||
file.seek(lutOffset + sizeof(uint32_t) * currentPage);
|
||||
@@ -286,7 +308,7 @@ std::optional<uint16_t> Section::getPageForAnchor(const std::string& anchor) con
|
||||
}
|
||||
|
||||
const uint32_t fileSize = f.size();
|
||||
f.seek(HEADER_SIZE - sizeof(uint32_t));
|
||||
f.seek(HEADER_SIZE - sizeof(uint32_t) * 2);
|
||||
uint32_t anchorMapOffset;
|
||||
serialization::readPod(f, anchorMapOffset);
|
||||
if (anchorMapOffset == 0 || anchorMapOffset >= fileSize) {
|
||||
@@ -311,3 +333,91 @@ std::optional<uint16_t> Section::getPageForAnchor(const std::string& anchor) con
|
||||
f.close();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
std::optional<uint16_t> Section::getPageForParagraphIndex(const uint16_t pIndex) const {
|
||||
FsFile f;
|
||||
if (!Storage.openFileForRead("SCT", filePath, f)) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
const uint32_t fileSize = f.size();
|
||||
|
||||
// Read paragraph LUT offset from end of header
|
||||
f.seek(HEADER_SIZE - sizeof(uint32_t));
|
||||
uint32_t paragraphLutOffset;
|
||||
serialization::readPod(f, paragraphLutOffset);
|
||||
if (paragraphLutOffset == 0 || paragraphLutOffset >= fileSize) {
|
||||
f.close();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
f.seek(paragraphLutOffset);
|
||||
uint16_t count;
|
||||
serialization::readPod(f, count);
|
||||
if (count == 0) {
|
||||
f.close();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
// Validate that all entries fit within the file
|
||||
const uint32_t lutEnd = paragraphLutOffset + sizeof(uint16_t) + count * sizeof(uint16_t);
|
||||
if (lutEnd > fileSize) {
|
||||
f.close();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
// Find the first page whose paragraph index >= pIndex.
|
||||
// Each entry stores the <p> index at the time that page was completed.
|
||||
uint16_t resultPage = count - 1; // default to last page
|
||||
for (uint16_t i = 0; i < count; i++) {
|
||||
uint16_t pagePIdx;
|
||||
serialization::readPod(f, pagePIdx);
|
||||
if (pagePIdx >= pIndex) {
|
||||
resultPage = i;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
f.close();
|
||||
return resultPage;
|
||||
}
|
||||
|
||||
std::optional<uint16_t> Section::getParagraphIndexForPage(const uint16_t page) const {
|
||||
FsFile f;
|
||||
if (!Storage.openFileForRead("SCT", filePath, f)) {
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
const uint32_t fileSize = f.size();
|
||||
|
||||
f.seek(HEADER_SIZE - sizeof(uint32_t));
|
||||
uint32_t paragraphLutOffset;
|
||||
serialization::readPod(f, paragraphLutOffset);
|
||||
if (paragraphLutOffset == 0 || paragraphLutOffset >= fileSize) {
|
||||
f.close();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
f.seek(paragraphLutOffset);
|
||||
uint16_t count;
|
||||
serialization::readPod(f, count);
|
||||
if (count == 0 || page >= count) {
|
||||
f.close();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
// Validate that the target entry fits within the file
|
||||
const uint32_t entryEnd = paragraphLutOffset + sizeof(uint16_t) + (page + 1) * sizeof(uint16_t);
|
||||
if (entryEnd > fileSize) {
|
||||
f.close();
|
||||
return std::nullopt;
|
||||
}
|
||||
|
||||
// Seek to the entry for the requested page
|
||||
f.seek(paragraphLutOffset + sizeof(uint16_t) + page * sizeof(uint16_t));
|
||||
uint16_t pIdx;
|
||||
serialization::readPod(f, pIdx);
|
||||
|
||||
f.close();
|
||||
return pIdx;
|
||||
}
|
||||
|
||||
@@ -42,4 +42,14 @@ class Section {
|
||||
|
||||
// Look up the page number for an anchor id from the section cache file.
|
||||
std::optional<uint16_t> getPageForAnchor(const std::string& anchor) const;
|
||||
|
||||
// Look up the page number for a paragraph index (1-based, from XPath p[N]).
|
||||
// Uses the per-page paragraph index LUT stored in the section cache.
|
||||
// Returns nullopt if the paragraph LUT is not available (old cache format).
|
||||
std::optional<uint16_t> getPageForParagraphIndex(uint16_t pIndex) const;
|
||||
|
||||
// Look up the paragraph index for a given page number.
|
||||
// Returns the 1-based paragraph index of the last <p> element on or before the page.
|
||||
// Returns nullopt if the paragraph LUT is not available (old cache format).
|
||||
std::optional<uint16_t> getParagraphIndexForPage(uint16_t page) const;
|
||||
};
|
||||
|
||||
@@ -491,6 +491,19 @@ void XMLCALL ChapterHtmlSlimParser::startElement(void* userData, const XML_Char*
|
||||
}
|
||||
}
|
||||
|
||||
// Track body element depth for paragraph index counting
|
||||
if (strcmp(name, "body") == 0 && self->xpathBodyDepth < 0) {
|
||||
self->xpathBodyDepth = self->depth;
|
||||
}
|
||||
|
||||
// Count <p> sibling indices at body-child level. Must happen BEFORE the display:none
|
||||
// check so that hidden <p> elements are still counted, matching ChapterXPathIndexer's
|
||||
// counting (pure XML, no CSS). This ensures paragraph indices in the section cache LUT
|
||||
// align with KOReader's crengine XPath indices.
|
||||
if (self->xpathBodyDepth >= 0 && self->depth == self->xpathBodyDepth + 1 && strcmp(name, "p") == 0) {
|
||||
self->xpathParagraphIndex++;
|
||||
}
|
||||
|
||||
if (matches(name, SKIP_TAGS, NUM_SKIP_TAGS)) {
|
||||
// start skip
|
||||
self->skipUntilDepth = self->depth;
|
||||
@@ -1062,6 +1075,7 @@ bool ChapterHtmlSlimParser::parseAndBuildPages() {
|
||||
anchorData.push_back({std::move(pendingAnchorId), static_cast<uint16_t>(completedPageCount)});
|
||||
pendingAnchorId.clear();
|
||||
}
|
||||
paragraphIndexPerPage.push_back(xpathParagraphIndex);
|
||||
completePageFn(std::move(currentPage));
|
||||
completedPageCount++;
|
||||
currentPage.reset();
|
||||
@@ -1080,6 +1094,7 @@ void ChapterHtmlSlimParser::addLineToPage(std::shared_ptr<TextBlock> line) {
|
||||
}
|
||||
|
||||
if (currentPageNextY + lineHeight > viewportHeight) {
|
||||
paragraphIndexPerPage.push_back(xpathParagraphIndex);
|
||||
completePageFn(std::move(currentPage));
|
||||
completedPageCount++;
|
||||
currentPage.reset(new Page());
|
||||
|
||||
@@ -75,6 +75,14 @@ class ChapterHtmlSlimParser {
|
||||
std::vector<std::pair<std::string, uint16_t>> anchorData;
|
||||
std::string pendingAnchorId; // deferred until after previous text block is flushed
|
||||
|
||||
// Paragraph index tracking for XPath-to-page lookup table.
|
||||
// Counts <p> sibling indices (1-based, matching XPath convention) during page building.
|
||||
// Stored per page in the section cache so that XPath p[N] can be resolved to a page
|
||||
// without reparsing, and current page can generate an XPath without reparsing.
|
||||
uint16_t xpathParagraphIndex = 0; // current <p> sibling index (1-based)
|
||||
int xpathBodyDepth = -1; // depth of the <body> element (-1 = not yet seen)
|
||||
std::vector<uint16_t> paragraphIndexPerPage; // <p> index at each page completion
|
||||
|
||||
// Footnote link tracking
|
||||
bool insideFootnoteLink = false;
|
||||
int footnoteLinkDepth = -1;
|
||||
@@ -126,4 +134,5 @@ class ChapterHtmlSlimParser {
|
||||
bool parseAndBuildPages();
|
||||
void addLineToPage(std::shared_ptr<TextBlock> line);
|
||||
const std::vector<std::pair<std::string, uint16_t>>& getAnchors() const { return anchorData; }
|
||||
const std::vector<uint16_t>& getParagraphIndexPerPage() const { return paragraphIndexPerPage; }
|
||||
};
|
||||
|
||||
@@ -266,6 +266,8 @@ STR_SYNCING_TIME: "Syncing time..."
|
||||
STR_CALC_HASH: "Calculating document hash..."
|
||||
STR_HASH_FAILED: "Failed to calculate document hash"
|
||||
STR_FETCH_PROGRESS: "Fetching remote progress..."
|
||||
STR_MAPPING_REMOTE: "Mapping remote position..."
|
||||
STR_MAPPING_LOCAL: "Calculating local position..."
|
||||
STR_UPLOAD_PROGRESS: "Uploading progress..."
|
||||
STR_NO_CREDENTIALS_MSG: "No credentials configured"
|
||||
STR_KOREADER_SETUP_HINT: "Set up KOReader account in Settings"
|
||||
|
||||
@@ -599,3 +599,43 @@ bool ChapterXPathIndexer::tryExtractSpineIndexFromXPath(const std::string& xpath
|
||||
outSpineIndex = static_cast<int>(parsed) - 1;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool ChapterXPathIndexer::tryExtractParagraphIndexFromXPath(const std::string& xpath, uint16_t& outParagraphIndex) {
|
||||
outParagraphIndex = 0;
|
||||
if (xpath.empty()) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const std::string normalized = normalizeXPath(xpath);
|
||||
|
||||
// Find /p[ after the second /body/ (the inner body inside DocFragment)
|
||||
const std::string bodyKey = "/body";
|
||||
size_t secondBody = normalized.find(bodyKey);
|
||||
if (secondBody != std::string::npos) {
|
||||
secondBody = normalized.find(bodyKey, secondBody + bodyKey.size());
|
||||
}
|
||||
|
||||
const std::string pKey = "/p[";
|
||||
const size_t pos = normalized.find(pKey, secondBody != std::string::npos ? secondBody : 0);
|
||||
if (pos == std::string::npos) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const size_t start = pos + pKey.size();
|
||||
size_t end = start;
|
||||
while (end < normalized.size() && std::isdigit(static_cast<unsigned char>(normalized[end]))) {
|
||||
end++;
|
||||
}
|
||||
|
||||
if (end == start || end >= normalized.size() || normalized[end] != ']') {
|
||||
return false;
|
||||
}
|
||||
|
||||
const long parsed = std::strtol(normalized.substr(start, end - start).c_str(), nullptr, 10);
|
||||
if (parsed < 1 || parsed > UINT16_MAX) {
|
||||
return false;
|
||||
}
|
||||
|
||||
outParagraphIndex = static_cast<uint16_t>(parsed);
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -64,4 +64,16 @@ class ChapterXPathIndexer {
|
||||
* (converted to 0-based outSpineIndex); false otherwise
|
||||
*/
|
||||
static bool tryExtractSpineIndexFromXPath(const std::string& xpath, int& outSpineIndex);
|
||||
|
||||
/**
|
||||
* Extract the paragraph index from a KOReader XPath.
|
||||
* Looks for the first /p[N] segment after /body/ and returns N (1-based).
|
||||
*
|
||||
* Example: "/body/DocFragment[7]/body/p[685]/text().96" → outParagraphIndex = 685
|
||||
*
|
||||
* @param xpath KOReader XPath
|
||||
* @param outParagraphIndex 1-based paragraph index
|
||||
* @return true if a /p[N] segment was found
|
||||
*/
|
||||
static bool tryExtractParagraphIndexFromXPath(const std::string& xpath, uint16_t& outParagraphIndex);
|
||||
};
|
||||
|
||||
@@ -19,12 +19,17 @@ KOReaderPosition ProgressMapper::toKOReader(const std::shared_ptr<Epub>& epub, c
|
||||
// Calculate overall book progress (0.0-1.0)
|
||||
result.percentage = epub->calculateProgress(pos.spineIndex, intraSpineProgress);
|
||||
|
||||
// Generate the best available XPath for the current chapter position.
|
||||
// Prefer element-level XPaths from a lightweight XHTML reparse; fall back
|
||||
// to a synthetic chapter-level path if parsing fails.
|
||||
result.xpath = ChapterXPathIndexer::findXPathForProgress(epub, pos.spineIndex, intraSpineProgress);
|
||||
if (result.xpath.empty()) {
|
||||
result.xpath = generateXPath(pos.spineIndex);
|
||||
// Generate XPath for the current position.
|
||||
// Prefer paragraph index from the section cache LUT (exact element mapping) over
|
||||
// byte-offset estimation (which can drift in chapters with non-uniform content density).
|
||||
if (pos.hasParagraphIndex && pos.paragraphIndex > 0) {
|
||||
result.xpath = "/body/DocFragment[" + std::to_string(pos.spineIndex + 1) + "]/body/p[" +
|
||||
std::to_string(pos.paragraphIndex) + "]";
|
||||
} else {
|
||||
result.xpath = ChapterXPathIndexer::findXPathForProgress(epub, pos.spineIndex, intraSpineProgress);
|
||||
if (result.xpath.empty()) {
|
||||
result.xpath = generateXPath(pos.spineIndex);
|
||||
}
|
||||
}
|
||||
|
||||
// Get chapter info for logging
|
||||
@@ -64,6 +69,12 @@ CrossPointPosition ProgressMapper::toCrossPoint(const std::shared_ptr<Epub>& epu
|
||||
resolvedIntraSpineProgress = intraFromXPath;
|
||||
usedXPathMapping = true;
|
||||
}
|
||||
// Extract paragraph index from XPath for direct page lookup via section cache
|
||||
uint16_t pIndex = 0;
|
||||
if (ChapterXPathIndexer::tryExtractParagraphIndexFromXPath(koPos.xpath, pIndex)) {
|
||||
result.paragraphIndex = pIndex;
|
||||
result.hasParagraphIndex = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (!usedXPathMapping) {
|
||||
|
||||
@@ -8,9 +8,11 @@
|
||||
* CrossPoint position representation.
|
||||
*/
|
||||
struct CrossPointPosition {
|
||||
int spineIndex; // Current spine item (chapter) index
|
||||
int pageNumber; // Current page within the spine item
|
||||
int totalPages; // Total pages in the current spine item
|
||||
int spineIndex; // Current spine item (chapter) index
|
||||
int pageNumber; // Current page within the spine item (estimated if no paragraph LUT)
|
||||
int totalPages; // Total pages in the current spine item
|
||||
uint16_t paragraphIndex = 0; // 1-based <p> index from XPath (0 if unavailable)
|
||||
bool hasParagraphIndex = false; // True when paragraphIndex was resolved from XPath
|
||||
};
|
||||
|
||||
/**
|
||||
|
||||
@@ -39,7 +39,9 @@ struct PageResult {
|
||||
|
||||
struct SyncResult {
|
||||
int spineIndex = 0;
|
||||
int page = 0;
|
||||
int page = 0; // estimated page (fallback)
|
||||
uint16_t paragraphIndex = 0; // 1-based <p> index from XPath
|
||||
bool hasParagraphIndex = false; // true when paragraphIndex is available
|
||||
};
|
||||
|
||||
enum class NetworkMode;
|
||||
|
||||
@@ -395,9 +395,18 @@ void EpubReaderActivity::onReaderMenuConfirm(EpubReaderMenuActivity::MenuAction
|
||||
if (KOREADER_STORE.hasCredentials()) {
|
||||
const int currentPage = section ? section->currentPage : 0;
|
||||
const int totalPages = section ? section->pageCount : 0;
|
||||
// Look up paragraph index from section cache for accurate XPath generation on upload
|
||||
uint16_t paragraphIdx = 0;
|
||||
bool hasParagraphIdx = false;
|
||||
if (section) {
|
||||
if (const auto pIdx = section->getParagraphIndexForPage(currentPage)) {
|
||||
paragraphIdx = *pIdx;
|
||||
hasParagraphIdx = true;
|
||||
}
|
||||
}
|
||||
startActivityForResult(
|
||||
std::make_unique<KOReaderSyncActivity>(renderer, mappedInput, epub, epub->getPath(), currentSpineIndex,
|
||||
currentPage, totalPages),
|
||||
currentPage, totalPages, paragraphIdx, hasParagraphIdx),
|
||||
[this](const ActivityResult& result) {
|
||||
if (!result.isCancelled) {
|
||||
const auto& sync = std::get<SyncResult>(result.data);
|
||||
@@ -405,6 +414,10 @@ void EpubReaderActivity::onReaderMenuConfirm(EpubReaderMenuActivity::MenuAction
|
||||
RenderLock lock(*this);
|
||||
currentSpineIndex = sync.spineIndex;
|
||||
nextPageNumber = sync.page;
|
||||
if (sync.hasParagraphIndex) {
|
||||
pendingParagraphLookup = true;
|
||||
pendingParagraphIndex = sync.paragraphIndex;
|
||||
}
|
||||
section.reset();
|
||||
}
|
||||
}
|
||||
@@ -623,6 +636,17 @@ void EpubReaderActivity::render(RenderLock&& lock) {
|
||||
pendingAnchor.clear();
|
||||
}
|
||||
|
||||
// Resolve pending KOReader sync paragraph index to accurate page via Section paragraph LUT
|
||||
if (pendingParagraphLookup) {
|
||||
if (const auto page = section->getPageForParagraphIndex(pendingParagraphIndex)) {
|
||||
section->currentPage = *page;
|
||||
LOG_DBG("ERS", "Resolved p[%u] to page %d (was %d)", pendingParagraphIndex, *page, nextPageNumber);
|
||||
} else {
|
||||
LOG_DBG("ERS", "Paragraph LUT not available, using estimated page %d", nextPageNumber);
|
||||
}
|
||||
pendingParagraphLookup = false;
|
||||
}
|
||||
|
||||
// handles changes in reader settings and reset to approximate position based on cached progress
|
||||
if (cachedChapterTotalPageCount > 0) {
|
||||
// only goes to relative position if spine index matches cached value
|
||||
|
||||
@@ -24,6 +24,9 @@ class EpubReaderActivity final : public Activity {
|
||||
bool pendingPercentJump = false;
|
||||
// Normalized 0.0-1.0 progress within the target spine item, computed from book percentage.
|
||||
float pendingSpineProgress = 0.0f;
|
||||
// Pending paragraph index from KOReader sync (resolved to page via Section paragraph LUT)
|
||||
bool pendingParagraphLookup = false;
|
||||
uint16_t pendingParagraphIndex = 0;
|
||||
bool pendingScreenshot = false;
|
||||
bool skipNextButtonCheck = false; // Skip button processing for one frame after subactivity exit
|
||||
bool automaticPageTurnActive = false;
|
||||
|
||||
@@ -132,11 +132,24 @@ void KOReaderSyncActivity::performSync() {
|
||||
|
||||
// Convert remote progress to CrossPoint position
|
||||
hasRemoteProgress = true;
|
||||
{
|
||||
RenderLock lock(*this);
|
||||
statusMessage = tr(STR_MAPPING_REMOTE);
|
||||
}
|
||||
requestUpdateAndWait();
|
||||
|
||||
KOReaderPosition koPos = {remoteProgress.progress, remoteProgress.percentage};
|
||||
remotePosition = ProgressMapper::toCrossPoint(epub, koPos, currentSpineIndex, totalPagesInSpine);
|
||||
|
||||
// Calculate local progress in KOReader format (for display)
|
||||
CrossPointPosition localPos = {currentSpineIndex, currentPage, totalPagesInSpine};
|
||||
{
|
||||
RenderLock lock(*this);
|
||||
statusMessage = tr(STR_MAPPING_LOCAL);
|
||||
}
|
||||
requestUpdateAndWait();
|
||||
|
||||
CrossPointPosition localPos = {currentSpineIndex, currentPage, totalPagesInSpine, localParagraphIndex,
|
||||
hasLocalParagraphIndex};
|
||||
localProgress = ProgressMapper::toKOReader(epub, localPos);
|
||||
|
||||
{
|
||||
@@ -162,7 +175,8 @@ void KOReaderSyncActivity::performUpload() {
|
||||
requestUpdateAndWait();
|
||||
|
||||
// Convert current position to KOReader format
|
||||
CrossPointPosition localPos = {currentSpineIndex, currentPage, totalPagesInSpine};
|
||||
CrossPointPosition localPos = {currentSpineIndex, currentPage, totalPagesInSpine, localParagraphIndex,
|
||||
hasLocalParagraphIndex};
|
||||
KOReaderPosition koPos = ProgressMapper::toKOReader(epub, localPos);
|
||||
|
||||
KOReaderProgress progress;
|
||||
@@ -361,7 +375,8 @@ void KOReaderSyncActivity::loop() {
|
||||
if (mappedInput.wasReleased(MappedInputManager::Button::Confirm)) {
|
||||
if (selectedOption == 0) {
|
||||
// Wifi will be turned off in onExit()
|
||||
setResult(SyncResult{remotePosition.spineIndex, remotePosition.pageNumber});
|
||||
setResult(SyncResult{remotePosition.spineIndex, remotePosition.pageNumber, remotePosition.paragraphIndex,
|
||||
remotePosition.hasParagraphIndex});
|
||||
finish();
|
||||
} else if (selectedOption == 1) {
|
||||
// Upload local progress
|
||||
|
||||
@@ -22,13 +22,16 @@ class KOReaderSyncActivity final : public Activity {
|
||||
public:
|
||||
explicit KOReaderSyncActivity(GfxRenderer& renderer, MappedInputManager& mappedInput,
|
||||
const std::shared_ptr<Epub>& epub, const std::string& epubPath, int currentSpineIndex,
|
||||
int currentPage, int totalPagesInSpine)
|
||||
int currentPage, int totalPagesInSpine, uint16_t paragraphIndex = 0,
|
||||
bool hasParagraphIndex = false)
|
||||
: Activity("KOReaderSync", renderer, mappedInput),
|
||||
epub(epub),
|
||||
epubPath(epubPath),
|
||||
currentSpineIndex(currentSpineIndex),
|
||||
currentPage(currentPage),
|
||||
totalPagesInSpine(totalPagesInSpine),
|
||||
localParagraphIndex(paragraphIndex),
|
||||
hasLocalParagraphIndex(hasParagraphIndex),
|
||||
remoteProgress{},
|
||||
remotePosition{},
|
||||
localProgress{} {}
|
||||
@@ -57,6 +60,8 @@ class KOReaderSyncActivity final : public Activity {
|
||||
int currentSpineIndex;
|
||||
int currentPage;
|
||||
int totalPagesInSpine;
|
||||
uint16_t localParagraphIndex;
|
||||
bool hasLocalParagraphIndex;
|
||||
|
||||
State state = WIFI_SELECTION;
|
||||
std::string statusMessage;
|
||||
|
||||
Reference in New Issue
Block a user