168 lines
12 KiB
Markdown
168 lines
12 KiB
Markdown
# EPUB TOC Anchor Navigation
|
|
|
|
This document describes how the reader handles EPUB Table of Contents (TOC) entries that use fragment anchors to point into spine files, enabling navigation to sub-chapters within a single XHTML file.
|
|
|
|
## Background: EPUB spine and TOC structure
|
|
|
|
An EPUB's **spine** is an ordered list of XHTML files that define reading order. The **TOC** (table of contents) maps chapter names to positions in the spine, optionally with fragment anchors (e.g. `chapter1.xhtml#section-5`).
|
|
|
|
Two layouts are relevant here:
|
|
|
|
- **1:1** -- one TOC entry per spine item (most common, no anchors needed)
|
|
- **Multi-TOC-per-spine** -- multiple TOC entries point into a single spine file using fragment anchors (e.g. Moby Dick from Project Gutenberg packs 3-9 chapters per file)
|
|
|
|
Spine items before the first TOC entry (cover pages) and after the last (appendices, copyright) have no TOC entry of their own.
|
|
|
|
## BookMetadataCache and TOC-to-spine mapping
|
|
|
|
`BookMetadataCache` builds the mapping between spine items and TOC entries at epub open time. Key details:
|
|
|
|
- Each `SpineEntry` has a `tocIndex` field set during cache building. For spines with no matching TOC entry, `tocIndex` inherits the previous spine's value (`lastSpineTocIndex`). This means orphan spines (cover pages, appendices) are treated as continuations of the nearest preceding chapter.
|
|
- `getTocIndexForSpineIndex(i)` returns the stored `tocIndex` for spine `i` -- a file seek into BookMetadataCache, not computed on the fly.
|
|
- `getTocItem(i)` returns the TOC entry (title, spineIndex, anchor) for TOC index `i` -- also a file seek per call, not cached in memory. Code that queries TOC metadata in a loop should cache the results locally first.
|
|
- `getSpineIndexForTocIndex(i)` does the reverse lookup (TOC index to spine index).
|
|
|
|
### Cached TOC reliability flag
|
|
|
|
`hasReliableToc()` answers whether the TOC has enough spine coverage (>=25% of spines referenced) to drive chapter UX, with short-circuits for `tocCount <= 0` and the "large book with one TOC entry" pathology.
|
|
|
|
The result is computed once during `buildBookBin` (folded into the existing `spineIndex->tocIndex` scan, so no extra disk pass) and persisted as a single byte in book.bin's header A. `Epub::hasReliableToc()` reads `BookMetadataCache::isTocReliable()` and caches the bool in `tocReliabilityState`.
|
|
|
|
This matters because the check used to recompute the answer on demand by calling `getTocEntry(i)` for every TOC entry, which does two SD-card seeks per call. On a 2858-entry web-novel TOC that was ~5700 seeks (~7 seconds) added to first-page latency. `BOOK_CACHE_VERSION` was bumped to 7 for this layout change; older caches are rebuilt on next open.
|
|
|
|
## Section cache file format
|
|
|
|
The section cache (`.bin`) stores pre-rendered page data for a spine item. The file layout:
|
|
|
|
```
|
|
[header: version, render parameters, pageCount, lutOffset, anchorMapOffset]
|
|
[serialized pages...]
|
|
[page LUT: array of uint32_t file offsets, one per page]
|
|
[anchor map: uint16_t count, then (string, uint16_t) pairs]
|
|
```
|
|
|
|
The header size is defined by `HEADER_SIZE` (a constexpr computed via `sizeof` sum) and validated with a `static_assert`. Three functions read this header independently and must stay in sync:
|
|
|
|
- `loadSectionFile` -- full section load, reads header + builds TOC boundaries from anchor map
|
|
- `getPageForAnchor` -- seeks directly to anchor map offset from header
|
|
- `writeSectionFileHeader` -- writes the header during cache creation
|
|
|
|
When modifying the header layout, bump `SECTION_FILE_VERSION` to invalidate stale caches and update all read paths.
|
|
|
|
## Anchor-to-page mapping
|
|
|
|
### Recording anchors during parsing
|
|
|
|
`ChapterHtmlSlimParser` records every HTML `id` attribute and its corresponding page number into `anchorData` (a flat `std::vector<std::pair<std::string, uint16_t>>`). Recording is deferred via `pendingAnchorId` until `startNewTextBlock()`, after the previous text block is flushed to pages via `makePages()`. This ensures `completedPageCount` reflects the correct page.
|
|
|
|
For TOC anchors specifically, `startNewTextBlock` also forces a page break before recording, so chapters start on fresh pages rather than mid-page. The parser receives the set of TOC anchor strings via `tocAnchors` (a `std::vector<std::string>`) from `Section::createSectionFile`.
|
|
|
|
### On-disk format
|
|
|
|
The anchor data is serialized at the end of the section cache file (`.bin`), after the page LUT. The header stores the anchor map offset. Format:
|
|
|
|
```
|
|
[uint16_t count]
|
|
[string anchor_1][uint16_t page_1]
|
|
[string anchor_2][uint16_t page_2]
|
|
...
|
|
```
|
|
|
|
This data serves two purposes:
|
|
- **Footnote navigation** (`getPageForAnchor`): on-demand linear scan for a single anchor
|
|
- **TOC boundary resolution** (`buildTocBoundariesFromFile`): scan matching only TOC anchors
|
|
|
|
### Data structure choices
|
|
|
|
All anchor storage uses flat vectors, not `std::map` or `std::set`. On the ESP32-C3, each `std::map`/`std::set` node requires its own heap allocation, causing fragmentation. Vectors use a single contiguous allocation. The entry counts are small enough (typically 1-10 TOC anchors per spine, dozens to hundreds of total anchors) that linear scans are faster than tree lookups at these sizes.
|
|
|
|
## TOC boundaries in Section
|
|
|
|
When a section is loaded or created, `Section` builds an in-memory `tocBoundaries` vector mapping each TOC entry in that spine to its start page. This is a small vector (1-3 entries typically) that enables O(1) lookups without file I/O.
|
|
|
|
### Two build paths
|
|
|
|
**From in-memory anchors** (`buildTocBoundaries`): Called after `createSectionFile` when the parser's anchor vector is still in memory. Iterates TOC entries and does linear scans against the anchor vector.
|
|
|
|
**From disk** (`buildTocBoundariesFromFile`): Called from `loadSectionFile` when loading a cached section. Caches the small set of TOC anchor strings first (since `getTocItem()` does file I/O to `BookMetadataCache`), then streams through on-disk anchors matching only those, stopping early once all are resolved. Uses a reusable `std::string` buffer to avoid per-entry heap allocation.
|
|
|
|
The two functions are kept separate because their iteration patterns differ fundamentally: in-memory iterates TOC entries with inner scans of anchors, while the disk path iterates disk entries with inner scans of the small TOC anchor set.
|
|
|
|
### Early exit optimization
|
|
|
|
If no TOC entries in the spine have anchors (`unresolvedCount == 0`), both functions return immediately without storing any boundaries. `getTocIndexForPage` falls back to `epub->getTocIndexForSpineIndex`, which gives the correct answer for the common 1:1 case.
|
|
|
|
### Query methods
|
|
|
|
- `getTocIndexForPage(page)` -- binary search on sorted `tocBoundaries` to find which chapter a page belongs to
|
|
- `getPageForTocIndex(tocIndex)` -- linear scan to find a chapter's start page
|
|
- `getPageRangeForTocIndex(tocIndex)` -- returns `[startPage, endPage)` range for a chapter within this spine
|
|
|
|
All are in-memory, no file I/O.
|
|
|
|
## Chapter navigation in EpubReaderActivity
|
|
|
|
### Chapter skip (long-press)
|
|
|
|
Navigates by TOC index, not spine index. Uses `getTocIndexForPage` to determine the current chapter, then increments or decrements.
|
|
|
|
- **Same-spine skip**: Resolves the target page via `getPageForTocIndex` entirely in memory
|
|
- **Cross-spine skip**: Sets `pendingTocIndex` (a `std::optional<int>`) which is resolved after the target section loads in `render()`
|
|
- **Forward past last TOC entry**: Jumps to end-of-book (spine index clamped in `render()`)
|
|
- **Backward before first TOC entry**: Jumps to the spine before the current chapter's first spine (clamped to 0 in `render()`)
|
|
- **No TOC entry for spine** (`curTocIndex < 0`): Falls back to spine-level skip
|
|
|
|
### Chapter selector
|
|
|
|
The chapter selection activity receives `currentTocIndex` (per-page, not per-spine) so it highlights the correct sub-chapter. Returns `ChapterResult` with both `spineIndex` and `std::optional<int> tocIndex`. The reader resolves the page via `getPageForTocIndex` for same-spine navigation or defers via `pendingTocIndex` for cross-spine.
|
|
|
|
### Footnote navigation
|
|
|
|
Uses the existing `pendingAnchor` mechanism from the footnote anchor navigation commit (4d222567). `getPageForAnchor` does an on-demand linear scan of the on-disk anchor data. This is separate from TOC boundaries -- it reads all anchors (not just TOC ones) and is only called for footnote jumps.
|
|
|
|
### Status bar
|
|
|
|
Uses `getTocIndexForPage()` for the chapter title, so the status bar shows the correct sub-chapter name when reading a multi-TOC-per-spine file.
|
|
|
|
## Orphan spine handling
|
|
|
|
Spine items without a TOC entry inherit the previous spine's `tocIndex` in `BookMetadataCache`. This means:
|
|
|
|
- Pre-TOC spines (cover pages) may have `tocIndex == -1` if they're before any chapter
|
|
- Post-TOC spines (appendices, copyright) inherit the last chapter's `tocIndex`
|
|
|
|
The chapter skip logic guards against `curTocIndex < 0` and falls back to spine-level navigation.
|
|
|
|
## Implementation pitfalls and edge cases
|
|
|
|
### Anchor recording timing
|
|
|
|
The `pendingAnchorId` deferred recording pattern is critical for correctness. Anchors must be recorded *after* `makePages()` flushes the previous text block (so `completedPageCount` reflects the right page) but the TOC page break must happen *before* recording (so the anchor lands on the new page). Both of these happen inside `startNewTextBlock()`. An earlier design used a `recordAnchor` lambda called at various points in `startElement()`, but this had wrong timing for headings and block elements -- `startNewTextBlock` would consume `pendingAnchorId` before `recordAnchor` could force the page break. Moving all page-break logic into `startNewTextBlock` fixed this.
|
|
|
|
### pendingAnchorId overwrite on consecutive elements
|
|
|
|
If two elements with `id` attributes appear before any `startNewTextBlock` call (e.g. nested divs), the second `id` overwrites `pendingAnchorId` and the first anchor is never recorded. This is a known limitation inherited from the footnote anchor navigation commit (4d222567) on master. In practice, TOC anchors are on chapter headings which trigger `startNewTextBlock`, so this doesn't affect TOC navigation.
|
|
|
|
### wordsExtractedInBlock reset on empty block reuse
|
|
|
|
When `startNewTextBlock` reuses an empty text block (the early-return path), `wordsExtractedInBlock` must be reset to 0. Without this, footnotes in the reused block could be assigned to wrong pages based on stale word counts from a prior block.
|
|
|
|
### getTocItem() does file I/O
|
|
|
|
`epub->getTocItem()` reads from `BookMetadataCache` via file seek on every call. This is why `buildTocBoundariesFromFile` caches the TOC anchor strings into a small vector before entering the disk scan loop -- otherwise the inner loop would do file I/O (BookMetadataCache) for every on-disk anchor entry.
|
|
|
|
### Defensive sort on tocBoundaries
|
|
|
|
`tocBoundaries` is sorted by `startPage` after building. In well-formed EPUBs, entries are already in order (TOC follows document order). The sort is a safety net for malformed EPUBs where TOC entries might be out of document order. With 1-3 entries it has no measurable cost.
|
|
|
|
## Test epub
|
|
|
|
`scripts/generate_spine_toc_edges_epub.py` generates `test/epubs/test_spine_toc_edges.epub`, a purpose-built epub that exercises spine/TOC relationship patterns. See the script header for the full list of edge cases covered.
|
|
|
|
## Performance characteristics
|
|
|
|
- **Per page turn**: All in-memory. `getTocIndexForPage` (binary search on 1-3 entries), `getTocItem` for title (one file seek to BookMetadataCache -- noted as a future optimization opportunity).
|
|
- **Section load**: One file open for the section cache. `buildTocBoundariesFromFile` scans the anchor map for a few TOC entries with early exit.
|
|
- **Footnote navigation**: One additional file open to scan the anchor map for a single anchor.
|
|
- **1:1 TOC-to-spine (common case)**: No overhead. `unresolvedCount == 0`, `tocBoundaries` stays empty, all queries fall back to spine-level methods.
|