diff --git a/README.md b/README.md index ddf70792..0287645d 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ CrossPoint is open-source e-reader firmware - community-built, fully hackable, f ## What can CrossPoint do? -- **Reader engine**: EPUB 2/3 rendering with embedded-style option, image handling, hyphenation, kerning, chapter navigation, footnotes, bookmarks, go-to-percent, auto page turn, orientation control, focus reading, KOReader progress sync and more. +- **Reader engine**: EPUB 2/3 rendering with embedded-style option, image handling, hyphenation, kerning, chapter navigation, footnotes, bookmarks, dictionary lookups ([StarDict](docs/dictionary.md)), go-to-percent, auto page turn, orientation control, focus reading, KOReader progress sync and more. - **Various formats**: native handling for `.epub`, `.xtc/.xtch`, `.txt`, and `.bmp`. @@ -42,8 +42,6 @@ CrossPoint is open-source e-reader firmware - community-built, fully hackable, f ### Coming soon: -- Dictionary lookup — inline word lookup without leaving the reader. - - More themes. - Much more! stay tuned. diff --git a/USER_GUIDE.md b/USER_GUIDE.md index a67ee999..1b8dc531 100644 --- a/USER_GUIDE.md +++ b/USER_GUIDE.md @@ -263,6 +263,8 @@ The Settings screen allows you to configure the device's behavior. There are a f - "ON" - Vertical space will be added between paragraphs in Reading Mode - "OFF" - Paragraphs will not have vertical space added, but will have first-line indentation +- **Dictionary**: Select the StarDict dictionary used for word lookups while reading, or "None" to disable lookups. *(Only shown when at least one dictionary folder exists under `/dictionaries/` on the SD card — see [docs/dictionary.md](docs/dictionary.md) for setup and usage.)* + - **Text Anti-Aliasing**: Whether to show smooth grey edges (anti-aliasing) on text in reading mode. Note this slows down page turns slightly. - **Images**: Whether to display embedded images (JPG/PNG) found in EPUB files; options are "ON" (default) or "OFF". @@ -282,6 +284,7 @@ The Settings screen allows you to configure the device's behavior. There are a f - **Long-press Menu**: Selects the function bound to holding the menu button (Confirm) while reading an EPUB. **Cycles through the available functions** each time the setting is selected — additional functions may be added in future releases, so this is not a binary on/off toggle. A short press of Confirm always opens the reader menu as normal: - "Bookmark" (default) - Hold Confirm (~0.4 second) to drop a bookmark at the current page. - "KOSync" - Hold Confirm (~1 second) to launch KOReader sync directly. + - "Dictionary" - Hold Confirm (~0.4 second) to start dictionary word selection on the current page (see [docs/dictionary.md](docs/dictionary.md)). - "Disabled" - Long-press is ignored; only short-press opens the reader menu. - **Short Power Button Click**: Controls the effect of a short click of the power button: @@ -583,7 +586,7 @@ If the device goes to sleep or you close the book while viewing a footnote, the * **Return to Home:** Press the **Back** button to close the book and return to the **[Home](#31-home-screen)** screen. * **Return to Browse Files:** Press and hold the **Back** button to close the book and return to the **[Browse Files](#33-browse-files-screen)** screen. * **Reader Menu:** Press **Confirm** to open the **[Reader Menu](#5-reader-menu)**, which includes chapter navigation, reading options, and more. -* **Long-press Confirm (configurable):** Holding **Confirm** runs the function chosen by the **Long-press Menu** setting in **[Controls Settings](#363-controls)** — "Bookmark" (default) drops a bookmark, "KOSync" launches KOReader Sync, "Disabled" does nothing. A short press always opens the Reader Menu. +* **Long-press Confirm (configurable):** Holding **Confirm** runs the function chosen by the **Long-press Menu** setting in **[Controls Settings](#363-controls)** — "Bookmark" (default) drops a bookmark, "KOSync" launches KOReader Sync, "Dictionary" starts a word lookup, "Disabled" does nothing. A short press always opens the Reader Menu. ### Supported Languages @@ -605,6 +608,7 @@ Available options include: - **Select Chapter** – Open the table of contents to jump to a specific chapter (see [Chapter Selection](#51-chapter-selection) below). - **Footnotes** – Navigate to the footnotes for the current section *(only shown in books that contain footnotes)*. +- **Look Up** – Select a word on the current page and show its dictionary definition (see [docs/dictionary.md](docs/dictionary.md)). Requires a dictionary to be selected in **Settings → Reader → Dictionary**. - **Reading Orientation** – Cycle through screen orientations without leaving the reader. - **Auto Turn (Pages Per Minute)** – Cycle through automatic page turn speed options for hands-free reading. - **Go to %** – Jump to a specific position in the book by percentage. diff --git a/docs/dictionary.md b/docs/dictionary.md new file mode 100644 index 00000000..c32a763f --- /dev/null +++ b/docs/dictionary.md @@ -0,0 +1,51 @@ +# Dictionary + +Look up words while reading an EPUB using an offline StarDict dictionary stored on the SD card. + +## Supported Format + +The reader supports **StarDict** dictionaries. When searching for dictionaries online, look for "StarDict format" or files with `.dict`, `.idx`, and `.ifo` extensions. + +A dictionary folder must contain: + +- `.idx` — word index (required, **must be uncompressed** — a `.idx.gz` will not work; decompress it on your computer with `gzip -d` first) +- `.dict` or `.dict.dz` — definition data (`.dict.dz` is supported as-is; entries are decompressed on the fly during lookup) +- `.ifo` — metadata (optional) + +Not supported: `.syn` synonym files (ignored), dictionaries with 64-bit index offsets (`idxoffsetbits=64` in the `.ifo` — rare, and rejected with an error), and HTML-formatted definitions render as raw markup rather than styled text. + +## Setting Up a Dictionary + +1. Copy your dictionary folder(s) to `/dictionaries/` on the SD card — one dictionary per folder, e.g. `/dictionaries/webster/webster.idx` + `webster.dict.dz`. A hidden `/.dictionaries/` folder (dot-prefixed) works the same way, for keeping it out of the file browser. +2. Open **Settings → Reader → Dictionary** on the device. +3. Select a dictionary from the list, or **None** to disable lookups. + +The Dictionary setting only appears when at least one usable dictionary folder exists. Folders containing more than one dictionary (multiple `.idx` stems) are skipped as ambiguous. + +## Looking Up a Word + +Two ways to start a lookup while reading: + +- Open the reader menu (**Confirm**) and choose **Look Up**. +- Or set **Settings → Controls → Long-press Menu** to "Dictionary", then hold **Confirm** (~0.4s) on the reading page. + +One word on the page becomes highlighted: + +1. Use **Left/Right** to move between words in reading order, and the side **Up/Down** buttons to jump between lines. +2. Press **Confirm** to look up the highlighted word. +3. Press **Back** to return to the reader. + +On the very first lookup with a dictionary (and again if the dictionary file changes), the reader shows *"Indexing dictionary…"* while it builds a small `.qidx` sidecar file next to the `.idx`. This takes a few seconds for large dictionaries and makes all subsequent lookups fast. The sidecar can be deleted safely at any time — it will simply be rebuilt. + +### How Lookup Works + +1. **Direct match** — the word is found as-is (case-insensitive) in the dictionary index. Surrounding punctuation is ignored. +2. **Stemming** — on a miss, common English word forms are retried automatically: possessives and plurals (`dogs` → `dog`, `stories` → `story`) and verb endings (`walked` → `walk`, `running` → `run`, `making` → `make`). +3. **Not found** — a short popup appears and you return to word selection. + +## The Definition Screen + +When a word is found, the definition screen shows the matched headword at the top and the definition text below, with a page counter for long definitions. + +- **Left/Right** or side **Up/Down** — previous / next page +- **Back** — return to word selection diff --git a/lib/GfxRenderer/GfxRenderer.cpp b/lib/GfxRenderer/GfxRenderer.cpp index e63ad51a..be580b10 100644 --- a/lib/GfxRenderer/GfxRenderer.cpp +++ b/lib/GfxRenderer/GfxRenderer.cpp @@ -213,6 +213,61 @@ static inline void rotateCoordinates(const GfxRenderer::Orientation orientation, } } +// Output of screenRectToAlignedMemRect: a rectangle in panel-memory +// coordinates whose x and width are guaranteed to be multiples of 8 (the +// SDK's EInkDisplay::displayWindow alignment requirement). `valid == false` +// means the input was empty or fully outside the panel. +struct AlignedMemRect { + uint16_t x = 0; + uint16_t y = 0; + uint16_t w = 0; + uint16_t h = 0; + bool valid = false; +}; + +// Translate a screen-coordinate rectangle (the coordinate system used by +// fillRect / drawText / the rest of the renderer's public API) into a +// panel-memory rectangle suitable for direct framebuffer indexing. Rotates +// the rectangle's two opposite corners with rotateCoordinates(), takes the +// bounding box (which naturally swaps width/height in Portrait / +// PortraitInverted), then snaps the x extent outward to multiples of 8 and +// clamps to panel bounds. Precondition: panel dims are multiples of 8 (true +// for the 800x480 panel), so clamping cannot re-break alignment. +static AlignedMemRect screenRectToAlignedMemRect(GfxRenderer::Orientation orientation, int sx, int sy, int sw, int sh, + uint16_t panelWidth, uint16_t panelHeight) { + AlignedMemRect out; + if (sw <= 0 || sh <= 0) return out; + + int x0, y0, x1, y1; + rotateCoordinates(orientation, sx, sy, &x0, &y0, panelWidth, panelHeight); + rotateCoordinates(orientation, sx + sw - 1, sy + sh - 1, &x1, &y1, panelWidth, panelHeight); + + const int memXLo = std::min(x0, x1); + const int memYLo = std::min(y0, y1); + const int memXHi = std::max(x0, x1) + 1; // exclusive upper bound + const int memYHi = std::max(y0, y1) + 1; + + // Snap x outward to multiples of 8. + int alignedXLo = memXLo & ~0x7; // round down + int alignedXHi = (memXHi + 7) & ~0x7; // round up + + if (alignedXLo < 0) alignedXLo = 0; + if (alignedXHi > panelWidth) alignedXHi = panelWidth; + int clampedYLo = memYLo; + int clampedYHi = memYHi; + if (clampedYLo < 0) clampedYLo = 0; + if (clampedYHi > panelHeight) clampedYHi = panelHeight; + + if (alignedXHi <= alignedXLo || clampedYHi <= clampedYLo) return out; + + out.x = static_cast(alignedXLo); + out.y = static_cast(clampedYLo); + out.w = static_cast(alignedXHi - alignedXLo); + out.h = static_cast(clampedYHi - clampedYLo); + out.valid = true; + return out; +} + enum class TextRotation { None, Rotated90CW }; // Shared glyph rendering logic for normal and rotated text. @@ -1451,6 +1506,39 @@ void GfxRenderer::displayBuffer(const HalDisplay::RefreshMode refreshMode) const display.displayBuffer(refreshMode, fadingFix); } +size_t GfxRenderer::readFramebufferRegion(int x, int y, int w, int h, uint8_t* dst, size_t dstCapacity) const { + if (dst == nullptr || w <= 0 || h <= 0) return 0; + + const AlignedMemRect mem = screenRectToAlignedMemRect(orientation, x, y, w, h, panelWidth, panelHeight); + if (!mem.valid) return 0; + + const size_t rowBytes = mem.w / 8; // exact: mem.w is a multiple of 8 + const size_t needed = rowBytes * mem.h; + if (needed > dstCapacity) return 0; + + for (uint16_t row = 0; row < mem.h; ++row) { + const uint8_t* srcRow = frameBuffer + (static_cast(mem.y + row) * panelWidthBytes) + (mem.x / 8); + uint8_t* dstRow = dst + (static_cast(row) * rowBytes); + memcpy(dstRow, srcRow, rowBytes); + } + return needed; +} + +void GfxRenderer::writeFramebufferRegion(int x, int y, int w, int h, const uint8_t* src) { + if (src == nullptr || w <= 0 || h <= 0) return; + + const AlignedMemRect mem = screenRectToAlignedMemRect(orientation, x, y, w, h, panelWidth, panelHeight); + if (!mem.valid) return; + + const size_t rowBytes = mem.w / 8; // exact: mem.w is a multiple of 8 + + for (uint16_t row = 0; row < mem.h; ++row) { + const uint8_t* srcRow = src + (static_cast(row) * rowBytes); + uint8_t* dstRow = frameBuffer + (static_cast(mem.y + row) * panelWidthBytes) + (mem.x / 8); + memcpy(dstRow, srcRow, rowBytes); + } +} + std::string GfxRenderer::truncatedText(const int fontId, const char* text, const int maxWidth, const EpdFontFamily::Style style) const { if (!text || maxWidth <= 0) return ""; diff --git a/lib/GfxRenderer/GfxRenderer.h b/lib/GfxRenderer/GfxRenderer.h index e5829679..d4158910 100644 --- a/lib/GfxRenderer/GfxRenderer.h +++ b/lib/GfxRenderer/GfxRenderer.h @@ -189,6 +189,15 @@ class GfxRenderer { void drawBitmap1Bit(const Bitmap& bitmap, int x, int y, int maxWidth, int maxHeight) const; void fillPolygon(const int* xPoints, const int* yPoints, int numPoints, bool state = true) const; + // Snapshot / restore a screen-coordinate framebuffer region (byte-aligned in + // panel memory). readFramebufferRegion returns the bytes written to dst, or + // 0 when the region is empty, offscreen, or exceeds dstCapacity. Pass the + // same rectangle to writeFramebufferRegion to restore the saved pixels. + // Enables partial-repaint patterns (e.g. moving a selection highlight) + // without re-rendering the whole page. + size_t readFramebufferRegion(int x, int y, int w, int h, uint8_t* dst, size_t dstCapacity) const; + void writeFramebufferRegion(int x, int y, int w, int h, const uint8_t* src); + // Text int getTextWidth(int fontId, const char* text, EpdFontFamily::Style style = EpdFontFamily::REGULAR, BidiUtils::BidiBaseDir baseDir = BidiUtils::BidiBaseDir::AUTO) const; diff --git a/lib/I18n/translations/english.yaml b/lib/I18n/translations/english.yaml index e5900ddc..88e8feda 100644 --- a/lib/I18n/translations/english.yaml +++ b/lib/I18n/translations/english.yaml @@ -26,6 +26,12 @@ STR_EMPTY_FILE: "Empty file" STR_OUT_OF_BOUNDS: "Out of bounds" STR_LOADING: "Loading..." STR_LOADING_POPUP: "Loading" +STR_LOOKUP: "Look Up" +STR_DICT_LOOKING_UP: "Looking up..." +STR_DICT_INDEXING: "Indexing dictionary..." +STR_DICT_NOT_FOUND: "Not found" +STR_DICT_NO_DICT_SET: "No dictionary set" +STR_DICT_ERROR: "Dictionary error" STR_WIFI_NETWORKS: "Wi-Fi Networks" STR_NO_NETWORKS: "No networks found" STR_NETWORKS_FOUND: "%zu networks found" @@ -163,6 +169,7 @@ STR_PREV_NEXT: "Prev/Next" STR_NEXT_PREV: "Next/Prev" STR_KOSYNC: "KOSync" STR_BOOKMARK_OPTION: "Bookmark" +STR_DICTIONARY: "Dictionary" STR_DISABLED: "Disabled" STR_NOTO_SERIF: "Noto Serif" STR_NOTO_SANS: "Noto Sans" diff --git a/src/CrossPointSettings.h b/src/CrossPointSettings.h index b6580a64..6f2f71e3 100644 --- a/src/CrossPointSettings.h +++ b/src/CrossPointSettings.h @@ -153,6 +153,7 @@ class CrossPointSettings { LP_MENU_KOSYNC = 0, LP_MENU_DISABLED = 1, LP_MENU_BOOKMARK = 2, + LP_MENU_DICTIONARY = 3, LONG_PRESS_MENU_FUNCTION_COUNT }; @@ -265,6 +266,8 @@ class CrossPointSettings { uint8_t focusReadingEnabled = 0; // SD card font family name (empty = use built-in fontFamily) char sdFontFamilyName[32] = ""; + // Dictionary folder name under /dictionaries (empty = no dictionary) + char dictionaryName[32] = ""; // Show hidden files/directories (starting with '.') in the file browser (0 = hidden, 1 = show) uint8_t showHiddenFiles = 0; // Remove a book from the Recent Books list when its End-of-Book screen is reached (0 = off, 1 = on) diff --git a/src/JsonSettingsIO.cpp b/src/JsonSettingsIO.cpp index 21701efd..57ad5f22 100644 --- a/src/JsonSettingsIO.cpp +++ b/src/JsonSettingsIO.cpp @@ -149,6 +149,10 @@ bool JsonSettingsIO::saveSettings(const CrossPointSettings& s, const char* path) if (s.sdFontFamilyName[0] != '\0') { doc["sdFontFamilyName"] = s.sdFontFamilyName; } + // Dictionary folder name — uses dynamic getter/setter in SettingsList, save manually + if (s.dictionaryName[0] != '\0') { + doc["dictionaryName"] = s.dictionaryName; + } // Language -- managed by LanguageSelectActivity, not in SettingsList. // Stored as ISO code string ("EN", "DE", ...) for stability across enum reorders. @@ -256,6 +260,11 @@ bool JsonSettingsIO::loadSettings(CrossPointSettings& s, const char* json, bool* if (needsResave) *needsResave = true; } + // Dictionary folder name — uses dynamic getter/setter in SettingsList, load manually + const char* dictName = doc["dictionaryName"] | ""; + strncpy(s.dictionaryName, dictName, sizeof(s.dictionaryName) - 1); + s.dictionaryName[sizeof(s.dictionaryName) - 1] = '\0'; + // Language -- stored as code string for stability across enum reorders. if (doc["language"].is()) { s.language = static_cast(I18n::languageFromCode(doc["language"].as())); diff --git a/src/SettingsList.h b/src/SettingsList.h index e7e97ad4..f57b8e2a 100644 --- a/src/SettingsList.h +++ b/src/SettingsList.h @@ -13,6 +13,7 @@ #include "CrossPointSettings.h" #include "KOReaderCredentialStore.h" #include "activities/settings/SettingsActivity.h" +#include "util/DictionaryRegistry.h" // Build the font family setting dynamically. When registry is non-null, SD card fonts // are appended after the built-in fonts. Otherwise only built-in fonts are listed. @@ -90,6 +91,47 @@ inline SettingInfo buildFontFamilySetting(const SdCardFontRegistry* registry) { return s; } +// Build the dictionary selection setting dynamically from the folders discovered +// under /dictionaries. "None" plus one option per dictionary; the selected folder +// name persists in SETTINGS.dictionaryName (saved/loaded manually in +// JsonSettingsIO — the generic loop skips dynamic entries). +inline SettingInfo buildDictionarySetting(const std::vector& dictionaries) { + std::vector folderNames; + folderNames.reserve(dictionaries.size()); + std::transform(dictionaries.begin(), dictionaries.end(), std::back_inserter(folderNames), + [](const DictionaryEntry& d) { return d.name; }); + + SettingInfo s; + s.nameId = StrId::STR_DICTIONARY; + s.type = SettingType::ENUM; + s.enumStringValues.reserve(folderNames.size() + 1); + s.enumStringValues.push_back(I18N.get(StrId::STR_NONE_OPT)); + s.enumStringValues.insert(s.enumStringValues.end(), folderNames.begin(), folderNames.end()); + s.category = StrId::STR_CAT_READER; + + s.valueGetter = [folderNames]() -> uint8_t { + for (size_t i = 0; i < folderNames.size(); i++) { + // Compare within the settings field capacity: an over-long folder name is + // stored truncated, and must still match its list entry. + if (strncmp(folderNames[i].c_str(), SETTINGS.dictionaryName, sizeof(SETTINGS.dictionaryName) - 1) == 0) { + return static_cast(i + 1); + } + } + return 0; // "None", also when the stored folder no longer exists + }; + + s.valueSetter = [folderNames](uint8_t v) { + if (v == 0 || v > folderNames.size()) { + SETTINGS.dictionaryName[0] = '\0'; + return; + } + strncpy(SETTINGS.dictionaryName, folderNames[v - 1].c_str(), sizeof(SETTINGS.dictionaryName) - 1); + SETTINGS.dictionaryName[sizeof(SETTINGS.dictionaryName) - 1] = '\0'; + }; + + return s; +} + // Shared settings list used by both the device settings UI and the web settings API. // Each entry has a key (for JSON API) and category (for grouping). // ACTION-type entries and entries without a key are device-only. @@ -99,7 +141,8 @@ inline SettingInfo buildFontFamilySetting(const SdCardFontRegistry* registry) { // SdCardFontRegistry is supplied AND has SD card fonts installed, the // font-family entry is replaced in a per-call copy with a registry-aware // version. Callers without SD fonts pay only a vector copy. -inline std::vector getSettingsList(const SdCardFontRegistry* registry = nullptr) { +inline std::vector getSettingsList(const SdCardFontRegistry* registry = nullptr, + const std::vector* dictionaries = nullptr) { static const std::vector baseList = [] { std::vector v = { // --- Display --- @@ -173,8 +216,8 @@ inline std::vector getSettingsList(const SdCardFontRegistry* regist StrId::STR_LONG_PRESS_BEHAVIOR_ORIENTATION}, "longPressButtonBehavior", StrId::STR_CAT_CONTROLS), SettingInfo::Enum(StrId::STR_LONG_PRESS_MENU, &CrossPointSettings::longPressMenuFunction, - {StrId::STR_KOSYNC, StrId::STR_DISABLED, StrId::STR_BOOKMARK_OPTION}, "longPressMenuFunction", - StrId::STR_CAT_CONTROLS), + {StrId::STR_KOSYNC, StrId::STR_DISABLED, StrId::STR_BOOKMARK_OPTION, StrId::STR_DICTIONARY}, + "longPressMenuFunction", StrId::STR_CAT_CONTROLS), SettingInfo::Enum( StrId::STR_SHORT_PWR_BTN, &CrossPointSettings::shortPwrBtn, {StrId::STR_IGNORE, StrId::STR_SLEEP, StrId::STR_PAGE_TURN, StrId::STR_FORCE_REFRESH, StrId::STR_FOOTNOTES}, @@ -308,5 +351,11 @@ inline std::vector getSettingsList(const SdCardFontRegistry* regist *it = buildFontFamilySetting(registry); } } + if (dictionaries && !dictionaries->empty()) { + // Insert at the end of the Reader category (just before the first Controls entry). + auto it = + std::find_if(v.begin(), v.end(), [](const SettingInfo& s) { return s.category == StrId::STR_CAT_CONTROLS; }); + v.insert(it, buildDictionarySetting(*dictionaries)); + } return v; } diff --git a/src/activities/reader/DictionaryDefinitionActivity.cpp b/src/activities/reader/DictionaryDefinitionActivity.cpp new file mode 100644 index 00000000..a9f65ebf --- /dev/null +++ b/src/activities/reader/DictionaryDefinitionActivity.cpp @@ -0,0 +1,237 @@ +#include "DictionaryDefinitionActivity.h" + +#include +#include +#include + +#include +#include +#include + +#include "CrossPointSettings.h" +#include "components/UITheme.h" +#include "fontIds.h" +#include "util/HtmlToPlainText.h" + +namespace { + +// Longest measurable/drawable span. Wrapped lines stay under the screen width +// (far below this); only pathological unbreakable tokens are split at this cap. +constexpr size_t MAX_LINE_BYTES = 191; + +// Body text left/right inset, matching the reader's default feel. +constexpr int SIDE_PADDING = 20; + +} // namespace + +void DictionaryDefinitionActivity::onEnter() { + Activity::onEnter(); + // Normalize StarDict multi-type separators so the wrap loop and the + // C-string font APIs below both see the whole definition. + std::replace(definition.begin(), definition.end(), '\0', '\n'); + definition = htmlToPlainText(definition); + wrapText(); + requestUpdate(); +} + +int DictionaryDefinitionActivity::measureSpan(const int fontId, const char* text, size_t len) const { + char buf[MAX_LINE_BYTES + 1]; + len = std::min(len, MAX_LINE_BYTES); + memcpy(buf, text, len); + buf[len] = '\0'; + return renderer.getTextAdvanceX(fontId, buf, EpdFontFamily::REGULAR); +} + +// Greedy word-wrap of `definition` into byte spans. '\n' breaks lines (blank +// lines survive as paragraph spacing; NULs from multi-type StarDict entries +// were normalized to newlines in onEnter); '\r' is dropped by treating it as +// a space at a token edge. +void DictionaryDefinitionActivity::wrapText() { + lines.clear(); + lines.reserve(definition.size() / 32 + 8); + + const int fontId = SETTINGS.getReaderFontId(); + // SD-card fonts: merge every definition codepoint into the persistent + // advance table up front. Otherwise each unseen codepoint measured below + // falls back to an on-demand glyph load from SD (8-slot overflow ring). + renderer.ensureSdCardFontReady(fontId, definition.c_str(), 0x01 /* REGULAR */); + + const auto& metrics = UITheme::getInstance().getMetrics(); + const auto orientation = renderer.getOrientation(); + const bool isLandscape = orientation == GfxRenderer::Orientation::LandscapeClockwise || + orientation == GfxRenderer::Orientation::LandscapeCounterClockwise; + const bool isInverted = orientation == GfxRenderer::Orientation::PortraitInverted; + const int hintGutterWidth = isLandscape ? metrics.sideButtonHintsWidth : 0; + const int maxWidth = renderer.getScreenWidth() - hintGutterWidth - 2 * SIDE_PADDING; + const int spaceWidth = renderer.getSpaceWidth(fontId, EpdFontFamily::REGULAR); + + const int lineHeight = renderer.getLineHeight(fontId); + const int topArea = (isInverted ? metrics.buttonHintsHeight : 0) + metrics.topPadding + metrics.headerHeight; + const int bottomArea = metrics.buttonHintsHeight + metrics.verticalSpacing; + linesPerPage = std::max(1, (renderer.getScreenHeight() - topArea - bottomArea) / lineHeight); + + const char* text = definition.c_str(); + const uint32_t n = static_cast(definition.size()); + uint32_t lineStart = 0; + uint32_t lineEnd = 0; // one past the last token byte on the current line + int lineWidth = 0; + + const auto flushLine = [&](uint32_t nextStart) { + lines.push_back({lineStart, static_cast(lineEnd - lineStart)}); + lineStart = nextStart; + lineEnd = nextStart; + lineWidth = 0; + }; + + uint32_t i = 0; + while (i < n) { + const char c = text[i]; + if (c == '\n' || c == '\0') { + flushLine(i + 1); + i++; + continue; + } + if (c == ' ' || c == '\t' || c == '\r') { + i++; + continue; + } + + // Token: run of non-whitespace bytes, capped at the measure buffer. + const uint32_t tokenStart = i; + while (i < n && text[i] != ' ' && text[i] != '\t' && text[i] != '\r' && text[i] != '\n' && text[i] != '\0' && + i - tokenStart < MAX_LINE_BYTES) { + i++; + } + // If the byte cap cut the token mid-UTF-8-sequence, back off to the last + // complete codepoint so measure/draw never see a partial sequence. A + // natural stop lands on whitespace or the terminating NUL, never on a + // continuation byte, so this is a no-op there. + while (i - tokenStart > 1 && (text[i] & 0xC0) == 0x80) i--; + const uint32_t tokenLen = i - tokenStart; + const int tokenWidth = measureSpan(fontId, text + tokenStart, tokenLen); + + if (lineEnd == lineStart) { + lineStart = tokenStart; + lineEnd = tokenStart + tokenLen; + lineWidth = tokenWidth; + } else if (lineWidth + spaceWidth + tokenWidth <= maxWidth && + tokenStart + tokenLen - lineStart <= UINT16_MAX) { // span len must fit Line::len + lineEnd = tokenStart + tokenLen; + lineWidth += spaceWidth + tokenWidth; + } else { + flushLine(tokenStart); + lineEnd = tokenStart + tokenLen; + lineWidth = tokenWidth; + } + + // An unbreakable token wider than the screen is now alone on the line + // (any previous content was flushed above): split it at the widest + // fitting UTF-8 boundary and carry the remainder forward. + while (lineWidth > maxWidth && lineEnd - lineStart > 1) { + const uint32_t len = lineEnd - lineStart; + uint32_t lastFit = 0; + for (uint32_t f = 1; f <= len; f++) { + if (f == len || (text[lineStart + f] & 0xC0) != 0x80) { // codepoint boundary + if (measureSpan(fontId, text + lineStart, f) > maxWidth) break; + lastFit = f; + } + } + if (lastFit == 0) { + // Even a single over-wide glyph must make progress; consume its whole + // UTF-8 sequence rather than splitting it into invalid fragments. + lastFit = 1; + while (lastFit < len && (text[lineStart + lastFit] & 0xC0) == 0x80) lastFit++; + } + const uint32_t rest = lineStart + lastFit; + lineEnd = rest; + flushLine(rest); + lineEnd = rest + (len - lastFit); + lineWidth = measureSpan(fontId, text + lineStart, lineEnd - lineStart); + } + } + if (lineEnd > lineStart) flushLine(n); + + // Trim trailing blank lines so the last page is not empty padding. + while (!lines.empty() && lines.back().len == 0) lines.pop_back(); + + totalPages = std::max(1, (static_cast(lines.size()) + linesPerPage - 1) / linesPerPage); + currentPage = 0; +} + +void DictionaryDefinitionActivity::loop() { + if (mappedInput.wasReleased(MappedInputManager::Button::Back)) { + finish(); + return; + } + + buttonNavigator.onNext([this] { + if (currentPage + 1 < totalPages) { + currentPage++; + requestUpdate(); + } + }); + + buttonNavigator.onPrevious([this] { + if (currentPage > 0) { + currentPage--; + requestUpdate(); + } + }); +} + +// Draws the current page's line spans (copied into a stack buffer for NUL +// termination). Called twice per render: once in font-cache scan mode, once +// for the real paint. +void DictionaryDefinitionActivity::drawBody(const int fontId, const int x, const int startY) const { + const int lineHeight = renderer.getLineHeight(fontId); + char buf[MAX_LINE_BYTES + 1]; + const int firstLine = currentPage * linesPerPage; + const int lastLine = std::min(firstLine + linesPerPage, static_cast(lines.size())); + for (int i = firstLine; i < lastLine; i++) { + if (lines[i].len == 0) continue; + const size_t len = std::min(static_cast(lines[i].len), MAX_LINE_BYTES); + memcpy(buf, definition.c_str() + lines[i].start, len); + buf[len] = '\0'; + renderer.drawText(fontId, x, startY + (i - firstLine) * lineHeight, buf); + } +} + +void DictionaryDefinitionActivity::render(RenderLock&&) { + renderer.clearScreen(); + + const auto& metrics = UITheme::getInstance().getMetrics(); + const auto orientation = renderer.getOrientation(); + const bool isLandscapeCw = orientation == GfxRenderer::Orientation::LandscapeClockwise; + const bool isLandscapeCcw = orientation == GfxRenderer::Orientation::LandscapeCounterClockwise; + const bool isInverted = orientation == GfxRenderer::Orientation::PortraitInverted; + const int hintGutterWidth = (isLandscapeCw || isLandscapeCcw) ? metrics.sideButtonHintsWidth : 0; + const int contentX = isLandscapeCw ? hintGutterWidth : 0; + const int contentWidth = renderer.getScreenWidth() - hintGutterWidth; + const int contentY = isInverted ? metrics.buttonHintsHeight : 0; + + // Header: matched headword left, page counter right. + const int headerY = contentY + metrics.topPadding + 10; + renderer.drawText(UI_12_FONT_ID, contentX + SIDE_PADDING, headerY, headword.c_str(), true, EpdFontFamily::BOLD); + if (totalPages > 1) { + char counter[16]; + snprintf(counter, sizeof(counter), "%d/%d", currentPage + 1, totalPages); + const int counterWidth = renderer.getTextWidth(UI_10_FONT_ID, counter); + renderer.drawText(UI_10_FONT_ID, contentX + contentWidth - SIDE_PADDING - counterWidth, headerY, counter); + } + + // Body: two-pass draw inside a prewarm scope (same pattern as the reader's + // renderContents) so SD-card font glyphs load from SD in one batch instead + // of one on-demand overflow read per character on every page turn. + const int fontId = SETTINGS.getReaderFontId(); + const int bodyStartY = contentY + metrics.topPadding + metrics.headerHeight; + auto* fcm = renderer.getFontCacheManager(); + auto scope = fcm->createPrewarmScope(); + drawBody(fontId, contentX + SIDE_PADDING, bodyStartY); // scan pass: records codepoints only + scope.endScanAndPrewarm(); + drawBody(fontId, contentX + SIDE_PADDING, bodyStartY); + + const auto labels = + mappedInput.mapLabels(tr(STR_BACK), "", (currentPage > 0 ? "<" : ""), (currentPage + 1 < totalPages ? ">" : "")); + GUI.drawButtonHints(renderer, labels.btn1, labels.btn2, labels.btn3, labels.btn4); + renderer.displayBuffer(); +} diff --git a/src/activities/reader/DictionaryDefinitionActivity.h b/src/activities/reader/DictionaryDefinitionActivity.h new file mode 100644 index 00000000..ef2c1330 --- /dev/null +++ b/src/activities/reader/DictionaryDefinitionActivity.h @@ -0,0 +1,46 @@ +#pragma once + +#include +#include +#include + +#include "activities/Activity.h" +#include "util/ButtonNavigator.h" + +// Paged plain-text viewer for one dictionary definition. The definition is +// word-wrapped once on entry; each page renders spans of the original string, +// so no per-line copies are held. +class DictionaryDefinitionActivity final : public Activity { + public: + explicit DictionaryDefinitionActivity(GfxRenderer& renderer, MappedInputManager& mappedInput, std::string headword, + std::string definition) + : Activity("DictionaryDefinition", renderer, mappedInput), + headword(std::move(headword)), + definition(std::move(definition)) {} + + void onEnter() override; + void loop() override; + void render(RenderLock&&) override; + + private: + // One wrapped display line: a byte span of `definition`. Wrapping keeps + // lines under the screen width, so uint16_t length is ample. + struct Line { + uint32_t start; + uint16_t len; + }; + + void wrapText(); + int measureSpan(int fontId, const char* text, size_t len) const; + void drawBody(int fontId, int x, int startY) const; + + const std::string headword; + // Not const: onEnter() normalizes embedded NULs (StarDict multi-type + // separators) to newlines so C-string APIs see the whole text. + std::string definition; + std::vector lines; + int currentPage = 0; + int totalPages = 1; + int linesPerPage = 1; + ButtonNavigator buttonNavigator; +}; diff --git a/src/activities/reader/DictionaryWordSelectActivity.cpp b/src/activities/reader/DictionaryWordSelectActivity.cpp new file mode 100644 index 00000000..42cb87ed --- /dev/null +++ b/src/activities/reader/DictionaryWordSelectActivity.cpp @@ -0,0 +1,301 @@ +#include "DictionaryWordSelectActivity.h" + +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "CrossPointSettings.h" +#include "DictionaryDefinitionActivity.h" +#include "components/UITheme.h" + +namespace { + +constexpr unsigned long POPUP_DURATION_MS = 1500; + +// A token is selectable when it has an ASCII alphanumeric or a non-ASCII +// codepoint outside U+2000-U+206F (dashes, bullets and other General +// Punctuation that appear as standalone tokens are not words). +bool isSelectableToken(const char* text) { + for (const uint8_t* p = reinterpret_cast(text); *p != 0; p++) { + if (*p < 0x80) { + if (std::isalnum(*p)) return true; + } else if (*p == 0xE2 && (p[1] == 0x80 || p[1] == 0x81)) { + if (p[2] == 0) break; // truncated sequence: skipping would step past the NUL + p += 2; // skip the 3-byte General Punctuation codepoint + } else { + return true; + } + } + return false; +} + +void indexBuildYield(void*) { vTaskDelay(1); } + +} // namespace + +void DictionaryWordSelectActivity::onEnter() { + Activity::onEnter(); + fontId = SETTINGS.getReaderFontId(); + lineHeight = renderer.getLineHeight(fontId); + // No null check: a failed allocation just disables the differential + // fast path (drawHighlightWithSnapshot skips the read), keeping the + // full-repaint path as the fallback. + snapshot = makeUniqueNoThrow(SNAPSHOT_CAPACITY); + extractWords(); + // Start on the middle row's word nearest mid-screen instead of top-left: + // any word on the page is then at most half a page of moves away. + if (!words.empty()) { + const int initial = closestInRow(rowCount / 2, renderer.getScreenWidth() / 2); + if (initial >= 0) selected = initial; + } + requestUpdate(); +} + +void DictionaryWordSelectActivity::extractWords() { + words.clear(); + words.reserve(128); + rowCount = 0; + + // Single walk: collect the selectable words while accumulating their text + // and styles (~2KB transient string, freed on return). Widths are measured + // afterwards: merging the page's codepoints into the SD font's persistent + // advance table first keeps getTextAdvanceX on the in-RAM path instead of + // loading glyphs from SD one overflow slot at a time. + std::string pageText; + pageText.reserve(2048); + uint8_t styleMask = 0; + + for (const auto& element : page->elements) { + if (element->getTag() != TAG_PageLine) continue; + const auto* line = static_cast(element.get()); + const auto& block = line->getBlock(); + if (!block || !block->valid()) continue; + + bool rowHasWords = false; + for (uint16_t i = 0; i < block->wordCount(); i++) { + const char* text = block->wordText(i); + if (!isSelectableToken(text)) continue; + + WordBox box; + box.x = static_cast(line->xPos + block->wordXpos(i) + marginLeft); + box.y = static_cast(line->yPos + marginTop); + box.style = block->wordStyle(i); + box.width = 0; // measured below, once the advance table is ready + box.row = rowCount; + box.text = text; + words.push_back(box); + rowHasWords = true; + + pageText.append(text); + pageText.push_back(' '); + styleMask |= static_cast(1u << (static_cast(box.style) & 0x03)); + } + if (rowHasWords) rowCount++; + } + + if (styleMask == 0) styleMask = 0x01; // REGULAR + renderer.ensureSdCardFontReady(fontId, pageText.c_str(), styleMask); + for (auto& word : words) { + word.width = static_cast(renderer.getTextAdvanceX(fontId, word.text, word.style)); + } +} + +// Index of the word in `row` whose horizontal center is closest to centerX; +// -1 when the row has no words. +int DictionaryWordSelectActivity::closestInRow(const uint16_t row, const int centerX) const { + int best = -1; + int bestDistance = INT_MAX; + for (int i = 0; i < static_cast(words.size()); i++) { + if (words[i].row != row) continue; + const int distance = std::abs(words[i].x + words[i].width / 2 - centerX); + if (distance < bestDistance) { + bestDistance = distance; + best = i; + } + } + return best; +} + +void DictionaryWordSelectActivity::moveVertical(const int direction) { + const WordBox& current = words[selected]; + const int targetRow = static_cast(current.row) + direction; + if (targetRow < 0 || targetRow >= static_cast(rowCount)) return; + + const int best = closestInRow(static_cast(targetRow), current.x + current.width / 2); + if (best >= 0 && best != selected) { + selected = best; + requestUpdate(); + } +} + +void DictionaryWordSelectActivity::performLookup() { + popup = Popup::Busy; + if (!dictOpenAttempted) { + dictOpenAttempted = true; + dictOpenOk = dict.open(SETTINGS.dictionaryName); + } + const bool indexing = dictOpenOk && dict.needsIndex(); + popupMsg = indexing ? StrId::STR_DICT_INDEXING : StrId::STR_DICT_LOOKING_UP; + requestUpdateAndWait(); // paint the page + busy popup before blocking on SD + + bool ok = dictOpenOk; + if (ok && indexing) ok = dict.buildIndex(&indexBuildYield); + + std::string definition; + std::string headword; + const bool found = ok && dict.lookup(words[selected].text, definition, headword); + + if (found) { + popup = Popup::None; + startActivityForResult(std::make_unique(renderer, mappedInput, std::move(headword), + std::move(definition)), + [this](const ActivityResult&) { requestUpdate(); }); + return; + } + popup = ok ? Popup::NotFound : Popup::Error; + popupMsg = ok ? StrId::STR_DICT_NOT_FOUND : StrId::STR_DICT_ERROR; + popupTime = millis(); + requestUpdate(); +} + +void DictionaryWordSelectActivity::loop() { + if (popup == Popup::NotFound || popup == Popup::Error) { + if (millis() - popupTime >= POPUP_DURATION_MS) { + popup = Popup::None; + requestUpdate(); + } + return; + } + + if (mappedInput.wasPressed(MappedInputManager::Button::Confirm)) confirmPressSeen = true; + + if (mappedInput.wasReleased(MappedInputManager::Button::Back)) { + finish(); + return; + } + if (mappedInput.wasReleased(MappedInputManager::Button::Confirm) && confirmPressSeen && !words.empty()) { + performLookup(); + return; + } + + if (words.empty()) return; + if (mappedInput.wasPressed(MappedInputManager::Button::Left) && selected > 0) { + selected--; + requestUpdate(); + } else if (mappedInput.wasPressed(MappedInputManager::Button::Right) && + selected + 1 < static_cast(words.size())) { + selected++; + requestUpdate(); + } else if (mappedInput.wasPressed(MappedInputManager::Button::Up)) { + moveVertical(-1); + } else if (mappedInput.wasPressed(MappedInputManager::Button::Down)) { + moveVertical(1); + } +} + +// Saves the pixels under words[selected]'s highlight box, then draws the +// highlight over them. Returns false when the pixels could not be saved +// (no buffer / oversize box) — the highlight is drawn regardless, but the +// next cursor move must do a full repaint. +bool DictionaryWordSelectActivity::drawHighlightWithSnapshot() { + const WordBox& word = words[selected]; + int hx = word.x - 2; + int hy = word.y - 2; + int hw = word.width + 4; + int hh = lineHeight + 4; + // Clamp to the panel so save, draw and restore all use the same box. + if (hx < 0) { + hw += hx; + hx = 0; + } + if (hy < 0) { + hh += hy; + hy = 0; + } + + bool saved = false; + if (snapshot && hw > 0 && hh > 0) { + saved = renderer.readFramebufferRegion(hx, hy, hw, hh, snapshot.get(), SNAPSHOT_CAPACITY) > 0; + } + snapshotX = static_cast(hx); + snapshotY = static_cast(hy); + snapshotW = static_cast(hw); + snapshotH = static_cast(hh); + snapshotIdx = saved ? selected : -1; + + renderer.fillRect(hx, hy, hw, hh, true); + renderer.drawText(fontId, word.x, word.y, word.text, false, word.style); + return saved; +} + +// Front-button bar (Back/Confirm/Left/Right). Drawn last on every repaint +// path, including the differential highlight-only path, so it always ends +// up as the top layer even when a highlighted word's box falls under a +// hint's screen area. No side-button hints: Up/Down row jump has no spare +// screen area on this page (it reuses the reader's full-bleed layout), and +// a hint box there would hide text instead of sitting in a reserved gutter. +void DictionaryWordSelectActivity::drawHints() const { + // No selectable word on this page: Confirm/Left/Right are all no-ops + // (guarded by words.empty() in loop()/performLookup), so only Back does + // anything and only Back is hinted. + if (words.empty()) { + const auto labels = mappedInput.mapLabels(tr(STR_BACK), "", "", ""); + GUI.drawButtonHints(renderer, labels.btn1, labels.btn2, labels.btn3, labels.btn4); + return; + } + const auto labels = mappedInput.mapLabels(tr(STR_BACK), tr(STR_LOOKUP), tr(STR_DIR_LEFT), tr(STR_DIR_RIGHT)); + GUI.drawButtonHints(renderer, labels.btn1, labels.btn2, labels.btn3, labels.btn4); +} + +void DictionaryWordSelectActivity::render(RenderLock&&) { + // Differential fast path: only the highlight moved and the framebuffer + // still holds a clean page (no popup or sub-activity since the last full + // repaint). Restore the pixels under the old highlight, draw the new one, + // and push — skipping the two-pass page render entirely. + if (popup == Popup::None && snapshotIdx >= 0 && !words.empty() && selected != snapshotIdx) { + renderer.writeFramebufferRegion(snapshotX, snapshotY, snapshotW, snapshotH, snapshot.get()); + // The full path's PrewarmScope cleared the glyph cache on exit; batch-load + // just the highlighted word's glyphs before drawing them white-on-black. + renderer.getFontCacheManager()->prewarmCache( + fontId, words[selected].text, static_cast(1u << (static_cast(words[selected].style) & 0x03))); + if (drawHighlightWithSnapshot()) { + drawHints(); + renderer.displayBuffer(HalDisplay::FAST_REFRESH); + return; + } + // Snapshot failed (oversize box) — fall through to a full repaint. + } + + renderer.clearScreen(); + + // Same prewarm-scan-then-render pass the reader uses, so SD-card fonts hit + // the in-RAM glyph cache during the real draw. + auto* fcm = renderer.getFontCacheManager(); + auto scope = fcm->createPrewarmScope(); + page->render(renderer, fontId, marginLeft, marginTop); + scope.endScanAndPrewarm(); + page->render(renderer, fontId, marginLeft, marginTop); + + if (!words.empty()) { + drawHighlightWithSnapshot(); + } + + drawHints(); + + if (popup != Popup::None) { + // The popup overdraws the page, so the snapshot no longer matches the + // framebuffer — force the next render onto the full-repaint path. + snapshotIdx = -1; + // drawPopup overlays the framebuffer and refreshes the display itself. + // I18N.get directly: tr() only accepts literal key names. + GUI.drawPopup(renderer, I18N.get(popupMsg)); + return; + } + renderer.displayBuffer(HalDisplay::FAST_REFRESH); +} diff --git a/src/activities/reader/DictionaryWordSelectActivity.h b/src/activities/reader/DictionaryWordSelectActivity.h new file mode 100644 index 00000000..164cf323 --- /dev/null +++ b/src/activities/reader/DictionaryWordSelectActivity.h @@ -0,0 +1,84 @@ +#pragma once + +#include +#include + +#include +#include + +#include "activities/Activity.h" +#include "util/Dictionary.h" + +// Button-driven word selection over the current reader page: Left/Right step +// through words in reading order, Up/Down jump rows, Confirm looks the word up +// and opens DictionaryDefinitionActivity, Back returns to the reader. +class DictionaryWordSelectActivity final : public Activity { + public: + explicit DictionaryWordSelectActivity(GfxRenderer& renderer, MappedInputManager& mappedInput, + std::unique_ptr page, int marginLeft, int marginTop) + : Activity("DictionaryWordSelect", renderer, mappedInput), + page(std::move(page)), + marginLeft(marginLeft), + marginTop(marginTop) {} + + void onEnter() override; + void loop() override; + void render(RenderLock&&) override; + + private: + // Screen box of one selectable word. `text` points into the owned Page's + // TextBlock arena (NUL-terminated), valid for this activity's lifetime. + struct WordBox { + int16_t x; + int16_t y; + int16_t width; + uint16_t row; + const char* text; + EpdFontFamily::Style style; + }; + + enum class Popup : uint8_t { None, Busy, NotFound, Error }; + + void extractWords(); + int closestInRow(uint16_t row, int centerX) const; + void moveVertical(int direction); + void performLookup(); + bool drawHighlightWithSnapshot(); + void drawHints() const; + + std::unique_ptr page; + const int marginLeft; + const int marginTop; + int fontId = 0; + int lineHeight = 0; + + std::vector words; + int selected = 0; + uint16_t rowCount = 0; + + Dictionary dict; + bool dictOpenAttempted = false; + bool dictOpenOk = false; + + Popup popup = Popup::None; + StrId popupMsg = StrId::STR_DICT_NOT_FOUND; + unsigned long popupTime = 0; + + // Differential highlight repaint: the pixels under the current highlight + // box, so a cursor move restores them and repaints only the two affected + // boxes instead of re-running the full two-pass page render (which also + // reloads every SD-font glyph on the page). snapshotIdx is the word whose + // under-pixels are saved; -1 means the framebuffer no longer holds a clean + // page (popup drawn, sub-activity shown) and the next render must be full. + static constexpr size_t SNAPSHOT_CAPACITY = 4096; + std::unique_ptr snapshot; + int16_t snapshotX = 0; + int16_t snapshotY = 0; + int16_t snapshotW = 0; + int16_t snapshotH = 0; + int snapshotIdx = -1; + + // The activity is entered while Confirm is still held (long-press trigger): + // ignore the stale release until a fresh press is seen. + bool confirmPressSeen = false; +}; diff --git a/src/activities/reader/EpubReaderActivity.cpp b/src/activities/reader/EpubReaderActivity.cpp index 6d8cc417..eaf310d4 100644 --- a/src/activities/reader/EpubReaderActivity.cpp +++ b/src/activities/reader/EpubReaderActivity.cpp @@ -20,6 +20,7 @@ #include "BookmarkEntry.h" #include "CrossPointSettings.h" #include "CrossPointState.h" +#include "DictionaryWordSelectActivity.h" #include "EpubReaderBookmarksActivity.h" #include "EpubReaderChapterSelectionActivity.h" #include "EpubReaderFootnotesActivity.h" @@ -257,6 +258,29 @@ void EpubReaderActivity::openReaderMenu() { }); } +void EpubReaderActivity::openDictionaryWordSelect() { + if (SETTINGS.dictionaryName[0] == '\0') { + showDictionaryMessage = true; + dictionaryMessageTime = millis(); + requestUpdate(); + return; + } + if (!section) return; + auto page = section->loadPage(section->currentPage); + if (!page) return; + + // Word geometry must match render(): viewable-area margins plus screen margin. + int orientedMarginTop, orientedMarginRight, orientedMarginBottom, orientedMarginLeft; + renderer.getOrientedViewableTRBL(&orientedMarginTop, &orientedMarginRight, &orientedMarginBottom, + &orientedMarginLeft); + orientedMarginTop += SETTINGS.screenMargin; + orientedMarginLeft += SETTINGS.screenMargin; + + startActivityForResult(std::make_unique(renderer, mappedInput, std::move(page), + orientedMarginLeft, orientedMarginTop), + [this](const ActivityResult&) { requestUpdate(); }); +} + void EpubReaderActivity::loop() { if (!epub) { // Should never happen @@ -379,6 +403,11 @@ void EpubReaderActivity::loop() { requestUpdate(); } + if (showDictionaryMessage && (millis() - dictionaryMessageTime) >= ReaderUtils::BOOKMARK_MESSAGE_DURATION_MS) { + showDictionaryMessage = false; + requestUpdate(); + } + // While the end screen suggestion menu is showing it owns Confirm/Back/navigation // input. Anything it doesn't handle (e.g. long-press Back to the file browser) falls // through to the regular handlers below; page turns are absorbed by the end-of-book @@ -442,6 +471,14 @@ void EpubReaderActivity::loop() { } } break; + case CrossPointSettings::LP_MENU_DICTIONARY: + // Hold ~0.4s starts dictionary word selection on the current page. + if (mappedInput.getHeldTime() >= ReaderUtils::BOOKMARK_HOLD_MS && !showDictionaryMessage) { + ignoreNextConfirmRelease = true; // Prevent menu open on the release that follows + openDictionaryWordSelect(); + return; + } + break; case CrossPointSettings::LP_MENU_DISABLED: default: break; @@ -710,6 +747,10 @@ void EpubReaderActivity::onReaderMenuConfirm(EpubReaderMenuActivity::MenuAction }); break; } + case EpubReaderMenuActivity::MenuAction::DICTIONARY: { + openDictionaryWordSelect(); + break; + } case EpubReaderMenuActivity::MenuAction::DISPLAY_QR: { if (section && section->currentPage >= 0 && section->currentPage < section->pageCount) { std::string fullText = section->getTextFromSectionFile(); @@ -1300,6 +1341,10 @@ void EpubReaderActivity::render(RenderLock&& lock) { if (showBookmarkMessage) { GUI.drawPopup(renderer, bookmarkRemoved ? tr(STR_BOOKMARK_REMOVED) : tr(STR_BOOKMARK_ADDED)); } + + if (showDictionaryMessage) { + GUI.drawPopup(renderer, tr(STR_DICT_NO_DICT_SET)); + } } bool EpubReaderActivity::applyDeferredReposition() { diff --git a/src/activities/reader/EpubReaderActivity.h b/src/activities/reader/EpubReaderActivity.h index e8799018..e9cd0179 100644 --- a/src/activities/reader/EpubReaderActivity.h +++ b/src/activities/reader/EpubReaderActivity.h @@ -39,6 +39,9 @@ class EpubReaderActivity final : public Activity { bool skipNextButtonCheck = false; // Skip button processing for one frame after subactivity exit bool automaticPageTurnActive = false; bool showBookmarkMessage = false; + // "No dictionary set" popup, shown when a lookup is triggered without a configured dictionary. + bool showDictionaryMessage = false; + unsigned long dictionaryMessageTime = 0UL; bool ignoreNextConfirmRelease = false; bool currentPageBookmarked = false; bool bookmarkRemoved = false; // true when last toggle removed (controls popup text) @@ -119,6 +122,7 @@ class EpubReaderActivity final : public Activity { void onReaderMenuConfirm(EpubReaderMenuActivity::MenuAction action); // Opens the reader menu for the current position (short-press Confirm) void openReaderMenu(); + void openDictionaryWordSelect(); // Returns true if sync acted (launched, or surfaced a save error); false if it was a no-op // because no KOReader credentials are stored. bool launchKOReaderSync(); diff --git a/src/activities/reader/EpubReaderMenuActivity.cpp b/src/activities/reader/EpubReaderMenuActivity.cpp index 2bf1df41..293c055b 100644 --- a/src/activities/reader/EpubReaderMenuActivity.cpp +++ b/src/activities/reader/EpubReaderMenuActivity.cpp @@ -22,7 +22,7 @@ EpubReaderMenuActivity::EpubReaderMenuActivity(GfxRenderer& renderer, MappedInpu std::vector EpubReaderMenuActivity::buildMenuItems(bool hasFootnotes, bool hasBookmarks) { std::vector items; - items.reserve(12); + items.reserve(13); items.push_back({MenuAction::SELECT_CHAPTER, StrId::STR_SELECT_CHAPTER}); if (hasFootnotes) { items.push_back({MenuAction::FOOTNOTES, StrId::STR_FOOTNOTES}); @@ -31,6 +31,7 @@ std::vector EpubReaderMenuActivity::buildMenuI items.push_back({MenuAction::BOOKMARKS, StrId::STR_BOOKMARKS}); } items.push_back({MenuAction::TOGGLE_BOOKMARK, StrId::STR_TOGGLE_BOOKMARK}); + items.push_back({MenuAction::DICTIONARY, StrId::STR_LOOKUP}); items.push_back({MenuAction::ROTATE_SCREEN, StrId::STR_ORIENTATION}); items.push_back({MenuAction::AUTO_PAGE_TURN, StrId::STR_AUTO_TURN_PAGES_PER_MIN}); items.push_back({MenuAction::GO_TO_PERCENT, StrId::STR_GO_TO_PERCENT}); diff --git a/src/activities/reader/EpubReaderMenuActivity.h b/src/activities/reader/EpubReaderMenuActivity.h index 1920ac8e..732fa144 100644 --- a/src/activities/reader/EpubReaderMenuActivity.h +++ b/src/activities/reader/EpubReaderMenuActivity.h @@ -24,7 +24,8 @@ class EpubReaderMenuActivity final : public Activity { DISPLAY_QR, GO_HOME, SYNC, - DELETE_CACHE + DELETE_CACHE, + DICTIONARY }; explicit EpubReaderMenuActivity(GfxRenderer& renderer, MappedInputManager& mappedInput, const std::string& title, diff --git a/src/activities/settings/SettingsActivity.cpp b/src/activities/settings/SettingsActivity.cpp index 1294c8d9..5212dab2 100644 --- a/src/activities/settings/SettingsActivity.cpp +++ b/src/activities/settings/SettingsActivity.cpp @@ -39,7 +39,12 @@ void SettingsActivity::rebuildSettingsLists() { // reader activity ran — otherwise the font-family picker shows stale list. sdFontSystem.refreshIfDirty(); - for (auto& setting : getSettingsList(&sdFontSystem.registry())) { + // Rescan /dictionaries on every rebuild: cheap (one directory listing) and + // picks up dictionaries copied to the SD card since the last visit. + std::vector dictionaries; + DictionaryRegistry::discover(dictionaries); + + for (auto& setting : getSettingsList(&sdFontSystem.registry(), &dictionaries)) { if (setting.category == StrId::STR_NONE_OPT) continue; if (setting.category == StrId::STR_CAT_DISPLAY) { displaySettings.push_back(setting); diff --git a/src/util/DictZip.cpp b/src/util/DictZip.cpp new file mode 100644 index 00000000..620310a7 --- /dev/null +++ b/src/util/DictZip.cpp @@ -0,0 +1,180 @@ +#include "DictZip.h" + +#include +#include + +namespace DictZip { +namespace { + +// Caps the chunk table at 32KB of heap (8192 * 4 bytes); at the typical ~58KB +// chunk length that still allows ~460MB of uncompressed dictionary data. +constexpr uint16_t MAX_CHUNK_COUNT = 8192; + +bool readLe16(HalFile& file, uint16_t* out) { + uint8_t raw[2]; + if (file.read(raw, 2) != 2) return false; + *out = static_cast(raw[0] | (static_cast(raw[1]) << 8)); + return true; +} + +bool extractChunkSlice(HalFile& file, uint32_t compressedOffset, uint32_t compressedSize, uint32_t discardSize, + uint32_t extractSize, HalFile& outFile) { + if (extractSize == 0) return true; + auto compBuf = makeUniqueNoThrow(compressedSize); + if (!compBuf) return false; + + file.seekSet(compressedOffset); + if (file.read(compBuf.get(), static_cast(compressedSize)) != static_cast(compressedSize)) return false; + + InflateReader reader; + if (!reader.init(true)) return false; + reader.setSource(compBuf.get(), compressedSize); + + auto buf = makeUniqueNoThrow(512); + if (!buf) return false; + + uint32_t batch; + while (discardSize > 0) { + batch = discardSize < 512 ? discardSize : 512; + if (!reader.read(buf.get(), batch)) return false; + discardSize -= batch; + } + + while (extractSize > 0) { + batch = extractSize < 512 ? extractSize : 512; + if (!reader.read(buf.get(), batch)) return false; + if (outFile.write(buf.get(), batch) != batch) return false; + extractSize -= batch; + } + + return true; +} + +} // namespace + +bool parse(HalFile& file, Info* info) { + if (!info) return false; + *info = {}; + + uint8_t header[10]; + if (file.read(header, sizeof(header)) != static_cast(sizeof(header))) return false; + if (header[0] != 0x1f || header[1] != 0x8b || header[2] != 8) return false; + + const uint8_t flags = header[3]; + if ((flags & 0x04) == 0) return false; // dictzip requires FEXTRA + + uint16_t xlen = 0; + if (!readLe16(file, &xlen)) return false; + + uint32_t extraRead = 0; + bool foundRa = false; + while (extraRead + 4 <= xlen) { + uint8_t subHeader[4]; + if (file.read(subHeader, sizeof(subHeader)) != static_cast(sizeof(subHeader))) return false; + extraRead += 4; + const uint16_t subLen = static_cast(subHeader[2] | (static_cast(subHeader[3]) << 8)); + if (extraRead + subLen > xlen) return false; + + if (subHeader[0] == 'R' && subHeader[1] == 'A') { + if (subLen < 6) return false; + + uint16_t version = 0; + uint16_t chunkLen = 0; + uint16_t chunkCount = 0; + if (!readLe16(file, &version) || !readLe16(file, &chunkLen) || !readLe16(file, &chunkCount)) return false; + extraRead += 6; + if (version != 1 || chunkLen == 0 || chunkCount == 0 || chunkCount > MAX_CHUNK_COUNT) return false; + if (subLen != static_cast(6 + chunkCount * 2)) return false; + + info->chunkLength = chunkLen; + info->chunkOffsets.reserve(static_cast(chunkCount) + 1); + info->chunkOffsets.push_back(0); + uint32_t cumulative = 0; + for (uint16_t i = 0; i < chunkCount; i++) { + uint16_t compLen = 0; + if (!readLe16(file, &compLen)) return false; + extraRead += 2; + cumulative += compLen; + info->chunkOffsets.push_back(cumulative); + } + foundRa = true; + } else { + file.seekSet(file.position() + subLen); + extraRead += subLen; + } + } + if (extraRead != xlen || !foundRa) return false; + + if (flags & 0x08) { // FNAME + int b; + do { + b = file.read(); + if (b < 0) return false; + } while (b != 0); + } + if (flags & 0x10) { // FCOMMENT + int b; + do { + b = file.read(); + if (b < 0) return false; + } while (b != 0); + } + if (flags & 0x02) { // FHCRC + uint8_t crc[2]; + if (file.read(crc, 2) != 2) return false; + } + + info->dataOffset = static_cast(file.position()); + const uint32_t fileSize = static_cast(file.fileSize()); + if (fileSize < 4) return false; + file.seekSet(fileSize - 4); + uint8_t isizeRaw[4]; + if (file.read(isizeRaw, 4) != 4) return false; + info->totalSize = static_cast(isizeRaw[0]) | (static_cast(isizeRaw[1]) << 8) | + (static_cast(isizeRaw[2]) << 16) | (static_cast(isizeRaw[3]) << 24); + if (info->totalSize == 0) return false; + info->valid = true; + return true; +} + +bool extractEntry(const char* path, uint32_t offset, uint32_t size, HalFile& outFile) { + if (size == 0) return true; + + HalFile file; + if (!Storage.openFileForRead("DICTZIP", path, file)) return false; + + Info info; + if (!parse(file, &info)) return false; + + // Reject ranges outside the uncompressed data (offset/size come from the + // untrusted .idx). Subtraction form avoids uint32 overflow in offset + size + // and guarantees localOffset < chunkOutSize in the loop below. + if (offset > info.totalSize || size > info.totalSize - offset) return false; + + const uint32_t startChunk = offset / info.chunkLength; + const uint32_t endChunk = (offset + size - 1) / info.chunkLength; + if (endChunk + 1 >= info.chunkOffsets.size()) return false; + + uint32_t remaining = size; + const uint32_t lastChunk = static_cast(info.chunkOffsets.size() - 2); + for (uint32_t chunk = startChunk; chunk <= endChunk; chunk++) { + uint32_t chunkOutSize = info.chunkLength; + if (chunk == lastChunk) chunkOutSize = info.totalSize - chunk * info.chunkLength; + if (chunkOutSize == 0 || chunkOutSize > info.chunkLength) chunkOutSize = info.chunkLength; + + const uint32_t localOffset = (chunk == startChunk) ? (offset % info.chunkLength) : 0; + const uint32_t available = chunkOutSize - localOffset; + const uint32_t take = remaining < available ? remaining : available; + + const uint32_t compOffset = info.dataOffset + info.chunkOffsets[chunk]; + const uint32_t compSize = info.chunkOffsets[chunk + 1] - info.chunkOffsets[chunk]; + if (!extractChunkSlice(file, compOffset, compSize, localOffset, take, outFile)) return false; + + remaining -= take; + if (remaining == 0) break; + } + + return remaining == 0; +} + +} // namespace DictZip diff --git a/src/util/DictZip.h b/src/util/DictZip.h new file mode 100644 index 00000000..47f41dfe --- /dev/null +++ b/src/util/DictZip.h @@ -0,0 +1,26 @@ +#pragma once + +#include + +#include +#include + +// Random-access reader for dictzip (.dict.dz) files: gzip with an extra "RA" +// field holding a chunk table, so any byte range can be decompressed without +// inflating the whole file. Format: https://linux.die.net/man/1/dictzip +namespace DictZip { + +struct Info { + uint32_t dataOffset = 0; // file offset where compressed chunk data starts + uint32_t totalSize = 0; // uncompressed size (gzip ISIZE trailer) + uint16_t chunkLength = 0; // uncompressed bytes per chunk + std::vector chunkOffsets; // cumulative compressed offsets, chunkCount+1 entries + bool valid = false; +}; + +bool parse(HalFile& file, Info* info); + +// Decompress the uncompressed byte range [offset, offset+size) into outFile. +bool extractEntry(const char* path, uint32_t offset, uint32_t size, HalFile& outFile); + +} // namespace DictZip diff --git a/src/util/Dictionary.cpp b/src/util/Dictionary.cpp new file mode 100644 index 00000000..4d4d2b1f --- /dev/null +++ b/src/util/Dictionary.cpp @@ -0,0 +1,378 @@ +#include "Dictionary.h" + +#include +#include +#include + +#include +#include +#include + +#include "DictZip.h" +#include "DictionaryRegistry.h" +#include "StringUtils.h" + +namespace { + +// Shared temp file for entries lazily extracted from .dict.dz. +constexpr const char* DICT_TMP_FILE = "/.crosspoint/dict.tmp"; + +// .qidx sidecar header: magic, version, sample interval, sample count, and the +// .idx file size the sidecar was built from (staleness check). +constexpr uint32_t QIDX_MAGIC = 0x58444951; // "QIDX" little-endian +constexpr uint32_t QIDX_VERSION = 1; +constexpr size_t QIDX_HEADER_BYTES = 5 * sizeof(uint32_t); + +struct QidxHeader { + uint32_t sampleCount = 0; + uint32_t idxFileSize = 0; + bool valid = false; +}; + +QidxHeader readQidxHeader(HalFile& qidx, uint32_t sampleInterval) { + QidxHeader header; + uint32_t raw[5]; + if (!qidx.seekSet(0) || qidx.read(raw, sizeof(raw)) != static_cast(sizeof(raw))) return header; + if (raw[0] != QIDX_MAGIC || raw[1] != QIDX_VERSION || raw[2] != sampleInterval) return header; + header.sampleCount = raw[3]; + header.idxFileSize = raw[4]; + header.valid = true; + return header; +} + +bool readSampleOffset(HalFile& qidx, uint32_t sampleIndex, uint32_t* out) { + if (!qidx.seekSet(QIDX_HEADER_BYTES + static_cast(sampleIndex) * sizeof(uint32_t))) return false; + return qidx.read(out, sizeof(*out)) == static_cast(sizeof(*out)); +} + +uint32_t readBe32(const uint8_t* p) { + return (static_cast(p[0]) << 24) | (static_cast(p[1]) << 16) | + (static_cast(p[2]) << 8) | static_cast(p[3]); +} + +// Word characters for cleaning: ASCII alphanumerics plus any UTF-8 +// continuation/lead byte, so accented words keep their edges. +bool isWordByte(unsigned char c) { return c >= 0x80 || std::isalnum(c) != 0; } + +// True when the .ifo declares 64-bit index offsets, which this reader does not +// support (only scans the first 2KB — idxoffsetbits always appears early). +bool ifoDeclares64BitOffsets(const std::string& ifoPath) { + HalFile ifo; + if (!Storage.openFileForRead("DICT", ifoPath, ifo)) return false; + char buf[2048]; + const int n = ifo.read(buf, sizeof(buf) - 1); + if (n <= 0) return false; + buf[n] = '\0'; + const char* line = strstr(buf, "idxoffsetbits"); + if (!line) return false; + const char* eq = strchr(line, '='); + return eq && strtol(eq + 1, nullptr, 10) == 64; +} + +} // namespace + +bool Dictionary::open(const char* folderName) { + basePath.clear(); + std::string resolved; + if (!DictionaryRegistry::resolveBasePath(folderName, resolved)) { + LOG_ERR("DICT", "No dictionary found in folder '%s'", folderName ? folderName : ""); + return false; + } + + if (!Storage.exists((resolved + ".idx").c_str())) { + LOG_ERR("DICT", "%s.idx missing (compressed .idx.gz is not supported)", resolved.c_str()); + return false; + } + hasPlainDict = Storage.exists((resolved + ".dict").c_str()); + if (!hasPlainDict && !Storage.exists((resolved + ".dict.dz").c_str())) { + LOG_ERR("DICT", "%s has no .dict or .dict.dz", resolved.c_str()); + return false; + } + if (ifoDeclares64BitOffsets(resolved + ".ifo")) { + LOG_ERR("DICT", "%s uses 64-bit index offsets (unsupported)", resolved.c_str()); + return false; + } + + basePath = std::move(resolved); + return true; +} + +bool Dictionary::needsIndex() { + if (!isOpen()) return false; + + HalFile idx; + if (!Storage.openFileForRead("DICT", basePath + ".idx", idx)) return false; + const uint32_t idxSize = static_cast(idx.fileSize()); + + HalFile qidx; + if (!Storage.openFileForRead("DICT", basePath + ".qidx", qidx)) return true; + const QidxHeader header = readQidxHeader(qidx, SAMPLE_INTERVAL); + return !header.valid || header.idxFileSize != idxSize; +} + +bool Dictionary::buildIndex(void (*yieldFn)(void*), void* ctx) { + if (!isOpen()) return false; + + HalFile idx; + if (!Storage.openFileForRead("DICT", basePath + ".idx", idx)) return false; + const uint32_t idxSize = static_cast(idx.fileSize()); + + constexpr size_t CHUNK_BYTES = 4096; + auto buf = makeUniqueNoThrow(CHUNK_BYTES); + if (!buf) { + LOG_ERR("DICT", "OOM: %u byte index scan buffer", CHUNK_BYTES); + return false; + } + + // Stream each sample offset straight to the sidecar instead of accumulating + // them in RAM: a large .idx would otherwise cost tens of KB of vector heap, + // and vector growth aborts on OOM under -fno-exceptions. The header slot is + // zero-filled until the scan succeeds, so an interrupted build leaves a file + // readQidxHeader rejects (magic mismatch) and needsIndex() triggers a rebuild. + const std::string qidxPath = basePath + ".qidx"; + HalFile out; + if (!Storage.openFileForWrite("DICT", qidxPath, out)) return false; + const auto writeU32 = [&out](uint32_t v) { return out.write(&v, sizeof(v)) == static_cast(sizeof(v)); }; + const uint32_t placeholder[5] = {}; + bool ok = out.write(placeholder, sizeof(placeholder)) == sizeof(placeholder); + uint32_t sampleCount = 0; + if (ok) { + ok = writeU32(0); // entry 0 always starts at byte 0 + sampleCount = 1; + } + + const unsigned long startMs = millis(); + uint32_t entryCount = 0; + uint32_t pos = 0; + uint32_t suffixLeft = 0; // 0 while scanning a headword, else suffix bytes remaining + uint32_t sinceYield = 0; + while (ok && pos < idxSize) { + const int n = idx.read(buf.get(), CHUNK_BYTES); + if (n <= 0) { + LOG_ERR("DICT", "Index scan read failed at %lu", static_cast(pos)); + ok = false; + break; + } + for (int i = 0; ok && i < n; i++) { + if (suffixLeft == 0) { + if (buf[i] == 0) suffixLeft = 8; + } else if (--suffixLeft == 0) { + entryCount++; + const uint32_t nextEntryStart = pos + i + 1; + if (entryCount % SAMPLE_INTERVAL == 0 && nextEntryStart < idxSize) { + ok = writeU32(nextEntryStart); + sampleCount++; + } + } + } + pos += n; + sinceYield += n; + if (yieldFn && sinceYield >= 64 * 1024) { + sinceYield = 0; + yieldFn(ctx); + } + } + + if (ok) { + // Backpatch the now-valid header over the placeholder. + const uint32_t header[5] = {QIDX_MAGIC, QIDX_VERSION, SAMPLE_INTERVAL, sampleCount, idxSize}; + ok = out.seekSet(0) && out.write(header, sizeof(header)) == sizeof(header); + } + if (!ok) { + LOG_ERR("DICT", "Index build failed, removing %s", qidxPath.c_str()); + out.close(); // close before remove of the same path + Storage.remove(qidxPath.c_str()); + return false; + } + + LOG_INF("DICT", "Indexed %lu entries (%lu samples) in %lu ms", static_cast(entryCount), + static_cast(sampleCount), millis() - startMs); + return true; +} + +int Dictionary::readWordInto(HalFile& file, char* buf, size_t bufSize) { + size_t i = 0; + while (i < bufSize - 1) { + const int ch = file.read(); + if (ch < 0) return -1; // EOF or I/O error + if (ch == 0) { + buf[i] = '\0'; + return static_cast(i); + } + buf[i++] = static_cast(ch); + } + // Word too long for buffer — consume remaining bytes to stay in sync + buf[bufSize - 1] = '\0'; + int ch; + do { + ch = file.read(); + } while (ch > 0); + return static_cast(bufSize - 1); +} + +DictLocation Dictionary::locate(const char* target, std::string* matchedHeadwordOut) { + DictLocation result; + if (!isOpen()) return result; + + HalFile idx; + if (!Storage.openFileForRead("DICT", basePath + ".idx", idx)) return result; + const uint32_t idxSize = static_cast(idx.fileSize()); + + // Bisect the sampled offsets to the last sample whose headword <= target. + // Falls back to a full scan from byte 0 when the sidecar is unusable. + uint32_t startByte = 0; + HalFile qidx; + if (Storage.openFileForRead("DICT", basePath + ".qidx", qidx)) { + const QidxHeader header = readQidxHeader(qidx, SAMPLE_INTERVAL); + if (header.valid && header.idxFileSize == idxSize && header.sampleCount > 0) { + uint32_t lo = 0; + uint32_t hi = header.sampleCount - 1; + while (lo < hi) { + const uint32_t mid = (lo + hi + 1) / 2; + uint32_t offset = 0; + if (!readSampleOffset(qidx, mid, &offset) || !idx.seekSet(offset) || + readWordInto(idx, wordBuf, sizeof(wordBuf)) < 0) { + lo = 0; + break; + } + if (StringUtils::asciiCaseCmp(wordBuf, target) <= 0) { + lo = mid; + } else { + hi = mid - 1; + } + } + readSampleOffset(qidx, lo, &startByte); + } + } + + // Linear scan of at most SAMPLE_INTERVAL entries: headword NUL, BE32 offset, + // BE32 size. The index is sorted, so stop at the first headword > target. + idx.seekSet(startByte); + while (static_cast(idx.position()) < idxSize) { + if (readWordInto(idx, wordBuf, sizeof(wordBuf)) < 0) break; + uint8_t suffix[8]; + if (idx.read(suffix, 8) != 8) break; + + const int cmp = StringUtils::asciiCaseCmp(wordBuf, target); + if (cmp == 0) { + result.offset = readBe32(suffix); + result.size = readBe32(suffix + 4); + result.found = true; + if (matchedHeadwordOut) *matchedHeadwordOut = wordBuf; + return result; + } + if (cmp > 0) break; + } + return result; +} + +bool Dictionary::readDefinition(const DictLocation& location, std::string& out) { + if (!location.found) return false; + const uint32_t size = std::min(location.size, MAX_DEFINITION_BYTES); + + std::string path; + uint32_t offset = 0; + if (hasPlainDict) { + path = basePath + ".dict"; + offset = location.offset; + } else { + HalFile tmp = Storage.open(DICT_TMP_FILE, O_WRITE | O_CREAT | O_TRUNC); + if (!tmp) { + LOG_ERR("DICT", "Failed to open %s", DICT_TMP_FILE); + return false; + } + if (!DictZip::extractEntry((basePath + ".dict.dz").c_str(), location.offset, size, tmp)) { + LOG_ERR("DICT", "dictzip extraction failed for %s", basePath.c_str()); + return false; + } + tmp.close(); // close before reopening the same path for read + path = DICT_TMP_FILE; + } + + HalFile dict; + if (!Storage.openFileForRead("DICT", path, dict)) return false; + const uint32_t dictSize = static_cast(dict.fileSize()); + if (offset > dictSize || size > dictSize - offset) { + LOG_ERR("DICT", "Definition out of bounds (%lu+%lu > %lu)", static_cast(offset), + static_cast(size), static_cast(dictSize)); + return false; + } + + // std::string growth aborts on OOM (-fno-exceptions); refuse up front unless + // the allocation fits comfortably in the largest free block. + if (ESP.getMaxAllocHeap() < size + 8 * 1024) { + LOG_ERR("DICT", "Low heap for %lu byte definition", static_cast(size)); + return false; + } + + dict.seekSet(offset); + out.assign(size, '\0'); + const int bytesRead = dict.read(&out[0], size); + if (bytesRead < 0) { + out.clear(); + return false; + } + if (static_cast(bytesRead) < size) out.resize(bytesRead); + return true; +} + +std::string Dictionary::cleanWord(const char* word) { + if (!word) return ""; + size_t start = 0; + size_t end = strlen(word); + while (start < end && !isWordByte(static_cast(word[start]))) start++; + while (end > start && !isWordByte(static_cast(word[end - 1]))) end--; + if (start >= end) return ""; + + std::string result(word + start, end - start); + std::transform(result.begin(), result.end(), result.begin(), + [](unsigned char c) { return c >= 0x80 ? c : static_cast(std::tolower(c)); }); + return result; +} + +void Dictionary::stemVariants(const std::string& word, std::vector& out) { + out.clear(); + out.reserve(6); + const size_t n = word.size(); + const auto add = [&out](std::string v) { + if (std::find(out.begin(), out.end(), v) == out.end()) out.push_back(std::move(v)); + }; + // endsWith requires a non-empty remainder so variants never come out empty. + const auto endsWith = [&word, n](const char* suffix) { + const size_t len = strlen(suffix); + return n > len && word.compare(n - len, len, suffix) == 0; + }; + + if (endsWith("'s")) add(word.substr(0, n - 2)); + if (endsWith("\xE2\x80\x99s")) add(word.substr(0, n - 4)); // U+2019 apostrophe + if (endsWith("ies")) add(word.substr(0, n - 3) + "y"); // stories -> story + if (endsWith("es")) add(word.substr(0, n - 2)); // boxes -> box + if (endsWith("s")) add(word.substr(0, n - 1)); // dogs -> dog + if (endsWith("ed")) { + add(word.substr(0, n - 2)); // walked -> walk + add(word.substr(0, n - 1)); // loved -> love + if (n >= 4 && word[n - 3] == word[n - 4]) add(word.substr(0, n - 3)); // stopped -> stop + } + if (endsWith("ing")) { + add(word.substr(0, n - 3)); // walking -> walk + add(word.substr(0, n - 3) + "e"); // making -> make + if (n >= 5 && word[n - 4] == word[n - 5]) add(word.substr(0, n - 4)); // running -> run + } +} + +bool Dictionary::lookup(const char* word, std::string& definitionOut, std::string& matchedHeadwordOut) { + const std::string cleaned = cleanWord(word); + if (cleaned.empty() || !isOpen()) return false; + + DictLocation location = locate(cleaned.c_str(), &matchedHeadwordOut); + if (!location.found) { + std::vector variants; + stemVariants(cleaned, variants); + for (const auto& variant : variants) { + location = locate(variant.c_str(), &matchedHeadwordOut); + if (location.found) break; + } + } + if (!location.found) return false; + return readDefinition(location, definitionOut); +} diff --git a/src/util/Dictionary.h b/src/util/Dictionary.h new file mode 100644 index 00000000..0a4c27e8 --- /dev/null +++ b/src/util/Dictionary.h @@ -0,0 +1,65 @@ +#pragma once + +#include + +#include +#include +#include + +// Result of an index search — file location of a definition without reading it. +struct DictLocation { + uint32_t offset = 0; // byte offset in .dict data + uint32_t size = 0; // byte length in .dict data + bool found = false; +}; + +// Slim StarDict reader: exact-match lookup with a mini stemming fallback. +// +// Expects /dictionaries//.idx (uncompressed) plus .dict or +// .dict.dz. Lookups binary-search a lazily built sampled-offset sidecar +// (.qidx, byte offset of every SAMPLE_INTERVAL-th .idx entry), then +// linear-scan at most SAMPLE_INTERVAL entries. Everything streams from SD; no +// index is held in RAM. +class Dictionary { + public: + // Resolve the dictionary folder and validate its files. Rejects + // dictionaries with 64-bit index offsets (idxoffsetbits=64 in .ifo). + bool open(const char* folderName); + bool isOpen() const { return !basePath.empty(); } + + // True when the .qidx sidecar is missing or stale — call buildIndex() first + // so the UI can show an "Indexing…" message for the slow first pass. + bool needsIndex(); + + // One streaming pass over .idx writing the .qidx sidecar. yieldFn (optional) + // is called every ~64KB consumed to feed the watchdog / repaint the UI. + bool buildIndex(void (*yieldFn)(void*) = nullptr, void* ctx = nullptr); + + // Clean the word, look it up, and on a miss retry mini stem variants + // (-'s/-s/-es/-ies/-ed/-ing). On a hit fills the definition text (capped at + // MAX_DEFINITION_BYTES) and the headword as stored in the index. + bool lookup(const char* word, std::string& definitionOut, std::string& matchedHeadwordOut); + + static std::string cleanWord(const char* word); + + static constexpr uint32_t MAX_DEFINITION_BYTES = 64 * 1024; + + private: + static constexpr uint32_t SAMPLE_INTERVAL = 256; + + DictLocation locate(const char* target, std::string* matchedHeadwordOut); + bool readDefinition(const DictLocation& location, std::string& out); + static void stemVariants(const std::string& word, std::vector& out); + + // Read a null-terminated word from an open file into buf (max bufSize-1 + // chars). Returns the number of characters read (excluding null), or -1 on + // EOF/error. Over-long words are truncated but the stream stays in sync. + static int readWordInto(HalFile& file, char* buf, size_t bufSize); + + std::string basePath; // "/dictionaries//", empty when not open + bool hasPlainDict = false; + + // Shared scan buffer: lookups are single-threaded and this avoids a + // 256-byte array on the stack of every locate() call. + char wordBuf[256] = {}; +}; diff --git a/src/util/DictionaryRegistry.cpp b/src/util/DictionaryRegistry.cpp new file mode 100644 index 00000000..239113be --- /dev/null +++ b/src/util/DictionaryRegistry.cpp @@ -0,0 +1,113 @@ +#include "DictionaryRegistry.h" + +#include +#include + +#include +#include + +#include "StringUtils.h" + +namespace DictionaryRegistry { +namespace { + +// Dictionaries are looked up in both roots, in order. The hidden variant +// lets users keep the folder out of the file browser (hidden by default, +// see FileBrowserActivity's showHiddenFiles check). +constexpr const char* DICT_ROOTS[] = {"/dictionaries", "/.dictionaries"}; + +// Find the single .idx stem inside one dictionary folder. Returns false when +// the folder holds no .idx or more than one distinct stem (ambiguous). +bool findStem(const char* folderPath, std::string& stemOut) { + auto dir = Storage.open(folderPath); + if (!dir || !dir.isDirectory()) return false; + + dir.rewindDirectory(); + char name[128]; + char foundStem[128]; + foundStem[0] = '\0'; + for (auto entry = dir.openNextFile(); entry; entry = dir.openNextFile()) { + entry.getName(name, sizeof(name)); + // Skip macOS metadata files (AppleDouble resource forks) + if (entry.isDirectory() || strncmp(name, "._", 2) == 0) continue; + + const size_t len = strlen(name); + if (len <= 4 || strcmp(name + len - 4, ".idx") != 0) continue; + + name[len - 4] = '\0'; + if (foundStem[0] != '\0' && strcmp(foundStem, name) != 0) { + LOG_DBG("DREG", "Skipping %s: multiple index stems found", folderPath); + return false; + } + strncpy(foundStem, name, sizeof(foundStem) - 1); + foundStem[sizeof(foundStem) - 1] = '\0'; + } + + if (foundStem[0] == '\0') return false; + + // Require dictionary data next to the index, so folders holding only an + // .idx never surface as selectable dictionaries that fail at lookup time. + const std::string base = std::string(folderPath) + "/" + foundStem; + if (!Storage.exists((base + ".dict").c_str()) && !Storage.exists((base + ".dict.dz").c_str())) { + LOG_DBG("DREG", "Skipping %s: no .dict or .dict.dz", folderPath); + return false; + } + + stemOut = foundStem; + return true; +} + +} // namespace + +void discover(std::vector& out) { + out.clear(); + out.reserve(8); + + for (const char* dictRoot : DICT_ROOTS) { + auto rootDir = Storage.open(dictRoot); + if (!rootDir || !rootDir.isDirectory()) { + LOG_DBG("DREG", "No %s directory on SD card", dictRoot); + continue; + } + + rootDir.rewindDirectory(); + char name[128]; + for (auto entry = rootDir.openNextFile(); entry; entry = rootDir.openNextFile()) { + entry.getName(name, sizeof(name)); + if (!entry.isDirectory() || name[0] == '.') continue; + + std::string folderPath = std::string(dictRoot) + "/" + name; + std::string stem; + if (!findStem(folderPath.c_str(), stem)) continue; + + DictionaryEntry e; + e.name = name; + e.stem = std::move(stem); + out.push_back(std::move(e)); + LOG_DBG("DREG", "Found dictionary: %s", name); + } + } + + // Case-insensitive sort by folder name (matches FileBrowserActivity ordering). + std::sort(out.begin(), out.end(), [](const DictionaryEntry& a, const DictionaryEntry& b) { + return StringUtils::asciiCaseCmp(a.name.c_str(), b.name.c_str()) < 0; + }); +} + +bool resolveBasePath(const char* folderName, std::string& basePathOut) { + if (!folderName || folderName[0] == '\0') return false; + // folderName is persisted in the settings JSON: reject separators and dot + // prefixes so a crafted value cannot escape the dictionary roots. + if (folderName[0] == '.' || strpbrk(folderName, "/\\") != nullptr) return false; + + for (const char* dictRoot : DICT_ROOTS) { + std::string folderPath = std::string(dictRoot) + "/" + folderName; + std::string stem; + if (!findStem(folderPath.c_str(), stem)) continue; + basePathOut = folderPath + "/" + stem; + return true; + } + return false; +} + +} // namespace DictionaryRegistry diff --git a/src/util/DictionaryRegistry.h b/src/util/DictionaryRegistry.h new file mode 100644 index 00000000..3fca48ed --- /dev/null +++ b/src/util/DictionaryRegistry.h @@ -0,0 +1,25 @@ +#pragma once + +#include +#include + +// One StarDict dictionary found under /dictionaries or /.dictionaries: a +// subfolder holding .idx plus .dict or .dict.dz. +struct DictionaryEntry { + std::string name; // subfolder name (shown to the user, stored in settings) + std::string stem; // index basename without .idx +}; + +namespace DictionaryRegistry { + +// Scan /dictionaries/*/ and /.dictionaries/*/ for dictionaries. Folders with +// multiple index stems are ambiguous and skipped. Result is sorted +// case-insensitively by name. +void discover(std::vector& out); + +// Resolve a folder name to its extensionless base path +// ("/dictionaries//" or "/.dictionaries//"). +// Returns false if the folder holds no usable dictionary in either root. +bool resolveBasePath(const char* folderName, std::string& basePathOut); + +} // namespace DictionaryRegistry diff --git a/src/util/HtmlToPlainText.cpp b/src/util/HtmlToPlainText.cpp new file mode 100644 index 00000000..a558a03b --- /dev/null +++ b/src/util/HtmlToPlainText.cpp @@ -0,0 +1,150 @@ +#include "HtmlToPlainText.h" + +#include + +#include +#include +#include + +namespace { + +bool isTagStart(const std::string& input, size_t pos) { + if (pos + 1 >= input.size()) return false; + const unsigned char next = input[pos + 1]; + return next == '/' || next == '!' || next == '?' || std::isalpha(next); +} + +enum class TagBreak : uint8_t { None, Line, Paragraph }; + +TagBreak tagBreak(const std::string& input, size_t start, size_t end) { + while (start < end && (input[start] == '/' || std::isspace(static_cast(input[start])))) start++; + const size_t nameStart = start; + while (start < end && std::isalpha(static_cast(input[start]))) start++; + const size_t len = start - nameStart; + if (len == 0) return TagBreak::None; + + const auto equals = [&](const char* name) { + const size_t nameLen = strlen(name); + if (len != nameLen) return false; + for (size_t i = 0; i < len; i++) { + if (std::tolower(static_cast(input[nameStart + i])) != name[i]) return false; + } + return true; + }; + + if (equals("p") || equals("h1") || equals("h2") || equals("h3") || equals("h4") || equals("h5") || equals("h6") || + equals("hr")) { + return TagBreak::Paragraph; + } + if (equals("br") || equals("div") || equals("li") || equals("tr")) return TagBreak::Line; + return TagBreak::None; +} + +void appendCodepoint(std::string& output, uint32_t codepoint) { + if (codepoint == 0 || codepoint > 0x10FFFF || (codepoint >= 0xD800 && codepoint <= 0xDFFF)) return; + if (codepoint <= 0x7F) { + output.push_back(static_cast(codepoint)); + } else if (codepoint <= 0x7FF) { + output.push_back(static_cast(0xC0 | (codepoint >> 6))); + output.push_back(static_cast(0x80 | (codepoint & 0x3F))); + } else if (codepoint <= 0xFFFF) { + output.push_back(static_cast(0xE0 | (codepoint >> 12))); + output.push_back(static_cast(0x80 | ((codepoint >> 6) & 0x3F))); + output.push_back(static_cast(0x80 | (codepoint & 0x3F))); + } else { + output.push_back(static_cast(0xF0 | (codepoint >> 18))); + output.push_back(static_cast(0x80 | ((codepoint >> 12) & 0x3F))); + output.push_back(static_cast(0x80 | ((codepoint >> 6) & 0x3F))); + output.push_back(static_cast(0x80 | (codepoint & 0x3F))); + } +} + +bool appendNumericEntity(std::string& output, const char* entity, size_t len) { + if (len < 4 || entity[0] != '&' || entity[1] != '#' || entity[len - 1] != ';') return false; + size_t pos = 2; + uint32_t base = 10; + if (entity[pos] == 'x' || entity[pos] == 'X') { + base = 16; + pos++; + } + if (pos == len - 1) return false; + + uint32_t value = 0; + for (; pos < len - 1; pos++) { + const unsigned char c = entity[pos]; + uint32_t digit; + if (c >= '0' && c <= '9') + digit = c - '0'; + else if (base == 16 && c >= 'a' && c <= 'f') + digit = c - 'a' + 10; + else if (base == 16 && c >= 'A' && c <= 'F') + digit = c - 'A' + 10; + else + return false; + if (value > (0x10FFFF - digit) / base) return false; + value = value * base + digit; + } + appendCodepoint(output, value); + return true; +} + +void appendBreak(std::string& output, size_t count = 1) { + while (!output.empty() && output.back() == ' ') output.pop_back(); + if (output.empty()) return; + size_t existing = 0; + while (existing < output.size() && output[output.size() - existing - 1] == '\n') existing++; + while (existing++ < count) output.push_back('\n'); +} + +} // namespace + +std::string htmlToPlainText(const std::string& html) { + std::string output; + output.reserve(html.size()); + + for (size_t i = 0; i < html.size();) { + if (html[i] == '<' && isTagStart(html, i)) { + const size_t close = html.find('>', i + 1); + if (close == std::string::npos) { + output.push_back(html[i++]); + continue; + } + const TagBreak separator = tagBreak(html, i + 1, close); + if (separator == TagBreak::Line) + appendBreak(output); + else if (separator == TagBreak::Paragraph) + appendBreak(output, 2); + i = close + 1; + continue; + } + + if (html[i] == '&') { + const size_t semicolon = html.find(';', i + 1); + if (semicolon != std::string::npos && semicolon - i <= 16) { + const size_t len = semicolon - i + 1; + const char* value = lookupHtmlEntity(html.data() + i, len); + if (value != nullptr) { + output.append(value); + i = semicolon + 1; + continue; + } + if (appendNumericEntity(output, html.data() + i, len)) { + i = semicolon + 1; + continue; + } + } + } + + const unsigned char c = html[i++]; + if (c == '\r' || c == '\t') { + if (!output.empty() && output.back() != ' ' && output.back() != '\n') output.push_back(' '); + } else if (c == '\n') { + appendBreak(output); + } else { + output.push_back(static_cast(c)); + } + } + + while (!output.empty() && (output.back() == ' ' || output.back() == '\n')) output.pop_back(); + return output; +} diff --git a/src/util/HtmlToPlainText.h b/src/util/HtmlToPlainText.h new file mode 100644 index 00000000..9f374712 --- /dev/null +++ b/src/util/HtmlToPlainText.h @@ -0,0 +1,7 @@ +#pragma once + +#include + +// Convert an HTML fragment to readable plain text. This intentionally ignores +// styling; block elements become line breaks and HTML entities are decoded. +std::string htmlToPlainText(const std::string& html); diff --git a/src/util/StringUtils.h b/src/util/StringUtils.h index 1fa6cc01..f82ec97e 100644 --- a/src/util/StringUtils.h +++ b/src/util/StringUtils.h @@ -1,9 +1,32 @@ #pragma once +#include #include namespace StringUtils { +/** + * Case-insensitive ASCII strcmp. Returns <0, 0, or >0 like strcmp, comparing + * each byte by its lowercased value. + * + * Used wherever data is sorted case-insensitively: StarDict indexes (including + * wiktionary-derived dictionaries) and the on-disk dictionary folder list. + * Plain strcmp would land a binary search on the wrong page for any word whose + * alphabetic neighbourhood contains mixed-case boundaries. + * + * Inline (header) because Dictionary's binary search calls it per comparison + * step; a cross-TU call here would defeat inlining on a hot path. + */ +inline int asciiCaseCmp(const char* a, const char* b) { + while (*a && *b) { + int diff = std::tolower(static_cast(*a)) - std::tolower(static_cast(*b)); + if (diff != 0) return diff; + ++a; + ++b; + } + return std::tolower(static_cast(*a)) - std::tolower(static_cast(*b)); +} + /** * Sanitize a string for use as a filename. * Replaces invalid characters with underscores, trims spaces/dots,