feat: Slim dictionary (#2583)
Co-authored-by: Ryan Hitchman <hitchmanr@gmail.com> Co-authored-by: Kurtis Grant <kurtis.b.grant@gmail.com> Co-authored-by: kemonine <kemonine@kemonine.info> Co-authored-by: DustinHu <hu.dustin@gmail.com> Co-authored-by: Justin Mitchell <justin@jmitch.com>
This commit is contained in:
Uri Tauber
co-authored by
Ryan Hitchman
Kurtis Grant
kemonine
DustinHu
Justin Mitchell
Ryan Hitchman
Kurtis Grant
kemonine
DustinHu
Justin Mitchell
parent
5ba1d5747f
commit
b1d1d757ba
@@ -12,7 +12,7 @@ CrossPoint is open-source e-reader firmware - community-built, fully hackable, f
|
||||
|
||||
## What can CrossPoint do?
|
||||
|
||||
- **Reader engine**: EPUB 2/3 rendering with embedded-style option, image handling, hyphenation, kerning, chapter navigation, footnotes, bookmarks, go-to-percent, auto page turn, orientation control, focus reading, KOReader progress sync and more.
|
||||
- **Reader engine**: EPUB 2/3 rendering with embedded-style option, image handling, hyphenation, kerning, chapter navigation, footnotes, bookmarks, dictionary lookups ([StarDict](docs/dictionary.md)), go-to-percent, auto page turn, orientation control, focus reading, KOReader progress sync and more.
|
||||
|
||||
- **Various formats**: native handling for `.epub`, `.xtc/.xtch`, `.txt`, and `.bmp`.
|
||||
|
||||
@@ -42,8 +42,6 @@ CrossPoint is open-source e-reader firmware - community-built, fully hackable, f
|
||||
|
||||
### Coming soon:
|
||||
|
||||
- Dictionary lookup — inline word lookup without leaving the reader.
|
||||
|
||||
- More themes.
|
||||
|
||||
- Much more! stay tuned.
|
||||
|
||||
+5
-1
@@ -263,6 +263,8 @@ The Settings screen allows you to configure the device's behavior. There are a f
|
||||
- "ON" - Vertical space will be added between paragraphs in Reading Mode
|
||||
- "OFF" - Paragraphs will not have vertical space added, but will have first-line indentation
|
||||
|
||||
- **Dictionary**: Select the StarDict dictionary used for word lookups while reading, or "None" to disable lookups. *(Only shown when at least one dictionary folder exists under `/dictionaries/` on the SD card — see [docs/dictionary.md](docs/dictionary.md) for setup and usage.)*
|
||||
|
||||
- **Text Anti-Aliasing**: Whether to show smooth grey edges (anti-aliasing) on text in reading mode. Note this slows down page turns slightly.
|
||||
|
||||
- **Images**: Whether to display embedded images (JPG/PNG) found in EPUB files; options are "ON" (default) or "OFF".
|
||||
@@ -282,6 +284,7 @@ The Settings screen allows you to configure the device's behavior. There are a f
|
||||
- **Long-press Menu**: Selects the function bound to holding the menu button (Confirm) while reading an EPUB. **Cycles through the available functions** each time the setting is selected — additional functions may be added in future releases, so this is not a binary on/off toggle. A short press of Confirm always opens the reader menu as normal:
|
||||
- "Bookmark" (default) - Hold Confirm (~0.4 second) to drop a bookmark at the current page.
|
||||
- "KOSync" - Hold Confirm (~1 second) to launch KOReader sync directly.
|
||||
- "Dictionary" - Hold Confirm (~0.4 second) to start dictionary word selection on the current page (see [docs/dictionary.md](docs/dictionary.md)).
|
||||
- "Disabled" - Long-press is ignored; only short-press opens the reader menu.
|
||||
|
||||
- **Short Power Button Click**: Controls the effect of a short click of the power button:
|
||||
@@ -583,7 +586,7 @@ If the device goes to sleep or you close the book while viewing a footnote, the
|
||||
* **Return to Home:** Press the **Back** button to close the book and return to the **[Home](#31-home-screen)** screen.
|
||||
* **Return to Browse Files:** Press and hold the **Back** button to close the book and return to the **[Browse Files](#33-browse-files-screen)** screen.
|
||||
* **Reader Menu:** Press **Confirm** to open the **[Reader Menu](#5-reader-menu)**, which includes chapter navigation, reading options, and more.
|
||||
* **Long-press Confirm (configurable):** Holding **Confirm** runs the function chosen by the **Long-press Menu** setting in **[Controls Settings](#363-controls)** — "Bookmark" (default) drops a bookmark, "KOSync" launches KOReader Sync, "Disabled" does nothing. A short press always opens the Reader Menu.
|
||||
* **Long-press Confirm (configurable):** Holding **Confirm** runs the function chosen by the **Long-press Menu** setting in **[Controls Settings](#363-controls)** — "Bookmark" (default) drops a bookmark, "KOSync" launches KOReader Sync, "Dictionary" starts a word lookup, "Disabled" does nothing. A short press always opens the Reader Menu.
|
||||
|
||||
### Supported Languages
|
||||
|
||||
@@ -605,6 +608,7 @@ Available options include:
|
||||
|
||||
- **Select Chapter** – Open the table of contents to jump to a specific chapter (see [Chapter Selection](#51-chapter-selection) below).
|
||||
- **Footnotes** – Navigate to the footnotes for the current section *(only shown in books that contain footnotes)*.
|
||||
- **Look Up** – Select a word on the current page and show its dictionary definition (see [docs/dictionary.md](docs/dictionary.md)). Requires a dictionary to be selected in **Settings → Reader → Dictionary**.
|
||||
- **Reading Orientation** – Cycle through screen orientations without leaving the reader.
|
||||
- **Auto Turn (Pages Per Minute)** – Cycle through automatic page turn speed options for hands-free reading.
|
||||
- **Go to %** – Jump to a specific position in the book by percentage.
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
# Dictionary
|
||||
|
||||
Look up words while reading an EPUB using an offline StarDict dictionary stored on the SD card.
|
||||
|
||||
## Supported Format
|
||||
|
||||
The reader supports **StarDict** dictionaries. When searching for dictionaries online, look for "StarDict format" or files with `.dict`, `.idx`, and `.ifo` extensions.
|
||||
|
||||
A dictionary folder must contain:
|
||||
|
||||
- `.idx` — word index (required, **must be uncompressed** — a `.idx.gz` will not work; decompress it on your computer with `gzip -d` first)
|
||||
- `.dict` or `.dict.dz` — definition data (`.dict.dz` is supported as-is; entries are decompressed on the fly during lookup)
|
||||
- `.ifo` — metadata (optional)
|
||||
|
||||
Not supported: `.syn` synonym files (ignored), dictionaries with 64-bit index offsets (`idxoffsetbits=64` in the `.ifo` — rare, and rejected with an error), and HTML-formatted definitions render as raw markup rather than styled text.
|
||||
|
||||
## Setting Up a Dictionary
|
||||
|
||||
1. Copy your dictionary folder(s) to `/dictionaries/` on the SD card — one dictionary per folder, e.g. `/dictionaries/webster/webster.idx` + `webster.dict.dz`. A hidden `/.dictionaries/` folder (dot-prefixed) works the same way, for keeping it out of the file browser.
|
||||
2. Open **Settings → Reader → Dictionary** on the device.
|
||||
3. Select a dictionary from the list, or **None** to disable lookups.
|
||||
|
||||
The Dictionary setting only appears when at least one usable dictionary folder exists. Folders containing more than one dictionary (multiple `.idx` stems) are skipped as ambiguous.
|
||||
|
||||
## Looking Up a Word
|
||||
|
||||
Two ways to start a lookup while reading:
|
||||
|
||||
- Open the reader menu (**Confirm**) and choose **Look Up**.
|
||||
- Or set **Settings → Controls → Long-press Menu** to "Dictionary", then hold **Confirm** (~0.4s) on the reading page.
|
||||
|
||||
One word on the page becomes highlighted:
|
||||
|
||||
1. Use **Left/Right** to move between words in reading order, and the side **Up/Down** buttons to jump between lines.
|
||||
2. Press **Confirm** to look up the highlighted word.
|
||||
3. Press **Back** to return to the reader.
|
||||
|
||||
On the very first lookup with a dictionary (and again if the dictionary file changes), the reader shows *"Indexing dictionary…"* while it builds a small `.qidx` sidecar file next to the `.idx`. This takes a few seconds for large dictionaries and makes all subsequent lookups fast. The sidecar can be deleted safely at any time — it will simply be rebuilt.
|
||||
|
||||
### How Lookup Works
|
||||
|
||||
1. **Direct match** — the word is found as-is (case-insensitive) in the dictionary index. Surrounding punctuation is ignored.
|
||||
2. **Stemming** — on a miss, common English word forms are retried automatically: possessives and plurals (`dogs` → `dog`, `stories` → `story`) and verb endings (`walked` → `walk`, `running` → `run`, `making` → `make`).
|
||||
3. **Not found** — a short popup appears and you return to word selection.
|
||||
|
||||
## The Definition Screen
|
||||
|
||||
When a word is found, the definition screen shows the matched headword at the top and the definition text below, with a page counter for long definitions.
|
||||
|
||||
- **Left/Right** or side **Up/Down** — previous / next page
|
||||
- **Back** — return to word selection
|
||||
@@ -213,6 +213,61 @@ static inline void rotateCoordinates(const GfxRenderer::Orientation orientation,
|
||||
}
|
||||
}
|
||||
|
||||
// Output of screenRectToAlignedMemRect: a rectangle in panel-memory
|
||||
// coordinates whose x and width are guaranteed to be multiples of 8 (the
|
||||
// SDK's EInkDisplay::displayWindow alignment requirement). `valid == false`
|
||||
// means the input was empty or fully outside the panel.
|
||||
struct AlignedMemRect {
|
||||
uint16_t x = 0;
|
||||
uint16_t y = 0;
|
||||
uint16_t w = 0;
|
||||
uint16_t h = 0;
|
||||
bool valid = false;
|
||||
};
|
||||
|
||||
// Translate a screen-coordinate rectangle (the coordinate system used by
|
||||
// fillRect / drawText / the rest of the renderer's public API) into a
|
||||
// panel-memory rectangle suitable for direct framebuffer indexing. Rotates
|
||||
// the rectangle's two opposite corners with rotateCoordinates(), takes the
|
||||
// bounding box (which naturally swaps width/height in Portrait /
|
||||
// PortraitInverted), then snaps the x extent outward to multiples of 8 and
|
||||
// clamps to panel bounds. Precondition: panel dims are multiples of 8 (true
|
||||
// for the 800x480 panel), so clamping cannot re-break alignment.
|
||||
static AlignedMemRect screenRectToAlignedMemRect(GfxRenderer::Orientation orientation, int sx, int sy, int sw, int sh,
|
||||
uint16_t panelWidth, uint16_t panelHeight) {
|
||||
AlignedMemRect out;
|
||||
if (sw <= 0 || sh <= 0) return out;
|
||||
|
||||
int x0, y0, x1, y1;
|
||||
rotateCoordinates(orientation, sx, sy, &x0, &y0, panelWidth, panelHeight);
|
||||
rotateCoordinates(orientation, sx + sw - 1, sy + sh - 1, &x1, &y1, panelWidth, panelHeight);
|
||||
|
||||
const int memXLo = std::min(x0, x1);
|
||||
const int memYLo = std::min(y0, y1);
|
||||
const int memXHi = std::max(x0, x1) + 1; // exclusive upper bound
|
||||
const int memYHi = std::max(y0, y1) + 1;
|
||||
|
||||
// Snap x outward to multiples of 8.
|
||||
int alignedXLo = memXLo & ~0x7; // round down
|
||||
int alignedXHi = (memXHi + 7) & ~0x7; // round up
|
||||
|
||||
if (alignedXLo < 0) alignedXLo = 0;
|
||||
if (alignedXHi > panelWidth) alignedXHi = panelWidth;
|
||||
int clampedYLo = memYLo;
|
||||
int clampedYHi = memYHi;
|
||||
if (clampedYLo < 0) clampedYLo = 0;
|
||||
if (clampedYHi > panelHeight) clampedYHi = panelHeight;
|
||||
|
||||
if (alignedXHi <= alignedXLo || clampedYHi <= clampedYLo) return out;
|
||||
|
||||
out.x = static_cast<uint16_t>(alignedXLo);
|
||||
out.y = static_cast<uint16_t>(clampedYLo);
|
||||
out.w = static_cast<uint16_t>(alignedXHi - alignedXLo);
|
||||
out.h = static_cast<uint16_t>(clampedYHi - clampedYLo);
|
||||
out.valid = true;
|
||||
return out;
|
||||
}
|
||||
|
||||
enum class TextRotation { None, Rotated90CW };
|
||||
|
||||
// Shared glyph rendering logic for normal and rotated text.
|
||||
@@ -1451,6 +1506,39 @@ void GfxRenderer::displayBuffer(const HalDisplay::RefreshMode refreshMode) const
|
||||
display.displayBuffer(refreshMode, fadingFix);
|
||||
}
|
||||
|
||||
size_t GfxRenderer::readFramebufferRegion(int x, int y, int w, int h, uint8_t* dst, size_t dstCapacity) const {
|
||||
if (dst == nullptr || w <= 0 || h <= 0) return 0;
|
||||
|
||||
const AlignedMemRect mem = screenRectToAlignedMemRect(orientation, x, y, w, h, panelWidth, panelHeight);
|
||||
if (!mem.valid) return 0;
|
||||
|
||||
const size_t rowBytes = mem.w / 8; // exact: mem.w is a multiple of 8
|
||||
const size_t needed = rowBytes * mem.h;
|
||||
if (needed > dstCapacity) return 0;
|
||||
|
||||
for (uint16_t row = 0; row < mem.h; ++row) {
|
||||
const uint8_t* srcRow = frameBuffer + (static_cast<uint32_t>(mem.y + row) * panelWidthBytes) + (mem.x / 8);
|
||||
uint8_t* dstRow = dst + (static_cast<size_t>(row) * rowBytes);
|
||||
memcpy(dstRow, srcRow, rowBytes);
|
||||
}
|
||||
return needed;
|
||||
}
|
||||
|
||||
void GfxRenderer::writeFramebufferRegion(int x, int y, int w, int h, const uint8_t* src) {
|
||||
if (src == nullptr || w <= 0 || h <= 0) return;
|
||||
|
||||
const AlignedMemRect mem = screenRectToAlignedMemRect(orientation, x, y, w, h, panelWidth, panelHeight);
|
||||
if (!mem.valid) return;
|
||||
|
||||
const size_t rowBytes = mem.w / 8; // exact: mem.w is a multiple of 8
|
||||
|
||||
for (uint16_t row = 0; row < mem.h; ++row) {
|
||||
const uint8_t* srcRow = src + (static_cast<size_t>(row) * rowBytes);
|
||||
uint8_t* dstRow = frameBuffer + (static_cast<uint32_t>(mem.y + row) * panelWidthBytes) + (mem.x / 8);
|
||||
memcpy(dstRow, srcRow, rowBytes);
|
||||
}
|
||||
}
|
||||
|
||||
std::string GfxRenderer::truncatedText(const int fontId, const char* text, const int maxWidth,
|
||||
const EpdFontFamily::Style style) const {
|
||||
if (!text || maxWidth <= 0) return "";
|
||||
|
||||
@@ -189,6 +189,15 @@ class GfxRenderer {
|
||||
void drawBitmap1Bit(const Bitmap& bitmap, int x, int y, int maxWidth, int maxHeight) const;
|
||||
void fillPolygon(const int* xPoints, const int* yPoints, int numPoints, bool state = true) const;
|
||||
|
||||
// Snapshot / restore a screen-coordinate framebuffer region (byte-aligned in
|
||||
// panel memory). readFramebufferRegion returns the bytes written to dst, or
|
||||
// 0 when the region is empty, offscreen, or exceeds dstCapacity. Pass the
|
||||
// same rectangle to writeFramebufferRegion to restore the saved pixels.
|
||||
// Enables partial-repaint patterns (e.g. moving a selection highlight)
|
||||
// without re-rendering the whole page.
|
||||
size_t readFramebufferRegion(int x, int y, int w, int h, uint8_t* dst, size_t dstCapacity) const;
|
||||
void writeFramebufferRegion(int x, int y, int w, int h, const uint8_t* src);
|
||||
|
||||
// Text
|
||||
int getTextWidth(int fontId, const char* text, EpdFontFamily::Style style = EpdFontFamily::REGULAR,
|
||||
BidiUtils::BidiBaseDir baseDir = BidiUtils::BidiBaseDir::AUTO) const;
|
||||
|
||||
@@ -26,6 +26,12 @@ STR_EMPTY_FILE: "Empty file"
|
||||
STR_OUT_OF_BOUNDS: "Out of bounds"
|
||||
STR_LOADING: "Loading..."
|
||||
STR_LOADING_POPUP: "Loading"
|
||||
STR_LOOKUP: "Look Up"
|
||||
STR_DICT_LOOKING_UP: "Looking up..."
|
||||
STR_DICT_INDEXING: "Indexing dictionary..."
|
||||
STR_DICT_NOT_FOUND: "Not found"
|
||||
STR_DICT_NO_DICT_SET: "No dictionary set"
|
||||
STR_DICT_ERROR: "Dictionary error"
|
||||
STR_WIFI_NETWORKS: "Wi-Fi Networks"
|
||||
STR_NO_NETWORKS: "No networks found"
|
||||
STR_NETWORKS_FOUND: "%zu networks found"
|
||||
@@ -163,6 +169,7 @@ STR_PREV_NEXT: "Prev/Next"
|
||||
STR_NEXT_PREV: "Next/Prev"
|
||||
STR_KOSYNC: "KOSync"
|
||||
STR_BOOKMARK_OPTION: "Bookmark"
|
||||
STR_DICTIONARY: "Dictionary"
|
||||
STR_DISABLED: "Disabled"
|
||||
STR_NOTO_SERIF: "Noto Serif"
|
||||
STR_NOTO_SANS: "Noto Sans"
|
||||
|
||||
@@ -153,6 +153,7 @@ class CrossPointSettings {
|
||||
LP_MENU_KOSYNC = 0,
|
||||
LP_MENU_DISABLED = 1,
|
||||
LP_MENU_BOOKMARK = 2,
|
||||
LP_MENU_DICTIONARY = 3,
|
||||
LONG_PRESS_MENU_FUNCTION_COUNT
|
||||
};
|
||||
|
||||
@@ -265,6 +266,8 @@ class CrossPointSettings {
|
||||
uint8_t focusReadingEnabled = 0;
|
||||
// SD card font family name (empty = use built-in fontFamily)
|
||||
char sdFontFamilyName[32] = "";
|
||||
// Dictionary folder name under /dictionaries (empty = no dictionary)
|
||||
char dictionaryName[32] = "";
|
||||
// Show hidden files/directories (starting with '.') in the file browser (0 = hidden, 1 = show)
|
||||
uint8_t showHiddenFiles = 0;
|
||||
// Remove a book from the Recent Books list when its End-of-Book screen is reached (0 = off, 1 = on)
|
||||
|
||||
@@ -149,6 +149,10 @@ bool JsonSettingsIO::saveSettings(const CrossPointSettings& s, const char* path)
|
||||
if (s.sdFontFamilyName[0] != '\0') {
|
||||
doc["sdFontFamilyName"] = s.sdFontFamilyName;
|
||||
}
|
||||
// Dictionary folder name — uses dynamic getter/setter in SettingsList, save manually
|
||||
if (s.dictionaryName[0] != '\0') {
|
||||
doc["dictionaryName"] = s.dictionaryName;
|
||||
}
|
||||
|
||||
// Language -- managed by LanguageSelectActivity, not in SettingsList.
|
||||
// Stored as ISO code string ("EN", "DE", ...) for stability across enum reorders.
|
||||
@@ -256,6 +260,11 @@ bool JsonSettingsIO::loadSettings(CrossPointSettings& s, const char* json, bool*
|
||||
if (needsResave) *needsResave = true;
|
||||
}
|
||||
|
||||
// Dictionary folder name — uses dynamic getter/setter in SettingsList, load manually
|
||||
const char* dictName = doc["dictionaryName"] | "";
|
||||
strncpy(s.dictionaryName, dictName, sizeof(s.dictionaryName) - 1);
|
||||
s.dictionaryName[sizeof(s.dictionaryName) - 1] = '\0';
|
||||
|
||||
// Language -- stored as code string for stability across enum reorders.
|
||||
if (doc["language"].is<const char*>()) {
|
||||
s.language = static_cast<uint8_t>(I18n::languageFromCode(doc["language"].as<const char*>()));
|
||||
|
||||
+52
-3
@@ -13,6 +13,7 @@
|
||||
#include "CrossPointSettings.h"
|
||||
#include "KOReaderCredentialStore.h"
|
||||
#include "activities/settings/SettingsActivity.h"
|
||||
#include "util/DictionaryRegistry.h"
|
||||
|
||||
// Build the font family setting dynamically. When registry is non-null, SD card fonts
|
||||
// are appended after the built-in fonts. Otherwise only built-in fonts are listed.
|
||||
@@ -90,6 +91,47 @@ inline SettingInfo buildFontFamilySetting(const SdCardFontRegistry* registry) {
|
||||
return s;
|
||||
}
|
||||
|
||||
// Build the dictionary selection setting dynamically from the folders discovered
|
||||
// under /dictionaries. "None" plus one option per dictionary; the selected folder
|
||||
// name persists in SETTINGS.dictionaryName (saved/loaded manually in
|
||||
// JsonSettingsIO — the generic loop skips dynamic entries).
|
||||
inline SettingInfo buildDictionarySetting(const std::vector<DictionaryEntry>& dictionaries) {
|
||||
std::vector<std::string> folderNames;
|
||||
folderNames.reserve(dictionaries.size());
|
||||
std::transform(dictionaries.begin(), dictionaries.end(), std::back_inserter(folderNames),
|
||||
[](const DictionaryEntry& d) { return d.name; });
|
||||
|
||||
SettingInfo s;
|
||||
s.nameId = StrId::STR_DICTIONARY;
|
||||
s.type = SettingType::ENUM;
|
||||
s.enumStringValues.reserve(folderNames.size() + 1);
|
||||
s.enumStringValues.push_back(I18N.get(StrId::STR_NONE_OPT));
|
||||
s.enumStringValues.insert(s.enumStringValues.end(), folderNames.begin(), folderNames.end());
|
||||
s.category = StrId::STR_CAT_READER;
|
||||
|
||||
s.valueGetter = [folderNames]() -> uint8_t {
|
||||
for (size_t i = 0; i < folderNames.size(); i++) {
|
||||
// Compare within the settings field capacity: an over-long folder name is
|
||||
// stored truncated, and must still match its list entry.
|
||||
if (strncmp(folderNames[i].c_str(), SETTINGS.dictionaryName, sizeof(SETTINGS.dictionaryName) - 1) == 0) {
|
||||
return static_cast<uint8_t>(i + 1);
|
||||
}
|
||||
}
|
||||
return 0; // "None", also when the stored folder no longer exists
|
||||
};
|
||||
|
||||
s.valueSetter = [folderNames](uint8_t v) {
|
||||
if (v == 0 || v > folderNames.size()) {
|
||||
SETTINGS.dictionaryName[0] = '\0';
|
||||
return;
|
||||
}
|
||||
strncpy(SETTINGS.dictionaryName, folderNames[v - 1].c_str(), sizeof(SETTINGS.dictionaryName) - 1);
|
||||
SETTINGS.dictionaryName[sizeof(SETTINGS.dictionaryName) - 1] = '\0';
|
||||
};
|
||||
|
||||
return s;
|
||||
}
|
||||
|
||||
// Shared settings list used by both the device settings UI and the web settings API.
|
||||
// Each entry has a key (for JSON API) and category (for grouping).
|
||||
// ACTION-type entries and entries without a key are device-only.
|
||||
@@ -99,7 +141,8 @@ inline SettingInfo buildFontFamilySetting(const SdCardFontRegistry* registry) {
|
||||
// SdCardFontRegistry is supplied AND has SD card fonts installed, the
|
||||
// font-family entry is replaced in a per-call copy with a registry-aware
|
||||
// version. Callers without SD fonts pay only a vector copy.
|
||||
inline std::vector<SettingInfo> getSettingsList(const SdCardFontRegistry* registry = nullptr) {
|
||||
inline std::vector<SettingInfo> getSettingsList(const SdCardFontRegistry* registry = nullptr,
|
||||
const std::vector<DictionaryEntry>* dictionaries = nullptr) {
|
||||
static const std::vector<SettingInfo> baseList = [] {
|
||||
std::vector<SettingInfo> v = {
|
||||
// --- Display ---
|
||||
@@ -173,8 +216,8 @@ inline std::vector<SettingInfo> getSettingsList(const SdCardFontRegistry* regist
|
||||
StrId::STR_LONG_PRESS_BEHAVIOR_ORIENTATION},
|
||||
"longPressButtonBehavior", StrId::STR_CAT_CONTROLS),
|
||||
SettingInfo::Enum(StrId::STR_LONG_PRESS_MENU, &CrossPointSettings::longPressMenuFunction,
|
||||
{StrId::STR_KOSYNC, StrId::STR_DISABLED, StrId::STR_BOOKMARK_OPTION}, "longPressMenuFunction",
|
||||
StrId::STR_CAT_CONTROLS),
|
||||
{StrId::STR_KOSYNC, StrId::STR_DISABLED, StrId::STR_BOOKMARK_OPTION, StrId::STR_DICTIONARY},
|
||||
"longPressMenuFunction", StrId::STR_CAT_CONTROLS),
|
||||
SettingInfo::Enum(
|
||||
StrId::STR_SHORT_PWR_BTN, &CrossPointSettings::shortPwrBtn,
|
||||
{StrId::STR_IGNORE, StrId::STR_SLEEP, StrId::STR_PAGE_TURN, StrId::STR_FORCE_REFRESH, StrId::STR_FOOTNOTES},
|
||||
@@ -308,5 +351,11 @@ inline std::vector<SettingInfo> getSettingsList(const SdCardFontRegistry* regist
|
||||
*it = buildFontFamilySetting(registry);
|
||||
}
|
||||
}
|
||||
if (dictionaries && !dictionaries->empty()) {
|
||||
// Insert at the end of the Reader category (just before the first Controls entry).
|
||||
auto it =
|
||||
std::find_if(v.begin(), v.end(), [](const SettingInfo& s) { return s.category == StrId::STR_CAT_CONTROLS; });
|
||||
v.insert(it, buildDictionarySetting(*dictionaries));
|
||||
}
|
||||
return v;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,237 @@
|
||||
#include "DictionaryDefinitionActivity.h"
|
||||
|
||||
#include <FontCacheManager.h>
|
||||
#include <GfxRenderer.h>
|
||||
#include <I18n.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cstdint>
|
||||
#include <cstdio>
|
||||
|
||||
#include "CrossPointSettings.h"
|
||||
#include "components/UITheme.h"
|
||||
#include "fontIds.h"
|
||||
#include "util/HtmlToPlainText.h"
|
||||
|
||||
namespace {
|
||||
|
||||
// Longest measurable/drawable span. Wrapped lines stay under the screen width
|
||||
// (far below this); only pathological unbreakable tokens are split at this cap.
|
||||
constexpr size_t MAX_LINE_BYTES = 191;
|
||||
|
||||
// Body text left/right inset, matching the reader's default feel.
|
||||
constexpr int SIDE_PADDING = 20;
|
||||
|
||||
} // namespace
|
||||
|
||||
void DictionaryDefinitionActivity::onEnter() {
|
||||
Activity::onEnter();
|
||||
// Normalize StarDict multi-type separators so the wrap loop and the
|
||||
// C-string font APIs below both see the whole definition.
|
||||
std::replace(definition.begin(), definition.end(), '\0', '\n');
|
||||
definition = htmlToPlainText(definition);
|
||||
wrapText();
|
||||
requestUpdate();
|
||||
}
|
||||
|
||||
int DictionaryDefinitionActivity::measureSpan(const int fontId, const char* text, size_t len) const {
|
||||
char buf[MAX_LINE_BYTES + 1];
|
||||
len = std::min(len, MAX_LINE_BYTES);
|
||||
memcpy(buf, text, len);
|
||||
buf[len] = '\0';
|
||||
return renderer.getTextAdvanceX(fontId, buf, EpdFontFamily::REGULAR);
|
||||
}
|
||||
|
||||
// Greedy word-wrap of `definition` into byte spans. '\n' breaks lines (blank
|
||||
// lines survive as paragraph spacing; NULs from multi-type StarDict entries
|
||||
// were normalized to newlines in onEnter); '\r' is dropped by treating it as
|
||||
// a space at a token edge.
|
||||
void DictionaryDefinitionActivity::wrapText() {
|
||||
lines.clear();
|
||||
lines.reserve(definition.size() / 32 + 8);
|
||||
|
||||
const int fontId = SETTINGS.getReaderFontId();
|
||||
// SD-card fonts: merge every definition codepoint into the persistent
|
||||
// advance table up front. Otherwise each unseen codepoint measured below
|
||||
// falls back to an on-demand glyph load from SD (8-slot overflow ring).
|
||||
renderer.ensureSdCardFontReady(fontId, definition.c_str(), 0x01 /* REGULAR */);
|
||||
|
||||
const auto& metrics = UITheme::getInstance().getMetrics();
|
||||
const auto orientation = renderer.getOrientation();
|
||||
const bool isLandscape = orientation == GfxRenderer::Orientation::LandscapeClockwise ||
|
||||
orientation == GfxRenderer::Orientation::LandscapeCounterClockwise;
|
||||
const bool isInverted = orientation == GfxRenderer::Orientation::PortraitInverted;
|
||||
const int hintGutterWidth = isLandscape ? metrics.sideButtonHintsWidth : 0;
|
||||
const int maxWidth = renderer.getScreenWidth() - hintGutterWidth - 2 * SIDE_PADDING;
|
||||
const int spaceWidth = renderer.getSpaceWidth(fontId, EpdFontFamily::REGULAR);
|
||||
|
||||
const int lineHeight = renderer.getLineHeight(fontId);
|
||||
const int topArea = (isInverted ? metrics.buttonHintsHeight : 0) + metrics.topPadding + metrics.headerHeight;
|
||||
const int bottomArea = metrics.buttonHintsHeight + metrics.verticalSpacing;
|
||||
linesPerPage = std::max(1, (renderer.getScreenHeight() - topArea - bottomArea) / lineHeight);
|
||||
|
||||
const char* text = definition.c_str();
|
||||
const uint32_t n = static_cast<uint32_t>(definition.size());
|
||||
uint32_t lineStart = 0;
|
||||
uint32_t lineEnd = 0; // one past the last token byte on the current line
|
||||
int lineWidth = 0;
|
||||
|
||||
const auto flushLine = [&](uint32_t nextStart) {
|
||||
lines.push_back({lineStart, static_cast<uint16_t>(lineEnd - lineStart)});
|
||||
lineStart = nextStart;
|
||||
lineEnd = nextStart;
|
||||
lineWidth = 0;
|
||||
};
|
||||
|
||||
uint32_t i = 0;
|
||||
while (i < n) {
|
||||
const char c = text[i];
|
||||
if (c == '\n' || c == '\0') {
|
||||
flushLine(i + 1);
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
if (c == ' ' || c == '\t' || c == '\r') {
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Token: run of non-whitespace bytes, capped at the measure buffer.
|
||||
const uint32_t tokenStart = i;
|
||||
while (i < n && text[i] != ' ' && text[i] != '\t' && text[i] != '\r' && text[i] != '\n' && text[i] != '\0' &&
|
||||
i - tokenStart < MAX_LINE_BYTES) {
|
||||
i++;
|
||||
}
|
||||
// If the byte cap cut the token mid-UTF-8-sequence, back off to the last
|
||||
// complete codepoint so measure/draw never see a partial sequence. A
|
||||
// natural stop lands on whitespace or the terminating NUL, never on a
|
||||
// continuation byte, so this is a no-op there.
|
||||
while (i - tokenStart > 1 && (text[i] & 0xC0) == 0x80) i--;
|
||||
const uint32_t tokenLen = i - tokenStart;
|
||||
const int tokenWidth = measureSpan(fontId, text + tokenStart, tokenLen);
|
||||
|
||||
if (lineEnd == lineStart) {
|
||||
lineStart = tokenStart;
|
||||
lineEnd = tokenStart + tokenLen;
|
||||
lineWidth = tokenWidth;
|
||||
} else if (lineWidth + spaceWidth + tokenWidth <= maxWidth &&
|
||||
tokenStart + tokenLen - lineStart <= UINT16_MAX) { // span len must fit Line::len
|
||||
lineEnd = tokenStart + tokenLen;
|
||||
lineWidth += spaceWidth + tokenWidth;
|
||||
} else {
|
||||
flushLine(tokenStart);
|
||||
lineEnd = tokenStart + tokenLen;
|
||||
lineWidth = tokenWidth;
|
||||
}
|
||||
|
||||
// An unbreakable token wider than the screen is now alone on the line
|
||||
// (any previous content was flushed above): split it at the widest
|
||||
// fitting UTF-8 boundary and carry the remainder forward.
|
||||
while (lineWidth > maxWidth && lineEnd - lineStart > 1) {
|
||||
const uint32_t len = lineEnd - lineStart;
|
||||
uint32_t lastFit = 0;
|
||||
for (uint32_t f = 1; f <= len; f++) {
|
||||
if (f == len || (text[lineStart + f] & 0xC0) != 0x80) { // codepoint boundary
|
||||
if (measureSpan(fontId, text + lineStart, f) > maxWidth) break;
|
||||
lastFit = f;
|
||||
}
|
||||
}
|
||||
if (lastFit == 0) {
|
||||
// Even a single over-wide glyph must make progress; consume its whole
|
||||
// UTF-8 sequence rather than splitting it into invalid fragments.
|
||||
lastFit = 1;
|
||||
while (lastFit < len && (text[lineStart + lastFit] & 0xC0) == 0x80) lastFit++;
|
||||
}
|
||||
const uint32_t rest = lineStart + lastFit;
|
||||
lineEnd = rest;
|
||||
flushLine(rest);
|
||||
lineEnd = rest + (len - lastFit);
|
||||
lineWidth = measureSpan(fontId, text + lineStart, lineEnd - lineStart);
|
||||
}
|
||||
}
|
||||
if (lineEnd > lineStart) flushLine(n);
|
||||
|
||||
// Trim trailing blank lines so the last page is not empty padding.
|
||||
while (!lines.empty() && lines.back().len == 0) lines.pop_back();
|
||||
|
||||
totalPages = std::max(1, (static_cast<int>(lines.size()) + linesPerPage - 1) / linesPerPage);
|
||||
currentPage = 0;
|
||||
}
|
||||
|
||||
void DictionaryDefinitionActivity::loop() {
|
||||
if (mappedInput.wasReleased(MappedInputManager::Button::Back)) {
|
||||
finish();
|
||||
return;
|
||||
}
|
||||
|
||||
buttonNavigator.onNext([this] {
|
||||
if (currentPage + 1 < totalPages) {
|
||||
currentPage++;
|
||||
requestUpdate();
|
||||
}
|
||||
});
|
||||
|
||||
buttonNavigator.onPrevious([this] {
|
||||
if (currentPage > 0) {
|
||||
currentPage--;
|
||||
requestUpdate();
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Draws the current page's line spans (copied into a stack buffer for NUL
|
||||
// termination). Called twice per render: once in font-cache scan mode, once
|
||||
// for the real paint.
|
||||
void DictionaryDefinitionActivity::drawBody(const int fontId, const int x, const int startY) const {
|
||||
const int lineHeight = renderer.getLineHeight(fontId);
|
||||
char buf[MAX_LINE_BYTES + 1];
|
||||
const int firstLine = currentPage * linesPerPage;
|
||||
const int lastLine = std::min(firstLine + linesPerPage, static_cast<int>(lines.size()));
|
||||
for (int i = firstLine; i < lastLine; i++) {
|
||||
if (lines[i].len == 0) continue;
|
||||
const size_t len = std::min(static_cast<size_t>(lines[i].len), MAX_LINE_BYTES);
|
||||
memcpy(buf, definition.c_str() + lines[i].start, len);
|
||||
buf[len] = '\0';
|
||||
renderer.drawText(fontId, x, startY + (i - firstLine) * lineHeight, buf);
|
||||
}
|
||||
}
|
||||
|
||||
void DictionaryDefinitionActivity::render(RenderLock&&) {
|
||||
renderer.clearScreen();
|
||||
|
||||
const auto& metrics = UITheme::getInstance().getMetrics();
|
||||
const auto orientation = renderer.getOrientation();
|
||||
const bool isLandscapeCw = orientation == GfxRenderer::Orientation::LandscapeClockwise;
|
||||
const bool isLandscapeCcw = orientation == GfxRenderer::Orientation::LandscapeCounterClockwise;
|
||||
const bool isInverted = orientation == GfxRenderer::Orientation::PortraitInverted;
|
||||
const int hintGutterWidth = (isLandscapeCw || isLandscapeCcw) ? metrics.sideButtonHintsWidth : 0;
|
||||
const int contentX = isLandscapeCw ? hintGutterWidth : 0;
|
||||
const int contentWidth = renderer.getScreenWidth() - hintGutterWidth;
|
||||
const int contentY = isInverted ? metrics.buttonHintsHeight : 0;
|
||||
|
||||
// Header: matched headword left, page counter right.
|
||||
const int headerY = contentY + metrics.topPadding + 10;
|
||||
renderer.drawText(UI_12_FONT_ID, contentX + SIDE_PADDING, headerY, headword.c_str(), true, EpdFontFamily::BOLD);
|
||||
if (totalPages > 1) {
|
||||
char counter[16];
|
||||
snprintf(counter, sizeof(counter), "%d/%d", currentPage + 1, totalPages);
|
||||
const int counterWidth = renderer.getTextWidth(UI_10_FONT_ID, counter);
|
||||
renderer.drawText(UI_10_FONT_ID, contentX + contentWidth - SIDE_PADDING - counterWidth, headerY, counter);
|
||||
}
|
||||
|
||||
// Body: two-pass draw inside a prewarm scope (same pattern as the reader's
|
||||
// renderContents) so SD-card font glyphs load from SD in one batch instead
|
||||
// of one on-demand overflow read per character on every page turn.
|
||||
const int fontId = SETTINGS.getReaderFontId();
|
||||
const int bodyStartY = contentY + metrics.topPadding + metrics.headerHeight;
|
||||
auto* fcm = renderer.getFontCacheManager();
|
||||
auto scope = fcm->createPrewarmScope();
|
||||
drawBody(fontId, contentX + SIDE_PADDING, bodyStartY); // scan pass: records codepoints only
|
||||
scope.endScanAndPrewarm();
|
||||
drawBody(fontId, contentX + SIDE_PADDING, bodyStartY);
|
||||
|
||||
const auto labels =
|
||||
mappedInput.mapLabels(tr(STR_BACK), "", (currentPage > 0 ? "<" : ""), (currentPage + 1 < totalPages ? ">" : ""));
|
||||
GUI.drawButtonHints(renderer, labels.btn1, labels.btn2, labels.btn3, labels.btn4);
|
||||
renderer.displayBuffer();
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "activities/Activity.h"
|
||||
#include "util/ButtonNavigator.h"
|
||||
|
||||
// Paged plain-text viewer for one dictionary definition. The definition is
|
||||
// word-wrapped once on entry; each page renders spans of the original string,
|
||||
// so no per-line copies are held.
|
||||
class DictionaryDefinitionActivity final : public Activity {
|
||||
public:
|
||||
explicit DictionaryDefinitionActivity(GfxRenderer& renderer, MappedInputManager& mappedInput, std::string headword,
|
||||
std::string definition)
|
||||
: Activity("DictionaryDefinition", renderer, mappedInput),
|
||||
headword(std::move(headword)),
|
||||
definition(std::move(definition)) {}
|
||||
|
||||
void onEnter() override;
|
||||
void loop() override;
|
||||
void render(RenderLock&&) override;
|
||||
|
||||
private:
|
||||
// One wrapped display line: a byte span of `definition`. Wrapping keeps
|
||||
// lines under the screen width, so uint16_t length is ample.
|
||||
struct Line {
|
||||
uint32_t start;
|
||||
uint16_t len;
|
||||
};
|
||||
|
||||
void wrapText();
|
||||
int measureSpan(int fontId, const char* text, size_t len) const;
|
||||
void drawBody(int fontId, int x, int startY) const;
|
||||
|
||||
const std::string headword;
|
||||
// Not const: onEnter() normalizes embedded NULs (StarDict multi-type
|
||||
// separators) to newlines so C-string APIs see the whole text.
|
||||
std::string definition;
|
||||
std::vector<Line> lines;
|
||||
int currentPage = 0;
|
||||
int totalPages = 1;
|
||||
int linesPerPage = 1;
|
||||
ButtonNavigator buttonNavigator;
|
||||
};
|
||||
@@ -0,0 +1,301 @@
|
||||
#include "DictionaryWordSelectActivity.h"
|
||||
|
||||
#include <FontCacheManager.h>
|
||||
#include <GfxRenderer.h>
|
||||
#include <Memory.h>
|
||||
#include <freertos/FreeRTOS.h>
|
||||
#include <freertos/task.h>
|
||||
|
||||
#include <cctype>
|
||||
#include <climits>
|
||||
#include <cstdlib>
|
||||
|
||||
#include "CrossPointSettings.h"
|
||||
#include "DictionaryDefinitionActivity.h"
|
||||
#include "components/UITheme.h"
|
||||
|
||||
namespace {
|
||||
|
||||
constexpr unsigned long POPUP_DURATION_MS = 1500;
|
||||
|
||||
// A token is selectable when it has an ASCII alphanumeric or a non-ASCII
|
||||
// codepoint outside U+2000-U+206F (dashes, bullets and other General
|
||||
// Punctuation that appear as standalone tokens are not words).
|
||||
bool isSelectableToken(const char* text) {
|
||||
for (const uint8_t* p = reinterpret_cast<const uint8_t*>(text); *p != 0; p++) {
|
||||
if (*p < 0x80) {
|
||||
if (std::isalnum(*p)) return true;
|
||||
} else if (*p == 0xE2 && (p[1] == 0x80 || p[1] == 0x81)) {
|
||||
if (p[2] == 0) break; // truncated sequence: skipping would step past the NUL
|
||||
p += 2; // skip the 3-byte General Punctuation codepoint
|
||||
} else {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
void indexBuildYield(void*) { vTaskDelay(1); }
|
||||
|
||||
} // namespace
|
||||
|
||||
void DictionaryWordSelectActivity::onEnter() {
|
||||
Activity::onEnter();
|
||||
fontId = SETTINGS.getReaderFontId();
|
||||
lineHeight = renderer.getLineHeight(fontId);
|
||||
// No null check: a failed allocation just disables the differential
|
||||
// fast path (drawHighlightWithSnapshot skips the read), keeping the
|
||||
// full-repaint path as the fallback.
|
||||
snapshot = makeUniqueNoThrow<uint8_t[]>(SNAPSHOT_CAPACITY);
|
||||
extractWords();
|
||||
// Start on the middle row's word nearest mid-screen instead of top-left:
|
||||
// any word on the page is then at most half a page of moves away.
|
||||
if (!words.empty()) {
|
||||
const int initial = closestInRow(rowCount / 2, renderer.getScreenWidth() / 2);
|
||||
if (initial >= 0) selected = initial;
|
||||
}
|
||||
requestUpdate();
|
||||
}
|
||||
|
||||
void DictionaryWordSelectActivity::extractWords() {
|
||||
words.clear();
|
||||
words.reserve(128);
|
||||
rowCount = 0;
|
||||
|
||||
// Single walk: collect the selectable words while accumulating their text
|
||||
// and styles (~2KB transient string, freed on return). Widths are measured
|
||||
// afterwards: merging the page's codepoints into the SD font's persistent
|
||||
// advance table first keeps getTextAdvanceX on the in-RAM path instead of
|
||||
// loading glyphs from SD one overflow slot at a time.
|
||||
std::string pageText;
|
||||
pageText.reserve(2048);
|
||||
uint8_t styleMask = 0;
|
||||
|
||||
for (const auto& element : page->elements) {
|
||||
if (element->getTag() != TAG_PageLine) continue;
|
||||
const auto* line = static_cast<const PageLine*>(element.get());
|
||||
const auto& block = line->getBlock();
|
||||
if (!block || !block->valid()) continue;
|
||||
|
||||
bool rowHasWords = false;
|
||||
for (uint16_t i = 0; i < block->wordCount(); i++) {
|
||||
const char* text = block->wordText(i);
|
||||
if (!isSelectableToken(text)) continue;
|
||||
|
||||
WordBox box;
|
||||
box.x = static_cast<int16_t>(line->xPos + block->wordXpos(i) + marginLeft);
|
||||
box.y = static_cast<int16_t>(line->yPos + marginTop);
|
||||
box.style = block->wordStyle(i);
|
||||
box.width = 0; // measured below, once the advance table is ready
|
||||
box.row = rowCount;
|
||||
box.text = text;
|
||||
words.push_back(box);
|
||||
rowHasWords = true;
|
||||
|
||||
pageText.append(text);
|
||||
pageText.push_back(' ');
|
||||
styleMask |= static_cast<uint8_t>(1u << (static_cast<uint8_t>(box.style) & 0x03));
|
||||
}
|
||||
if (rowHasWords) rowCount++;
|
||||
}
|
||||
|
||||
if (styleMask == 0) styleMask = 0x01; // REGULAR
|
||||
renderer.ensureSdCardFontReady(fontId, pageText.c_str(), styleMask);
|
||||
for (auto& word : words) {
|
||||
word.width = static_cast<int16_t>(renderer.getTextAdvanceX(fontId, word.text, word.style));
|
||||
}
|
||||
}
|
||||
|
||||
// Index of the word in `row` whose horizontal center is closest to centerX;
|
||||
// -1 when the row has no words.
|
||||
int DictionaryWordSelectActivity::closestInRow(const uint16_t row, const int centerX) const {
|
||||
int best = -1;
|
||||
int bestDistance = INT_MAX;
|
||||
for (int i = 0; i < static_cast<int>(words.size()); i++) {
|
||||
if (words[i].row != row) continue;
|
||||
const int distance = std::abs(words[i].x + words[i].width / 2 - centerX);
|
||||
if (distance < bestDistance) {
|
||||
bestDistance = distance;
|
||||
best = i;
|
||||
}
|
||||
}
|
||||
return best;
|
||||
}
|
||||
|
||||
void DictionaryWordSelectActivity::moveVertical(const int direction) {
|
||||
const WordBox& current = words[selected];
|
||||
const int targetRow = static_cast<int>(current.row) + direction;
|
||||
if (targetRow < 0 || targetRow >= static_cast<int>(rowCount)) return;
|
||||
|
||||
const int best = closestInRow(static_cast<uint16_t>(targetRow), current.x + current.width / 2);
|
||||
if (best >= 0 && best != selected) {
|
||||
selected = best;
|
||||
requestUpdate();
|
||||
}
|
||||
}
|
||||
|
||||
void DictionaryWordSelectActivity::performLookup() {
|
||||
popup = Popup::Busy;
|
||||
if (!dictOpenAttempted) {
|
||||
dictOpenAttempted = true;
|
||||
dictOpenOk = dict.open(SETTINGS.dictionaryName);
|
||||
}
|
||||
const bool indexing = dictOpenOk && dict.needsIndex();
|
||||
popupMsg = indexing ? StrId::STR_DICT_INDEXING : StrId::STR_DICT_LOOKING_UP;
|
||||
requestUpdateAndWait(); // paint the page + busy popup before blocking on SD
|
||||
|
||||
bool ok = dictOpenOk;
|
||||
if (ok && indexing) ok = dict.buildIndex(&indexBuildYield);
|
||||
|
||||
std::string definition;
|
||||
std::string headword;
|
||||
const bool found = ok && dict.lookup(words[selected].text, definition, headword);
|
||||
|
||||
if (found) {
|
||||
popup = Popup::None;
|
||||
startActivityForResult(std::make_unique<DictionaryDefinitionActivity>(renderer, mappedInput, std::move(headword),
|
||||
std::move(definition)),
|
||||
[this](const ActivityResult&) { requestUpdate(); });
|
||||
return;
|
||||
}
|
||||
popup = ok ? Popup::NotFound : Popup::Error;
|
||||
popupMsg = ok ? StrId::STR_DICT_NOT_FOUND : StrId::STR_DICT_ERROR;
|
||||
popupTime = millis();
|
||||
requestUpdate();
|
||||
}
|
||||
|
||||
void DictionaryWordSelectActivity::loop() {
|
||||
if (popup == Popup::NotFound || popup == Popup::Error) {
|
||||
if (millis() - popupTime >= POPUP_DURATION_MS) {
|
||||
popup = Popup::None;
|
||||
requestUpdate();
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
if (mappedInput.wasPressed(MappedInputManager::Button::Confirm)) confirmPressSeen = true;
|
||||
|
||||
if (mappedInput.wasReleased(MappedInputManager::Button::Back)) {
|
||||
finish();
|
||||
return;
|
||||
}
|
||||
if (mappedInput.wasReleased(MappedInputManager::Button::Confirm) && confirmPressSeen && !words.empty()) {
|
||||
performLookup();
|
||||
return;
|
||||
}
|
||||
|
||||
if (words.empty()) return;
|
||||
if (mappedInput.wasPressed(MappedInputManager::Button::Left) && selected > 0) {
|
||||
selected--;
|
||||
requestUpdate();
|
||||
} else if (mappedInput.wasPressed(MappedInputManager::Button::Right) &&
|
||||
selected + 1 < static_cast<int>(words.size())) {
|
||||
selected++;
|
||||
requestUpdate();
|
||||
} else if (mappedInput.wasPressed(MappedInputManager::Button::Up)) {
|
||||
moveVertical(-1);
|
||||
} else if (mappedInput.wasPressed(MappedInputManager::Button::Down)) {
|
||||
moveVertical(1);
|
||||
}
|
||||
}
|
||||
|
||||
// Saves the pixels under words[selected]'s highlight box, then draws the
|
||||
// highlight over them. Returns false when the pixels could not be saved
|
||||
// (no buffer / oversize box) — the highlight is drawn regardless, but the
|
||||
// next cursor move must do a full repaint.
|
||||
bool DictionaryWordSelectActivity::drawHighlightWithSnapshot() {
|
||||
const WordBox& word = words[selected];
|
||||
int hx = word.x - 2;
|
||||
int hy = word.y - 2;
|
||||
int hw = word.width + 4;
|
||||
int hh = lineHeight + 4;
|
||||
// Clamp to the panel so save, draw and restore all use the same box.
|
||||
if (hx < 0) {
|
||||
hw += hx;
|
||||
hx = 0;
|
||||
}
|
||||
if (hy < 0) {
|
||||
hh += hy;
|
||||
hy = 0;
|
||||
}
|
||||
|
||||
bool saved = false;
|
||||
if (snapshot && hw > 0 && hh > 0) {
|
||||
saved = renderer.readFramebufferRegion(hx, hy, hw, hh, snapshot.get(), SNAPSHOT_CAPACITY) > 0;
|
||||
}
|
||||
snapshotX = static_cast<int16_t>(hx);
|
||||
snapshotY = static_cast<int16_t>(hy);
|
||||
snapshotW = static_cast<int16_t>(hw);
|
||||
snapshotH = static_cast<int16_t>(hh);
|
||||
snapshotIdx = saved ? selected : -1;
|
||||
|
||||
renderer.fillRect(hx, hy, hw, hh, true);
|
||||
renderer.drawText(fontId, word.x, word.y, word.text, false, word.style);
|
||||
return saved;
|
||||
}
|
||||
|
||||
// Front-button bar (Back/Confirm/Left/Right). Drawn last on every repaint
|
||||
// path, including the differential highlight-only path, so it always ends
|
||||
// up as the top layer even when a highlighted word's box falls under a
|
||||
// hint's screen area. No side-button hints: Up/Down row jump has no spare
|
||||
// screen area on this page (it reuses the reader's full-bleed layout), and
|
||||
// a hint box there would hide text instead of sitting in a reserved gutter.
|
||||
void DictionaryWordSelectActivity::drawHints() const {
|
||||
// No selectable word on this page: Confirm/Left/Right are all no-ops
|
||||
// (guarded by words.empty() in loop()/performLookup), so only Back does
|
||||
// anything and only Back is hinted.
|
||||
if (words.empty()) {
|
||||
const auto labels = mappedInput.mapLabels(tr(STR_BACK), "", "", "");
|
||||
GUI.drawButtonHints(renderer, labels.btn1, labels.btn2, labels.btn3, labels.btn4);
|
||||
return;
|
||||
}
|
||||
const auto labels = mappedInput.mapLabels(tr(STR_BACK), tr(STR_LOOKUP), tr(STR_DIR_LEFT), tr(STR_DIR_RIGHT));
|
||||
GUI.drawButtonHints(renderer, labels.btn1, labels.btn2, labels.btn3, labels.btn4);
|
||||
}
|
||||
|
||||
void DictionaryWordSelectActivity::render(RenderLock&&) {
|
||||
// Differential fast path: only the highlight moved and the framebuffer
|
||||
// still holds a clean page (no popup or sub-activity since the last full
|
||||
// repaint). Restore the pixels under the old highlight, draw the new one,
|
||||
// and push — skipping the two-pass page render entirely.
|
||||
if (popup == Popup::None && snapshotIdx >= 0 && !words.empty() && selected != snapshotIdx) {
|
||||
renderer.writeFramebufferRegion(snapshotX, snapshotY, snapshotW, snapshotH, snapshot.get());
|
||||
// The full path's PrewarmScope cleared the glyph cache on exit; batch-load
|
||||
// just the highlighted word's glyphs before drawing them white-on-black.
|
||||
renderer.getFontCacheManager()->prewarmCache(
|
||||
fontId, words[selected].text, static_cast<uint8_t>(1u << (static_cast<uint8_t>(words[selected].style) & 0x03)));
|
||||
if (drawHighlightWithSnapshot()) {
|
||||
drawHints();
|
||||
renderer.displayBuffer(HalDisplay::FAST_REFRESH);
|
||||
return;
|
||||
}
|
||||
// Snapshot failed (oversize box) — fall through to a full repaint.
|
||||
}
|
||||
|
||||
renderer.clearScreen();
|
||||
|
||||
// Same prewarm-scan-then-render pass the reader uses, so SD-card fonts hit
|
||||
// the in-RAM glyph cache during the real draw.
|
||||
auto* fcm = renderer.getFontCacheManager();
|
||||
auto scope = fcm->createPrewarmScope();
|
||||
page->render(renderer, fontId, marginLeft, marginTop);
|
||||
scope.endScanAndPrewarm();
|
||||
page->render(renderer, fontId, marginLeft, marginTop);
|
||||
|
||||
if (!words.empty()) {
|
||||
drawHighlightWithSnapshot();
|
||||
}
|
||||
|
||||
drawHints();
|
||||
|
||||
if (popup != Popup::None) {
|
||||
// The popup overdraws the page, so the snapshot no longer matches the
|
||||
// framebuffer — force the next render onto the full-repaint path.
|
||||
snapshotIdx = -1;
|
||||
// drawPopup overlays the framebuffer and refreshes the display itself.
|
||||
// I18N.get directly: tr() only accepts literal key names.
|
||||
GUI.drawPopup(renderer, I18N.get(popupMsg));
|
||||
return;
|
||||
}
|
||||
renderer.displayBuffer(HalDisplay::FAST_REFRESH);
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
#pragma once
|
||||
|
||||
#include <Epub/Page.h>
|
||||
#include <I18n.h>
|
||||
|
||||
#include <memory>
|
||||
#include <vector>
|
||||
|
||||
#include "activities/Activity.h"
|
||||
#include "util/Dictionary.h"
|
||||
|
||||
// Button-driven word selection over the current reader page: Left/Right step
|
||||
// through words in reading order, Up/Down jump rows, Confirm looks the word up
|
||||
// and opens DictionaryDefinitionActivity, Back returns to the reader.
|
||||
class DictionaryWordSelectActivity final : public Activity {
|
||||
public:
|
||||
explicit DictionaryWordSelectActivity(GfxRenderer& renderer, MappedInputManager& mappedInput,
|
||||
std::unique_ptr<Page> page, int marginLeft, int marginTop)
|
||||
: Activity("DictionaryWordSelect", renderer, mappedInput),
|
||||
page(std::move(page)),
|
||||
marginLeft(marginLeft),
|
||||
marginTop(marginTop) {}
|
||||
|
||||
void onEnter() override;
|
||||
void loop() override;
|
||||
void render(RenderLock&&) override;
|
||||
|
||||
private:
|
||||
// Screen box of one selectable word. `text` points into the owned Page's
|
||||
// TextBlock arena (NUL-terminated), valid for this activity's lifetime.
|
||||
struct WordBox {
|
||||
int16_t x;
|
||||
int16_t y;
|
||||
int16_t width;
|
||||
uint16_t row;
|
||||
const char* text;
|
||||
EpdFontFamily::Style style;
|
||||
};
|
||||
|
||||
enum class Popup : uint8_t { None, Busy, NotFound, Error };
|
||||
|
||||
void extractWords();
|
||||
int closestInRow(uint16_t row, int centerX) const;
|
||||
void moveVertical(int direction);
|
||||
void performLookup();
|
||||
bool drawHighlightWithSnapshot();
|
||||
void drawHints() const;
|
||||
|
||||
std::unique_ptr<Page> page;
|
||||
const int marginLeft;
|
||||
const int marginTop;
|
||||
int fontId = 0;
|
||||
int lineHeight = 0;
|
||||
|
||||
std::vector<WordBox> words;
|
||||
int selected = 0;
|
||||
uint16_t rowCount = 0;
|
||||
|
||||
Dictionary dict;
|
||||
bool dictOpenAttempted = false;
|
||||
bool dictOpenOk = false;
|
||||
|
||||
Popup popup = Popup::None;
|
||||
StrId popupMsg = StrId::STR_DICT_NOT_FOUND;
|
||||
unsigned long popupTime = 0;
|
||||
|
||||
// Differential highlight repaint: the pixels under the current highlight
|
||||
// box, so a cursor move restores them and repaints only the two affected
|
||||
// boxes instead of re-running the full two-pass page render (which also
|
||||
// reloads every SD-font glyph on the page). snapshotIdx is the word whose
|
||||
// under-pixels are saved; -1 means the framebuffer no longer holds a clean
|
||||
// page (popup drawn, sub-activity shown) and the next render must be full.
|
||||
static constexpr size_t SNAPSHOT_CAPACITY = 4096;
|
||||
std::unique_ptr<uint8_t[]> snapshot;
|
||||
int16_t snapshotX = 0;
|
||||
int16_t snapshotY = 0;
|
||||
int16_t snapshotW = 0;
|
||||
int16_t snapshotH = 0;
|
||||
int snapshotIdx = -1;
|
||||
|
||||
// The activity is entered while Confirm is still held (long-press trigger):
|
||||
// ignore the stale release until a fresh press is seen.
|
||||
bool confirmPressSeen = false;
|
||||
};
|
||||
@@ -20,6 +20,7 @@
|
||||
#include "BookmarkEntry.h"
|
||||
#include "CrossPointSettings.h"
|
||||
#include "CrossPointState.h"
|
||||
#include "DictionaryWordSelectActivity.h"
|
||||
#include "EpubReaderBookmarksActivity.h"
|
||||
#include "EpubReaderChapterSelectionActivity.h"
|
||||
#include "EpubReaderFootnotesActivity.h"
|
||||
@@ -257,6 +258,29 @@ void EpubReaderActivity::openReaderMenu() {
|
||||
});
|
||||
}
|
||||
|
||||
void EpubReaderActivity::openDictionaryWordSelect() {
|
||||
if (SETTINGS.dictionaryName[0] == '\0') {
|
||||
showDictionaryMessage = true;
|
||||
dictionaryMessageTime = millis();
|
||||
requestUpdate();
|
||||
return;
|
||||
}
|
||||
if (!section) return;
|
||||
auto page = section->loadPage(section->currentPage);
|
||||
if (!page) return;
|
||||
|
||||
// Word geometry must match render(): viewable-area margins plus screen margin.
|
||||
int orientedMarginTop, orientedMarginRight, orientedMarginBottom, orientedMarginLeft;
|
||||
renderer.getOrientedViewableTRBL(&orientedMarginTop, &orientedMarginRight, &orientedMarginBottom,
|
||||
&orientedMarginLeft);
|
||||
orientedMarginTop += SETTINGS.screenMargin;
|
||||
orientedMarginLeft += SETTINGS.screenMargin;
|
||||
|
||||
startActivityForResult(std::make_unique<DictionaryWordSelectActivity>(renderer, mappedInput, std::move(page),
|
||||
orientedMarginLeft, orientedMarginTop),
|
||||
[this](const ActivityResult&) { requestUpdate(); });
|
||||
}
|
||||
|
||||
void EpubReaderActivity::loop() {
|
||||
if (!epub) {
|
||||
// Should never happen
|
||||
@@ -379,6 +403,11 @@ void EpubReaderActivity::loop() {
|
||||
requestUpdate();
|
||||
}
|
||||
|
||||
if (showDictionaryMessage && (millis() - dictionaryMessageTime) >= ReaderUtils::BOOKMARK_MESSAGE_DURATION_MS) {
|
||||
showDictionaryMessage = false;
|
||||
requestUpdate();
|
||||
}
|
||||
|
||||
// While the end screen suggestion menu is showing it owns Confirm/Back/navigation
|
||||
// input. Anything it doesn't handle (e.g. long-press Back to the file browser) falls
|
||||
// through to the regular handlers below; page turns are absorbed by the end-of-book
|
||||
@@ -442,6 +471,14 @@ void EpubReaderActivity::loop() {
|
||||
}
|
||||
}
|
||||
break;
|
||||
case CrossPointSettings::LP_MENU_DICTIONARY:
|
||||
// Hold ~0.4s starts dictionary word selection on the current page.
|
||||
if (mappedInput.getHeldTime() >= ReaderUtils::BOOKMARK_HOLD_MS && !showDictionaryMessage) {
|
||||
ignoreNextConfirmRelease = true; // Prevent menu open on the release that follows
|
||||
openDictionaryWordSelect();
|
||||
return;
|
||||
}
|
||||
break;
|
||||
case CrossPointSettings::LP_MENU_DISABLED:
|
||||
default:
|
||||
break;
|
||||
@@ -710,6 +747,10 @@ void EpubReaderActivity::onReaderMenuConfirm(EpubReaderMenuActivity::MenuAction
|
||||
});
|
||||
break;
|
||||
}
|
||||
case EpubReaderMenuActivity::MenuAction::DICTIONARY: {
|
||||
openDictionaryWordSelect();
|
||||
break;
|
||||
}
|
||||
case EpubReaderMenuActivity::MenuAction::DISPLAY_QR: {
|
||||
if (section && section->currentPage >= 0 && section->currentPage < section->pageCount) {
|
||||
std::string fullText = section->getTextFromSectionFile();
|
||||
@@ -1300,6 +1341,10 @@ void EpubReaderActivity::render(RenderLock&& lock) {
|
||||
if (showBookmarkMessage) {
|
||||
GUI.drawPopup(renderer, bookmarkRemoved ? tr(STR_BOOKMARK_REMOVED) : tr(STR_BOOKMARK_ADDED));
|
||||
}
|
||||
|
||||
if (showDictionaryMessage) {
|
||||
GUI.drawPopup(renderer, tr(STR_DICT_NO_DICT_SET));
|
||||
}
|
||||
}
|
||||
|
||||
bool EpubReaderActivity::applyDeferredReposition() {
|
||||
|
||||
@@ -39,6 +39,9 @@ class EpubReaderActivity final : public Activity {
|
||||
bool skipNextButtonCheck = false; // Skip button processing for one frame after subactivity exit
|
||||
bool automaticPageTurnActive = false;
|
||||
bool showBookmarkMessage = false;
|
||||
// "No dictionary set" popup, shown when a lookup is triggered without a configured dictionary.
|
||||
bool showDictionaryMessage = false;
|
||||
unsigned long dictionaryMessageTime = 0UL;
|
||||
bool ignoreNextConfirmRelease = false;
|
||||
bool currentPageBookmarked = false;
|
||||
bool bookmarkRemoved = false; // true when last toggle removed (controls popup text)
|
||||
@@ -119,6 +122,7 @@ class EpubReaderActivity final : public Activity {
|
||||
void onReaderMenuConfirm(EpubReaderMenuActivity::MenuAction action);
|
||||
// Opens the reader menu for the current position (short-press Confirm)
|
||||
void openReaderMenu();
|
||||
void openDictionaryWordSelect();
|
||||
// Returns true if sync acted (launched, or surfaced a save error); false if it was a no-op
|
||||
// because no KOReader credentials are stored.
|
||||
bool launchKOReaderSync();
|
||||
|
||||
@@ -22,7 +22,7 @@ EpubReaderMenuActivity::EpubReaderMenuActivity(GfxRenderer& renderer, MappedInpu
|
||||
std::vector<EpubReaderMenuActivity::MenuItem> EpubReaderMenuActivity::buildMenuItems(bool hasFootnotes,
|
||||
bool hasBookmarks) {
|
||||
std::vector<MenuItem> items;
|
||||
items.reserve(12);
|
||||
items.reserve(13);
|
||||
items.push_back({MenuAction::SELECT_CHAPTER, StrId::STR_SELECT_CHAPTER});
|
||||
if (hasFootnotes) {
|
||||
items.push_back({MenuAction::FOOTNOTES, StrId::STR_FOOTNOTES});
|
||||
@@ -31,6 +31,7 @@ std::vector<EpubReaderMenuActivity::MenuItem> EpubReaderMenuActivity::buildMenuI
|
||||
items.push_back({MenuAction::BOOKMARKS, StrId::STR_BOOKMARKS});
|
||||
}
|
||||
items.push_back({MenuAction::TOGGLE_BOOKMARK, StrId::STR_TOGGLE_BOOKMARK});
|
||||
items.push_back({MenuAction::DICTIONARY, StrId::STR_LOOKUP});
|
||||
items.push_back({MenuAction::ROTATE_SCREEN, StrId::STR_ORIENTATION});
|
||||
items.push_back({MenuAction::AUTO_PAGE_TURN, StrId::STR_AUTO_TURN_PAGES_PER_MIN});
|
||||
items.push_back({MenuAction::GO_TO_PERCENT, StrId::STR_GO_TO_PERCENT});
|
||||
|
||||
@@ -24,7 +24,8 @@ class EpubReaderMenuActivity final : public Activity {
|
||||
DISPLAY_QR,
|
||||
GO_HOME,
|
||||
SYNC,
|
||||
DELETE_CACHE
|
||||
DELETE_CACHE,
|
||||
DICTIONARY
|
||||
};
|
||||
|
||||
explicit EpubReaderMenuActivity(GfxRenderer& renderer, MappedInputManager& mappedInput, const std::string& title,
|
||||
|
||||
@@ -39,7 +39,12 @@ void SettingsActivity::rebuildSettingsLists() {
|
||||
// reader activity ran — otherwise the font-family picker shows stale list.
|
||||
sdFontSystem.refreshIfDirty();
|
||||
|
||||
for (auto& setting : getSettingsList(&sdFontSystem.registry())) {
|
||||
// Rescan /dictionaries on every rebuild: cheap (one directory listing) and
|
||||
// picks up dictionaries copied to the SD card since the last visit.
|
||||
std::vector<DictionaryEntry> dictionaries;
|
||||
DictionaryRegistry::discover(dictionaries);
|
||||
|
||||
for (auto& setting : getSettingsList(&sdFontSystem.registry(), &dictionaries)) {
|
||||
if (setting.category == StrId::STR_NONE_OPT) continue;
|
||||
if (setting.category == StrId::STR_CAT_DISPLAY) {
|
||||
displaySettings.push_back(setting);
|
||||
|
||||
@@ -0,0 +1,180 @@
|
||||
#include "DictZip.h"
|
||||
|
||||
#include <InflateReader.h>
|
||||
#include <Memory.h>
|
||||
|
||||
namespace DictZip {
|
||||
namespace {
|
||||
|
||||
// Caps the chunk table at 32KB of heap (8192 * 4 bytes); at the typical ~58KB
|
||||
// chunk length that still allows ~460MB of uncompressed dictionary data.
|
||||
constexpr uint16_t MAX_CHUNK_COUNT = 8192;
|
||||
|
||||
bool readLe16(HalFile& file, uint16_t* out) {
|
||||
uint8_t raw[2];
|
||||
if (file.read(raw, 2) != 2) return false;
|
||||
*out = static_cast<uint16_t>(raw[0] | (static_cast<uint16_t>(raw[1]) << 8));
|
||||
return true;
|
||||
}
|
||||
|
||||
bool extractChunkSlice(HalFile& file, uint32_t compressedOffset, uint32_t compressedSize, uint32_t discardSize,
|
||||
uint32_t extractSize, HalFile& outFile) {
|
||||
if (extractSize == 0) return true;
|
||||
auto compBuf = makeUniqueNoThrow<uint8_t[]>(compressedSize);
|
||||
if (!compBuf) return false;
|
||||
|
||||
file.seekSet(compressedOffset);
|
||||
if (file.read(compBuf.get(), static_cast<int>(compressedSize)) != static_cast<int>(compressedSize)) return false;
|
||||
|
||||
InflateReader reader;
|
||||
if (!reader.init(true)) return false;
|
||||
reader.setSource(compBuf.get(), compressedSize);
|
||||
|
||||
auto buf = makeUniqueNoThrow<uint8_t[]>(512);
|
||||
if (!buf) return false;
|
||||
|
||||
uint32_t batch;
|
||||
while (discardSize > 0) {
|
||||
batch = discardSize < 512 ? discardSize : 512;
|
||||
if (!reader.read(buf.get(), batch)) return false;
|
||||
discardSize -= batch;
|
||||
}
|
||||
|
||||
while (extractSize > 0) {
|
||||
batch = extractSize < 512 ? extractSize : 512;
|
||||
if (!reader.read(buf.get(), batch)) return false;
|
||||
if (outFile.write(buf.get(), batch) != batch) return false;
|
||||
extractSize -= batch;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool parse(HalFile& file, Info* info) {
|
||||
if (!info) return false;
|
||||
*info = {};
|
||||
|
||||
uint8_t header[10];
|
||||
if (file.read(header, sizeof(header)) != static_cast<int>(sizeof(header))) return false;
|
||||
if (header[0] != 0x1f || header[1] != 0x8b || header[2] != 8) return false;
|
||||
|
||||
const uint8_t flags = header[3];
|
||||
if ((flags & 0x04) == 0) return false; // dictzip requires FEXTRA
|
||||
|
||||
uint16_t xlen = 0;
|
||||
if (!readLe16(file, &xlen)) return false;
|
||||
|
||||
uint32_t extraRead = 0;
|
||||
bool foundRa = false;
|
||||
while (extraRead + 4 <= xlen) {
|
||||
uint8_t subHeader[4];
|
||||
if (file.read(subHeader, sizeof(subHeader)) != static_cast<int>(sizeof(subHeader))) return false;
|
||||
extraRead += 4;
|
||||
const uint16_t subLen = static_cast<uint16_t>(subHeader[2] | (static_cast<uint16_t>(subHeader[3]) << 8));
|
||||
if (extraRead + subLen > xlen) return false;
|
||||
|
||||
if (subHeader[0] == 'R' && subHeader[1] == 'A') {
|
||||
if (subLen < 6) return false;
|
||||
|
||||
uint16_t version = 0;
|
||||
uint16_t chunkLen = 0;
|
||||
uint16_t chunkCount = 0;
|
||||
if (!readLe16(file, &version) || !readLe16(file, &chunkLen) || !readLe16(file, &chunkCount)) return false;
|
||||
extraRead += 6;
|
||||
if (version != 1 || chunkLen == 0 || chunkCount == 0 || chunkCount > MAX_CHUNK_COUNT) return false;
|
||||
if (subLen != static_cast<uint16_t>(6 + chunkCount * 2)) return false;
|
||||
|
||||
info->chunkLength = chunkLen;
|
||||
info->chunkOffsets.reserve(static_cast<size_t>(chunkCount) + 1);
|
||||
info->chunkOffsets.push_back(0);
|
||||
uint32_t cumulative = 0;
|
||||
for (uint16_t i = 0; i < chunkCount; i++) {
|
||||
uint16_t compLen = 0;
|
||||
if (!readLe16(file, &compLen)) return false;
|
||||
extraRead += 2;
|
||||
cumulative += compLen;
|
||||
info->chunkOffsets.push_back(cumulative);
|
||||
}
|
||||
foundRa = true;
|
||||
} else {
|
||||
file.seekSet(file.position() + subLen);
|
||||
extraRead += subLen;
|
||||
}
|
||||
}
|
||||
if (extraRead != xlen || !foundRa) return false;
|
||||
|
||||
if (flags & 0x08) { // FNAME
|
||||
int b;
|
||||
do {
|
||||
b = file.read();
|
||||
if (b < 0) return false;
|
||||
} while (b != 0);
|
||||
}
|
||||
if (flags & 0x10) { // FCOMMENT
|
||||
int b;
|
||||
do {
|
||||
b = file.read();
|
||||
if (b < 0) return false;
|
||||
} while (b != 0);
|
||||
}
|
||||
if (flags & 0x02) { // FHCRC
|
||||
uint8_t crc[2];
|
||||
if (file.read(crc, 2) != 2) return false;
|
||||
}
|
||||
|
||||
info->dataOffset = static_cast<uint32_t>(file.position());
|
||||
const uint32_t fileSize = static_cast<uint32_t>(file.fileSize());
|
||||
if (fileSize < 4) return false;
|
||||
file.seekSet(fileSize - 4);
|
||||
uint8_t isizeRaw[4];
|
||||
if (file.read(isizeRaw, 4) != 4) return false;
|
||||
info->totalSize = static_cast<uint32_t>(isizeRaw[0]) | (static_cast<uint32_t>(isizeRaw[1]) << 8) |
|
||||
(static_cast<uint32_t>(isizeRaw[2]) << 16) | (static_cast<uint32_t>(isizeRaw[3]) << 24);
|
||||
if (info->totalSize == 0) return false;
|
||||
info->valid = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool extractEntry(const char* path, uint32_t offset, uint32_t size, HalFile& outFile) {
|
||||
if (size == 0) return true;
|
||||
|
||||
HalFile file;
|
||||
if (!Storage.openFileForRead("DICTZIP", path, file)) return false;
|
||||
|
||||
Info info;
|
||||
if (!parse(file, &info)) return false;
|
||||
|
||||
// Reject ranges outside the uncompressed data (offset/size come from the
|
||||
// untrusted .idx). Subtraction form avoids uint32 overflow in offset + size
|
||||
// and guarantees localOffset < chunkOutSize in the loop below.
|
||||
if (offset > info.totalSize || size > info.totalSize - offset) return false;
|
||||
|
||||
const uint32_t startChunk = offset / info.chunkLength;
|
||||
const uint32_t endChunk = (offset + size - 1) / info.chunkLength;
|
||||
if (endChunk + 1 >= info.chunkOffsets.size()) return false;
|
||||
|
||||
uint32_t remaining = size;
|
||||
const uint32_t lastChunk = static_cast<uint32_t>(info.chunkOffsets.size() - 2);
|
||||
for (uint32_t chunk = startChunk; chunk <= endChunk; chunk++) {
|
||||
uint32_t chunkOutSize = info.chunkLength;
|
||||
if (chunk == lastChunk) chunkOutSize = info.totalSize - chunk * info.chunkLength;
|
||||
if (chunkOutSize == 0 || chunkOutSize > info.chunkLength) chunkOutSize = info.chunkLength;
|
||||
|
||||
const uint32_t localOffset = (chunk == startChunk) ? (offset % info.chunkLength) : 0;
|
||||
const uint32_t available = chunkOutSize - localOffset;
|
||||
const uint32_t take = remaining < available ? remaining : available;
|
||||
|
||||
const uint32_t compOffset = info.dataOffset + info.chunkOffsets[chunk];
|
||||
const uint32_t compSize = info.chunkOffsets[chunk + 1] - info.chunkOffsets[chunk];
|
||||
if (!extractChunkSlice(file, compOffset, compSize, localOffset, take, outFile)) return false;
|
||||
|
||||
remaining -= take;
|
||||
if (remaining == 0) break;
|
||||
}
|
||||
|
||||
return remaining == 0;
|
||||
}
|
||||
|
||||
} // namespace DictZip
|
||||
@@ -0,0 +1,26 @@
|
||||
#pragma once
|
||||
|
||||
#include <HalStorage.h>
|
||||
|
||||
#include <cstdint>
|
||||
#include <vector>
|
||||
|
||||
// Random-access reader for dictzip (.dict.dz) files: gzip with an extra "RA"
|
||||
// field holding a chunk table, so any byte range can be decompressed without
|
||||
// inflating the whole file. Format: https://linux.die.net/man/1/dictzip
|
||||
namespace DictZip {
|
||||
|
||||
struct Info {
|
||||
uint32_t dataOffset = 0; // file offset where compressed chunk data starts
|
||||
uint32_t totalSize = 0; // uncompressed size (gzip ISIZE trailer)
|
||||
uint16_t chunkLength = 0; // uncompressed bytes per chunk
|
||||
std::vector<uint32_t> chunkOffsets; // cumulative compressed offsets, chunkCount+1 entries
|
||||
bool valid = false;
|
||||
};
|
||||
|
||||
bool parse(HalFile& file, Info* info);
|
||||
|
||||
// Decompress the uncompressed byte range [offset, offset+size) into outFile.
|
||||
bool extractEntry(const char* path, uint32_t offset, uint32_t size, HalFile& outFile);
|
||||
|
||||
} // namespace DictZip
|
||||
@@ -0,0 +1,378 @@
|
||||
#include "Dictionary.h"
|
||||
|
||||
#include <Arduino.h>
|
||||
#include <Logging.h>
|
||||
#include <Memory.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cctype>
|
||||
#include <cstring>
|
||||
|
||||
#include "DictZip.h"
|
||||
#include "DictionaryRegistry.h"
|
||||
#include "StringUtils.h"
|
||||
|
||||
namespace {
|
||||
|
||||
// Shared temp file for entries lazily extracted from .dict.dz.
|
||||
constexpr const char* DICT_TMP_FILE = "/.crosspoint/dict.tmp";
|
||||
|
||||
// .qidx sidecar header: magic, version, sample interval, sample count, and the
|
||||
// .idx file size the sidecar was built from (staleness check).
|
||||
constexpr uint32_t QIDX_MAGIC = 0x58444951; // "QIDX" little-endian
|
||||
constexpr uint32_t QIDX_VERSION = 1;
|
||||
constexpr size_t QIDX_HEADER_BYTES = 5 * sizeof(uint32_t);
|
||||
|
||||
struct QidxHeader {
|
||||
uint32_t sampleCount = 0;
|
||||
uint32_t idxFileSize = 0;
|
||||
bool valid = false;
|
||||
};
|
||||
|
||||
QidxHeader readQidxHeader(HalFile& qidx, uint32_t sampleInterval) {
|
||||
QidxHeader header;
|
||||
uint32_t raw[5];
|
||||
if (!qidx.seekSet(0) || qidx.read(raw, sizeof(raw)) != static_cast<int>(sizeof(raw))) return header;
|
||||
if (raw[0] != QIDX_MAGIC || raw[1] != QIDX_VERSION || raw[2] != sampleInterval) return header;
|
||||
header.sampleCount = raw[3];
|
||||
header.idxFileSize = raw[4];
|
||||
header.valid = true;
|
||||
return header;
|
||||
}
|
||||
|
||||
bool readSampleOffset(HalFile& qidx, uint32_t sampleIndex, uint32_t* out) {
|
||||
if (!qidx.seekSet(QIDX_HEADER_BYTES + static_cast<size_t>(sampleIndex) * sizeof(uint32_t))) return false;
|
||||
return qidx.read(out, sizeof(*out)) == static_cast<int>(sizeof(*out));
|
||||
}
|
||||
|
||||
uint32_t readBe32(const uint8_t* p) {
|
||||
return (static_cast<uint32_t>(p[0]) << 24) | (static_cast<uint32_t>(p[1]) << 16) |
|
||||
(static_cast<uint32_t>(p[2]) << 8) | static_cast<uint32_t>(p[3]);
|
||||
}
|
||||
|
||||
// Word characters for cleaning: ASCII alphanumerics plus any UTF-8
|
||||
// continuation/lead byte, so accented words keep their edges.
|
||||
bool isWordByte(unsigned char c) { return c >= 0x80 || std::isalnum(c) != 0; }
|
||||
|
||||
// True when the .ifo declares 64-bit index offsets, which this reader does not
|
||||
// support (only scans the first 2KB — idxoffsetbits always appears early).
|
||||
bool ifoDeclares64BitOffsets(const std::string& ifoPath) {
|
||||
HalFile ifo;
|
||||
if (!Storage.openFileForRead("DICT", ifoPath, ifo)) return false;
|
||||
char buf[2048];
|
||||
const int n = ifo.read(buf, sizeof(buf) - 1);
|
||||
if (n <= 0) return false;
|
||||
buf[n] = '\0';
|
||||
const char* line = strstr(buf, "idxoffsetbits");
|
||||
if (!line) return false;
|
||||
const char* eq = strchr(line, '=');
|
||||
return eq && strtol(eq + 1, nullptr, 10) == 64;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool Dictionary::open(const char* folderName) {
|
||||
basePath.clear();
|
||||
std::string resolved;
|
||||
if (!DictionaryRegistry::resolveBasePath(folderName, resolved)) {
|
||||
LOG_ERR("DICT", "No dictionary found in folder '%s'", folderName ? folderName : "");
|
||||
return false;
|
||||
}
|
||||
|
||||
if (!Storage.exists((resolved + ".idx").c_str())) {
|
||||
LOG_ERR("DICT", "%s.idx missing (compressed .idx.gz is not supported)", resolved.c_str());
|
||||
return false;
|
||||
}
|
||||
hasPlainDict = Storage.exists((resolved + ".dict").c_str());
|
||||
if (!hasPlainDict && !Storage.exists((resolved + ".dict.dz").c_str())) {
|
||||
LOG_ERR("DICT", "%s has no .dict or .dict.dz", resolved.c_str());
|
||||
return false;
|
||||
}
|
||||
if (ifoDeclares64BitOffsets(resolved + ".ifo")) {
|
||||
LOG_ERR("DICT", "%s uses 64-bit index offsets (unsupported)", resolved.c_str());
|
||||
return false;
|
||||
}
|
||||
|
||||
basePath = std::move(resolved);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool Dictionary::needsIndex() {
|
||||
if (!isOpen()) return false;
|
||||
|
||||
HalFile idx;
|
||||
if (!Storage.openFileForRead("DICT", basePath + ".idx", idx)) return false;
|
||||
const uint32_t idxSize = static_cast<uint32_t>(idx.fileSize());
|
||||
|
||||
HalFile qidx;
|
||||
if (!Storage.openFileForRead("DICT", basePath + ".qidx", qidx)) return true;
|
||||
const QidxHeader header = readQidxHeader(qidx, SAMPLE_INTERVAL);
|
||||
return !header.valid || header.idxFileSize != idxSize;
|
||||
}
|
||||
|
||||
bool Dictionary::buildIndex(void (*yieldFn)(void*), void* ctx) {
|
||||
if (!isOpen()) return false;
|
||||
|
||||
HalFile idx;
|
||||
if (!Storage.openFileForRead("DICT", basePath + ".idx", idx)) return false;
|
||||
const uint32_t idxSize = static_cast<uint32_t>(idx.fileSize());
|
||||
|
||||
constexpr size_t CHUNK_BYTES = 4096;
|
||||
auto buf = makeUniqueNoThrow<uint8_t[]>(CHUNK_BYTES);
|
||||
if (!buf) {
|
||||
LOG_ERR("DICT", "OOM: %u byte index scan buffer", CHUNK_BYTES);
|
||||
return false;
|
||||
}
|
||||
|
||||
// Stream each sample offset straight to the sidecar instead of accumulating
|
||||
// them in RAM: a large .idx would otherwise cost tens of KB of vector heap,
|
||||
// and vector growth aborts on OOM under -fno-exceptions. The header slot is
|
||||
// zero-filled until the scan succeeds, so an interrupted build leaves a file
|
||||
// readQidxHeader rejects (magic mismatch) and needsIndex() triggers a rebuild.
|
||||
const std::string qidxPath = basePath + ".qidx";
|
||||
HalFile out;
|
||||
if (!Storage.openFileForWrite("DICT", qidxPath, out)) return false;
|
||||
const auto writeU32 = [&out](uint32_t v) { return out.write(&v, sizeof(v)) == static_cast<int>(sizeof(v)); };
|
||||
const uint32_t placeholder[5] = {};
|
||||
bool ok = out.write(placeholder, sizeof(placeholder)) == sizeof(placeholder);
|
||||
uint32_t sampleCount = 0;
|
||||
if (ok) {
|
||||
ok = writeU32(0); // entry 0 always starts at byte 0
|
||||
sampleCount = 1;
|
||||
}
|
||||
|
||||
const unsigned long startMs = millis();
|
||||
uint32_t entryCount = 0;
|
||||
uint32_t pos = 0;
|
||||
uint32_t suffixLeft = 0; // 0 while scanning a headword, else suffix bytes remaining
|
||||
uint32_t sinceYield = 0;
|
||||
while (ok && pos < idxSize) {
|
||||
const int n = idx.read(buf.get(), CHUNK_BYTES);
|
||||
if (n <= 0) {
|
||||
LOG_ERR("DICT", "Index scan read failed at %lu", static_cast<unsigned long>(pos));
|
||||
ok = false;
|
||||
break;
|
||||
}
|
||||
for (int i = 0; ok && i < n; i++) {
|
||||
if (suffixLeft == 0) {
|
||||
if (buf[i] == 0) suffixLeft = 8;
|
||||
} else if (--suffixLeft == 0) {
|
||||
entryCount++;
|
||||
const uint32_t nextEntryStart = pos + i + 1;
|
||||
if (entryCount % SAMPLE_INTERVAL == 0 && nextEntryStart < idxSize) {
|
||||
ok = writeU32(nextEntryStart);
|
||||
sampleCount++;
|
||||
}
|
||||
}
|
||||
}
|
||||
pos += n;
|
||||
sinceYield += n;
|
||||
if (yieldFn && sinceYield >= 64 * 1024) {
|
||||
sinceYield = 0;
|
||||
yieldFn(ctx);
|
||||
}
|
||||
}
|
||||
|
||||
if (ok) {
|
||||
// Backpatch the now-valid header over the placeholder.
|
||||
const uint32_t header[5] = {QIDX_MAGIC, QIDX_VERSION, SAMPLE_INTERVAL, sampleCount, idxSize};
|
||||
ok = out.seekSet(0) && out.write(header, sizeof(header)) == sizeof(header);
|
||||
}
|
||||
if (!ok) {
|
||||
LOG_ERR("DICT", "Index build failed, removing %s", qidxPath.c_str());
|
||||
out.close(); // close before remove of the same path
|
||||
Storage.remove(qidxPath.c_str());
|
||||
return false;
|
||||
}
|
||||
|
||||
LOG_INF("DICT", "Indexed %lu entries (%lu samples) in %lu ms", static_cast<unsigned long>(entryCount),
|
||||
static_cast<unsigned long>(sampleCount), millis() - startMs);
|
||||
return true;
|
||||
}
|
||||
|
||||
int Dictionary::readWordInto(HalFile& file, char* buf, size_t bufSize) {
|
||||
size_t i = 0;
|
||||
while (i < bufSize - 1) {
|
||||
const int ch = file.read();
|
||||
if (ch < 0) return -1; // EOF or I/O error
|
||||
if (ch == 0) {
|
||||
buf[i] = '\0';
|
||||
return static_cast<int>(i);
|
||||
}
|
||||
buf[i++] = static_cast<char>(ch);
|
||||
}
|
||||
// Word too long for buffer — consume remaining bytes to stay in sync
|
||||
buf[bufSize - 1] = '\0';
|
||||
int ch;
|
||||
do {
|
||||
ch = file.read();
|
||||
} while (ch > 0);
|
||||
return static_cast<int>(bufSize - 1);
|
||||
}
|
||||
|
||||
DictLocation Dictionary::locate(const char* target, std::string* matchedHeadwordOut) {
|
||||
DictLocation result;
|
||||
if (!isOpen()) return result;
|
||||
|
||||
HalFile idx;
|
||||
if (!Storage.openFileForRead("DICT", basePath + ".idx", idx)) return result;
|
||||
const uint32_t idxSize = static_cast<uint32_t>(idx.fileSize());
|
||||
|
||||
// Bisect the sampled offsets to the last sample whose headword <= target.
|
||||
// Falls back to a full scan from byte 0 when the sidecar is unusable.
|
||||
uint32_t startByte = 0;
|
||||
HalFile qidx;
|
||||
if (Storage.openFileForRead("DICT", basePath + ".qidx", qidx)) {
|
||||
const QidxHeader header = readQidxHeader(qidx, SAMPLE_INTERVAL);
|
||||
if (header.valid && header.idxFileSize == idxSize && header.sampleCount > 0) {
|
||||
uint32_t lo = 0;
|
||||
uint32_t hi = header.sampleCount - 1;
|
||||
while (lo < hi) {
|
||||
const uint32_t mid = (lo + hi + 1) / 2;
|
||||
uint32_t offset = 0;
|
||||
if (!readSampleOffset(qidx, mid, &offset) || !idx.seekSet(offset) ||
|
||||
readWordInto(idx, wordBuf, sizeof(wordBuf)) < 0) {
|
||||
lo = 0;
|
||||
break;
|
||||
}
|
||||
if (StringUtils::asciiCaseCmp(wordBuf, target) <= 0) {
|
||||
lo = mid;
|
||||
} else {
|
||||
hi = mid - 1;
|
||||
}
|
||||
}
|
||||
readSampleOffset(qidx, lo, &startByte);
|
||||
}
|
||||
}
|
||||
|
||||
// Linear scan of at most SAMPLE_INTERVAL entries: headword NUL, BE32 offset,
|
||||
// BE32 size. The index is sorted, so stop at the first headword > target.
|
||||
idx.seekSet(startByte);
|
||||
while (static_cast<uint32_t>(idx.position()) < idxSize) {
|
||||
if (readWordInto(idx, wordBuf, sizeof(wordBuf)) < 0) break;
|
||||
uint8_t suffix[8];
|
||||
if (idx.read(suffix, 8) != 8) break;
|
||||
|
||||
const int cmp = StringUtils::asciiCaseCmp(wordBuf, target);
|
||||
if (cmp == 0) {
|
||||
result.offset = readBe32(suffix);
|
||||
result.size = readBe32(suffix + 4);
|
||||
result.found = true;
|
||||
if (matchedHeadwordOut) *matchedHeadwordOut = wordBuf;
|
||||
return result;
|
||||
}
|
||||
if (cmp > 0) break;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
bool Dictionary::readDefinition(const DictLocation& location, std::string& out) {
|
||||
if (!location.found) return false;
|
||||
const uint32_t size = std::min(location.size, MAX_DEFINITION_BYTES);
|
||||
|
||||
std::string path;
|
||||
uint32_t offset = 0;
|
||||
if (hasPlainDict) {
|
||||
path = basePath + ".dict";
|
||||
offset = location.offset;
|
||||
} else {
|
||||
HalFile tmp = Storage.open(DICT_TMP_FILE, O_WRITE | O_CREAT | O_TRUNC);
|
||||
if (!tmp) {
|
||||
LOG_ERR("DICT", "Failed to open %s", DICT_TMP_FILE);
|
||||
return false;
|
||||
}
|
||||
if (!DictZip::extractEntry((basePath + ".dict.dz").c_str(), location.offset, size, tmp)) {
|
||||
LOG_ERR("DICT", "dictzip extraction failed for %s", basePath.c_str());
|
||||
return false;
|
||||
}
|
||||
tmp.close(); // close before reopening the same path for read
|
||||
path = DICT_TMP_FILE;
|
||||
}
|
||||
|
||||
HalFile dict;
|
||||
if (!Storage.openFileForRead("DICT", path, dict)) return false;
|
||||
const uint32_t dictSize = static_cast<uint32_t>(dict.fileSize());
|
||||
if (offset > dictSize || size > dictSize - offset) {
|
||||
LOG_ERR("DICT", "Definition out of bounds (%lu+%lu > %lu)", static_cast<unsigned long>(offset),
|
||||
static_cast<unsigned long>(size), static_cast<unsigned long>(dictSize));
|
||||
return false;
|
||||
}
|
||||
|
||||
// std::string growth aborts on OOM (-fno-exceptions); refuse up front unless
|
||||
// the allocation fits comfortably in the largest free block.
|
||||
if (ESP.getMaxAllocHeap() < size + 8 * 1024) {
|
||||
LOG_ERR("DICT", "Low heap for %lu byte definition", static_cast<unsigned long>(size));
|
||||
return false;
|
||||
}
|
||||
|
||||
dict.seekSet(offset);
|
||||
out.assign(size, '\0');
|
||||
const int bytesRead = dict.read(&out[0], size);
|
||||
if (bytesRead < 0) {
|
||||
out.clear();
|
||||
return false;
|
||||
}
|
||||
if (static_cast<uint32_t>(bytesRead) < size) out.resize(bytesRead);
|
||||
return true;
|
||||
}
|
||||
|
||||
std::string Dictionary::cleanWord(const char* word) {
|
||||
if (!word) return "";
|
||||
size_t start = 0;
|
||||
size_t end = strlen(word);
|
||||
while (start < end && !isWordByte(static_cast<unsigned char>(word[start]))) start++;
|
||||
while (end > start && !isWordByte(static_cast<unsigned char>(word[end - 1]))) end--;
|
||||
if (start >= end) return "";
|
||||
|
||||
std::string result(word + start, end - start);
|
||||
std::transform(result.begin(), result.end(), result.begin(),
|
||||
[](unsigned char c) { return c >= 0x80 ? c : static_cast<unsigned char>(std::tolower(c)); });
|
||||
return result;
|
||||
}
|
||||
|
||||
void Dictionary::stemVariants(const std::string& word, std::vector<std::string>& out) {
|
||||
out.clear();
|
||||
out.reserve(6);
|
||||
const size_t n = word.size();
|
||||
const auto add = [&out](std::string v) {
|
||||
if (std::find(out.begin(), out.end(), v) == out.end()) out.push_back(std::move(v));
|
||||
};
|
||||
// endsWith requires a non-empty remainder so variants never come out empty.
|
||||
const auto endsWith = [&word, n](const char* suffix) {
|
||||
const size_t len = strlen(suffix);
|
||||
return n > len && word.compare(n - len, len, suffix) == 0;
|
||||
};
|
||||
|
||||
if (endsWith("'s")) add(word.substr(0, n - 2));
|
||||
if (endsWith("\xE2\x80\x99s")) add(word.substr(0, n - 4)); // U+2019 apostrophe
|
||||
if (endsWith("ies")) add(word.substr(0, n - 3) + "y"); // stories -> story
|
||||
if (endsWith("es")) add(word.substr(0, n - 2)); // boxes -> box
|
||||
if (endsWith("s")) add(word.substr(0, n - 1)); // dogs -> dog
|
||||
if (endsWith("ed")) {
|
||||
add(word.substr(0, n - 2)); // walked -> walk
|
||||
add(word.substr(0, n - 1)); // loved -> love
|
||||
if (n >= 4 && word[n - 3] == word[n - 4]) add(word.substr(0, n - 3)); // stopped -> stop
|
||||
}
|
||||
if (endsWith("ing")) {
|
||||
add(word.substr(0, n - 3)); // walking -> walk
|
||||
add(word.substr(0, n - 3) + "e"); // making -> make
|
||||
if (n >= 5 && word[n - 4] == word[n - 5]) add(word.substr(0, n - 4)); // running -> run
|
||||
}
|
||||
}
|
||||
|
||||
bool Dictionary::lookup(const char* word, std::string& definitionOut, std::string& matchedHeadwordOut) {
|
||||
const std::string cleaned = cleanWord(word);
|
||||
if (cleaned.empty() || !isOpen()) return false;
|
||||
|
||||
DictLocation location = locate(cleaned.c_str(), &matchedHeadwordOut);
|
||||
if (!location.found) {
|
||||
std::vector<std::string> variants;
|
||||
stemVariants(cleaned, variants);
|
||||
for (const auto& variant : variants) {
|
||||
location = locate(variant.c_str(), &matchedHeadwordOut);
|
||||
if (location.found) break;
|
||||
}
|
||||
}
|
||||
if (!location.found) return false;
|
||||
return readDefinition(location, definitionOut);
|
||||
}
|
||||
@@ -0,0 +1,65 @@
|
||||
#pragma once
|
||||
|
||||
#include <HalStorage.h>
|
||||
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
// Result of an index search — file location of a definition without reading it.
|
||||
struct DictLocation {
|
||||
uint32_t offset = 0; // byte offset in .dict data
|
||||
uint32_t size = 0; // byte length in .dict data
|
||||
bool found = false;
|
||||
};
|
||||
|
||||
// Slim StarDict reader: exact-match lookup with a mini stemming fallback.
|
||||
//
|
||||
// Expects /dictionaries/<folder>/<stem>.idx (uncompressed) plus <stem>.dict or
|
||||
// <stem>.dict.dz. Lookups binary-search a lazily built sampled-offset sidecar
|
||||
// (<stem>.qidx, byte offset of every SAMPLE_INTERVAL-th .idx entry), then
|
||||
// linear-scan at most SAMPLE_INTERVAL entries. Everything streams from SD; no
|
||||
// index is held in RAM.
|
||||
class Dictionary {
|
||||
public:
|
||||
// Resolve the dictionary folder and validate its files. Rejects
|
||||
// dictionaries with 64-bit index offsets (idxoffsetbits=64 in .ifo).
|
||||
bool open(const char* folderName);
|
||||
bool isOpen() const { return !basePath.empty(); }
|
||||
|
||||
// True when the .qidx sidecar is missing or stale — call buildIndex() first
|
||||
// so the UI can show an "Indexing…" message for the slow first pass.
|
||||
bool needsIndex();
|
||||
|
||||
// One streaming pass over .idx writing the .qidx sidecar. yieldFn (optional)
|
||||
// is called every ~64KB consumed to feed the watchdog / repaint the UI.
|
||||
bool buildIndex(void (*yieldFn)(void*) = nullptr, void* ctx = nullptr);
|
||||
|
||||
// Clean the word, look it up, and on a miss retry mini stem variants
|
||||
// (-'s/-s/-es/-ies/-ed/-ing). On a hit fills the definition text (capped at
|
||||
// MAX_DEFINITION_BYTES) and the headword as stored in the index.
|
||||
bool lookup(const char* word, std::string& definitionOut, std::string& matchedHeadwordOut);
|
||||
|
||||
static std::string cleanWord(const char* word);
|
||||
|
||||
static constexpr uint32_t MAX_DEFINITION_BYTES = 64 * 1024;
|
||||
|
||||
private:
|
||||
static constexpr uint32_t SAMPLE_INTERVAL = 256;
|
||||
|
||||
DictLocation locate(const char* target, std::string* matchedHeadwordOut);
|
||||
bool readDefinition(const DictLocation& location, std::string& out);
|
||||
static void stemVariants(const std::string& word, std::vector<std::string>& out);
|
||||
|
||||
// Read a null-terminated word from an open file into buf (max bufSize-1
|
||||
// chars). Returns the number of characters read (excluding null), or -1 on
|
||||
// EOF/error. Over-long words are truncated but the stream stays in sync.
|
||||
static int readWordInto(HalFile& file, char* buf, size_t bufSize);
|
||||
|
||||
std::string basePath; // "/dictionaries/<folder>/<stem>", empty when not open
|
||||
bool hasPlainDict = false;
|
||||
|
||||
// Shared scan buffer: lookups are single-threaded and this avoids a
|
||||
// 256-byte array on the stack of every locate() call.
|
||||
char wordBuf[256] = {};
|
||||
};
|
||||
@@ -0,0 +1,113 @@
|
||||
#include "DictionaryRegistry.h"
|
||||
|
||||
#include <HalStorage.h>
|
||||
#include <Logging.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cstring>
|
||||
|
||||
#include "StringUtils.h"
|
||||
|
||||
namespace DictionaryRegistry {
|
||||
namespace {
|
||||
|
||||
// Dictionaries are looked up in both roots, in order. The hidden variant
|
||||
// lets users keep the folder out of the file browser (hidden by default,
|
||||
// see FileBrowserActivity's showHiddenFiles check).
|
||||
constexpr const char* DICT_ROOTS[] = {"/dictionaries", "/.dictionaries"};
|
||||
|
||||
// Find the single .idx stem inside one dictionary folder. Returns false when
|
||||
// the folder holds no .idx or more than one distinct stem (ambiguous).
|
||||
bool findStem(const char* folderPath, std::string& stemOut) {
|
||||
auto dir = Storage.open(folderPath);
|
||||
if (!dir || !dir.isDirectory()) return false;
|
||||
|
||||
dir.rewindDirectory();
|
||||
char name[128];
|
||||
char foundStem[128];
|
||||
foundStem[0] = '\0';
|
||||
for (auto entry = dir.openNextFile(); entry; entry = dir.openNextFile()) {
|
||||
entry.getName(name, sizeof(name));
|
||||
// Skip macOS metadata files (AppleDouble resource forks)
|
||||
if (entry.isDirectory() || strncmp(name, "._", 2) == 0) continue;
|
||||
|
||||
const size_t len = strlen(name);
|
||||
if (len <= 4 || strcmp(name + len - 4, ".idx") != 0) continue;
|
||||
|
||||
name[len - 4] = '\0';
|
||||
if (foundStem[0] != '\0' && strcmp(foundStem, name) != 0) {
|
||||
LOG_DBG("DREG", "Skipping %s: multiple index stems found", folderPath);
|
||||
return false;
|
||||
}
|
||||
strncpy(foundStem, name, sizeof(foundStem) - 1);
|
||||
foundStem[sizeof(foundStem) - 1] = '\0';
|
||||
}
|
||||
|
||||
if (foundStem[0] == '\0') return false;
|
||||
|
||||
// Require dictionary data next to the index, so folders holding only an
|
||||
// .idx never surface as selectable dictionaries that fail at lookup time.
|
||||
const std::string base = std::string(folderPath) + "/" + foundStem;
|
||||
if (!Storage.exists((base + ".dict").c_str()) && !Storage.exists((base + ".dict.dz").c_str())) {
|
||||
LOG_DBG("DREG", "Skipping %s: no .dict or .dict.dz", folderPath);
|
||||
return false;
|
||||
}
|
||||
|
||||
stemOut = foundStem;
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
void discover(std::vector<DictionaryEntry>& out) {
|
||||
out.clear();
|
||||
out.reserve(8);
|
||||
|
||||
for (const char* dictRoot : DICT_ROOTS) {
|
||||
auto rootDir = Storage.open(dictRoot);
|
||||
if (!rootDir || !rootDir.isDirectory()) {
|
||||
LOG_DBG("DREG", "No %s directory on SD card", dictRoot);
|
||||
continue;
|
||||
}
|
||||
|
||||
rootDir.rewindDirectory();
|
||||
char name[128];
|
||||
for (auto entry = rootDir.openNextFile(); entry; entry = rootDir.openNextFile()) {
|
||||
entry.getName(name, sizeof(name));
|
||||
if (!entry.isDirectory() || name[0] == '.') continue;
|
||||
|
||||
std::string folderPath = std::string(dictRoot) + "/" + name;
|
||||
std::string stem;
|
||||
if (!findStem(folderPath.c_str(), stem)) continue;
|
||||
|
||||
DictionaryEntry e;
|
||||
e.name = name;
|
||||
e.stem = std::move(stem);
|
||||
out.push_back(std::move(e));
|
||||
LOG_DBG("DREG", "Found dictionary: %s", name);
|
||||
}
|
||||
}
|
||||
|
||||
// Case-insensitive sort by folder name (matches FileBrowserActivity ordering).
|
||||
std::sort(out.begin(), out.end(), [](const DictionaryEntry& a, const DictionaryEntry& b) {
|
||||
return StringUtils::asciiCaseCmp(a.name.c_str(), b.name.c_str()) < 0;
|
||||
});
|
||||
}
|
||||
|
||||
bool resolveBasePath(const char* folderName, std::string& basePathOut) {
|
||||
if (!folderName || folderName[0] == '\0') return false;
|
||||
// folderName is persisted in the settings JSON: reject separators and dot
|
||||
// prefixes so a crafted value cannot escape the dictionary roots.
|
||||
if (folderName[0] == '.' || strpbrk(folderName, "/\\") != nullptr) return false;
|
||||
|
||||
for (const char* dictRoot : DICT_ROOTS) {
|
||||
std::string folderPath = std::string(dictRoot) + "/" + folderName;
|
||||
std::string stem;
|
||||
if (!findStem(folderPath.c_str(), stem)) continue;
|
||||
basePathOut = folderPath + "/" + stem;
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
} // namespace DictionaryRegistry
|
||||
@@ -0,0 +1,25 @@
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
// One StarDict dictionary found under /dictionaries or /.dictionaries: a
|
||||
// subfolder holding <stem>.idx plus <stem>.dict or <stem>.dict.dz.
|
||||
struct DictionaryEntry {
|
||||
std::string name; // subfolder name (shown to the user, stored in settings)
|
||||
std::string stem; // index basename without .idx
|
||||
};
|
||||
|
||||
namespace DictionaryRegistry {
|
||||
|
||||
// Scan /dictionaries/*/ and /.dictionaries/*/ for dictionaries. Folders with
|
||||
// multiple index stems are ambiguous and skipped. Result is sorted
|
||||
// case-insensitively by name.
|
||||
void discover(std::vector<DictionaryEntry>& out);
|
||||
|
||||
// Resolve a folder name to its extensionless base path
|
||||
// ("/dictionaries/<folder>/<stem>" or "/.dictionaries/<folder>/<stem>").
|
||||
// Returns false if the folder holds no usable dictionary in either root.
|
||||
bool resolveBasePath(const char* folderName, std::string& basePathOut);
|
||||
|
||||
} // namespace DictionaryRegistry
|
||||
@@ -0,0 +1,150 @@
|
||||
#include "HtmlToPlainText.h"
|
||||
|
||||
#include <Epub/htmlEntities.h>
|
||||
|
||||
#include <cctype>
|
||||
#include <cstdint>
|
||||
#include <cstring>
|
||||
|
||||
namespace {
|
||||
|
||||
bool isTagStart(const std::string& input, size_t pos) {
|
||||
if (pos + 1 >= input.size()) return false;
|
||||
const unsigned char next = input[pos + 1];
|
||||
return next == '/' || next == '!' || next == '?' || std::isalpha(next);
|
||||
}
|
||||
|
||||
enum class TagBreak : uint8_t { None, Line, Paragraph };
|
||||
|
||||
TagBreak tagBreak(const std::string& input, size_t start, size_t end) {
|
||||
while (start < end && (input[start] == '/' || std::isspace(static_cast<unsigned char>(input[start])))) start++;
|
||||
const size_t nameStart = start;
|
||||
while (start < end && std::isalpha(static_cast<unsigned char>(input[start]))) start++;
|
||||
const size_t len = start - nameStart;
|
||||
if (len == 0) return TagBreak::None;
|
||||
|
||||
const auto equals = [&](const char* name) {
|
||||
const size_t nameLen = strlen(name);
|
||||
if (len != nameLen) return false;
|
||||
for (size_t i = 0; i < len; i++) {
|
||||
if (std::tolower(static_cast<unsigned char>(input[nameStart + i])) != name[i]) return false;
|
||||
}
|
||||
return true;
|
||||
};
|
||||
|
||||
if (equals("p") || equals("h1") || equals("h2") || equals("h3") || equals("h4") || equals("h5") || equals("h6") ||
|
||||
equals("hr")) {
|
||||
return TagBreak::Paragraph;
|
||||
}
|
||||
if (equals("br") || equals("div") || equals("li") || equals("tr")) return TagBreak::Line;
|
||||
return TagBreak::None;
|
||||
}
|
||||
|
||||
void appendCodepoint(std::string& output, uint32_t codepoint) {
|
||||
if (codepoint == 0 || codepoint > 0x10FFFF || (codepoint >= 0xD800 && codepoint <= 0xDFFF)) return;
|
||||
if (codepoint <= 0x7F) {
|
||||
output.push_back(static_cast<char>(codepoint));
|
||||
} else if (codepoint <= 0x7FF) {
|
||||
output.push_back(static_cast<char>(0xC0 | (codepoint >> 6)));
|
||||
output.push_back(static_cast<char>(0x80 | (codepoint & 0x3F)));
|
||||
} else if (codepoint <= 0xFFFF) {
|
||||
output.push_back(static_cast<char>(0xE0 | (codepoint >> 12)));
|
||||
output.push_back(static_cast<char>(0x80 | ((codepoint >> 6) & 0x3F)));
|
||||
output.push_back(static_cast<char>(0x80 | (codepoint & 0x3F)));
|
||||
} else {
|
||||
output.push_back(static_cast<char>(0xF0 | (codepoint >> 18)));
|
||||
output.push_back(static_cast<char>(0x80 | ((codepoint >> 12) & 0x3F)));
|
||||
output.push_back(static_cast<char>(0x80 | ((codepoint >> 6) & 0x3F)));
|
||||
output.push_back(static_cast<char>(0x80 | (codepoint & 0x3F)));
|
||||
}
|
||||
}
|
||||
|
||||
bool appendNumericEntity(std::string& output, const char* entity, size_t len) {
|
||||
if (len < 4 || entity[0] != '&' || entity[1] != '#' || entity[len - 1] != ';') return false;
|
||||
size_t pos = 2;
|
||||
uint32_t base = 10;
|
||||
if (entity[pos] == 'x' || entity[pos] == 'X') {
|
||||
base = 16;
|
||||
pos++;
|
||||
}
|
||||
if (pos == len - 1) return false;
|
||||
|
||||
uint32_t value = 0;
|
||||
for (; pos < len - 1; pos++) {
|
||||
const unsigned char c = entity[pos];
|
||||
uint32_t digit;
|
||||
if (c >= '0' && c <= '9')
|
||||
digit = c - '0';
|
||||
else if (base == 16 && c >= 'a' && c <= 'f')
|
||||
digit = c - 'a' + 10;
|
||||
else if (base == 16 && c >= 'A' && c <= 'F')
|
||||
digit = c - 'A' + 10;
|
||||
else
|
||||
return false;
|
||||
if (value > (0x10FFFF - digit) / base) return false;
|
||||
value = value * base + digit;
|
||||
}
|
||||
appendCodepoint(output, value);
|
||||
return true;
|
||||
}
|
||||
|
||||
void appendBreak(std::string& output, size_t count = 1) {
|
||||
while (!output.empty() && output.back() == ' ') output.pop_back();
|
||||
if (output.empty()) return;
|
||||
size_t existing = 0;
|
||||
while (existing < output.size() && output[output.size() - existing - 1] == '\n') existing++;
|
||||
while (existing++ < count) output.push_back('\n');
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::string htmlToPlainText(const std::string& html) {
|
||||
std::string output;
|
||||
output.reserve(html.size());
|
||||
|
||||
for (size_t i = 0; i < html.size();) {
|
||||
if (html[i] == '<' && isTagStart(html, i)) {
|
||||
const size_t close = html.find('>', i + 1);
|
||||
if (close == std::string::npos) {
|
||||
output.push_back(html[i++]);
|
||||
continue;
|
||||
}
|
||||
const TagBreak separator = tagBreak(html, i + 1, close);
|
||||
if (separator == TagBreak::Line)
|
||||
appendBreak(output);
|
||||
else if (separator == TagBreak::Paragraph)
|
||||
appendBreak(output, 2);
|
||||
i = close + 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
if (html[i] == '&') {
|
||||
const size_t semicolon = html.find(';', i + 1);
|
||||
if (semicolon != std::string::npos && semicolon - i <= 16) {
|
||||
const size_t len = semicolon - i + 1;
|
||||
const char* value = lookupHtmlEntity(html.data() + i, len);
|
||||
if (value != nullptr) {
|
||||
output.append(value);
|
||||
i = semicolon + 1;
|
||||
continue;
|
||||
}
|
||||
if (appendNumericEntity(output, html.data() + i, len)) {
|
||||
i = semicolon + 1;
|
||||
continue;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const unsigned char c = html[i++];
|
||||
if (c == '\r' || c == '\t') {
|
||||
if (!output.empty() && output.back() != ' ' && output.back() != '\n') output.push_back(' ');
|
||||
} else if (c == '\n') {
|
||||
appendBreak(output);
|
||||
} else {
|
||||
output.push_back(static_cast<char>(c));
|
||||
}
|
||||
}
|
||||
|
||||
while (!output.empty() && (output.back() == ' ' || output.back() == '\n')) output.pop_back();
|
||||
return output;
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
|
||||
// Convert an HTML fragment to readable plain text. This intentionally ignores
|
||||
// styling; block elements become line breaks and HTML entities are decoded.
|
||||
std::string htmlToPlainText(const std::string& html);
|
||||
@@ -1,9 +1,32 @@
|
||||
#pragma once
|
||||
|
||||
#include <cctype>
|
||||
#include <string>
|
||||
|
||||
namespace StringUtils {
|
||||
|
||||
/**
|
||||
* Case-insensitive ASCII strcmp. Returns <0, 0, or >0 like strcmp, comparing
|
||||
* each byte by its lowercased value.
|
||||
*
|
||||
* Used wherever data is sorted case-insensitively: StarDict indexes (including
|
||||
* wiktionary-derived dictionaries) and the on-disk dictionary folder list.
|
||||
* Plain strcmp would land a binary search on the wrong page for any word whose
|
||||
* alphabetic neighbourhood contains mixed-case boundaries.
|
||||
*
|
||||
* Inline (header) because Dictionary's binary search calls it per comparison
|
||||
* step; a cross-TU call here would defeat inlining on a hot path.
|
||||
*/
|
||||
inline int asciiCaseCmp(const char* a, const char* b) {
|
||||
while (*a && *b) {
|
||||
int diff = std::tolower(static_cast<unsigned char>(*a)) - std::tolower(static_cast<unsigned char>(*b));
|
||||
if (diff != 0) return diff;
|
||||
++a;
|
||||
++b;
|
||||
}
|
||||
return std::tolower(static_cast<unsigned char>(*a)) - std::tolower(static_cast<unsigned char>(*b));
|
||||
}
|
||||
|
||||
/**
|
||||
* Sanitize a string for use as a filename.
|
||||
* Replaces invalid characters with underscores, trims spaces/dots,
|
||||
|
||||
Reference in New Issue
Block a user