## Summary
**What is the goal of this PR?**
Restore compatibility with older XTC files that use the legacy 48-byte
header layout, so they open correctly instead of failing
header/page-table validation.
**What changes are included?**
- Updates the XTC parser to accept legacy files where pageTableOffset
starts at 0x30 instead of the newer full header size.
- Avoids treating legacy header bytes as a valid chapterOffset, so older
files are not misclassified as having chapters.
- Switches XTC file size and seek operations to 64-bit-safe wrapper
calls in HalStorage, which matches the XTC format’s 64-bit offset fields
and keeps page/chapter lookups from narrowing offsets.
- Keeps the existing bounds checks and improves related logging when XTC
page loads fail.
## Additional Context
- This is a narrow compatibility fix for XTC handling. The functional
change is in the XTC parser; the HalStorage changes are supporting
wrapper methods needed by that parser update.
- Risk should be low outside XTC reading, since the HAL changes only
expose existing underlying file APIs rather than changing storage
behavior.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **YES**
Opening XTC files with a high page count (e.g. *The Magic Mountain* at
4,187 pages) causes an immediate `abort()` crash and reboot loop. The
device becomes unusable until the book is removed from the SD card.
**Crash log:**
```
abort() was called at 0x4214a5fb on core 0
```
### Root cause
During `XtcParser::open()`, the parser calls
`m_pageTable.resize(pageCount)` to load the entire page table into RAM.
Each `PageInfo` entry is 16 bytes, so:
- 4,187 pages x 16 bytes = **66,992 bytes (~65KB)** as a single
contiguous heap allocation
On the ESP32-C3 with ~380KB total RAM (no PSRAM), this allocation fails
after firmware, fonts, and the activity system are already loaded.
Because the firmware is compiled with `-fno-exceptions`, the failed
`new` inside `std::vector::resize()` calls `abort()` instead of
throwing.
This affects any XTC file with roughly 3,000+ pages, depending on heap
state at the time of loading.
## Solution
Replace the bulk page table allocation with on-demand reads from the SD
card. Instead of loading all page table entries into a vector at file
open, we now:
1. Read only the **first** page table entry at open time (to get default
page dimensions)
2. Read a **single** 16-byte entry from the SD card each time a page is
loaded
This reduces page table memory usage from `pageCount * 16` bytes to
**zero bytes**, regardless of how many pages the file contains.
### Changes
| File | What changed |
|------|-------------|
| `XtcParser.h` | Removed `std::vector<PageInfo> m_pageTable`. Added
`readPageTableEntry()` for on-demand reads. |
| `XtcParser.cpp` | Replaced `readPageTable()` with
`readFirstPageInfo()`. Updated `getPageInfo()`, `loadPage()`, and
`loadPageStreaming()` to seek and read individual entries from the file.
|
## Trade-offs
### Performance
Each page turn now requires one additional SD card seek + 16-byte read
to look up the page table entry before reading the page data itself.
- SD card sequential read latency: ~0.1-0.5ms for a 16-byte read
- E-ink full display refresh: ~1,000-2,000ms
I personally can't see any performance difference while reading and the
trade off of not boot looping seems to make this well worth it.
### Memory
| Metric | Before | After |
|--------|--------|-------|
| Page table RAM (4,187 pages) | ~65KB | 0 bytes |
| Page table RAM (1,000 pages) | ~16KB | 0 bytes |
| Page table RAM (max 65,535 pages) | ~1MB (impossible) | 0 bytes |
## Summary
**What is the goal of this PR?** (e.g., Implements the new feature for
file uploading.)
`DESTRUCTOR_CLOSES_FILE=1` is set in platformio.ini, which makes SdFat's
FsBaseFile destructor call close() automatically when a file goes out of
scope.
Three categories of file close calls remain untouched:
1. Close before Storage.remove() on the same path: ScreenshotUtil.cpp
closes the file before deleting it on write error. The remove might fail
if the file is still open.
2. Close before reopening the same variable: Epub.cpp writes a temp
NCX/nav file, closes it, then reopens it for reading. The
RecentBooksStore.cpp close before saveToFile() is the same pattern, it
rewrites the same file.
3. Close on member variables: BookMetadataCache.cpp (bookFile,
spineFile, tocFile), Section.cpp (file), XtcParser.cpp (m_file),
ZipFile.cpp (file). These persist beyond any single function scope, so
the destructor timing doesn't match the intended close point.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**PARTIALLY**_
## Summary
**What is the goal of this PR?**
This change avoids the pattern of creating a `std::string` using
`.substr` in order to compare against a file extension literal.
```c++
std::string path;
if (path.length() >= 4 && path.substr(path.length() - 4) == ".ext")
```
The `checkFileExtension` utility has moved from StringUtils to
FsHelpers, to be available to code in lib/. The signature now accepts a
`std::string_view` instead of `std::string`, which makes the single
implementation reusable for Arduino `String`.
Added utility functions for commonly repeated extensions.
These changes **save about 2 KB of flash (5,999,427 to 5,997,343)**.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _**NO**_
## Summary
* Definition and use of a central LOG function, that can later be
extended or completely be removed (for public use where debugging
information may not be required) to save flash by suppressing the
-DENABLE_SERIAL_LOG like in the slim branch
* **What changes are included?**
## Additional Context
* By using the central logger the usual:
```
#include <HardwareSerial.h>
...
Serial.printf("[%lu] [WCS] Obfuscating/deobfuscating %zu bytes\n", millis(), data.size());
```
would then become
```
#include <Logging.h>
...
LOG_DBG("WCS", "Obfuscating/deobfuscating %zu bytes", data.size());
```
You do have ``LOG_DBG`` for debug messages, ``LOG_ERR`` for error
messages and ``LOG_INF`` for informational messages. Depending on the
verbosity level defined (see below) soe of these message types will be
suppressed/not-compiled.
* The normal compilation (default) will create a firmware.elf file of
42.194.356 bytes, the same code via slim will create 42.024.048 bytes -
170.308 bytes less
* Firmware.bin : 6.469.984 bytes for default, 6.418.672 bytes for slim -
51.312 bytes less
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _NO_
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
## Summary
Continue my changes to introduce the HAL infrastructure from
https://github.com/crosspoint-reader/crosspoint-reader/pull/522
This PR touches quite a lot of files, but most of them are just name
changing. It should not have any impacts to the end behavior.
## Additional Context
My plan is to firstly add this small shim layer, which sounds useless at
first, but then I'll implement an emulated driver which can be helpful
for testing and for development.
Currently, on my fork, I'm using a FS driver that allow "mounting" a
local directory from my computer to the device, much like the `-v` mount
option on docker. This allows me to quickly reset `.crosspoint`
directory if anything goes wrong. I plan to upstream this feature when
this PR get merged.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? NO
## Summary
* Extract author from XTC/XTCH files
## Additional Context
* Based on updated details in
https://gist.github.com/CrazyCoder/b125f26d6987c0620058249f59f1327d
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? No
## Summary
* Swap to updated SDCardManager which uses SdFat
* Add exFAT support
* Swap to using FsFile everywhere
* Use newly exposed `SdMan` macro to get to static instance of
SDCardManager
* Move a bunch of FsHelpers up to SDCardManager
## Summary
- **What is the goal of this PR?** Add chapter selection support to the
XTC reader activity, including parsing chapter metadata from XTC files.
- **What changes are included?** Implemented XTC chapter parsing and
exposure in the XTC library, added a chapter selection activity for XTC,
integrated it into XtcReaderActivity, and normalized chapter page
indices by shifting them to 0-based.
## Additional Context
- The reader uses 0-based page indexing (first page = 0), but the XTC
chapter table appears to be 1-based (first page = 1), so chapter
start/end pages are shifted down by 1 during parsing.
## Summary
* **What is the goal of this PR?**
Add support for XTC (XTeink X4 native) ebook format, which contains
pre-rendered 480x800 1-bit bitmap pages optimized for e-ink displays.
* **What changes are included?**
- New `lib/Xtc/` library with XtcParser for reading XTC files
- XtcReaderActivity for displaying XTC pages on e-ink display
- XTC file detection in FileSelectionActivity
- Cover BMP generation from first XTC page
- Correct XTG page header structure (22 bytes) and bit polarity handling
## Additional Context
- XTC files contain pre-rendered bitmap pages with embedded status bar
(page numbers, progress %)
- XTG page header: 22 bytes (magic + dimensions + reserved fields +
bitmap size)
- Bit polarity: 0 = black, 1 = white
- No runtime text rendering needed - pages display directly on e-ink
- Faster page display compared to EPUB since no parsing/rendering
required
- Memory efficient: loads one page at a time (48KB per page)
- Tested with XTC files generated from https://x4converter.rho.sh/
- Verified correct page alignment and color rendering
- Please report any issues if you test with XTC files from other
sources.
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
Zach Nelson