Justin Mitchell
fedcb2f53d
Opening XTC files with a high page count (e.g. *The Magic Mountain* at 4,187 pages) causes an immediate `abort()` crash and reboot loop. The device becomes unusable until the book is removed from the SD card. **Crash log:** ``` abort() was called at 0x4214a5fb on core 0 ``` ### Root cause During `XtcParser::open()`, the parser calls `m_pageTable.resize(pageCount)` to load the entire page table into RAM. Each `PageInfo` entry is 16 bytes, so: - 4,187 pages x 16 bytes = **66,992 bytes (~65KB)** as a single contiguous heap allocation On the ESP32-C3 with ~380KB total RAM (no PSRAM), this allocation fails after firmware, fonts, and the activity system are already loaded. Because the firmware is compiled with `-fno-exceptions`, the failed `new` inside `std::vector::resize()` calls `abort()` instead of throwing. This affects any XTC file with roughly 3,000+ pages, depending on heap state at the time of loading. ## Solution Replace the bulk page table allocation with on-demand reads from the SD card. Instead of loading all page table entries into a vector at file open, we now: 1. Read only the **first** page table entry at open time (to get default page dimensions) 2. Read a **single** 16-byte entry from the SD card each time a page is loaded This reduces page table memory usage from `pageCount * 16` bytes to **zero bytes**, regardless of how many pages the file contains. ### Changes | File | What changed | |------|-------------| | `XtcParser.h` | Removed `std::vector<PageInfo> m_pageTable`. Added `readPageTableEntry()` for on-demand reads. | | `XtcParser.cpp` | Replaced `readPageTable()` with `readFirstPageInfo()`. Updated `getPageInfo()`, `loadPage()`, and `loadPageStreaming()` to seek and read individual entries from the file. | ## Trade-offs ### Performance Each page turn now requires one additional SD card seek + 16-byte read to look up the page table entry before reading the page data itself. - SD card sequential read latency: ~0.1-0.5ms for a 16-byte read - E-ink full display refresh: ~1,000-2,000ms I personally can't see any performance difference while reading and the trade off of not boot looping seems to make this well worth it. ### Memory | Metric | Before | After | |--------|--------|-------| | Page table RAM (4,187 pages) | ~65KB | 0 bytes | | Page table RAM (1,000 pages) | ~16KB | 0 bytes | | Page table RAM (max 65,535 pages) | ~1MB (impossible) | 0 bytes |
115 lines
3.1 KiB
C++
115 lines
3.1 KiB
C++
/**
|
|
* XtcParser.h
|
|
*
|
|
* XTC file parsing and page data extraction
|
|
* XTC ebook support for CrossPoint Reader
|
|
*/
|
|
|
|
#pragma once
|
|
|
|
#include <HalStorage.h>
|
|
|
|
#include <functional>
|
|
#include <memory>
|
|
#include <string>
|
|
#include <vector>
|
|
|
|
#include "XtcTypes.h"
|
|
|
|
namespace xtc {
|
|
|
|
/**
|
|
* XTC File Parser
|
|
*
|
|
* Reads XTC files from SD card and extracts page data.
|
|
* Designed for ESP32-C3's limited RAM (~380KB) using streaming.
|
|
*
|
|
* The source file is kept closed between reads to free heap for rendering.
|
|
* It is reopened on-demand for page table lookups and bitmap data reads.
|
|
*/
|
|
class XtcParser {
|
|
public:
|
|
XtcParser();
|
|
~XtcParser();
|
|
|
|
// File open/close
|
|
XtcError open(const char* filepath);
|
|
void close();
|
|
bool isOpen() const { return m_isOpen; }
|
|
|
|
// Header information access
|
|
const XtcHeader& getHeader() const { return m_header; }
|
|
uint16_t getPageCount() const { return m_header.pageCount; }
|
|
uint16_t getWidth() const { return m_defaultWidth; }
|
|
uint16_t getHeight() const { return m_defaultHeight; }
|
|
uint8_t getBitDepth() const { return m_bitDepth; } // 1 = XTC/XTG, 2 = XTCH/XTH
|
|
|
|
// Page information
|
|
bool getPageInfo(uint32_t pageIndex, PageInfo& info);
|
|
|
|
/**
|
|
* Load page bitmap (raw 1-bit data, skipping XTG header)
|
|
*
|
|
* @param pageIndex Page index (0-based)
|
|
* @param buffer Output buffer (caller allocated)
|
|
* @param bufferSize Buffer size
|
|
* @return Number of bytes read on success, 0 on failure
|
|
*/
|
|
size_t loadPage(uint32_t pageIndex, uint8_t* buffer, size_t bufferSize);
|
|
|
|
/**
|
|
* Streaming page load
|
|
* Memory-efficient method that reads page data in chunks.
|
|
*
|
|
* @param pageIndex Page index
|
|
* @param callback Callback function to receive data chunks
|
|
* @param chunkSize Chunk size (default: 1024 bytes)
|
|
* @return Error code
|
|
*/
|
|
XtcError loadPageStreaming(uint32_t pageIndex,
|
|
std::function<void(const uint8_t* data, size_t size, size_t offset)> callback,
|
|
size_t chunkSize = 1024);
|
|
|
|
// Get title/author from metadata
|
|
std::string getTitle() const { return m_title; }
|
|
std::string getAuthor() const { return m_author; }
|
|
|
|
bool hasChapters() const { return m_hasChapters; }
|
|
const std::vector<ChapterInfo>& getChapters();
|
|
|
|
// Validation
|
|
static bool isValidXtcFile(const char* filepath);
|
|
|
|
// Error information
|
|
XtcError getLastError() const { return m_lastError; }
|
|
|
|
private:
|
|
FsFile m_file;
|
|
std::string m_filepath;
|
|
bool m_isOpen;
|
|
XtcHeader m_header;
|
|
std::vector<ChapterInfo> m_chapters;
|
|
std::string m_title;
|
|
std::string m_author;
|
|
uint16_t m_defaultWidth;
|
|
uint16_t m_defaultHeight;
|
|
uint8_t m_bitDepth; // 1 = XTC/XTG (1-bit), 2 = XTCH/XTH (2-bit)
|
|
bool m_hasChapters;
|
|
bool m_chaptersLoaded;
|
|
XtcError m_lastError;
|
|
|
|
// Internal helper functions
|
|
XtcError readHeader();
|
|
XtcError readFirstPageInfo();
|
|
XtcError readTitle();
|
|
XtcError readAuthor();
|
|
XtcError readChapters();
|
|
bool readPageTableEntry(uint32_t pageIndex, PageInfo& info);
|
|
|
|
// File handle management — reopen on demand, close after use
|
|
bool ensureFileOpen();
|
|
void closeFile();
|
|
};
|
|
|
|
} // namespace xtc
|