Compare commits

...
8 Commits
Author SHA1 Message Date
Justin Mitchell 98118d6e24 Improve heap management in EPUB rendering
Fix CPU pinning by tracking buildHeapPaused state when background build is blocked. Re-check heap gate after acquiring lock to handle race conditions where rendering expands glyph buffers. Refine plane buffer allocation to check both free heap and max contiguous block, preventing fragmentation issues that could cause OOM during page renders. Skip async refresh for image pages since they use blocking double-FAST path.
2026-07-17 12:16:32 -04:00
Justin Mitchell 3cc4069faa fix: suppress cppcheck missingInclude on fresh checkouts
On a fresh CI checkout cppcheck has no resolved include paths, so it
reports every project header as missing (~400 information-level lines)
and --fail-on-defect low fails the job. Suppress it alongside
missingIncludeSystem, matching the fix already proven on the BLE branch.
2026-07-14 21:37:11 -04:00
Justin Mitchell a3b8897977 fix: address review findings on prewarm gating and gitignore
Revalidate the section under RenderLock before the idle prewarm scan,
add a largest-block floor to its heap gate, rebuild the differential
baseline when the grayscale scratch allocation fails after an async
refresh, and anchor generated-file gitignore patterns to the repo root.
2026-07-14 16:29:33 -04:00
Justin Mitchell f7f1a6f8bf Merge remote-tracking branch 'origin/develop' into feat-deferred-refresh 2026-07-14 15:49:04 -04:00
Justin Mitchell 7cad3cfb0e Fix heap fragmentation issues
- Keep-if-fits font buffer reuse (SdCardFont), the page-turn fragmentation fix
- Background-build heap floors + buildTickHeapGate() (the ParsedText::addWord abort fix), with the BLE-shed branch removed
- KOSync TLS gate split (free ≥ 50K, largest block ≥ 20K)
- wolfSSL SP ECC flags + FP_MAX_BITS 8192 in the patch script
- custom_sdkconfig: timer-stack trims (~7KB) and, per your answer, the WiFi IRAM opts (~25-30KB); plus the cloud-component removal the hybrid build requires. All verified present in the generated sdkconfig after the build.
- Web server watchdog registration fix (this branch's handlers already call esp_task_wdt_reset without it)
2026-07-14 15:45:44 -04:00
Justin Mitchell e66de575a1 Add async display refresh support to HalDisplay
Introduces non-blocking refresh capability with displayBufferAsync() that starts panel waveform and returns immediately, plus waitRefreshComplete() to block until refresh finishes. Panels without async support fall back to blocking refresh. Requires framebuffer to remain untouched during refresh and differential baseline rebuild before next update.
2026-07-13 16:20:20 -04:00
Justin Mitchell bae29fc6ba Update freeink-sdk
1. X3 wb_gc passive byte (the one-byte 0x54 to 0x00, matching stock's passive WB table in the gc nudge bank) Every X3 grayscale render, reader AA, covers, and the sleep-cover nudge, gets it. 2. X4 blotchy grays on sleep covers: the displayGrayscaleBase(HALF_REFRESH) revert (the gray nudge LUT being calibrated against the HALF pixel state, not FULL's)
2026-07-13 02:23:59 -04:00
Justin Mitchell 379f49d165 Use HALF_REFRESH for sleep screens instead of FULL
Changes all sleep screen display refreshes from FULL_REFRESH to HALF_REFRESH to match OEM firmware behavior. The stock firmware uses a single-pass 0xD7 waveform for sleep screens, not the multi-flash GC waveform (0xF7) that FULL_REFRESH triggers. For grayscale mode, HALF_REFRESH is required because the gray nudge LUT is calibrated against the pixel charge state left by the single-pass waveform; using FULL_REFRESH causes blotchy noise in gray areas.
2026-07-13 01:08:05 -04:00
16 changed files with 438 additions and 70 deletions
+6
View File
@@ -25,3 +25,9 @@ lib/EpdFont/scripts/output/
# (worktrees, scheduled-task locks, settings.local, scout CLEANUP.md) out.
.claude/*
!.claude/skills/
/managed_components
/.dummy
/CMakeLists.txt
/dependencies.lock
/sdkconfig.default
/sdkconfig.defaults
+44 -17
View File
@@ -68,6 +68,22 @@ bool collectUniqueCodepoints(const char* text, uint32_t* codepoints, uint32_t& c
const char* asCStr(const std::string& s) { return s.c_str(); }
const char* asCStr(const char* s) { return s; }
// Keep-if-fits buffer reuse: only reallocate when the needed size exceeds the
// current capacity. Freeing + reallocating slightly different sizes every page
// turn punches non-coalescing holes in the heap (the freed block rarely fits the
// next page's need), eroding the largest contiguous block all session. With
// reuse, capacities converge on the book's max page after a few turns and page
// turns stop touching the allocator. Only three small instantiations exist
// (interval/glyph/byte arrays), so template bloat is negligible.
template <typename T, typename CapT>
bool ensureArrayCapacity(T*& buf, CapT& capacity, const uint32_t needed) {
if (buf && capacity >= needed) return true;
delete[] buf;
buf = new (std::nothrow) T[needed > 0 ? needed : 1];
capacity = buf ? static_cast<CapT>(needed) : 0;
return buf != nullptr;
}
} // namespace
SdCardFont::~SdCardFont() { freeAll(); }
@@ -83,6 +99,9 @@ void SdCardFont::freeStyleMiniData(PerStyle& s) {
s.miniBitmap = nullptr;
s.miniIntervalCount = 0;
s.miniGlyphCount = 0;
s.miniIntervalCapacity = 0;
s.miniGlyphCapacity = 0;
s.miniBitmapCapacity = 0;
freeStyleMiniKern(s);
memset(&s.miniData, 0, sizeof(s.miniData));
s.epdFont.data = &s.stubData;
@@ -109,6 +128,9 @@ void SdCardFont::freeStyleMiniKern(PerStyle& s) {
s.miniKernRightEntryCount = 0;
s.miniKernLeftClassCount = 0;
s.miniKernRightClassCount = 0;
s.miniKernLeftCapacity = 0;
s.miniKernRightCapacity = 0;
s.miniKernMatrixCapacity = 0;
}
void SdCardFont::freeStyleAll(PerStyle& s) {
@@ -311,13 +333,13 @@ bool SdCardFont::buildMiniKernMatrix(PerStyle& s, const uint32_t* codepoints, ui
if (miniLookupKernClass(s.kernRightClasses, s.header.kernRightEntryCount, codepoints[i]) != 0) miniRightCount++;
}
// Step 4: allocate the three mini buffers. The matrix is <1KB in practice
// (<30 × <30 × 1 byte) so fragmentation is a non-issue.
// Step 4: size the three mini buffers (reused across pages when they fit; the
// per-page sizes vary by a few entries, which as free+realloc churn was punching
// non-coalescing holes in the heap every page turn).
const uint32_t matrixBytes = static_cast<uint32_t>(numLeft) * numRight;
s.miniKernLeftClasses = new (std::nothrow) EpdKernClassEntry[miniLeftCount];
s.miniKernRightClasses = new (std::nothrow) EpdKernClassEntry[miniRightCount];
s.miniKernMatrix = new (std::nothrow) int8_t[matrixBytes];
if (!s.miniKernLeftClasses || !s.miniKernRightClasses || !s.miniKernMatrix) {
if (!ensureArrayCapacity(s.miniKernLeftClasses, s.miniKernLeftCapacity, miniLeftCount) ||
!ensureArrayCapacity(s.miniKernRightClasses, s.miniKernRightCapacity, miniRightCount) ||
!ensureArrayCapacity(s.miniKernMatrix, s.miniKernMatrixCapacity, matrixBytes)) {
LOG_ERR("SDCF", "Failed to allocate mini kern (%u+%u+%u bytes)", miniLeftCount * 3u, miniRightCount * 3u,
matrixBytes);
freeStyleMiniKern(s);
@@ -793,12 +815,19 @@ int SdCardFont::prewarmStyle(uint8_t styleIdx, const uint32_t* codepoints, uint3
return missed;
}
// Build mini intervals from sorted codepoints
freeStyleMiniData(s);
// Build mini intervals from sorted codepoints. Reset counts and fall back to the
// stub until the rebuild completes, but KEEP the existing buffers (keep-if-fits
// reuse) — the free-and-realloc-per-page pattern here was a primary fragmenter.
s.miniIntervalCount = 0;
s.miniGlyphCount = 0;
s.miniKernLeftEntryCount = 0;
s.miniKernRightEntryCount = 0;
s.miniKernLeftClassCount = 0;
s.miniKernRightClassCount = 0;
memset(&s.miniData, 0, sizeof(s.miniData));
s.epdFont.data = &s.stubData;
uint32_t intervalCapacity = validCount;
s.miniIntervals = new (std::nothrow) EpdUnicodeInterval[intervalCapacity];
if (!s.miniIntervals) {
if (!ensureArrayCapacity(s.miniIntervals, s.miniIntervalCapacity, validCount)) {
LOG_ERR("SDCF", "Failed to allocate mini intervals for style %u", styleIdx);
delete[] mappings;
return static_cast<int>(cpCount);
@@ -816,15 +845,14 @@ int SdCardFont::prewarmStyle(uint8_t styleIdx, const uint32_t* codepoints, uint3
}
}
// Allocate mini glyph array
s.miniGlyphCount = validCount;
s.miniGlyphs = new (std::nothrow) EpdGlyph[s.miniGlyphCount];
if (!s.miniGlyphs) {
// Mini glyph array (reused across pages when it fits)
if (!ensureArrayCapacity(s.miniGlyphs, s.miniGlyphCapacity, validCount)) {
LOG_ERR("SDCF", "Failed to allocate mini glyphs for style %u", styleIdx);
delete[] mappings;
freeStyleMiniData(s);
return static_cast<int>(cpCount);
}
s.miniGlyphCount = validCount;
// Build sorted read order for sequential I/O
uint32_t* readOrder = new (std::nothrow) uint32_t[validCount];
@@ -891,8 +919,7 @@ int SdCardFont::prewarmStyle(uint8_t styleIdx, const uint32_t* codepoints, uint3
totalBitmapSize += s.miniGlyphs[i].dataLength;
}
s.miniBitmap = new (std::nothrow) uint8_t[totalBitmapSize > 0 ? totalBitmapSize : 1];
if (!s.miniBitmap) {
if (!ensureArrayCapacity(s.miniBitmap, s.miniBitmapCapacity, totalBitmapSize)) {
LOG_ERR("SDCF", "Failed to allocate mini bitmap (%u bytes) for style %u", totalBitmapSize, styleIdx);
delete[] readOrder;
delete[] mappings;
+14 -1
View File
@@ -168,13 +168,22 @@ class SdCardFont {
// Stub EpdFontData returned when not prewarmed
EpdFontData stubData{};
// Mini EpdFontData built during prewarm
// Mini EpdFontData built during prewarm. Buffers are kept-if-fits across pages
// (capacities below track allocated sizes): freeing and reallocating slightly
// different sizes on every page turn was a primary heap fragmenter — each page's
// freed hole rarely fit the next page's need, so maxAlloc eroded all session.
// After a few pages the capacities converge on the book's max and page turns
// stop allocating entirely. freeStyleMiniData() still releases everything (and
// zeroes capacities) for style eviction / font unload.
EpdFontData miniData{};
EpdUnicodeInterval* miniIntervals = nullptr;
EpdGlyph* miniGlyphs = nullptr;
uint8_t* miniBitmap = nullptr;
uint32_t miniIntervalCount = 0;
uint32_t miniGlyphCount = 0;
uint32_t miniIntervalCapacity = 0;
uint32_t miniGlyphCapacity = 0;
uint32_t miniBitmapCapacity = 0;
// Per-page mini kern matrix (built by buildMiniKernMatrix on each full
// prewarm). miniKernLeftClasses/miniKernRightClasses map ONLY the codepoints
@@ -189,6 +198,10 @@ class SdCardFont {
uint8_t miniKernLeftClassCount = 0;
uint8_t miniKernRightClassCount = 0;
int8_t* miniKernMatrix = nullptr;
// Kept-if-fits capacities, same rationale as the mini glyph buffers above.
uint16_t miniKernLeftCapacity = 0;
uint16_t miniKernRightCapacity = 0;
uint32_t miniKernMatrixCapacity = 0;
// The EpdFont whose data pointer we manage
EpdFont epdFont{&stubData};
+14
View File
@@ -1451,6 +1451,20 @@ void GfxRenderer::displayBuffer(const HalDisplay::RefreshMode refreshMode) const
display.displayBuffer(refreshMode, fadingFix);
}
void GfxRenderer::displayBufferAsync(const HalDisplay::RefreshMode refreshMode) const {
// The async path has no turn-off-screen hook, which the sunlight fading fix
// relies on; keep those users on the blocking path.
if (fadingFix) {
display.displayBuffer(refreshMode, fadingFix);
return;
}
display.displayBufferAsync(refreshMode);
}
void GfxRenderer::waitRefreshComplete() const { display.waitRefreshComplete(); }
bool GfxRenderer::supportsAsyncRefresh() const { return !fadingFix && display.supportsAsyncRefresh(); }
std::string GfxRenderer::truncatedText(const int fontId, const char* text, const int maxWidth,
const EpdFontFamily::Style style) const {
if (!text || maxWidth <= 0) return "";
+11
View File
@@ -135,6 +135,17 @@ class GfxRenderer {
int getScreenWidth() const;
int getScreenHeight() const;
void displayBuffer(HalDisplay::RefreshMode refreshMode = HalDisplay::FAST_REFRESH) const;
// Non-blocking refresh: starts the waveform and returns so CPU work (e.g.
// grayscale strip rendering) can overlap the panel's refresh time. The
// framebuffer must stay untouched until waitRefreshComplete(). Falls back to
// a blocking refresh when fadingFix is enabled or the panel lacks deferral
// support. See HalDisplay::displayBufferAsync for the baseline contract.
void displayBufferAsync(HalDisplay::RefreshMode refreshMode = HalDisplay::FAST_REFRESH) const;
void waitRefreshComplete() const;
// True when displayBufferAsync() genuinely overlaps: panel defers and
// fadingFix isn't forcing the blocking path. Callers can skip overlap
// scaffolding (e.g. whole-plane grayscale buffers) when false.
bool supportsAsyncRefresh() const;
// EXPERIMENTAL: Windowed update - display only a rectangular region
// void displayWindow(int x, int y, int width, int height) const;
void invertScreen() const;
+18 -4
View File
@@ -22,7 +22,21 @@ constexpr char DEVICE_ID[] = "crosspoint-reader";
// footprint is smaller than mbedTLS's old ~48KB peak, but keep a conservative
// floor. Check both total free heap and largest contiguous block so fragmented
// heap does not fall through into a failed TLS allocation path.
constexpr uint32_t MIN_HEAP_FOR_TLS = 55000;
// MEMFIX-PORT: TLS heap gate; portable
// Field data (July 2026): launching sync from a reader session lands at
// 51.9-58.2 KB free / 42-53 KB maxAlloc after WiFi comes up. wolfSSL handles
// allocation failure by returning MEMORY_E (no abort under -fno-exceptions),
// so an optimistic attempt degrades to the same clean "sync failed" as the
// gate — the gate only needs to keep out states where a doomed handshake
// would waste tens of seconds, not guarantee success.
//
// Free and largest-block have separate requirements: with SP ECC
// (WOLFSSL_HAVE_SP_ECC) the handshake's crypto uses fixed 256-bit arrays, so
// the largest single TLS allocation is the ~17 KB wolfSSL record buffer, not
// a run of fast-math bignums. A handshake was measured succeeding inside a
// 43 KB largest block; requiring 50 KB contiguous refused syncs that fit.
constexpr uint32_t MIN_FREE_FOR_TLS = 50000;
constexpr uint32_t MIN_BLOCK_FOR_TLS = 20000;
// Apply the shared KOSync auth headers after begin(). x-auth-* is the native
// KOSync scheme; Basic auth is added for Calibre-Web-Automated compatibility.
@@ -39,9 +53,9 @@ void applyAuthHeaders(freeink::SecureHttpClient& http) {
bool insufficientHeap() {
const uint32_t freeHeap = ESP.getFreeHeap();
const uint32_t maxAllocHeap = ESP.getMaxAllocHeap();
if (freeHeap < MIN_HEAP_FOR_TLS || maxAllocHeap < MIN_HEAP_FOR_TLS) {
LOG_ERR("KOSync", "Insufficient heap for TLS handshake: %u bytes free, %u max alloc (need %u)", freeHeap,
maxAllocHeap, MIN_HEAP_FOR_TLS);
if (freeHeap < MIN_FREE_FOR_TLS || maxAllocHeap < MIN_BLOCK_FOR_TLS) {
LOG_ERR("KOSync", "Insufficient heap for TLS handshake: %u bytes free (need %u), %u max alloc (need %u)", freeHeap,
MIN_FREE_FOR_TLS, maxAllocHeap, MIN_BLOCK_FOR_TLS);
return true;
}
return false;
+12
View File
@@ -65,6 +65,18 @@ void HalDisplay::displayBuffer(HalDisplay::RefreshMode mode, bool turnOffScreen)
einkDisplay.displayBuffer(convertRefreshMode(mode), turnOffScreen);
}
void HalDisplay::displayBufferAsync(HalDisplay::RefreshMode mode) {
if (gpio.deviceIsX3() && mode == RefreshMode::HALF_REFRESH) {
einkDisplay.requestResync(1);
}
einkDisplay.displayBufferAsyncNoShadow(convertRefreshMode(mode));
}
void HalDisplay::waitRefreshComplete() { einkDisplay.waitRefreshComplete(); }
bool HalDisplay::supportsAsyncRefresh() const { return einkDisplay.supportsAsyncRefresh(); }
void HalDisplay::refreshDisplay(HalDisplay::RefreshMode mode, bool turnOffScreen) {
if (gpio.deviceIsX3() && mode == RefreshMode::HALF_REFRESH) {
einkDisplay.requestResync(1);
+11
View File
@@ -39,6 +39,17 @@ class HalDisplay {
bool fromProgmem = false) const;
void displayBuffer(RefreshMode mode = RefreshMode::FAST_REFRESH, bool turnOffScreen = false);
// Non-blocking refresh (shadow-free): starts the panel waveform and returns
// while the panel refreshes on its own. The framebuffer must stay untouched
// until waitRefreshComplete(), and the caller must rebuild the differential
// baseline before the next differential update (the tiled grayscale cleanup
// does). Panels without deferral fall back to a blocking refresh.
void displayBufferAsync(RefreshMode mode = RefreshMode::FAST_REFRESH);
// Block until a pending deferred refresh completes (no-op when none is).
void waitRefreshComplete();
// True when displayBufferAsync() genuinely overlaps (panel driver defers);
// false where it falls back to a blocking refresh.
bool supportsAsyncRefresh() const;
void refreshDisplay(RefreshMode mode = RefreshMode::FAST_REFRESH, bool turnOffScreen = false);
// Power management
+53 -2
View File
@@ -13,7 +13,10 @@ framework = arduino
monitor_speed = 115200
upload_speed = 921600
check_tool = cppcheck
check_flags = --enable=all --suppress=missingIncludeSystem --suppress=unusedFunction --suppress=unmatchedSuppression --suppress=*:*/.pio/* --inline-suppr
; missingInclude (project headers) is suppressed alongside missingIncludeSystem: on a
; fresh CI checkout cppcheck has no resolved include paths, so it reports every
; project header as missing (~400 information-level lines) and fails the job.
check_flags = --enable=all --suppress=missingIncludeSystem --suppress=missingInclude --suppress=unusedFunction --suppress=unmatchedSuppression --suppress=*:*/.pio/* --inline-suppr
check_skip_packages = yes
board_upload.flash_size = 16MB
@@ -42,7 +45,14 @@ build_flags =
-DWOLFSSL_OPTIONS_H
-DWOLFSSL_CLIENT_EXAMPLE
-DWOLFSSL_TLS13
-DWOLFSSL_SP_RISCV32
# MEMFIX-PORT: single-precision ECC (sp_c32.c). Without it every P-256 operation
# (TLS 1.3 key_share keygen, ECDHE, ECDSA cert verify) runs on fast-math bignums
# that WOLFSSL_SMALL_STACK heap-allocates at FP_MAX_BITS size -- tens of KB of
# temporaries, which OOMs (MP_MEM) at the ~50KB free heap a reading session
# leaves. SP uses fixed 256-bit arrays: a few KB, and several times faster.
# SP_SMALL trades the large precomputed point tables for smaller flash.
-DWOLFSSL_HAVE_SP_ECC
-DWOLFSSL_SP_SMALL
-DHAVE_TLS_EXTENSIONS
-DHAVE_SUPPORTED_CURVES
-DHAVE_HKDF
@@ -63,6 +73,47 @@ board_build.flash_mode = dio
board_build.flash_size = 16MB
board_build.partitions = partitions.csv
; MEMFIX-PORT: custom_sdkconfig heap reclamation (~32-37 KB). Rebuilds the
; Arduino core libs on first build (slower once, cached after; needs the CMake
; pin in platformio.local.ini on macOS).
;
; If an interrupted rebuild fails with "multiple definition of 'app_main'"
; (stale generated scaffold), clean it up with:
; rm -rf .dummy CMakeLists.txt sdkconfig.default sdkconfig.defaults .pio/build/default
; Do NOT use `git clean -fdX` — it deletes platformio.local.ini.
custom_sdkconfig =
; Task stack right-sizing from measured high-water marks (heap block map +
; per-task stack audit, July 2026): esp_timer used ~0.8 KB of 8 KB across
; every capture; the FreeRTOS timer service used ~0.5 KB
; of 4 KB. Neither runs TLS or app code. ~7 KB back to the heap.
CONFIG_ESP_TIMER_TASK_STACK_SIZE=4096
CONFIG_FREERTOS_TIMER_TASK_STACK_DEPTH=2560
; Move the WiFi stack's non-critical hot paths out of IRAM into flash.
; On the C3, IRAM and DRAM share one SRAM pool, so the ~25-30 KB this
; frees lands directly in the heap — paid for with lower WiFi throughput
; during transfers (occasional sync/OTA use, not streaming: acceptable).
; IRAM cost is static, so the heap gain applies even with WiFi off.
CONFIG_ESP_WIFI_IRAM_OPT=n
CONFIG_ESP_WIFI_RX_IRAM_OPT=n
; Keep the Arduino wrappers for the removed cloud components (below) out of
; the core source list; all other bundled libraries default to enabled.
CONFIG_ARDUINO_SELECTIVE_COMPILATION=y
CONFIG_ARDUINO_SELECTIVE_RainMaker=n
CONFIG_ARDUINO_SELECTIVE_Insights=n
; Drop unused cloud components from the core rebuild. esp_insights/rainmaker
; require embedded server certs the lib builder can't generate
; ("https_server.crt.S not found"); this firmware uses none of them.
custom_component_remove =
espressif/esp_insights
espressif/esp_rainmaker
espressif/esp_diagnostics
espressif/esp_diag_data_store
espressif/esp_schedule
espressif/esp_rcp_update
espressif/esp_secure_cert_mgr
espressif/cbor
extra_scripts =
pre:scripts/patch_wolfssl.py
pre:scripts/build_html.py
+5 -1
View File
@@ -12,8 +12,12 @@ OVERRIDES = f"""
#ifndef HAVE_FFDHE_2048
#define HAVE_FFDHE_2048
#endif
/* MEMFIX-PORT: 8192 handles up to RSA-4096 keys (the public-CA maximum,
ISRG Root X1 included) with half the per-bignum heap of 16384: with
WOLFSSL_SMALL_STACK each fast-math temp is FP_MAX_BITS/8 * 2 bytes on the
heap, and TLS cert verification allocates dozens at once. */
#undef FP_MAX_BITS
#define FP_MAX_BITS 16384
#define FP_MAX_BITS 8192
"""
+181 -39
View File
@@ -257,6 +257,18 @@ void EpubReaderActivity::openReaderMenu() {
});
}
bool EpubReaderActivity::buildTickHeapGate() {
const size_t freeHeap = ESP.getFreeHeap();
const size_t maxBlock = ESP.getMaxAllocHeap();
// Below the floors: just wait. The tick is deferrable — page-turn transients
// free up between turns and the tick retries every loop pass. Track the
// paused state so skipLoopDelay() stops pinning the CPU at full speed while
// no build work is actually happening (the gate can stay closed for a long
// stretch if the retained build context itself holds the heap down).
buildHeapPaused = freeHeap < BACKGROUND_BUILD_MIN_FREE_HEAP || maxBlock < BACKGROUND_BUILD_MIN_MAX_ALLOC;
return !buildHeapPaused;
}
void EpubReaderActivity::loop() {
if (!epub) {
// Should never happen
@@ -264,6 +276,40 @@ void EpubReaderActivity::loop() {
return;
}
// Idle glyph prewarm for the likely next page (currentPage + 1). The scan
// pass draws nothing (FCM scan mode suppresses pixels), so the displayed
// framebuffer is untouched; endScanAndPrewarm loads only glyphs not already
// cached. Debounced past rapid page-flipping, one attempt per position, and
// deferred while a render/build owns the CPU or the heap is at the render
// floor. Cross-chapter prewarm is deliberately out of scope (next spine's
// section isn't loaded).
constexpr unsigned long IDLE_PREWARM_DEBOUNCE_MS = 400;
if (section && !section->isBuilding() && !RenderLock::peek() && renderer.hasFrameBuffer() &&
lastRenderCompleteMs != 0 && millis() - lastRenderCompleteMs > IDLE_PREWARM_DEBOUNCE_MS &&
ESP.getFreeHeap() > RENDER_MIN_FREE_HEAP && ESP.getMaxAllocHeap() > BACKGROUND_BUILD_MIN_MAX_ALLOC &&
(idlePrewarmSpine != currentSpineIndex || idlePrewarmPage != section->currentPage)) {
RenderLock lock; // the page table must not change under the scan
// Re-check under the lock: peek() and acquisition are not atomic, so the render
// task may have reset/replaced the section or moved the page in between.
if (section && !section->isBuilding() &&
(idlePrewarmSpine != currentSpineIndex || idlePrewarmPage != section->currentPage)) {
idlePrewarmSpine = currentSpineIndex;
idlePrewarmPage = section->currentPage;
const int nextPage = section->currentPage + 1;
if (nextPage < static_cast<int>(section->pageCount)) {
if (const auto p = section->loadPage(nextPage)) {
if (auto* fcm = renderer.getFontCacheManager()) {
const auto t0 = millis();
auto scope = fcm->createPrewarmScope();
p->render(renderer, SETTINGS.getReaderFontId(), 0, 0); // scan only, no pixels
scope.endScanAndPrewarm();
LOG_DBG("ERS", "Idle prewarm: page %d in %lums", nextPage, millis() - t0);
}
}
}
}
}
// Lazily resume a partial's extension build once the reader nears its watermark. Far from
// it the rebuild is all cost (whole-chapter re-layout from page 0) and no benefit this
// session, so reopening a partial deliberately does NOT start it (see the deferral in
@@ -299,14 +345,17 @@ void EpubReaderActivity::loop() {
// "far enough ahead" and stall the build at 0 pages -- then the first turn past the
// watermark re-parses the whole chapter synchronously. Keep ticking until it finalizes.
if (section && section->isBuilding() && !RenderLock::peek() &&
(section->isPartial() || static_cast<int>(section->pageCount) < section->currentPage + BUILD_WINDOW_AHEAD)) {
(section->isPartial() || static_cast<int>(section->pageCount) < section->currentPage + BUILD_WINDOW_AHEAD) &&
buildTickHeapGate()) {
RenderLock lock;
// Re-check under the lock: render() (which also holds the RenderLock) may have finalized the
// build between the outer isBuilding() check and acquiring the lock here, in which case
// buildSomeMore() would fail and wrongly reset the section. cppcheck can't see the cross-task
// mutation, so it flags this as always true.
// buildSomeMore() would fail and wrongly reset the section. The heap gate must be re-read
// too: a render that won the lock race can expand retained glyph buffers, invalidating the
// pre-lock heap reading. cppcheck can't see the cross-task mutation, so it flags this as
// always true.
// cppcheck-suppress knownConditionTrueFalse
if (section->isBuilding()) {
if (section->isBuilding() && buildTickHeapGate()) {
if (!section->buildSomeMore(BACKGROUND_BUILD_PAGES_PER_TICK)) {
LOG_ERR("ERS", "Background section build failed");
section.reset();
@@ -1282,6 +1331,7 @@ void EpubReaderActivity::render(RenderLock&& lock) {
const auto start = millis();
renderContents(std::move(p), orientedMarginTop, orientedMarginRight, orientedMarginBottom, orientedMarginLeft);
LOG_DBG("ERS", "Rendered page in %dms", millis() - start);
lastRenderCompleteMs = millis();
}
// Only persist when the position actually changed. render() also runs on menu,
// bookmark and screenshot re-renders, and writeAtomic is several FAT ops for 6 bytes.
@@ -1351,6 +1401,13 @@ void EpubReaderActivity::renderContents(std::unique_ptr<Page> page, const int or
const bool pageHasImagesNeedingDecode = pageHasImages && page->hasImagesNeedingDecode();
const bool needsTextGrayscale = SETTINGS.textAntiAliasing;
const bool needsAnyGrayscale = needsTextGrayscale || pageHasImages;
const bool tiledGrayscale = needsAnyGrayscale && renderer.supportsStripGrayscale();
// Whole-plane buffering only pays when the BW refresh genuinely runs async
// underneath it; on blocking panels (X3) it would just spend ~50 KB for the
// identical serial timing. Image pages take the blocking double-FAST path
// below (no async refresh is ever started), so they'd spend the buffers with
// nothing in flight to overlap.
const bool overlapRefresh = tiledGrayscale && renderer.supportsAsyncRefresh() && !pageHasImages;
auto renderGrayscalePass = [&]() {
if (needsTextGrayscale) {
page->render(renderer, fontId, orientedMarginLeft, orientedMarginTop);
@@ -1396,50 +1453,79 @@ void EpubReaderActivity::renderContents(std::unique_ptr<Page> page, const int or
// regardless of residue.
pagesUntilFullRefresh = 1;
} else {
ReaderUtils::displayWithRefreshCycle(renderer, pagesUntilFullRefresh);
// Async form: start the waveform and return so the grayscale plane rendering
// below overlaps the panel's refresh time instead of following it.
ReaderUtils::displayWithRefreshCycle(renderer, pagesUntilFullRefresh, overlapRefresh);
}
const auto tDisplay = millis();
// Tiled grayscale: render each plane band-by-band into a small scratch and
// stream straight to the controller, leaving the BW framebuffer intact so no
// full-frame storeBwBuffer is needed; controller RAM is re-synced from the
// live framebuffer afterward. The page is re-rendered ceil(H/STRIP_ROWS) times
// per plane, but renderCharImpl culls out-of-band glyphs before decode so the
// cost stays close to one render. Both text (drawPixel) and images
// (DirectPixelWriter) honor the active strip target.
if (needsAnyGrayscale && renderer.supportsStripGrayscale()) {
// Tiled grayscale: render each plane band-by-band, leaving the BW
// framebuffer intact so no full-frame storeBwBuffer is needed; controller
// RAM is re-synced from the live framebuffer afterward. The page is
// re-rendered ceil(H/STRIP_ROWS) times per plane, but renderCharImpl culls
// out-of-band glyphs before decode so the cost stays close to one render.
// Both text (drawPixel) and images (DirectPixelWriter) honor the active
// strip target. When the BW refresh above went out async, the plane
// rendering below overlaps the panel's refresh time; only the controller
// RAM writes wait for BUSY.
if (tiledGrayscale) {
constexpr int STRIP_ROWS = 80;
const int gh = renderer.getDisplayHeight();
const int gwBytes = renderer.getDisplayWidthBytes();
const size_t planeBytes = static_cast<size_t>(gwBytes) * gh;
auto scratch = makeUniqueNoThrow<uint8_t[]>(static_cast<size_t>(gwBytes) * STRIP_ROWS);
if (!scratch) {
LOG_ERR("ERS", "OOM: grayscale strip scratch (%d bytes); skipping AA this page", gwBytes * STRIP_ROWS);
} else {
// Bands may be streamed in any order: X4 windows each via setRamArea, X3
// via PTL.
renderer.setRenderMode(GfxRenderer::GRAYSCALE_LSB);
// Render one plane band-by-band into a whole-plane buffer without touching
// the controller, so it can run while the refresh is still in flight.
auto renderPlaneToBuffer = [&](const bool lsbPlane, uint8_t* buf) {
renderer.setRenderMode(lsbPlane ? GfxRenderer::GRAYSCALE_LSB : GfxRenderer::GRAYSCALE_MSB);
for (int y = 0; y < gh; y += STRIP_ROWS) {
const int rows = (gh - y < STRIP_ROWS) ? (gh - y) : STRIP_ROWS;
renderer.beginStripTarget(scratch.get(), y, rows);
renderer.beginStripTarget(buf + static_cast<size_t>(y) * gwBytes, y, rows);
renderer.clearScreen(0x00);
renderGrayscalePass();
renderer.endStripTarget();
renderer.writeGrayscalePlaneStrip(true, scratch.get(), y, rows);
}
const auto tGrayLsb = millis();
};
// MSB plane.
renderer.setRenderMode(GfxRenderer::GRAYSCALE_MSB);
for (int y = 0; y < gh; y += STRIP_ROWS) {
const int rows = (gh - y < STRIP_ROWS) ? (gh - y) : STRIP_ROWS;
renderer.beginStripTarget(scratch.get(), y, rows);
renderer.clearScreen(0x00);
renderGrayscalePass();
renderer.endStripTarget();
renderer.writeGrayscalePlaneStrip(false, scratch.get(), y, rows);
// Tiered on heap pressure: two plane buffers hide both plane renders
// inside the refresh wait; one hides the LSB render (its buffer is reused
// for MSB after streaming); none falls back to the strip-scratch flow with
// no overlap. Each buffer is only attempted when it leaves ~60 KB free so
// the pass never starves concurrent allocations: the next page re-render
// allocates through throwing std::string paths that abort() on OOM under
// -fno-exceptions, so a plane buffer that "fits" but eats the render
// headroom is worse than the strip fallback. Blocking panels skip the
// buffers entirely (nothing to overlap).
constexpr size_t PLANE_BUF_HEADROOM = 60000;
// Free-heap alone ignores fragmentation: taking the largest block for a
// plane can leave only slivers behind even when total headroom looks fine.
// Require the block to fit the plane with 16 KB contiguous to spare, which
// also keeps the advance-table batch scratch viable mid-render (same
// rationale as BACKGROUND_BUILD_MIN_MAX_ALLOC).
constexpr size_t PLANE_BUF_MAX_ALLOC_RESERVE = 16 * 1024;
const auto planeBufFits = [planeBytes] {
return ESP.getFreeHeap() >= planeBytes + PLANE_BUF_HEADROOM &&
ESP.getMaxAllocHeap() >= planeBytes + PLANE_BUF_MAX_ALLOC_RESERVE;
};
auto lsbPlaneBuf = (overlapRefresh && planeBufFits()) ? makeUniqueNoThrow<uint8_t[]>(planeBytes) : nullptr;
auto msbPlaneBuf = (lsbPlaneBuf && planeBufFits()) ? makeUniqueNoThrow<uint8_t[]>(planeBytes) : nullptr;
if (lsbPlaneBuf) {
renderPlaneToBuffer(true, lsbPlaneBuf.get());
if (msbPlaneBuf) renderPlaneToBuffer(false, msbPlaneBuf.get());
const auto tGrayRender = millis();
renderer.waitRefreshComplete();
const auto tWait = millis();
renderer.writeGrayscalePlaneStrip(true, lsbPlaneBuf.get(), 0, gh);
if (msbPlaneBuf) {
renderer.writeGrayscalePlaneStrip(false, msbPlaneBuf.get(), 0, gh);
} else {
renderPlaneToBuffer(false, lsbPlaneBuf.get());
renderer.writeGrayscalePlaneStrip(false, lsbPlaneBuf.get(), 0, gh);
}
const auto tGrayMsb = millis();
const auto tGrayWrite = millis();
renderer.setRenderMode(GfxRenderer::BW);
renderer.displayGrayBuffer();
@@ -1448,14 +1534,70 @@ void EpubReaderActivity::renderContents(std::unique_ptr<Page> page, const int or
// BW framebuffer is intact; re-sync controller RAM for the next
// differential page turn directly from it.
renderer.cleanupGrayscaleWithFrameBuffer();
const auto tCleanup = millis();
const auto tEnd = millis();
LOG_DBG("ERS",
"Page render (tiled): prewarm=%lums bw_render=%lums display=%lums gray_lsb=%lums "
"gray_msb=%lums gray_display=%lums cleanup=%lums total=%lums",
tPrewarm - t0, tBwRender - tPrewarm, tDisplay - tBwRender, tGrayLsb - tDisplay, tGrayMsb - tGrayLsb,
tGrayDisplay - tGrayMsb, tCleanup - tGrayDisplay, tEnd - t0);
"Page render (tiled async): prewarm=%lums bw_render=%lums display=%lums gray_render=%lums "
"wait=%lums gray_write=%lums gray_display=%lums cleanup=%lums total=%lums (planes buffered: %d)",
tPrewarm - t0, tBwRender - tPrewarm, tDisplay - tBwRender, tGrayRender - tDisplay, tWait - tGrayRender,
tGrayWrite - tWait, tGrayDisplay - tGrayWrite, tEnd - tGrayDisplay, tEnd - t0, msbPlaneBuf ? 2 : 1);
} else {
// Per-strip scratch tier: blocking panels (X3) and the OOM fallback.
// The strip writes below need the panel idle, so wait out any pending
// async refresh first (no-op on blocking panels).
auto scratch = makeUniqueNoThrow<uint8_t[]>(static_cast<size_t>(gwBytes) * STRIP_ROWS);
renderer.waitRefreshComplete();
if (!scratch) {
LOG_ERR("ERS", "OOM: grayscale strip scratch (%d bytes); skipping AA this page", gwBytes * STRIP_ROWS);
if (overlapRefresh) {
// The BW refresh ran the shadow-free async path, so controller RAM's
// differential baseline was never rebuilt. Even with AA skipped it must
// be re-synced from the intact BW framebuffer, or the next differential
// update diffs against stale contents.
renderer.cleanupGrayscaleWithFrameBuffer();
}
} else {
// Bands may be streamed in any order: X4 windows each via setRamArea,
// X3 via PTL.
renderer.setRenderMode(GfxRenderer::GRAYSCALE_LSB);
for (int y = 0; y < gh; y += STRIP_ROWS) {
const int rows = (gh - y < STRIP_ROWS) ? (gh - y) : STRIP_ROWS;
renderer.beginStripTarget(scratch.get(), y, rows);
renderer.clearScreen(0x00);
renderGrayscalePass();
renderer.endStripTarget();
renderer.writeGrayscalePlaneStrip(true, scratch.get(), y, rows);
}
const auto tGrayLsb = millis();
// MSB plane.
renderer.setRenderMode(GfxRenderer::GRAYSCALE_MSB);
for (int y = 0; y < gh; y += STRIP_ROWS) {
const int rows = (gh - y < STRIP_ROWS) ? (gh - y) : STRIP_ROWS;
renderer.beginStripTarget(scratch.get(), y, rows);
renderer.clearScreen(0x00);
renderGrayscalePass();
renderer.endStripTarget();
renderer.writeGrayscalePlaneStrip(false, scratch.get(), y, rows);
}
const auto tGrayMsb = millis();
renderer.setRenderMode(GfxRenderer::BW);
renderer.displayGrayBuffer();
const auto tGrayDisplay = millis();
// BW framebuffer is intact; re-sync controller RAM for the next
// differential page turn directly from it.
renderer.cleanupGrayscaleWithFrameBuffer();
const auto tCleanup = millis();
const auto tEnd = millis();
LOG_DBG("ERS",
"Page render (tiled): prewarm=%lums bw_render=%lums display=%lums gray_lsb=%lums "
"gray_msb=%lums gray_display=%lums cleanup=%lums total=%lums",
tPrewarm - t0, tBwRender - tPrewarm, tDisplay - tBwRender, tGrayLsb - tDisplay, tGrayMsb - tGrayLsb,
tGrayDisplay - tGrayMsb, tCleanup - tGrayDisplay, tEnd - t0);
}
}
} else {
// Fallback path for a controller without strip support. grayscale rendering
+38 -2
View File
@@ -41,6 +41,13 @@ class EpubReaderActivity final : public Activity {
bool showBookmarkMessage = false;
bool ignoreNextConfirmRelease = false;
bool currentPageBookmarked = false;
// Idle-time glyph prewarm: after a page settles, scan the LIKELY next page
// (scan mode draws nothing) and load its missing glyphs from SD during idle,
// so the next turn's in-render prewarm is a cache hit instead of ~100 ms of
// SD reads on the page-turn critical path. One attempt per position.
int idlePrewarmSpine = -1;
int idlePrewarmPage = -1;
unsigned long lastRenderCompleteMs = 0;
bool bookmarkRemoved = false; // true when last toggle removed (controls popup text)
std::vector<BookmarkEntry> cachedBookmarks;
// Tracks whether this book is currently removed from Recent Books by the
@@ -86,6 +93,33 @@ class EpubReaderActivity final : public Activity {
// background build chunk never noticeably delays input or a pending render.
static constexpr int BUILD_PAGES_PER_CHUNK = 8;
static constexpr int BACKGROUND_BUILD_PAGES_PER_TICK = 2;
// MEMFIX-PORT: background-build heap floor; portable
// Skip background build ticks below this free-heap floor. The parse path grows
// word vectors of heap strings — throwing allocations that abort() on OOM under
// -fno-exceptions (field crash: bad_alloc in ParsedText::addWord during a
// background tick under heap pressure). The tick is deferrable work:
// page-turn transients free up between turns and the build resumes; the render
// path still builds the page it actually needs regardless of this floor.
static constexpr size_t BACKGROUND_BUILD_MIN_FREE_HEAP = 32 * 1024;
// Fragmentation floor for the same gate: a tick passed the free-heap floor at
// 34.7 KB free but the largest block was ~11 KB, and a parse allocation inside the
// tick aborted anyway. Free heap says how much memory exists; maxAlloc says whether
// any single allocation can actually have it. 16 KB also keeps the advance-table
// batch path (16 KB scratch) viable during builds.
static constexpr size_t BACKGROUND_BUILD_MIN_MAX_ALLOC = 16 * 1024;
// Gate for a background build tick: true when the heap can take parse allocations.
// Updates buildHeapPaused as a side effect.
bool buildTickHeapGate();
// True while the background build is gated on the heap floors. Lets skipLoopDelay()
// return the loop to normal delay/power-saving during the pause: isBuilding() stays
// true the whole time, and without this the loop would spin at full CPU speed doing
// no build work — indefinitely, if the build context itself keeps the heap low.
bool buildHeapPaused = false;
// Heap floor for optional render-adjacent work (idle prewarm). Page
// deserialization (TextBlock word vectors/strings) and glyph caching allocate
// through throwing paths that abort() on OOM; skip deferrable work below it.
static constexpr size_t RENDER_MIN_FREE_HEAP = 24 * 1024;
// How many pages to keep laid out ahead of the reader for a still-building section. A page
// turn is ~1s on e-ink and a page builds in ~30ms, so the reader can't out-click the builder
// -- a tiny buffer is enough. The background build stops once the watermark is this far
@@ -143,8 +177,10 @@ class EpubReaderActivity final : public Activity {
// Full CPU speed + fast loop ticks while a section build runs: at the low-power
// frequency a giant chapter's background rebuild stretches from ~40s to many
// minutes, so the reader exits before it can finalize and the next open restarts
// it from page 0. Reverts to normal power behavior the moment the build finishes.
bool skipLoopDelay() override { return section && section->isBuilding(); }
// it from page 0. Reverts to normal power behavior the moment the build finishes,
// and while the build is heap-paused (no work is happening, so spinning at full
// speed would only burn battery; the paused gate still retries every loop pass).
bool skipLoopDelay() override { return section && section->isBuilding() && !buildHeapPaused; }
bool isReaderActivity() const override { return true; }
ScreenshotInfo getScreenshotInfo() const override;
CrossPointPosition getCurrentPosition() const;
+12 -3
View File
@@ -59,12 +59,21 @@ inline PageTurnResult detectPageTurn(const MappedInputManager& input) {
return {prev, next, tiltPrev || tiltNext};
}
inline void displayWithRefreshCycle(const GfxRenderer& renderer, int& pagesUntilFullRefresh) {
// One helper, blocking or deferred: the async form starts the refresh and
// returns so the caller can overlap CPU work with the panel's refresh time.
// Async callers must not touch the framebuffer until
// renderer.waitRefreshComplete() and must rebuild the differential baseline
// before the next page turn (the tiled grayscale cleanup does).
inline void displayWithRefreshCycle(const GfxRenderer& renderer, int& pagesUntilFullRefresh, bool async = false) {
const auto mode = (pagesUntilFullRefresh <= 1) ? HalDisplay::HALF_REFRESH : HalDisplay::FAST_REFRESH;
if (async) {
renderer.displayBufferAsync(mode);
} else {
renderer.displayBuffer(mode);
}
if (pagesUntilFullRefresh <= 1) {
renderer.displayBuffer(HalDisplay::HALF_REFRESH);
pagesUntilFullRefresh = SETTINGS.getRefreshFrequency();
} else {
renderer.displayBuffer();
pagesUntilFullRefresh--;
}
}
+17
View File
@@ -200,6 +200,14 @@ void CrossPointWebServer::begin() {
udpActive = udp.begin(LOCAL_UDP_PORT);
LOG_DBG("WEB", "Discovery UDP %s on port %d", udpActive ? "enabled" : "failed", LOCAL_UDP_PORT);
// All request handlers run on the task that calls handleClient(). Register
// that task before any handler can call esp_task_wdt_reset().
const esp_err_t watchdogResult = esp_task_wdt_add(nullptr);
watchdogTaskRegistered = watchdogResult == ESP_OK;
if (!watchdogTaskRegistered) {
LOG_ERR("WEB", "Failed to register web server task with watchdog: %s", esp_err_to_name(watchdogResult));
}
running = true;
LOG_DBG("WEB", "Web server started on port %d", port);
@@ -229,6 +237,10 @@ void CrossPointWebServer::abortWsUpload(const char* tag) {
void CrossPointWebServer::stop() {
if (!running || !server) {
LOG_DBG("WEB", "stop() called but already stopped (running=%d, server=%p)", running, server.get());
if (watchdogTaskRegistered) {
esp_task_wdt_delete(nullptr);
watchdogTaskRegistered = false;
}
return;
}
@@ -269,6 +281,11 @@ void CrossPointWebServer::stop() {
LOG_DBG("WEB", "Web server stopped and deleted");
LOG_DBG("WEB", "[MEM] Free heap after delete server: %d bytes", ESP.getFreeHeap());
if (watchdogTaskRegistered) {
esp_task_wdt_delete(nullptr);
watchdogTaskRegistered = false;
}
// Note: Static upload variables (uploadFileName, uploadPath, uploadError) are declared
// later in the file and will be cleared when they go out of scope or on next upload
LOG_DBG("WEB", "[MEM] Free heap final: %d bytes", ESP.getFreeHeap());
+1
View File
@@ -72,6 +72,7 @@ class CrossPointWebServer {
std::unique_ptr<WebServer> server = nullptr;
std::unique_ptr<WebSocketsServer> wsServer = nullptr;
bool running = false;
bool watchdogTaskRegistered = false;
bool apMode = false; // true when running in AP mode, false for STA mode
uint16_t port = 80;
uint16_t wsPort = 81; // WebSocket port