## Summary
- Add embedded image support to EPUB rendering with JPEG and PNG
decoders
- Implement pixel caching system to cache decoded/dithered images to SD
card for faster re-rendering
- Add 4-level grayscale support for display
## Changes
### New Image Rendering System
- Add `ImageBlock` class to represent an image with its cached path and
display dimensions
- Add `PageImage` class as a new `PageElement` type for images on pages
- Add `ImageToFramebufferDecoder` interface for format-specific image
decoders
- Add `JpegToFramebufferConverter` - JPEG decoder with Bayer dithering
and scaling
- Add `PngToFramebufferConverter` - PNG decoder with Bayer dithering and
scaling
- Add `ImageDecoderFactory` to select appropriate decoder based on file
extension
- Add `getRenderMode()` to GfxRenderer for grayscale render mode queries
### Dithering and Grayscale
- Implement 4x4 Bayer ordered dithering for 4-level grayscale output
- Stateless algorithm works correctly with MCU block decoding
- Handles scaling without artifacts
- Add grayscale render mode support (BW, GRAYSCALE_LSB, GRAYSCALE_MSB)
- Image decoders and cache renderer respect current render mode
- Enables proper 4-level e-ink grayscale when anti-aliasing is enabled
### Pixel Caching
- Cache decoded/dithered images to `.pxc` files on SD card
- Cache format: 2-bit packed pixels (4 pixels per byte) with
width/height header
- On subsequent renders, load directly from cache instead of re-decoding
- Cache renderer supports grayscale render modes for multi-pass
rendering
- Significantly improves page navigation speed for image-heavy EPUBs
### HTML Parser Integration
- Update `ChapterHtmlSlimParser` to process `<img>` tags and extract
images from EPUB
- Resolve relative image paths within EPUB ZIP structure
- Extract images to cache directory before decoding
- Create `PageImage` elements with proper scaling to fit viewport
- Fall back to alt text display if image processing fails
### Build Configuration
- Add `PNG_MAX_BUFFERED_PIXELS=6402` to support up to 800px wide images
### Test Script
- Generate test EPUBs with annotated JPEG and PNG images
- Test cases cover: grayscale (4 levels), centering, scaling, cache
performance
## Test plan
- [x] Open EPUB with JPEG images - verify images display with proper
grayscale
- [x] Open EPUB with PNG images - verify images display correctly and no
crash
- [x] Navigate away from image page and back - verify faster load from
cache
- [x] Verify grayscale tones render correctly (not just black/white
dithering)
- [x] Verify large images are scaled down to fit screen
- [x] Verify images are centered horizontally
- [x] Verify page serialization/deserialization works with images
- [x] Verify images rendered in landscape mode
## Test Results
[png](https://photos.app.goo.gl/5zFUb8xA8db3dPd19)
[jpeg](https://photos.app.goo.gl/SwtwaL2DSQwKybhw7)








---
### AI Usage
Did you use AI tools to help write this code? _**< YES >**_
---------
Co-authored-by: Matthías Páll Gissurarson <mpg@mpg.is>
Co-authored-by: Dave Allie <dave@daveallie.com>
176 lines
5.8 KiB
C++
176 lines
5.8 KiB
C++
#include "ImageBlock.h"
|
|
|
|
#include <FsHelpers.h>
|
|
#include <GfxRenderer.h>
|
|
#include <HardwareSerial.h>
|
|
#include <SDCardManager.h>
|
|
#include <Serialization.h>
|
|
|
|
#include "../converters/DitherUtils.h"
|
|
#include "../converters/ImageDecoderFactory.h"
|
|
|
|
// Cache file format:
|
|
// - uint16_t width
|
|
// - uint16_t height
|
|
// - uint8_t pixels[...] - 2 bits per pixel, packed (4 pixels per byte), row-major order
|
|
|
|
ImageBlock::ImageBlock(const std::string& imagePath, int16_t width, int16_t height)
|
|
: imagePath(imagePath), width(width), height(height) {}
|
|
|
|
bool ImageBlock::imageExists() const { return Storage.exists(imagePath.c_str()); }
|
|
|
|
namespace {
|
|
|
|
std::string getCachePath(const std::string& imagePath) {
|
|
// Replace extension with .pxc (pixel cache)
|
|
size_t dotPos = imagePath.rfind('.');
|
|
if (dotPos != std::string::npos) {
|
|
return imagePath.substr(0, dotPos) + ".pxc";
|
|
}
|
|
return imagePath + ".pxc";
|
|
}
|
|
|
|
bool renderFromCache(GfxRenderer& renderer, const std::string& cachePath, int x, int y, int expectedWidth,
|
|
int expectedHeight) {
|
|
FsFile cacheFile;
|
|
if (!Storage.openFileForRead("IMG", cachePath, cacheFile)) {
|
|
return false;
|
|
}
|
|
|
|
uint16_t cachedWidth, cachedHeight;
|
|
if (cacheFile.read(&cachedWidth, 2) != 2 || cacheFile.read(&cachedHeight, 2) != 2) {
|
|
cacheFile.close();
|
|
return false;
|
|
}
|
|
|
|
// Verify dimensions are close (allow 1 pixel tolerance for rounding differences)
|
|
int widthDiff = abs(cachedWidth - expectedWidth);
|
|
int heightDiff = abs(cachedHeight - expectedHeight);
|
|
if (widthDiff > 1 || heightDiff > 1) {
|
|
Serial.printf("[%lu] [IMG] Cache dimension mismatch: %dx%d vs %dx%d\n", millis(), cachedWidth, cachedHeight,
|
|
expectedWidth, expectedHeight);
|
|
cacheFile.close();
|
|
return false;
|
|
}
|
|
|
|
// Use cached dimensions for rendering (they're the actual decoded size)
|
|
expectedWidth = cachedWidth;
|
|
expectedHeight = cachedHeight;
|
|
|
|
Serial.printf("[%lu] [IMG] Loading from cache: %s (%dx%d)\n", millis(), cachePath.c_str(), cachedWidth, cachedHeight);
|
|
|
|
// Read and render row by row to minimize memory usage
|
|
const int bytesPerRow = (cachedWidth + 3) / 4; // 2 bits per pixel, 4 pixels per byte
|
|
uint8_t* rowBuffer = (uint8_t*)malloc(bytesPerRow);
|
|
if (!rowBuffer) {
|
|
Serial.printf("[%lu] [IMG] Failed to allocate row buffer\n", millis());
|
|
cacheFile.close();
|
|
return false;
|
|
}
|
|
|
|
for (int row = 0; row < cachedHeight; row++) {
|
|
if (cacheFile.read(rowBuffer, bytesPerRow) != bytesPerRow) {
|
|
Serial.printf("[%lu] [IMG] Cache read error at row %d\n", millis(), row);
|
|
free(rowBuffer);
|
|
cacheFile.close();
|
|
return false;
|
|
}
|
|
|
|
int destY = y + row;
|
|
for (int col = 0; col < cachedWidth; col++) {
|
|
int byteIdx = col / 4;
|
|
int bitShift = 6 - (col % 4) * 2; // MSB first within byte
|
|
uint8_t pixelValue = (rowBuffer[byteIdx] >> bitShift) & 0x03;
|
|
|
|
drawPixelWithRenderMode(renderer, x + col, destY, pixelValue);
|
|
}
|
|
}
|
|
|
|
free(rowBuffer);
|
|
cacheFile.close();
|
|
Serial.printf("[%lu] [IMG] Cache render complete\n", millis());
|
|
return true;
|
|
}
|
|
|
|
} // namespace
|
|
|
|
void ImageBlock::render(GfxRenderer& renderer, const int x, const int y) {
|
|
Serial.printf("[%lu] [IMG] Rendering image at %d,%d: %s (%dx%d)\n", millis(), x, y, imagePath.c_str(), width, height);
|
|
|
|
const int screenWidth = renderer.getScreenWidth();
|
|
const int screenHeight = renderer.getScreenHeight();
|
|
|
|
// Bounds check render position using logical screen dimensions
|
|
if (x < 0 || y < 0 || x + width > screenWidth || y + height > screenHeight) {
|
|
Serial.printf("[%lu] [IMG] Invalid render position: (%d,%d) size (%dx%d) screen (%dx%d)\n", millis(), x, y, width,
|
|
height, screenWidth, screenHeight);
|
|
return;
|
|
}
|
|
|
|
// Try to render from cache first
|
|
std::string cachePath = getCachePath(imagePath);
|
|
if (renderFromCache(renderer, cachePath, x, y, width, height)) {
|
|
return; // Successfully rendered from cache
|
|
}
|
|
|
|
// No cache - need to decode the image
|
|
// Check if image file exists
|
|
FsFile file;
|
|
if (!Storage.openFileForRead("IMG", imagePath, file)) {
|
|
Serial.printf("[%lu] [IMG] Image file not found: %s\n", millis(), imagePath.c_str());
|
|
return;
|
|
}
|
|
size_t fileSize = file.size();
|
|
file.close();
|
|
|
|
if (fileSize == 0) {
|
|
Serial.printf("[%lu] [IMG] Image file is empty: %s\n", millis(), imagePath.c_str());
|
|
return;
|
|
}
|
|
|
|
Serial.printf("[%lu] [IMG] Decoding and caching: %s\n", millis(), imagePath.c_str());
|
|
|
|
RenderConfig config;
|
|
config.x = x;
|
|
config.y = y;
|
|
config.maxWidth = width;
|
|
config.maxHeight = height;
|
|
config.useGrayscale = true;
|
|
config.useDithering = true;
|
|
config.performanceMode = false;
|
|
config.useExactDimensions = true; // Use pre-calculated dimensions to avoid rounding mismatches
|
|
config.cachePath = cachePath; // Enable caching during decode
|
|
|
|
ImageToFramebufferDecoder* decoder = ImageDecoderFactory::getDecoder(imagePath);
|
|
if (!decoder) {
|
|
Serial.printf("[%lu] [IMG] No decoder found for image: %s\n", millis(), imagePath.c_str());
|
|
return;
|
|
}
|
|
|
|
Serial.printf("[%lu] [IMG] Using %s decoder\n", millis(), decoder->getFormatName());
|
|
|
|
bool success = decoder->decodeToFramebuffer(imagePath, renderer, config);
|
|
if (!success) {
|
|
Serial.printf("[%lu] [IMG] Failed to decode image: %s\n", millis(), imagePath.c_str());
|
|
return;
|
|
}
|
|
|
|
Serial.printf("[%lu] [IMG] Decode successful\n", millis());
|
|
}
|
|
|
|
bool ImageBlock::serialize(FsFile& file) {
|
|
serialization::writeString(file, imagePath);
|
|
serialization::writePod(file, width);
|
|
serialization::writePod(file, height);
|
|
return true;
|
|
}
|
|
|
|
std::unique_ptr<ImageBlock> ImageBlock::deserialize(FsFile& file) {
|
|
std::string path;
|
|
serialization::readString(file, path);
|
|
int16_t w, h;
|
|
serialization::readPod(file, w);
|
|
serialization::readPod(file, h);
|
|
return std::unique_ptr<ImageBlock>(new ImageBlock(path, w, h));
|
|
}
|