## Summary
**What is the goal of this PR?**
Restore compatibility with older XTC files that use the legacy 48-byte
header layout, so they open correctly instead of failing
header/page-table validation.
**What changes are included?**
- Updates the XTC parser to accept legacy files where pageTableOffset
starts at 0x30 instead of the newer full header size.
- Avoids treating legacy header bytes as a valid chapterOffset, so older
files are not misclassified as having chapters.
- Switches XTC file size and seek operations to 64-bit-safe wrapper
calls in HalStorage, which matches the XTC format’s 64-bit offset fields
and keeps page/chapter lookups from narrowing offsets.
- Keeps the existing bounds checks and improves related logging when XTC
page loads fail.
## Additional Context
- This is a narrow compatibility fix for XTC handling. The functional
change is in the XTC parser; the HalStorage changes are supporting
wrapper methods needed by that parser update.
- Risk should be low outside XTC reading, since the HAL changes only
expose existing underlying file APIs rather than changing storage
behavior.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **YES**
## Summary
Fix https://github.com/crosspoint-reader/crosspoint-reader/issues/1137
Introducing `HalFile`, a thin wrapper around `FsFile` that uses a global
mutex to protect file operations.
To test this PR, place the code below somewhere in the code base (I
placed it in `onGoToRecentBooks`)
```cpp
static auto testTask = [](void* param) {
for (int i = 0; i < 10; i++) {
String json = Storage.readFile("/.crosspoint/settings.json");
LOG_DBG("TEST_TASK", "Read settings.json, bytes read: %u", json.length());
}
vTaskDelete(nullptr);
};
xTaskCreate(testTask, "test0", 8192, nullptr, 1, nullptr);
xTaskCreate(testTask, "test1", 8192, nullptr, 1, nullptr);
xTaskCreate(testTask, "test2", 8192, nullptr, 1, nullptr);
xTaskCreate(testTask, "test3", 8192, nullptr, 1, nullptr);
delay(1000);
```
It will reliably lead to crash on `master`, but will function correctly
with this PR.
A macro renaming trick is used to avoid changing too many downstream
code files.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **PARTIALLY**, only to
help with tedious copy-paste tasks
---------
Co-authored-by: Zach Nelson <zach@zdnelson.com>
## Summary
Fix redefinition of `FILE_*` macro.
Note that there will still be 2 warning:
```
.pio/libdeps/default/WebSockets/src/WebSocketsClient.cpp: In member function 'void WebSocketsClient::clientDisconnect(WSclient_t*, const char*)':
.pio/libdeps/default/WebSockets/src/WebSocketsClient.cpp:573:31: warning: 'virtual void NetworkClient::flush()' is deprecated: Use clear() instead. [-Wdeprecated-declarations]
573 | client->tcp->flush();
| ~~~~~~~~~~~~~~~~~~^~
```
--> I assume the upstream library need to fix it
And:
```
src/activities/Activity.cpp:8:1: warning: 'noreturn' function does return
8 | }
| ^
```
Will be fixed in #1016
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? **NO**
## Summary
* This PR introduces a migration from binary file storage to JSON-based
storage for application settings, state, and various credential stores.
This improves readability, maintainability, and allows for easier manual
configuration editing.
* Benefits:
- Settings files are now JSON and can be easily read/edited manually
- Easier to inspect application state and settings during development
- JSON structure is more flexible for future changes
* Drawback: around 15k of additional flash usage
* Compatibility: Seamless migration preserves existing user data
## Additional Context
1. New JSON I/O Infrastructure files:
- JsonSettingsIO: Core JSON serialization/deserialization logic using
ArduinoJson library
- ObfuscationUtils: XOR-based password obfuscation for sensitive data
2. Migrated Components (now use JSON storage with automatic binary
migration):
- CrossPointSettings (settings.json): Main application settings
- CrossPointState (state.json): Application state (open book, sleep
mode, etc.)
- WifiCredentialStore (wifi.json): WiFi network credentials (Password
Obfuscation: Sensitive data like WiFi passwords, uses XOR encryption
with fixed keys. Note: This is obfuscation, not cryptographic security -
passwords can be recovered with the key)
- KOReaderCredentialStore (koreader.json): KOReader sync credentials
- RecentBooksStore (recent.json): Recently opened books list
3. Migration Logic
- Forward Compatibility: New installations use JSON format
- Backward Compatibility: Existing binary files are automatically
migrated to JSON on first load
- Backup Safety: Original binary files are renamed with .bak extension
after successful migration
- Fallback Handling: If JSON parsing fails, system falls back to binary
loading
4. Infrastructure Updates
- HalStorage: Added rename() method for backup operations
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? _** YES**_
---------
Co-authored-by: Dave Allie <dave@daveallie.com>
## Summary
Continue my changes to introduce the HAL infrastructure from
https://github.com/crosspoint-reader/crosspoint-reader/pull/522
This PR touches quite a lot of files, but most of them are just name
changing. It should not have any impacts to the end behavior.
## Additional Context
My plan is to firstly add this small shim layer, which sounds useless at
first, but then I'll implement an emulated driver which can be helpful
for testing and for development.
Currently, on my fork, I'm using a FS driver that allow "mounting" a
local directory from my computer to the device, much like the `-v` mount
option on docker. This allows me to quickly reset `.crosspoint`
directory if anything goes wrong. I plan to upstream this feature when
this PR get merged.
---
### AI Usage
While CrossPoint doesn't have restrictions on AI tools in contributing,
please be transparent about their usage as it
helps set the right context for reviewers.
Did you use AI tools to help write this code? NO
Julia