Files
Crosspoint/lib/I18n/I18n.cpp
T
jpirnayandZach Nelson d53c8b0e0e perf: replace i18n pointer tables with offset tables, strip unused strings (#1408)
## Summary

* **What is the goal of this PR?** Reduce the flash footprint of the
i18n string data and tooling improvements to `gen_i18n.py`.

* **What changes are included?**

### 1. `lib/I18n/I18n.cpp` — use offset-based lookup

`I18n::get()` previously dereferenced a `const char* const*` pointer
array. It now uses a two-field `LangStrings` struct (a flat char blob +
a `uint16_t` offset table) generated for each language:

```cpp
// before
const char* const* strings = getStringArray(_language);
return strings[index];

// after
const LangStrings lang = getLanguageStrings(_language);
return lang.data + lang.offsets[index];
```

Lookup cost is unchanged — still O(1), one array load and one addition.

### 2. `scripts/gen_i18n.py` — new generated layout

Each language's string data is now emitted as:

- **`STRINGS_XX_DATA[]`** — a single `const char[]` blob of all strings
concatenated with `\0` separators.
- **`OFFSETS_XX[]`** — a `uint16_t` array of one byte-offset per `StrId`
into the blob.

Previously each language had a `const char* const STRINGS_XX[]` pointer
array (4 bytes/entry on ESP32-C3).

#### Flash savings

| Table type | Size per language | 19 languages |
|---|---|---|
| `const char*` pointer array (before) | `339 × 4 = 1,356 B` | **25,764
B** |
| `uint16_t` offset table (after) | `339 × 2 = 678 B` | **12,882 B** |
| **Saved** | | **12,882 B (~12.6 KB)** |

String data size is unchanged — 133,092 B across 19 languages.

**Total: 158,856 B → 145,974 B** (pointer tables → offset tables).

### 3. Build-time stripping of unused strings

`gen_i18n.py` now scans the `src/` and `lib/` trees for `STR_*`
references and, during a PlatformIO build, automatically omits the 52
strings that are defined in YAML but never referenced in code. This
further reduces the compiled output from 339 → 287 string keys per
language.

---

## `gen_i18n.py` CLI reference

```
python gen_i18n.py [translations_dir [output_dir]] [options]
```

| Argument / Flag | Default | Description |
|---|---|---|
| `translations_dir` | `lib/I18n/translations` | Path to the
per-language YAML files |
| `output_dir` | `lib/I18n/` | Where to write the generated `.h` /
`.cpp` files |
| `--src-dirs DIR [DIR …]` | `src lib` | Directories scanned for `STR_*`
usage |
| `--strip-unused` | off | Remove unreferenced `STR_*` keys from
generated output |
| `--verbose` / `-v` | off | Print per-key INFO/WARNING messages and
`Generated:` lines |

The PlatformIO build (SCons `pre:` hook) calls `main(strip_unused=True)`
automatically, so unused strings are always stripped from firmware
builds without any manual flag.

**Default run output** (no flags) shows a per-language summary table:
```
Language            Code  Own  Fallback  Unused  Data (B)
------------------  ----  ---  --------  ------  --------
English             EN    339  0         52        5,266
Belarusian          BE    310  29        52        9,673
…

  Total: 339  |  Used in code: 287  |  Never used: 52
  Flash (now):    133,092 B strings  +  12,882 B offset tables (uint16_t)  =  145,974 B
  Flash (before): 133,092 B strings  +  25,764 B pointer tables (ptr32)    =  158,856 B
  Saved by offset tables: 12,882 B
```

---

### AI Usage

Did you use AI tools to help write this code? **YES** (GitHub Copilot)

---------

Co-authored-by: Zach Nelson <zach@zdnelson.com>
2026-04-30 13:57:19 -05:00

137 lines
3.6 KiB
C++

#include "I18n.h"
#include <HalStorage.h>
#include <HardwareSerial.h>
#include <Serialization.h>
#include <string>
#include "I18nStrings.h"
using namespace i18n_strings;
// Settings file path
static constexpr const char* SETTINGS_FILE = "/.crosspoint/language.bin";
static constexpr uint8_t SETTINGS_VERSION = 2;
I18n& I18n::getInstance() {
static I18n instance;
return instance;
}
const char* I18n::get(StrId id) const {
const auto index = static_cast<size_t>(id);
if (index >= static_cast<size_t>(StrId::_COUNT)) {
return "???";
}
// Use generated helper function - no hardcoded switch needed!
const LangStrings lang = getLanguageStrings(_language);
return lang.data + lang.offsets[index];
}
void I18n::setLanguage(Language lang) {
if (lang >= Language::_COUNT) {
return;
}
_language = lang;
saveSettings();
}
const char* I18n::getLanguageName(Language lang) const {
const auto index = static_cast<size_t>(lang);
if (index >= static_cast<size_t>(Language::_COUNT)) {
return "???";
}
return LANGUAGE_NAMES[index];
}
const char* I18n::getLanguageCode(Language lang) const {
const auto index = static_cast<size_t>(lang);
if (index >= static_cast<size_t>(Language::_COUNT)) {
return LANGUAGE_CODES[0];
}
return LANGUAGE_CODES[index];
}
void I18n::saveSettings() {
Storage.mkdir("/.crosspoint");
FsFile file;
if (!Storage.openFileForWrite("I18N", SETTINGS_FILE, file)) {
Serial.printf("[I18N] Failed to save settings\n");
return;
}
serialization::writePod(file, SETTINGS_VERSION);
serialization::writeString(file, getLanguageCode(_language));
file.close();
Serial.printf("[I18N] Settings saved: language=%d code=%s\n", static_cast<int>(_language),
getLanguageCode(_language));
}
void I18n::loadSettings() {
FsFile file;
if (!Storage.openFileForRead("I18N", SETTINGS_FILE, file)) {
Serial.printf("[I18N] No settings file, using default (English)\n");
return;
}
uint8_t version;
serialization::readPod(file, version);
if (version == SETTINGS_VERSION) {
std::string code;
serialization::readString(file, code);
bool found = false;
for (uint8_t i = 0; i < getLanguageCount(); i++) {
if (code == LANGUAGE_CODES[i]) {
_language = static_cast<Language>(i);
found = true;
break;
}
}
if (found) {
Serial.printf("[I18N] Loaded language code: %s (%d)\n", code.c_str(), static_cast<int>(_language));
} else {
Serial.printf("[I18N] Unknown language code in settings: %s\n", code.c_str());
}
file.close();
return;
}
// Legacy migration path: version 1 stored language enum index directly.
if (version == 1) {
uint8_t lang;
serialization::readPod(file, lang);
if (lang < static_cast<size_t>(Language::_COUNT)) {
_language = static_cast<Language>(lang);
Serial.printf("[I18N] Migrating v1 language index: %d -> %s\n", static_cast<int>(_language),
getLanguageCode(_language));
file.close();
saveSettings();
return;
}
file.close();
Serial.printf("[I18N] Invalid v1 language index: %d\n", static_cast<int>(lang));
return;
}
Serial.printf("[I18N] Settings version mismatch: %d\n", static_cast<int>(version));
file.close();
}
// Generate character set for a specific language
const char* I18n::getCharacterSet(Language lang) {
const auto langIndex = static_cast<size_t>(lang);
if (langIndex >= static_cast<size_t>(Language::_COUNT)) {
lang = Language::EN; // Fallback to first language
}
return CHARACTER_SETS[static_cast<size_t>(lang)];
}