feat: dict based Hyphenation (#305)
## Summary * Adds (optional) Hyphenation for English, French, German, Russian languages ## Additional Context * Included hyphenation dictionaries add approximately 280kb to the flash usage (German alone takes 200kb) * Trie encoded dictionaries are adopted from hypher project (https://github.com/typst/hypher) * Soft hyphens (and other explicit hyphens) take precedence over dict-based hyphenation. Overall, the hyphenation rules are quite aggressive, as I believe it makes more sense on our smaller screen. --------- Co-authored-by: Dave Allie <dave@daveallie.com>
This commit is contained in:
co-authored by
Dave Allie
parent
5fef99c641
commit
8824c87490
@@ -0,0 +1,26 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstddef>
|
||||
#include <string>
|
||||
|
||||
#include "LanguageHyphenator.h"
|
||||
|
||||
struct LanguageEntry {
|
||||
const char* cliName;
|
||||
const char* primaryTag;
|
||||
const LanguageHyphenator* hyphenator;
|
||||
};
|
||||
|
||||
struct LanguageEntryView {
|
||||
const LanguageEntry* data;
|
||||
size_t size;
|
||||
|
||||
const LanguageEntry* begin() const { return data; }
|
||||
const LanguageEntry* end() const { return data + size; }
|
||||
};
|
||||
|
||||
// Returns the Liang-backed hyphenator for a given primary language tag (e.g., "en", "fr").
|
||||
const LanguageHyphenator* getLanguageHyphenatorForPrimaryTag(const std::string& primaryTag);
|
||||
|
||||
// Exposes the list of supported languages primarily for tooling/tests.
|
||||
LanguageEntryView getLanguageEntries();
|
||||
Reference in New Issue
Block a user