generated from jric11/baseProject
164 lines
5.7 KiB
Markdown
164 lines
5.7 KiB
Markdown
# tc-lib-unicode
|
|
|
|
> UTF-8 and Unicode processing utilities, including bidirectional text handling.
|
|
|
|
[](https://packagist.org/packages/tecnickcom/tc-lib-unicode)
|
|
[](https://github.com/tecnickcom/tc-lib-unicode/actions/workflows/check.yml)
|
|
[](https://codecov.io/gh/tecnickcom/tc-lib-unicode)
|
|
[](https://packagist.org/packages/tecnickcom/tc-lib-unicode)
|
|
[](https://packagist.org/packages/tecnickcom/tc-lib-unicode)
|
|
|
|
[](https://github.com/sponsors/tecnickcom)
|
|
|
|
> 💖 Part of the [tc-lib-pdf / TCPDF](https://github.com/tecnickcom/tc-lib-pdf) ecosystem (100M+ installs). [Sponsor its maintenance →](https://github.com/sponsors/tecnickcom)
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
`tc-lib-unicode` provides Unicode conversion helpers and bidirectional algorithm support for robust multilingual text processing.
|
|
|
|
It is built to handle multilingual text paths where normalization, code-point handling, and bidirectional ordering directly affect rendering quality. By isolating Unicode-heavy operations, dependent libraries can keep text processing accurate and easier to audit.
|
|
|
|
| | |
|
|
|---|---|
|
|
| **Namespace** | `\Com\Tecnick\Unicode` |
|
|
| **Author** | Nicola Asuni <info@tecnick.com> |
|
|
| **License** | [GNU LGPL v3](https://www.gnu.org/copyleft/lesser.html) - see [LICENSE](LICENSE) |
|
|
| **API docs** | <https://tcpdf.org/docs/srcdoc/tc-lib-unicode> |
|
|
| **Packagist** | <https://packagist.org/packages/tecnickcom/tc-lib-unicode> |
|
|
|
|
---
|
|
|
|
## Features
|
|
|
|
### Unicode Utilities
|
|
- UTF-8 character and ordinal conversion helpers
|
|
- String/character array transformations
|
|
- Integration-ready conversion methods for document engines
|
|
|
|
### Bidirectional Support
|
|
- Unicode Bidirectional Algorithm (UAX #9) implementation, passing the full official `BidiCharacterTest.txt` conformance suite
|
|
- Right-to-left and mixed-direction text processing
|
|
- Arabic shaping driven by the Joining_Type property
|
|
|
|
### Character Substitution
|
|
- Context-sensitive codepoint-level substitution via `Substitution::replaceChars()`
|
|
- **Devanagari**: moves left-positional matras (U+093F, U+094E) to precede their base consonant cluster, including conjuncts joined by Virama (U+094D)
|
|
- **Hangul**: composes Hangul Jamo sequences (U+1100-U+11FF, U+A960-U+A97F, U+D7B0-U+D7FF) into precomposed syllables (U+AC00-U+D7A3) per section 3.12 of the Unicode standard
|
|
- **Thai**: returned unchanged, because Thai preposed vowels are already stored in visual order
|
|
|
|
---
|
|
|
|
## Requirements
|
|
|
|
- PHP 8.2 or later
|
|
- Extension: `mbstring`
|
|
- Composer
|
|
|
|
---
|
|
|
|
## Installation
|
|
|
|
```bash
|
|
composer require tecnickcom/tc-lib-unicode
|
|
```
|
|
|
|
---
|
|
|
|
## Quick Start
|
|
|
|
```php
|
|
<?php
|
|
|
|
require_once __DIR__ . '/vendor/autoload.php';
|
|
|
|
$bidi = new \Com\Tecnick\Unicode\Bidi('hello ', null, null, 'R', false);
|
|
echo $bidi->getString();
|
|
```
|
|
|
|
---
|
|
|
|
## Character substitution
|
|
|
|
`Substitution::replaceChars()` takes an array of Unicode codepoints and returns a transformed array with script-specific substitutions applied. It is a pure codepoint-level transform with no font or PDF dependency.
|
|
|
|
```php
|
|
<?php
|
|
|
|
require_once __DIR__ . '/vendor/autoload.php';
|
|
|
|
$sub = new \Com\Tecnick\Unicode\Substitution();
|
|
|
|
// Devanagari: left matra repositioned before its base consonant cluster
|
|
// Logical order: [U+0915 KA, U+093F VOWEL SIGN I]
|
|
// Visual order: [U+093F VOWEL SIGN I, U+0915 KA]
|
|
$result = $sub->replaceChars([0x0915, 0x093F]);
|
|
// $result === [0x093F, 0x0915]
|
|
|
|
// Hangul: Jamo composed into a precomposed syllable
|
|
// [U+1100 KIYEOK, U+1161 JUNGSEONG A, U+11A8 JONGSEONG KIYEOK] → [U+AC01 각]
|
|
$result = $sub->replaceChars([0x1100, 0x1161, 0x11A8]);
|
|
// $result === [0xAC01]
|
|
```
|
|
|
|
### Supported scripts and Unicode ranges
|
|
|
|
| Script | Unicode range(s) | Transformation |
|
|
|---|---|---|
|
|
| Devanagari | U+0900-U+097F | Left matras repositioned before consonant cluster |
|
|
| Hangul Jamo | U+1100-U+11FF, U+A960-U+A97F, U+D7B0-U+D7FF | Jamo composed to precomposed syllables (U+AC00-U+D7A3) |
|
|
| Thai | U+0E00-U+0E7F | None: the stored order is the display order |
|
|
|
|
Codepoints belonging to unsupported scripts are passed through unchanged.
|
|
|
|
---
|
|
|
|
## Limitations
|
|
|
|
- The paragraph separator is dropped during processing and appended again at the end of
|
|
the paragraph output instead of being reset by L1 and reversed by L2. A strict UAX #9
|
|
implementation would place it at the visual left edge of a right-to-left paragraph;
|
|
keeping it at the end of the string preserves line splitting for the consumers.
|
|
- Rule L3 (combining marks applied to characters shown in a different order) is not
|
|
implemented.
|
|
- Shaping is Arabic only. The other cursive scripts (Syriac, N'Ko, Mandaic, Adlam) are
|
|
returned unshaped.
|
|
- `Bidi` and `Convert` require valid UTF-8: malformed byte sequences raise an exception,
|
|
while code points that cannot be encoded are replaced with '?'.
|
|
|
|
---
|
|
|
|
## Development
|
|
|
|
```bash
|
|
make deps
|
|
make help
|
|
make qa
|
|
make server
|
|
```
|
|
|
|
`make server` starts the local PHP development server for the `example/` directory on `http://localhost:8000`.
|
|
Use a custom port with `make server PORT=8080`.
|
|
|
|
---
|
|
|
|
## Packaging
|
|
|
|
```bash
|
|
make rpm
|
|
make deb
|
|
```
|
|
|
|
For system packages, bootstrap with:
|
|
|
|
```php
|
|
require_once '/usr/share/php/Com/Tecnick/Unicode/autoload.php';
|
|
```
|
|
|
|
---
|
|
|
|
## Contributing
|
|
|
|
Contributions are welcome. Please review [CONTRIBUTING.md](CONTRIBUTING.md), [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md), and [SECURITY.md](SECURITY.md).
|