unicode-transforms
Unicode normalization
http://github.com/harendra-kumar/unicode-transforms
| Version on this page: | 0.3.0 |
| LTS Haskell 23.21: | 0.4.0.1@rev:7 |
| Stackage Nightly 2025-05-09: | 0.4.0.1@rev:7 |
| Latest on Hackage: | 0.4.0.1@rev:7 |
unicode-transforms-0.3.0@sha256:f3c842e25e52382dfcfc4cbf0be35209779e8f6df8d4f70de0e8eb5a1db4df31,5713Module documentation for 0.3.0
- Data
- Data.ByteString
- Data.ByteString.UTF8
- Data.Text
- Data.Unicode
- Data.ByteString
Unicode Transforms
Fast Unicode 9.0 normalization in Haskell (NFC, NFKC, NFD, NFKD).
What is normalization?
Unicode characters with adornments (e.g. Á) can be represented in two different forms, as a single composed character (U+00C1 = Á) or as multiple decomposed characters (U+0041(A) U+0301( ́ ) = Á). They are differently encoded byte sequences but for humans they have exactly the same visual appearance.
A regular byte comparison may tell that two strings are different even though
they might be equivalent. We need to convert both the strings in a
normalized form using the Unicode
Character Database before we can
compare them for equivalence. For example:
>> import Data.Text.Normalize
>> normalize NFC "\193" == normalize NFC "\65\769"
True
Contributing
Please use https://github.com/harendra-kumar/unicode-transforms to raise issues, or send pull requests.