Technical Unicode
5 min readUnicode Normalization Explained: NFC vs NFD Forms
Why identical-looking characters fail string comparisons in code and search engines
Executive Summary
In Unicode, the letter 'é' can be encoded as a single precomposed character (U+00E9) or as two characters: 'e' followed by a combining acute accent (U+0065 + U+0301). Without normalization, programming languages treat them as unequal.
The Four Unicode Normalization Forms
The Unicode Consortium defines four standard normalization forms:
1. NFC (Normalization Form Canonical Composition):
- Decomposes characters then recomposes them into standard precomposed forms.
- This is the standard for the World Wide Web (W3C recommended) and most databases.
2. NFD (Normalization Form Canonical Decomposition):
- Decomposes precomposed characters into their base character plus separate combining marks.
- macOS (Apple HFS+/APFS filenames) historically preferred NFD.
3. NFKC (Normalization Form Compatibility Composition):
- Performs compatibility decomposition (e.g. converting superscript ², ligatures like fi, and fullwidth characters to standard forms) and then recomposes.
4. NFKD (Normalization Form Compatibility Decomposition):
- Decomposes compatibility characters and combining marks into plain base ASCII equivalents. Ideal for building search indexes and URL slugs.
Comparison of Normalization in Code
Here is how string equality behaves before and after normalization in JavaScript:
| Representation | Codepoints | Length | Result of str1 === str2 |
|---|---|---|---|
| NFC ('é') | U+00E9 | 1 | false (without .normalize()) |
| NFD ('é') | U+0065 + U+0301 | 2 | false (without .normalize()) |
| str1.normalize('NFC') | U+00E9 | 1 | true (normalized) |
Best Practice for Developers
Always normalize user search queries and database inputs at the boundary using:
`const cleanQuery = query.normalize('NFC').trim();`
This guarantees that searches for accents, European languages, and emojis match accurately.
Frequently Asked Questions
Does Unicode normalization change how text looks on screen?
Canonical normalization (NFC and NFD) never changes visual appearance; browsers render precomposed and decomposed forms identically. Compatibility normalization (NFKC and NFKD) can simplify characters (e.g., turning 2² into 22 or ½ into 1/2).