PHPackages                             cleatsquad/php-text-normalizer - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Search & Filtering](/categories/search)
4. /
5. cleatsquad/php-text-normalizer

ActiveLibrary[Search & Filtering](/categories/search)

cleatsquad/php-text-normalizer
==============================

Text normalizer and tokenizer for search and deduplication: Unicode diacritic folding plus Latin and Arabic orthographic equivalences

v1.0.0(today)01↑2900%[1 issues](https://github.com/CleatSquad/php-text-normalizer/issues)MITPHPPHP >=8.2CI passing

Since Aug 14Pushed todayCompare

[ Source](https://github.com/CleatSquad/php-text-normalizer)[ Packagist](https://packagist.org/packages/cleatsquad/php-text-normalizer)[ Docs](https://github.com/CleatSquad/php-text-normalizer)[ RSS](/packages/cleatsquad-php-text-normalizer/feed)WikiDiscussions main Synced today

READMEChangelog (1)Dependencies (3)Versions (3)Used By (0)

PHP Text Normalizer
===================

[](#php-text-normalizer)

[![License: MIT](https://camo.githubusercontent.com/7013272bd27ece47364536a221edb554cd69683b68a46fc0ee96881174c4214c/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d626c75652e737667)](LICENSE)[![PHP Version](https://camo.githubusercontent.com/a04c4ba987a1efdcd8ce8fe345636b2eef0a31fdddb79b9b3f654dd38e4b2fb9/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f7068702d253345253344382e322d3737376262342e737667)](composer.json)

Folds text to a comparable form for **search and deduplication**: lowercase, diacritics removed, punctuation collapsed — while keeping the text **in its own script**.

```
$normalizer->normalize('Crème Brûlée');   // "creme brulee"
$normalizer->normalize('مَدْرَسَة');          // "مدرسه"  — still Arabic
```

Is this what you need?
----------------------

[](#is-this-what-you-need)

Most PHP libraries in this space produce an **ASCII slug for URLs**. This one produces a **comparison key in the original script**. Pick accordingly:

You wantUseA URL slug (`crème` → `creme`)[`cocur/slugify`](https://github.com/cocur/slugify) or `symfony/string`Transliteration into Latin (`مدرسة` → `madrasa`)`ext-intl` `Transliterator`A comparison key that stays Arabic (`مَدْرَسَة` → `مدرسه`)**this package**The distinction matters for search. Transliterating Arabic to Latin collapses unrelated roots onto the same consonant skeleton and produces a key you cannot display, highlight, or feed back into an Arabic index.

Installation
------------

[](#installation)

```
composer require cleatsquad/php-text-normalizer
```

PHP 8.2+. Uses `ext-intl` when present, and falls back to `symfony/polyfill-intl-normalizer` otherwise.

Usage
-----

[](#usage)

```
use CleatSquad\TextNormalizer\TextNormalizer;

$normalizer = new TextNormalizer();

$normalizer->normalize('Quelle est la MÉTÉO à Rabat ?');
// "quelle est la meteo a rabat"

$normalizer->tokenize('token-2024');
// ['token', '2024']
```

### Analyzing text with metadata

[](#analyzing-text-with-metadata)

```
$result = $normalizer->analyze('  Météo   à   RABAT !!! ');

$result->normalized;    // "meteo a rabat"
$result->original;      // "  Météo   à   RABAT !!! "
$result->wasModified(); // true
$result->length();      // 13
$result->profileName;   // "arabic_search_latin"
```

### Script profiles

[](#script-profiles)

```
use CleatSquad\TextNormalizer\NormalizerProfile;

new TextNormalizer(NormalizerProfile::latin());                    // Latin only
new TextNormalizer(NormalizerProfile::arabic());                   // Arabic search mode (ة -> ه)
new TextNormalizer(NormalizerProfile::arabic(searchEquivalences: false)); // Arabic strict mode (preserves ة)
new TextNormalizer(NormalizerProfile::cyrillic());                 // Cyrillic (ё -> е, і/ї -> i)
new TextNormalizer(NormalizerProfile::greek());                    // Greek (ς -> σ, tonos stripped)
new TextNormalizer(NormalizerProfile::all());                      // default (all scripts)
new TextNormalizer(new NormalizerProfile());                       // punctuation only, folds nothing
```

Profiles are scoped: a Latin profile leaves Arabic harakat exactly where they are. Compose your own, or extend a shipped one:

```
$profile = NormalizerProfile::latin()->merge(
    new NormalizerProfile(characterMap: ['ij' => 'ij'])
);
```

What it folds
-------------

[](#what-it-folds)

**Diacritics — by Unicode canonical decomposition, not a table.** `Košice`, `Ṣāliḥ`, `Đà Nẵng`, `Ĝangalo` all fold correctly, in every script, because NFD reaches every decomposable letter. A hand-written table only ever covers the ones someone remembered.

**Letters that carry no mark — by table**, since decomposition cannot reach them: `æ œ ß ø ł đ ð þ ħ ı ŋ ŧ ƶ`.

**Arabic orthographic equivalences — by table**, because Unicode considers them distinct letters and no normalization form unifies them:

FoldWhy`أ إ آ ٱ` → `ا`Alif variants`ة` → `ه`Ta Marbuta, as search indexes conventionally do`ى ی ې ۍ` → `ي`Alef Maksura, and Persian/Urdu/Pashto Yeh`ک ګ` → `ك`Keheh (Persian/Urdu Kaf)`ہ ھ` → `ه`Heh Goal, Heh Doachashmee`٠-٩` and `۰-۹` → `0-9`Arabic-Indic and Extended Arabic-Indic digitstatweel `ـ` removeddecorative elongation, never lexicalZWNJ/ZWJ removedinvisible, and Persian puts them inside wordsWithout these, `علي` typed on an Arabic keyboard and `علی` typed on a Persian one are two different strings, and your index answers nothing.

Design notes
------------

[](#design-notes)

**Idempotent.** Normalizing an already-normalized string returns it unchanged.

**Output stays in NFC** whenever valid UTF-8 Unicode normalization succeeds, making it safe to store and compare byte-wise. Malformed UTF-8 inputs are returned untouched rather than converted to an empty string.

**Combining marks a profile does not claim are preserved**, attached to their letter rather than treated as word boundaries.

**No clock, no I/O, no configuration files.** One object, two methods.

Testing
-------

[](#testing)

```
composer install
composer test      # PHPUnit
composer analyse   # PHPStan, max level
```

License
-------

[](#license)

MIT. See [LICENSE](LICENSE).

###  Health Score

41

—

FairBetter than 87% of packages

Maintenance100

Actively maintained with recent releases

Popularity2

Limited adoption so far

Community8

Small or concentrated contributor base

Maturity47

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 92.3% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~0 days

Total

2

Last Release

0d ago

Major Versions

v0.1.0 → v1.0.02026-08-14

### Community

Maintainers

![](https://www.gravatar.com/avatar/75b8cb31786be9b4017a0c617eebe3a0cd3b8d039069ffad5bb5007b5510fd9d?d=identicon)[mimou78](/maintainers/mimou78)

---

Top Contributors

[![mohaelmrabet](https://avatars.githubusercontent.com/u/3817628?v=4)](https://github.com/mohaelmrabet "mohaelmrabet (12 commits)")[![renovate[bot]](https://avatars.githubusercontent.com/in/2740?v=4)](https://github.com/renovate[bot] "renovate[bot] (1 commits)")

---

Tags

searchunicodetexttokenizertransliterationarabicnormalizationdiacritics

###  Code Quality

TestsPHPUnit

Static AnalysisPHPStan

Type Coverage Yes

### Embed Badge

![Health badge](/badges/cleatsquad-php-text-normalizer/health.svg)

```
[![Health](https://phpackages.com/badges/cleatsquad-php-text-normalizer/health.svg)](https://phpackages.com/packages/cleatsquad-php-text-normalizer)
```

###  Alternatives

[symfony/symfony

The Symfony PHP framework

31.4k87.4M2.2k](/packages/symfony-symfony)[symfony/string

Provides an object-oriented API to strings and deals with bytes, UTF-8 code points and grapheme clusters in a unified way

1.8k796.5M1.5k](/packages/symfony-string)[elasticsearch/elasticsearch

PHP Client for Elasticsearch

5.3k190.3M1.1k](/packages/elasticsearch-elasticsearch)[ruflin/elastica

Elasticsearch Client

2.3k53.1M236](/packages/ruflin-elastica)[solarium/solarium

PHP Solr client

93235.0M125](/packages/solarium-solarium)[netgen/query-translator

Query Translator is a search query translator with AST representation

2062.1M9](/packages/netgen-query-translator)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)