PHPackages                             yep/tokenizer - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. yep/tokenizer

ActiveLibrary[Utility & Helpers](/categories/utility)

yep/tokenizer
=============

Tokenizer

08PHP

Since Jan 17Pushed 10y ago1 watchersCompare

[ Source](https://github.com/YepFoundation/tokenizer)[ Packagist](https://packagist.org/packages/yep/tokenizer)[ RSS](/packages/yep-tokenizer/feed)WikiDiscussions master Synced 4w ago

READMEChangelogDependenciesVersions (1)Used By (0)

[![Build Status](https://camo.githubusercontent.com/2535537550d30dd411eae08d3ec09fb170145dca953be5964f50f722cc8093f8/68747470733a2f2f7472617669732d63692e6f72672f596570466f756e646174696f6e2f746f6b656e697a65722e7376673f6272616e63683d6d6173746572)](https://travis-ci.org/YepFoundation/tokenizer)[![Scrutinizer Code Quality](https://camo.githubusercontent.com/354e811c5ae1ded37c9d983a074403d8dbe5d8bbbdf48faf98eceba5f34b2eb8/68747470733a2f2f7363727574696e697a65722d63692e636f6d2f672f596570466f756e646174696f6e2f746f6b656e697a65722f6261646765732f7175616c6974792d73636f72652e706e673f623d6d6173746572)](https://scrutinizer-ci.com/g/YepFoundation/tokenizer/?branch=master)[![Scrutinizer Code Coverage](https://camo.githubusercontent.com/5e263ccacfbf691bfb51217520f6cce7821877e6054b270e9aac9e67d7b85fed/68747470733a2f2f7363727574696e697a65722d63692e636f6d2f672f596570466f756e646174696f6e2f746f6b656e697a65722f6261646765732f636f7665726167652e706e673f623d6d6173746572)](https://scrutinizer-ci.com/g/YepFoundation/tokenizer/?branch=master)[![Latest Stable Version](https://camo.githubusercontent.com/1a9fc75f18ad8f985865ef0783d8816a7acfa33bc420b62ca29f01f921dc49ea/68747470733a2f2f706f7365722e707567782e6f72672f7965702f746f6b656e697a65722f762f737461626c65)](https://packagist.org/packages/yep/tokenizer)[![Total Downloads](https://camo.githubusercontent.com/d7fcbbf25add8a67160780ee7da810cbcfa187d84c85f3b1915ffe90334f0c2b/68747470733a2f2f706f7365722e707567782e6f72672f7965702f746f6b656e697a65722f646f776e6c6f616473)](https://packagist.org/packages/yep/tokenizer)[![License](https://camo.githubusercontent.com/0a0dbbde0432bd03c938fb75fb47b0346101a931752da0e4a1bfb9e524059cec/68747470733a2f2f706f7365722e707567782e6f72672f7965702f746f6b656e697a65722f6c6963656e7365)](https://github.com/YepFoundation/tokenizer/blob/master/LICENSE.md)

Tokenizer ([docs](http://yepfoundation.github.io/tokenizer))
============================================================

[](#tokenizer-docs)

Packagist
---------

[](#packagist)

Tokenizer are available on [Packagist.org](https://packagist.org/packages/yep/tokenizer), just add the dependency to your composer.json.

```
{
  "require" : {
    "yep/tokenizer": "dev-master"
  }
}
```

Usage
-----

[](#usage)

No time to write manual. 😞

Try to look into tests folder.

###  Health Score

20

—

LowBetter than 13% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity4

Limited adoption so far

Community7

Small or concentrated contributor base

Maturity41

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

### Community

Maintainers

![](https://avatars.githubusercontent.com/u/545113?v=4)[yep](/maintainers/yep)[@yep](https://github.com/yep)

---

Top Contributors

[![Zemistr](https://avatars.githubusercontent.com/u/2613208?v=4)](https://github.com/Zemistr "Zemistr (6 commits)")

### Embed Badge

![Health badge](/badges/yep-tokenizer/health.svg)

```
[![Health](https://phpackages.com/badges/yep-tokenizer/health.svg)](https://phpackages.com/packages/yep-tokenizer)
```

###  Alternatives

[akeneo/pim-community-standard

The "Akeneo Community Standard Edition" distribution

42787.4k](/packages/akeneo-pim-community-standard)[silverstripe/html5

HTML5 support for SilverStripe

14232.8k4](/packages/silverstripe-html5)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)