PHPackages                             nexxes/tokenizer - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. nexxes/tokenizer

ActiveLibrary[Utility & Helpers](/categories/utility)

nexxes/tokenizer
================

Simple tokenizer to split a character steam (text) into character classes (tokens) written in PHP

140PHP

Since Sep 8Pushed 11y ago1 watchersCompare

[ Source](https://github.com/nexxes/php-tokenizer)[ Packagist](https://packagist.org/packages/nexxes/tokenizer)[ RSS](/packages/nexxes-tokenizer/feed)WikiDiscussions master Synced 1w ago

READMEChangelogDependenciesVersions (1)Used By (0)

Package nexxes/tokenizer
========================

[](#package-nexxestokenizer)

[![Build Status](https://camo.githubusercontent.com/17592592d0bf5c41ceb4c87a7d18668f444907ca96a6da382388d4fad55f4166/68747470733a2f2f7472617669732d63692e6f72672f6e65787865732f7068702d746f6b656e697a65722e7376673f6272616e63683d6d6173746572)](https://travis-ci.org/nexxes/php-tokenizer)

Simple tokenizer to split a character steam (text) into character classes (tokens) written in PHP

###  Health Score

21

—

LowBetter than 18% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity9

Limited adoption so far

Community7

Small or concentrated contributor base

Maturity41

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

### Community

Maintainers

![](https://www.gravatar.com/avatar/2b38daf7ea29726e8f479a49857e46ebe2396b354ce31dfabcf504633af43de1?d=identicon)[dennisbirkholz](/maintainers/dennisbirkholz)

---

Top Contributors

[![DennisBirkholz](https://avatars.githubusercontent.com/u/600655?v=4)](https://github.com/DennisBirkholz "DennisBirkholz (58 commits)")

### Embed Badge

![Health badge](/badges/nexxes-tokenizer/health.svg)

```
[![Health](https://phpackages.com/badges/nexxes-tokenizer/health.svg)](https://phpackages.com/packages/nexxes-tokenizer)
```

###  Alternatives

[tapp/filament-google-autocomplete-field

Filament plugin that provides a Google Autocomplete field

3098.1k](/packages/tapp-filament-google-autocomplete-field)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)