PHPackages                             scripturadesign/tokenizer - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. scripturadesign/tokenizer

ActiveLibrary[Utility & Helpers](/categories/utility)

scripturadesign/tokenizer
=========================

Tokenizer

07PHP

Since Jan 27Pushed 10y ago1 watchersCompare

[ Source](https://github.com/scripturadesign/tokenizer)[ Packagist](https://packagist.org/packages/scripturadesign/tokenizer)[ RSS](/packages/scripturadesign-tokenizer/feed)WikiDiscussions master Synced 4w ago

READMEChangelogDependenciesVersions (1)Used By (0)

Composer Skeleton
=================

[](#composer-skeleton)

[![Latest Version on Packagist](https://camo.githubusercontent.com/0724d30048d580944c3860bf7e87a85f03d8a707946e9519250af79ee53fec38/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f73637269707475726164657369676e2f746f6b656e697a65722e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/scripturadesign/tokenizer)[![Software License](https://camo.githubusercontent.com/55c0218c8f8009f06ad4ddae837ddd05301481fcf0dff8e0ed9dadda8780713e/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d627269676874677265656e2e7376673f7374796c653d666c61742d737175617265)](LICENSE.md)[![Build Status](https://camo.githubusercontent.com/08539620fa1af21703865b6b373789b9df9327ef7bc383192d58e6f044d41528/68747470733a2f2f696d672e736869656c64732e696f2f7472617669732f73637269707475726164657369676e2f746f6b656e697a65722f6d61737465722e7376673f7374796c653d666c61742d737175617265)](https://travis-ci.org/scripturadesign/tokenizer)[![Coverage Status](https://camo.githubusercontent.com/b3ed9b2b3394c146a64a64522ebfc275ba14d1e7a3a4c6ac560b96f030527cc3/68747470733a2f2f696d672e736869656c64732e696f2f7363727574696e697a65722f636f7665726167652f672f73637269707475726164657369676e2f746f6b656e697a65722e7376673f7374796c653d666c61742d737175617265)](https://scrutinizer-ci.com/g/scripturadesign/tokenizer/code-structure)[![Quality Score](https://camo.githubusercontent.com/906918643b3972290b6ffeb739395d75befdbd08ff8635ded637646c99f97dc2/68747470733a2f2f696d672e736869656c64732e696f2f7363727574696e697a65722f672f73637269707475726164657369676e2f746f6b656e697a65722e7376673f7374796c653d666c61742d737175617265)](https://scrutinizer-ci.com/g/scripturadesign/tokenizer)[![Total Downloads](https://camo.githubusercontent.com/edbf085d8b720c8ce14262985845b20d2c35928a759f732e64f4955bce6c962f/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f64742f73637269707475726164657369676e2f746f6b656e697a65722e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/scripturadesign/tokenizer)

Take a string an split it into a list of tokens.

Install
-------

[](#install)

Via Composer:

```
$ composer require scripturadesign/tokenizer
```

Usage
-----

[](#usage)

```
//
```

Change log
----------

[](#change-log)

Please see [CHANGELOG](CHANGELOG.md) for more information what has changed recently.

Testing
-------

[](#testing)

```
$ composer test
```

Code Style
----------

[](#code-style)

Easily check the code style against [PSR-2 Coding Standard](https://github.com/php-fig/fig-standards/blob/master/accepted/PSR-2-coding-style-guide.md) by running:

```
$ composer sniff
```

And automatically fix them with this:

```
$ composer fix
```

Contributing
------------

[](#contributing)

Please see [CONTRIBUTING](CONTRIBUTING.md) for details.

Security
--------

[](#security)

If you discover any security related issues, please email  instead of using the issue tracker.

Credits
-------

[](#credits)

- [Martin Dilling-Hansen](https://github.com/martindilling)
- [All Contributors](../../contributors)

License
-------

[](#license)

The MIT License (MIT). Please see [License File](LICENSE.md) for more information.

###  Health Score

20

—

LowBetter than 13% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity4

Limited adoption so far

Community7

Small or concentrated contributor base

Maturity41

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

### Community

Maintainers

![](https://www.gravatar.com/avatar/58a08b906afaef2d8891c0195c6ce4454d359854d2f0e126ff204cd02d158876?d=identicon)[martindilling](/maintainers/martindilling)

---

Top Contributors

[![martindilling](https://avatars.githubusercontent.com/u/1018838?v=4)](https://github.com/martindilling "martindilling (6 commits)")

### Embed Badge

![Health badge](/badges/scripturadesign-tokenizer/health.svg)

```
[![Health](https://phpackages.com/badges/scripturadesign-tokenizer/health.svg)](https://phpackages.com/packages/scripturadesign-tokenizer)
```

###  Alternatives

[dfridrich/php-mime-type

Simple PHP class for guessing file mime type based on file extension with ability to use in Symfony project.

17286.0k3](/packages/dfridrich-php-mime-type)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)