PHPackages                             mistralys/jtokenizer - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. mistralys/jtokenizer

AbandonedArchivedLibrary

mistralys/jtokenizer
====================

Modernized, composer compatible version of Tim Whitlock's JTokenizer PHP Javascript parser and tokenizer library.

1.0.1(5y ago)27.6k↓50%MITPHPPHP >=7.2

Since Mar 12Pushed 12mo ago2 watchersCompare

[ Source](https://github.com/Mistralys/jtokenizer)[ Packagist](https://packagist.org/packages/mistralys/jtokenizer)[ Docs](https://github.com/Mistralys/jtokenizer)[ RSS](/packages/mistralys-jtokenizer/feed)WikiDiscussions main Synced 1mo ago

READMEChangelog (2)Dependencies (2)Versions (3)Used By (0)

JTokenizer
==========

[](#jtokenizer)

**Archived project**: Please use [Peast](https://github.com/mck89/peast) instead, as this library is no longer actively maintained.

---

Modernized, composer compatible version of Tim Whitlock's JTokenizer PHP Javascript parser and tokenizer library.

> NOTE: This package is a good choice for very basic access to JavaScript tokens. A more modern and complete solution is [Peast](https://github.com/mck89/peast).

Overview
--------

[](#overview)

The JavaScript tokenizer is designed to mimic the PHP tokenizer. Possible uses for the tokenizer include code highlighting and simple manipulation of JavaScript source code.

Quickstart
----------

[](#quickstart)

Tokenize a JavaScript file:

```
$tokens = \JTokenizer\JTokenizer::getFileTokens('javascript.js');
```

Tokenize a JavaScript string:

```
$jsCode = '(JavaScript source code)';
$tokens = \JTokenizer\JTokenizer::getTokens($jsCode);
```

Both methods behave like the PHP `token_get_all` function, with the addition of a column number as well as a line number.

Token information
-----------------

[](#token-information)

The `getTokens()` method returns an indexed array with a list of token arrays. Each token array is an indexed array with the following information:

1. Token identifier
2. Matched literal
3. Line number
4. Column number

The token identifier can be either an Integer, or a string. To get the according token type name, use the `getTokenName()` method.

Source
------

[](#source)

Derived from Tim Whitlock's JTokenizer package, released back in 2009. As far as my research online showed, no one else published a modernized version of it that can run without deprecated messages on PHP7.4+.

The original sources are here:

###  Health Score

32

—

LowBetter than 72% of packages

Maintenance37

Infrequent updates — may be unmaintained

Popularity24

Limited adoption so far

Community8

Small or concentrated contributor base

Maturity49

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~0 days

Total

2

Last Release

1893d ago

PHP version history (2 changes)1.0.0PHP >=7.3

1.0.1PHP >=7.2

### Community

Maintainers

![](https://avatars.githubusercontent.com/u/8895528?v=4)[Mistralys](/maintainers/Mistralys)[@Mistralys](https://github.com/Mistralys)

---

Top Contributors

[![Mistralys](https://avatars.githubusercontent.com/u/8895528?v=4)](https://github.com/Mistralys "Mistralys (28 commits)")

---

Tags

javascriptlibraryparser-libraryphptokenizer

###  Code Quality

TestsPHPUnit

Static AnalysisPHPStan

Type Coverage Yes

### Embed Badge

![Health badge](/badges/mistralys-jtokenizer/health.svg)

```
[![Health](https://phpackages.com/badges/mistralys-jtokenizer/health.svg)](https://phpackages.com/packages/mistralys-jtokenizer)
```

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)