PHPackages                             cxrxexo/tiktoken - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Parsing & Serialization](/categories/parsing)
4. /
5. cxrxexo/tiktoken

ActiveLibrary[Parsing & Serialization](/categories/parsing)

cxrxexo/tiktoken
================

PHP version of tiktoken. PHP 7.4+ required.

1.0.1(2y ago)22.1k1MITPHPPHP ^7.4

Since Jun 24Pushed 2y ago1 watchersCompare

[ Source](https://github.com/CxRxExO/tiktoken-php-7.4)[ Packagist](https://packagist.org/packages/cxrxexo/tiktoken)[ RSS](/packages/cxrxexo-tiktoken/feed)WikiDiscussions master Synced 1mo ago

READMEChangelogDependencies (5)Versions (3)Used By (0)

tiktoken-php
============

[](#tiktoken-php)

[![Packagist Version](https://camo.githubusercontent.com/5ad80365e31b75a8cfd824f3e523cf9ba0cd5ff10164d571c5500c40c4a04f17/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f6378727865786f2f74696b746f6b656e)](https://camo.githubusercontent.com/5ad80365e31b75a8cfd824f3e523cf9ba0cd5ff10164d571c5500c40c4a04f17/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f6378727865786f2f74696b746f6b656e)[![Build status](https://camo.githubusercontent.com/df93138e6b02198eea588d2ab722b5d72fbb34bd0c514817d907943efd2be323/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f616374696f6e732f776f726b666c6f772f7374617475732f6378727865786f2f74696b746f6b656e2d7068702d372e342f63692e796d6c3f6272616e63683d6d6173746572)](https://camo.githubusercontent.com/df93138e6b02198eea588d2ab722b5d72fbb34bd0c514817d907943efd2be323/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f616374696f6e732f776f726b666c6f772f7374617475732f6378727865786f2f74696b746f6b656e2d7068702d372e342f63692e796d6c3f6272616e63683d6d6173746572)[![License](https://camo.githubusercontent.com/7173a525c7140757eb7f4d5935fd5a53950447920eab84b4cb09c3f11c9bd3be/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f6c6963656e73652f6378727865786f2f74696b746f6b656e2d7068702d372e34)](https://camo.githubusercontent.com/7173a525c7140757eb7f4d5935fd5a53950447920eab84b4cb09c3f11c9bd3be/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f6c6963656e73652f6378727865786f2f74696b746f6b656e2d7068702d372e34)

This is a port of the [tiktoken-php](https://github.com/yethee/tiktoken-php).

Installation
------------

[](#installation)

```
$ composer require cxrxexo/tiktoken
```

Usage
-----

[](#usage)

```
use CxRxExO\Tiktoken\EncoderProvider;

$provider = new EncoderProvider();

$encoder = $provider->getForModel('gpt-3.5-turbo-0301');
$tokens = $encoder->encode('Hello world!');
print_r($tokens);
// OUT: [9906, 1917, 0]

$encoder = $provider->get('p50k_base');
$tokens = $encoder->encode('Hello world!');
print_r($tokens);
// OUT: [15496, 995, 0]
```

Limitations
-----------

[](#limitations)

- Encoding for GPT-2 is not supported.
- Special tokens (like ``) are not supported.

License
-------

[](#license)

[MIT](./LICENSE)

###  Health Score

25

—

LowBetter than 37% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity19

Limited adoption so far

Community8

Small or concentrated contributor base

Maturity45

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~0 days

Total

2

Last Release

1051d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/8e067b3e461bb8128b2ac650fb5774b4dc5d1244d5f0e2af5a4a827c61a7e453?d=identicon)[CxRxExO](/maintainers/CxRxExO)

---

Top Contributors

[![CxRxExO](https://avatars.githubusercontent.com/u/11651565?v=4)](https://github.com/CxRxExO "CxRxExO (5 commits)")

---

Tags

encodedecodeopenaitokenizerbpetiktoken

###  Code Quality

TestsPHPUnit

Static AnalysisPsalm

Type Coverage Yes

### Embed Badge

![Health badge](/badges/cxrxexo-tiktoken/health.svg)

```
[![Health](https://phpackages.com/badges/cxrxexo-tiktoken/health.svg)](https://phpackages.com/packages/cxrxexo-tiktoken)
```

###  Alternatives

[yethee/tiktoken

PHP version of tiktoken

1583.1M15](/packages/yethee-tiktoken)[gioni06/gpt3-tokenizer

PHP package for Byte Pair Encoding (BPE) used by GPT-3.

85537.5k8](/packages/gioni06-gpt3-tokenizer)[rajentrivedi/tokenizer-x

TokenizerX calculates required tokens for given prompt

91214.0k3](/packages/rajentrivedi-tokenizer-x)[kherge/json

Encodes, decodes, and validates JSON data.

61226.6k6](/packages/kherge-json)[devium/toml

A PHP encoder/decoder for TOML compatible with specification 1.0.0

3968.9k13](/packages/devium-toml)[danny50610/bpe-tokeniser

PHP port for openai/tiktoken (most)

10422.0k1](/packages/danny50610-bpe-tokeniser)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)