PHPackages                             purewater2011/tiktoken-php7 - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. purewater2011/tiktoken-php7

ActiveLibrary[Utility & Helpers](/categories/utility)

purewater2011/tiktoken-php7
===========================

PHP 7.4+ compatible version of tiktoken - OpenAI's tiktoken tokenizer ported to PHP

03PHP

Since Sep 10Pushed 9mo agoCompare

[ Source](https://github.com/purewater2011/tiktoken-php7)[ Packagist](https://packagist.org/packages/purewater2011/tiktoken-php7)[ RSS](/packages/purewater2011-tiktoken-php7/feed)WikiDiscussions master Synced today

READMEChangelogDependenciesVersions (1)Used By (0)

tiktoken-php7
=============

[](#tiktoken-php7)

[![Packagist Version](https://camo.githubusercontent.com/c5e48b1da0b8c03a51628b0e394b51e38f6550294fd0c33b5cc1eae1937b3f01/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f707572657761746572323031312f74696b746f6b656e2d70687037)](https://camo.githubusercontent.com/c5e48b1da0b8c03a51628b0e394b51e38f6550294fd0c33b5cc1eae1937b3f01/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f707572657761746572323031312f74696b746f6b656e2d70687037)[![PHP Version](https://camo.githubusercontent.com/ddaa1ac738f86f37e80c28457324d539ea49ee928005bdf44b51444e5934a75a/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f7068702d762f707572657761746572323031312f74696b746f6b656e2d70687037)](https://camo.githubusercontent.com/ddaa1ac738f86f37e80c28457324d539ea49ee928005bdf44b51444e5934a75a/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f7068702d762f707572657761746572323031312f74696b746f6b656e2d70687037)[![License](https://camo.githubusercontent.com/cf12ae3b3d209067dc5c6405201619aeebc9d5eebcae95a17ba4cfa2a6d10f68/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f6c2f707572657761746572323031312f74696b746f6b656e2d70687037)](https://camo.githubusercontent.com/cf12ae3b3d209067dc5c6405201619aeebc9d5eebcae95a17ba4cfa2a6d10f68/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f6c2f707572657761746572323031312f74696b746f6b656e2d70687037)

A **PHP 7.4+ compatible** port of OpenAI's [tiktoken](https://github.com/openai/tiktoken) tokenizer.

This package is a backward-compatible fork that brings tiktoken functionality to PHP 7.4+, making it accessible to projects that haven't yet migrated to PHP 8.1+.

Features
--------

[](#features)

- ✅ **PHP 7.4+ compatibility** (downgraded from PHP 8.1+)
- ✅ Support for all OpenAI models (GPT-3.5, GPT-4, GPT-4o, etc.)
- ✅ Multiple encoding formats (r50k\_base, p50k\_base, cl100k\_base, o200k\_base)
- ✅ Efficient caching system
- ✅ Optional FFI-based native library support for better performance
- ✅ Full compatibility with original tiktoken API

Installation
------------

[](#installation)

```
composer require purewater2011/tiktoken-php7
```

### Requirements

[](#requirements)

- PHP 7.4 or higher
- `ext-ffi` (optional, for LibEncoder performance boost)

Quick Start
-----------

[](#quick-start)

```