PHPackages                             textualization/sentencepiece - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. textualization/sentencepiece

ActiveLibrary[Utility & Helpers](/categories/utility)

textualization/sentencepiece
============================

Google SentencePiece bindings using FFI and a C adapter.

v0.0.3(2y ago)11021[1 PRs](https://github.com/Textualization/php-sentencepiece/pulls)1MITPHP

Since Feb 12Pushed 2y agoCompare

[ Source](https://github.com/Textualization/php-sentencepiece)[ Packagist](https://packagist.org/packages/textualization/sentencepiece)[ Fund](https://ko-fi.com/textualization)[ RSS](/packages/textualization-sentencepiece/feed)WikiDiscussions main Synced 1mo ago

READMEChangelogDependencies (1)Versions (4)Used By (1)

PHP SentencePiece
=================

[](#php-sentencepiece)

This is a minimal wrapper on top of [Google SentencePiece](https://github.com/google/sentencepiece) to enable executing the [XLMRobertaTokenizer](https://github.com/huggingface/transformers/blob/v4.37.2/src/transformers/models/xlm_roberta/tokenization_xlm_roberta.py#L63) encode method.

It needs the dynamic library for SentencePiece built with aditional C wrapper functions, see the fork at \[\].

A binary for the library can be downloaded by doing:

```
composer exec -- php -r "require 'vendor/autoload.php'; Textualization\SentencePiece\Vendor::check();"

```

but depending on platform and GLIBC you might need to compile it yourself and copy to `vendor/textualization/sentencepiece/lib` (create the folder if it doesn't exist). See `src/Vendor.php` for details.

Running the tests
-----------------

[](#running-the-tests)

To run the tests you'll need to install the library per the instructions above.

To fully test it, download this file [sentencepiece.bpe.model](https://huggingface.co/intfloat/multilingual-e5-small/blob/main/sentencepiece.bpe.model) and place it in `tests/`.

###  Health Score

20

—

LowBetter than 14% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity14

Limited adoption so far

Community9

Small or concentrated contributor base

Maturity33

Early-stage or recently created project

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~1 days

Total

3

Last Release

815d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/970f356fe04869413a212efcf75d122352265e34d9cd35d85b0fe7d51ef25162?d=identicon)[textualization](/maintainers/textualization)

---

Top Contributors

[![DrDub](https://avatars.githubusercontent.com/u/315403?v=4)](https://github.com/DrDub "DrDub (5 commits)")

---

Tags

deep-learningffi-bindingsnatural-language-processingphp8tokenization

###  Code Quality

TestsPHPUnit

### Embed Badge

![Health badge](/badges/textualization-sentencepiece/health.svg)

```
[![Health](https://phpackages.com/badges/textualization-sentencepiece/health.svg)](https://phpackages.com/packages/textualization-sentencepiece)
```

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)