PHPackages                             codechap/yii3-context-trimmer - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Parsing & Serialization](/categories/parsing)
4. /
5. codechap/yii3-context-trimmer

ActiveLibrary[Parsing & Serialization](/categories/parsing)

codechap/yii3-context-trimmer
=============================

Tokenizer-agnostic text preprocessor for trimming and optimising content for LLM context windows.

1.0.0(1mo ago)04↓100%MITPHPPHP 8.2 - 8.5

Since Mar 13Pushed 1mo agoCompare

[ Source](https://github.com/codeChap/yii3-context-trimmer)[ Packagist](https://packagist.org/packages/codechap/yii3-context-trimmer)[ RSS](/packages/codechap-yii3-context-trimmer/feed)WikiDiscussions master Synced 1mo ago

READMEChangelogDependencies (3)Versions (2)Used By (0)

Yii3 Context Trimmer
====================

[](#yii3-context-trimmer)

Tokenizer-agnostic text preprocessor for trimming and optimising content to fit within LLM context windows. Built for Yii3 with full DI container integration, configurable params, and a console command.

Requirements
------------

[](#requirements)

- PHP 8.2 - 8.5

Installation
------------

[](#installation)

```
composer require codechap/yii3-context-trimmer
```

For Yii3 applications using the config plugin, the DI bindings and params are registered automatically.

Usage
-----

[](#usage)

### Via Dependency Injection (Yii3)

[](#via-dependency-injection-yii3)

Inject the interface to get a pre-configured trimmer from the DI container:

```
use Codechap\Yii3ContextTrimmer\ContextTrimmerInterface;

final class MyService
{
    public function __construct(
        private readonly ContextTrimmerInterface $trimmer,
    ) {}

    public function process(string $text): array
    {
        return $this->trimmer
            ->withMaxTokens(4096)
            ->withRemoveDuplicateLines(true)
            ->trim($text);
    }
}
```

Default configuration is handled via `params.php` — see [Configuration](#yii3-params) below.

### Standalone

[](#standalone)

```
use Codechap\Yii3ContextTrimmer\ContextTrimmer;

$trimmer = new ContextTrimmer();

$segments = $trimmer
    ->withMaxTokens(4096)
    ->withRemoveDuplicateLines(true)
    ->trim($longText);
```

### Custom Tokenizer

[](#custom-tokenizer)

The default tokenizer splits on spaces, which is a rough heuristic. For accurate token counting, provide a tokenizer matching your LLM's tokenization:

```
// Example: tiktoken-based tokenizer for OpenAI models
$trimmer = new ContextTrimmer(
    tokenizer: function (string $text): array {
        return your_tiktoken_encode($text);
    },
);
```

Configuration
-------------

[](#configuration)

### Yii3 Params

[](#yii3-params)

Override defaults in your application's `params.php`:

```
return [
    'codechap/yii3-context-trimmer' => [
        'maxTokens' => 4096,           // Max tokens per segment (default: 8192)
        'removeDuplicateLines' => true, // Remove duplicate lines (default: false)
        'removeShortWords' => false,    // Remove short words (default: false)
        'minWordLength' => 2,           // Min word length to keep (default: 2)
        'removeExtraneous' => false,    // Remove brackets/parens/etc (default: false)
        'compressWhitespace' => true,   // Compress whitespace (default: true)
    ],
];
```

### Options Reference

[](#options-reference)

OptionDefaultDescription`maxTokens``8192`Maximum tokens per output segment. Must be >= 2.`removeDuplicateLines``false`Remove duplicate non-blank lines. Blank lines are preserved as structural separators.`removeShortWords``false`Remove purely-alphabetical words shorter than `minWordLength`. **Warning:** This is aggressive and removes articles, prepositions, and pronouns.`minWordLength``2`Minimum word length to keep. Words shorter than this are removed.`removeExtraneous``false`Remove `[](){}*` characters. **Warning:** Destroys Markdown, HTML, and code syntax.`compressWhitespace``true`Collapse multiple whitespace characters into single spaces. Disable for code/formatted text.Console Command
---------------

[](#console-command)

Requires `yiisoft/yii-console` for the Yii3 console runner.

```
# Trim a file
./yii context:trim path/to/file.txt

# Pipe from stdin
cat document.txt | ./yii context:trim

# With options
./yii context:trim file.txt --max-tokens 4096 --remove-duplicates --json

# All options
./yii context:trim file.txt \
    -t 4096 \
    -d \                     # --remove-duplicates
    -s \                     # --remove-short-words
    -l 3 \                   # --min-word-length
    -x \                     # --remove-extraneous
    --no-compress \
    -j                       # --json output
```

Testing
-------

[](#testing)

```
composer test
```

Static Analysis
---------------

[](#static-analysis)

```
composer analyse
```

License
-------

[](#license)

MIT

###  Health Score

41

—

FairBetter than 88% of packages

Maintenance95

Actively maintained with recent releases

Popularity5

Limited adoption so far

Community6

Small or concentrated contributor base

Maturity51

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Unknown

Total

1

Last Release

57d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/eb9295442ae3080252d831794578a67b53968136834c0ebb1878d4f495a1f79a?d=identicon)[CodeChap](/maintainers/CodeChap)

---

Top Contributors

[![codeChap](https://avatars.githubusercontent.com/u/451621?v=4)](https://github.com/codeChap "codeChap (5 commits)")

---

Tags

aiContexttokenizerllmyii3trimmer

###  Code Quality

TestsPHPUnit

Static AnalysisPsalm

Type Coverage Yes

### Embed Badge

![Health badge](/badges/codechap-yii3-context-trimmer/health.svg)

```
[![Health](https://phpackages.com/badges/codechap-yii3-context-trimmer/health.svg)](https://phpackages.com/packages/codechap-yii3-context-trimmer)
```

###  Alternatives

[yethee/tiktoken

PHP version of tiktoken

1583.1M15](/packages/yethee-tiktoken)[helgesverre/toon

Token-Oriented Object Notation - A compact data format for reducing token consumption when sending structured data to LLMs

11841.4k9](/packages/helgesverre-toon)[mischasigtermans/laravel-toon

Token-Optimized Object Notation encoder/decoder for Laravel with intelligent nested object handling

13113.1k](/packages/mischasigtermans-laravel-toon)[gioni06/gpt3-tokenizer

PHP package for Byte Pair Encoding (BPE) used by GPT-3.

85537.5k8](/packages/gioni06-gpt3-tokenizer)[sbsaga/toon

🧠 TOON for Laravel — a compact, human-readable, and token-efficient data format for AI prompts & LLM contexts. Perfect for ChatGPT, Gemini, Claude, Mistral, and OpenAI integrations (JSON ⇄ TOON).

6115.6k](/packages/sbsaga-toon)[rajentrivedi/tokenizer-x

TokenizerX calculates required tokens for given prompt

91214.0k3](/packages/rajentrivedi-tokenizer-x)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)