PHPackages                             frolax/laravel-llm-tokenkit - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Parsing & Serialization](/categories/parsing)
4. /
5. frolax/laravel-llm-tokenkit

ActiveLibrary[Parsing & Serialization](/categories/parsing)

frolax/laravel-llm-tokenkit
===========================

Token estimation, cost calculation & context-window utilities for LLM-powered Laravel apps

v1.0.0(3mo ago)12MITPHPPHP ^8.2CI passing

Since Feb 11Pushed 2mo agoCompare

[ Source](https://github.com/frolaxhq/laravel-llm-tokenkit)[ Packagist](https://packagist.org/packages/frolax/laravel-llm-tokenkit)[ Docs](https://github.com/frolaxhq/laravel-llm-tokenkit)[ GitHub Sponsors](https://github.com/frolaxhq)[ RSS](/packages/frolax-laravel-llm-tokenkit/feed)WikiDiscussions main Synced 2mo ago

READMEChangelog (1)Dependencies (8)Versions (2)Used By (0)

LLM TokenKit for Laravel
========================

[](#llm-tokenkit-for-laravel)

[![Latest Version on Packagist](https://camo.githubusercontent.com/efa3c4e2f2dbb877c1c45e8674f4173ef8b4cba35996ddceb734ac561be41bf0/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f66726f6c61782f6c61726176656c2d6c6c6d2d746f6b656e6b69742e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/frolax/laravel-llm-tokenkit)[![GitHub Tests Action Status](https://camo.githubusercontent.com/f7872473ef61fbd497648946928fa0e947fdca1517d1fc57b417db1225813f33/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f616374696f6e732f776f726b666c6f772f7374617475732f66726f6c617868712f6c61726176656c2d6c6c6d2d746f6b656e6b69742f72756e2d74657374732e796d6c3f6272616e63683d6d61696e266c6162656c3d7465737473267374796c653d666c61742d737175617265)](https://github.com/frolaxhq/laravel-llm-tokenkit/actions?query=workflow%3Arun-tests+branch%3Amain)[![Total Downloads](https://camo.githubusercontent.com/793d46820afe04255d219ad1eb907a000eae104f2419bf1f43f0414033cda940/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f64742f66726f6c61782f6c61726176656c2d6c6c6d2d746f6b656e6b69742e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/frolax/laravel-llm-tokenkit)

**Token estimation, cost calculation & context-window utilities for LLM-powered Laravel apps.**

TokenKit is a **stateless** Laravel package that helps you:

- 📊 **Estimate tokens** for text and chat message arrays
- 💰 **Calculate costs** based on provider-specific pricing
- 📐 **Build context windows** with rolling window and token-budget truncation strategies

No database, no migrations, no external API calls. Works offline.

Installation
------------

[](#installation)

```
composer require frolax/laravel-llm-tokenkit
```

Publish the config file:

```
php artisan vendor:publish --tag="llm-tokenkit-config"
```

Quick Start
-----------

[](#quick-start)

```
use Frolax\LlmTokenKit\Facades\LlmTokenKit;
use Frolax\LlmTokenKit\Data\ModelRef;

$model = new ModelRef(provider: 'openai', model: 'gpt-4o');

// Or use a simple array:
$model = ['provider' => 'openai', 'model' => 'gpt-4o'];
```

### Estimate Tokens for Text

[](#estimate-tokens-for-text)

```
$estimate = LlmTokenKit::estimateText('Hello, world!', $model);

$estimate->inputTokensEstimated; // ~4
$estimate->confidence;           // Confidence::Medium
$estimate->method;               // EstimationMethod::Heuristic
$estimate->notes;                // ['...']
```

### Estimate Tokens for Chat Messages

[](#estimate-tokens-for-chat-messages)

```
$messages = [
    ['role' => 'system', 'content' => 'You are a helpful assistant.'],
    ['role' => 'user', 'content' => 'What is Laravel?'],
];

$estimate = LlmTokenKit::estimateMessages($messages, $model);

$estimate->inputTokensEstimated; // ~18
```

### Calculate Cost

[](#calculate-cost)

```
use Frolax\LlmTokenKit\Data\TokenUsage;

$cost = LlmTokenKit::cost(
    usage: new TokenUsage(promptTokens: 1500, completionTokens: 500),
    model: $model,
);

$cost->inputCost;   // 0.00375
$cost->outputCost;  // 0.005
$cost->totalCost;   // 0.00875
$cost->currency;    // 'USD'
```

You can also pass arrays:

```
$cost = LlmTokenKit::cost(
    usage: ['prompt_tokens' => 1500, 'completion_tokens' => 500],
    model: ['provider' => 'openai', 'model' => 'gpt-4o'],
);
```

Or provide explicit pricing:

```
use Frolax\LlmTokenKit\Data\Pricing;

$cost = LlmTokenKit::cost(
    usage: new TokenUsage(promptTokens: 1000, completionTokens: 200),
    pricing: new Pricing(inputPer1m: 15.00, outputPer1m: 60.00, reasoningPer1m: 60.00),
);
```

### Build Context Window

[](#build-context-window)

```
use Frolax\LlmTokenKit\Data\ContextBuildRequest;
use Frolax\LlmTokenKit\Enums\ContextStrategy;

$result = LlmTokenKit::buildContext(new ContextBuildRequest(
    system: 'You are a helpful assistant.',
    memorySummary: 'User prefers dark mode.',
    historyMessages: $chatHistory,
    newUserMessage: 'How do I use Eloquent?',
    modelRef: $model,
    tokenBudget: 4000,
    reservedOutputTokens: 1024,
    strategy: ContextStrategy::TruncateByTokens,
));

$result->messages;            // Ready-to-send message array
$result->tokenEstimate;       // TokenEstimate DTO
$result->wasTruncated;        // true if messages were dropped
$result->droppedMessageCount; // Number of dropped messages
$result->warnings;            // Any warnings
```

Or pass an array:

```
$result = LlmTokenKit::buildContext([
    'system' => 'You are helpful.',
    'history_messages' => $chatHistory,
    'new_user_message' => 'Hello!',
    'model_ref' => ['provider' => 'openai', 'model' => 'gpt-4o'],
    'token_budget' => 4000,
    'strategy' => 'truncate_by_tokens',
]);
```

Configuration
-------------

[](#configuration)

### Pricing

[](#pricing)

Pricing is resolved in this order:

1. **Exact model match** — `pricing.providers.{provider}.models.{model}`
2. **Wildcard match** — e.g. `gpt-4.1*` matches `gpt-4.1-mini`
3. **Provider default** — `pricing.providers.{provider}.default`
4. **Global default** — `pricing.default`

```
// config/llm-tokenkit.php
'pricing' => [
    'default' => [
        'input_per_1m' => 1.00,
        'output_per_1m' => 2.00,
        'currency' => 'USD',
    ],
    'providers' => [
        'openai' => [
            'models' => [
                'gpt-4o' => ['input_per_1m' => 2.50, 'output_per_1m' => 10.00],
                'gpt-4.1*' => ['input_per_1m' => 2.00, 'output_per_1m' => 8.00],
            ],
            'default' => ['input_per_1m' => 2.50, 'output_per_1m' => 10.00],
        ],
    ],
],
```

### Estimation

[](#estimation)

```
'estimation' => [
    'code_json_penalty' => 1.3,       // Multiplier for code/JSON
    'enable_code_json_penalty' => true,
    'language_multipliers' => [
        'bn' => 2.5,  // Bangla
        'hi' => 2.0,  // Hindi
        'ar' => 2.0,  // Arabic
        'zh' => 1.5,  // Chinese
    ],
],
```

### Context Builder

[](#context-builder)

```
'context' => [
    'reserved_output_tokens' => 4096,
    'window_size' => 50,
    'default_strategy' => 'rolling_window',
    'tool_token_threshold' => 200,
],
```

Artisan Command
---------------

[](#artisan-command)

Check which estimation backend is active:

```
php artisan tokenkit:check
```

This will report active estimators, detected tokenizer libraries, sample estimates, and pricing configuration. **No network calls are made.**

⚠️ Important Notes
------------------

[](#️-important-notes)

- Token estimates are **approximations**. Different providers/models tokenize differently.
- The heuristic estimator uses a `chars/4` base rule, which is reasonable for GPT models but may vary.
- For non-Latin scripts (Bangla, Hindi, Arabic, CJK), multipliers are applied but accuracy is lower.
- **Always compare estimates against actual provider usage** before relying on them for billing.

### Suggested Defaults for SaaS

[](#suggested-defaults-for-saas)

If building a SaaS product, consider:

- Adding a **10-15% buffer** to cost estimates for safety
- Using `Confidence::Medium` or higher estimates for billing purposes
- Regularly updating pricing in your config to match provider changes
- Implementing usage tracking with **actual provider-reported tokens** alongside estimates

Testing
-------

[](#testing)

```
composer test
```

License
-------

[](#license)

The MIT License (MIT). Please see [License File](LICENSE.md) for more information.

###  Health Score

37

—

LowBetter than 83% of packages

Maintenance84

Actively maintained with recent releases

Popularity5

Limited adoption so far

Community6

Small or concentrated contributor base

Maturity46

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Unknown

Total

1

Last Release

96d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/05f9a72ab9f1685e2d1e829714d6c3470ea38acc3bdf3b6608ac5edb4997b835?d=identicon)[bishwajitcadhikary](/maintainers/bishwajitcadhikary)

---

Top Contributors

[![bishwajitcadhikary](https://avatars.githubusercontent.com/u/55208330?v=4)](https://github.com/bishwajitcadhikary "bishwajitcadhikary (9 commits)")

---

Tags

laraveltokenizerllmfrolaxcontext-windowtoken-estimationcost-calculationlaravel-llm-tokenkit

###  Code Quality

TestsPest

Code StyleLaravel Pint

### Embed Badge

![Health badge](/badges/frolax-laravel-llm-tokenkit/health.svg)

```
[![Health](https://phpackages.com/badges/frolax-laravel-llm-tokenkit/health.svg)](https://phpackages.com/packages/frolax-laravel-llm-tokenkit)
```

###  Alternatives

[spatie/laravel-markdown

A highly configurable markdown renderer and Blade component for Laravel

4053.4M35](/packages/spatie-laravel-markdown)[rajentrivedi/tokenizer-x

TokenizerX calculates required tokens for given prompt

91214.0k3](/packages/rajentrivedi-tokenizer-x)[vormkracht10/laravel-mails

Laravel Mails can collect everything you might want to track about the mails that has been sent by your Laravel app.

24149.7k](/packages/vormkracht10-laravel-mails)[spatie/laravel-markdown-response

Serve markdown versions of your HTML pages to AI agents and bots

6512.6k](/packages/spatie-laravel-markdown-response)[mischasigtermans/laravel-toon

Token-Optimized Object Notation encoder/decoder for Laravel with intelligent nested object handling

13113.1k](/packages/mischasigtermans-laravel-toon)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)