PHPackages                             ferry-ai/php-inference - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [API Development](/categories/api)
4. /
5. ferry-ai/php-inference

ActiveProject[API Development](/categories/api)

ferry-ai/php-inference
======================

FerryAI — unified inference API for PHP applications

v0.1.1(1mo ago)3981↓44%22MITPHPPHP >=8.3CI passing

Since Jul 7Pushed 1w ago34 watchersCompare

[ Source](https://github.com/MADEVAL/FerryAI)[ Packagist](https://packagist.org/packages/ferry-ai/php-inference)[ Docs](https://github.com/MADEVAL/FerryAI)[ RSS](/packages/ferry-ai-php-inference/feed)WikiDiscussions main Synced 1w ago

READMEChangelog (1)Dependencies (14)Versions (3)Used By (0)

 [![FerryAI](img/ferryai_banner.svg)](img/ferryai_banner.svg)

FerryAI — native AI inference for PHP
=====================================

[](#ferryai--native-ai-inference-for-php)

**Run ONNX, GGUF, and RubixML models directly in PHP — no Python, no HTTP microservices, no Docker sidecars.**One API, full FFI bridge to native engines. Inference-only. PHP 8.3+.

[![CI](https://github.com/MADEVAL/FerryAI/actions/workflows/ci.yml/badge.svg)](https://github.com/MADEVAL/FerryAI/actions/workflows/ci.yml)[![Version](https://camo.githubusercontent.com/9330dda2615f6b5a6763a44826ad07826b432b885e64aa17d2f0dd0001cb074f/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f7461672f4d41444556414c2f466572727941493f6c6162656c3d76657273696f6e26636f6c6f723d626c7565)](https://github.com/MADEVAL/FerryAI/tags)[![PHP](https://camo.githubusercontent.com/92ba06fc4dfc5da0c66153448ac93520db2378e03de054a8a2b2554bae008fb4/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f7068702d382e332532422d3737374242343f6c6f676f3d706870266c6f676f436f6c6f723d7768697465)](https://www.php.net/)[![Tests](https://camo.githubusercontent.com/688472911bf8169ac81a833e98ea4dd4f15bc35a9c6e5c4482a82c02aa27b791/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f74657374732d3739332532463739332d627269676874677265656e)](https://github.com/MADEVAL/FerryAI/actions/workflows/ci.yml)[![License: MIT](https://camo.githubusercontent.com/8bb50fd2278f18fc326bf71f6e88ca8f884f72f179d3e555e20ed30157190d0d/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d677265656e2e737667)](LICENSE.md)[![PHPStan](https://camo.githubusercontent.com/d117944b58da8146f96b4ef7403807610a20eeb3fbcaaaf95157bbcdad1686eb/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5048505374616e2d6c6576656c253230382d627269676874677265656e2e737667)](phpstan.neon)[![Psalm](https://camo.githubusercontent.com/a48aaec5207c248adbc6f8b5eac677c9adc55d5d22b94310e4e3e4dc3615a839/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5073616c6d2d6c6576656c253230332d627269676874677265656e2e737667)](psalm.xml)

> **Status: early release (`v0.1.1`).** The public API is stabilizing and may change before `1.0` — pin a version and skim the [CHANGELOG](CHANGELOG.md) when upgrading. Code quality is production-grade (PHPStan level 8, Psalm level 3, 793 tests green on Windows + Linux).

Contents
--------

[](#contents)

- [Quick example](#quick-example)
- [Why FerryAI](#why-ferryai)
- [Backends](#backends)
- [Vector store](#vector-store)
- [Observability & model pool](#observability--model-pool)
- [Install](#install)
- [Dependencies](#dependencies)
- [Capabilities](#capabilities)
- [Packages](#packages)
- [Testing](#testing)
- [Examples](#examples)
- [Documentation](#documentation)
- [Contributing & license](#contributing--license)

---

Quick example
-------------

[](#quick-example)

**Embeddings & vector search** — semantic RAG in 8 lines:

```
use FerryAI\AI;

AI::config([
    'backend' => 'onnx',
    'backends' => ['embedding' => ['model_path' => '/models/all-MiniLM-L6-v2-onnx']],
]);

// embed → 384d vector, then store and search
$vec = AI::embed('Hello world');
$store = AI::vector('docs');
$store->add('doc1', $vec->vector, ['title' => 'Getting Started']);
$hits = $store->search(AI::embed('semantic query')->vector, k: 5);

// similarity between any two texts
echo AI::similarity('cat', 'kitten');   // 0.79

// compose a processing pipeline
$results = AI::pipeline()
    ->pipe(new TransformStage(strtoupper(...)))
    ->pipe(new FilterStage(fn($x) => strlen($x) > 3))
    ->run(['hi', 'hello', 'hey']);
```

**Chat & streaming** — local LLM in 3 lines:

```
AI::config(['backend' => 'llama', 'backends' => ['llama' => ['model_path' => '/models/qwen.gguf']]]);

echo AI::chat('Explain PHP FFI in one sentence.');        // full reply
foreach (AI::stream('Write a haiku about ferries.') as $token) { echo $token; }

// structured output via JSON Schema → GBNF grammar
$json = AI::chat('List 3 famous bridges with year and city.', [
    'grammar' => [
        'type' => 'object',
        'properties' => ['bridges' => [
            'type' => 'array',
            'items' => ['type' => 'object', 'properties' => [
                'name' => ['type' => 'string'],
                'year' => ['type' => 'integer'],
                'city' => ['type' => 'string'],
            ]],
        ]],
    ],
]);

// HTTP streaming response (PSR-7 SSE/NDJSON) for web apps
return AI::streamResponse([['role' => 'user', 'content' => $prompt]]);
```

---

Why FerryAI
-----------

[](#why-ferryai)

FerryAIPython sidecarDeploymentOne PHP process. `composer require`Python runtime + HTTP server + process managerLatencyZero-copy FFI → sub-ms overheadHTTP round-trip per inferenceMemoryShared weights across workers (shmop)Duplicated per processDebuggingPHP stack traces, xdebugCross-process tracingStructured outputJSON Schema → GBNF grammar, guaranteed valid JSONPrompt engineering + regex hacksModel cacheBuilt-in HuggingFace download + LRU cache + SHA-256 verifyManual pip + custom scriptsType safetyPHPStan level 8 + Psalm level 3mypy (optional)StreamingNative PHP Generator + SSE/NDJSON PSR-7 responseFlask/FastAPI streaming boilerplateFerryAI loads native shared libraries (`onnxruntime.dll`, `llama.dll`) directly via PHP FFI — the same C APIs that Python uses. No subprocess, no `shell_exec`, no Python. Tokenizers, vector search and tensor math all run in pure PHP when native equivalents are unavailable.

---

Backends
--------

[](#backends)

BackendDrivesHighlights**ONNX Runtime**`embed()` `similarity()` `classify()` `moderate()`Any `.onnx` model. CPU + CUDA/ROCm/DirectML/OpenVINO GPU. Auto-fallback to CPU when GPU deps are missing. All-MiniLM-L6-v2 → 384d vectors.**llama.cpp**`chat()` `stream()` `streamResponse()`Real LLM chat & token-by-token streaming. Runs on CPU and CUDA GPU (Windows + Linux). Samplers: greedy, top-k, top-p, **GBNF grammar**. JSON Schema → GBNF for guaranteed structured output. ChatFormatter with 5 message templates.**CPU Native**`predict()` + tensor opsPure-PHP tensor math (matmul, transpose, reshape, slice). Optional RubixML `.rbm` tabular inference. Always available, no native deps.### LLM in detail

[](#llm-in-detail)

PathSupport`AI::chat()` / `AI::stream()` (CPU)✅ real chat via `LlamaBackend` + `ferry_llama` wrapper, Windows and Linux`AI::chat()` / `AI::stream()` (GPU, CUDA)✅ layer offload via `GGML_CUDA=ON` buildSafetensors→GGUF models (e.g. Qwen3-0.6B)✅ one-time `convert_hf_to_gguf.py`, then native inferenceONNX embeddings (GPU, CUDA)✅ CUDA provider auto-detected, silent CPU fallback```
AI::config([
    'backend'  => 'llama',
    'device'   => 'cuda',   // or 'cpu'
    'backends' => ['llama' => ['model_path' => '/models/model.gguf', 'n_gpu_layers' => 35]],
]);

echo AI::chat('Summarize FFI in PHP.');
```

Configure the wrapper via `FERRY_AI_LLAMA_WRAPPER` (or `FERRY_AI_LLAMA_LIB`), add that dir to `PATH`. Sampling is per-request: `temperature: 0` → greedy, `> 0` → top-p; force one with `['sampler' => 'top_k']` or supply a `['grammar' => '']` / JSON Schema. Build steps: [`docs/DOCUMENTATION.md`](docs/DOCUMENTATION.md) · [native/llama-wrapper/README.md](native/llama-wrapper/README.md). Run: [`examples/03-chat.php`](examples/03-chat.php) · [`examples/04-streaming.php`](examples/04-streaming.php) · [`examples/09-grammar.php`](examples/09-grammar.php).

---

Vector store
------------

[](#vector-store)

Two interchangeable backends behind the same `VectorStore` contract — pick per environment:

BackendSearchBest for**SQLite**Brute-force, or native KNN via **sqlite-vec** (vec0 ANN) when availableDev, demos, embedded, single-file**PostgreSQL + pgvector**Native `` / `` / ``, HNSW / IVFFlat indexesProduction, large collections, concurrency```
AI::config(['vector' => [
    'driver' => 'pgsql',                                     // or omit for SQLite
    'dsn' => 'pgsql:host=127.0.0.1;port=5432',
    'user' => 'postgres', 'password' => 'postgres',
]]);

$store = AI::vector('docs');
$store->add('doc1', $vec->vector, ['lang' => 'en']);
$hits = $store->search($query, k: 5, filter: ['lang' => ['eq' => 'en']]);
```

SQLite transparently uses **sqlite-vec** (vec0 virtual tables) for native KNN on PHP 8.4+, and falls back to pure-PHP brute-force otherwise — filters always work. [`examples/21-postgres-vector.php`](examples/21-postgres-vector.php) · [`examples/23-sqlite-vec.php`](examples/23-sqlite-vec.php).

---

Observability & model pool
------------------------------

[](#observability--model-pool)

Instrumentation lives at the facade layer (backends stay isolated). **Off by default** — zero overhead when disabled:

```
AI::config(['observability' => ['metrics' => true, 'profiling' => true, 'logging' => true]]);

AI::embed('hello');                 // automatically timed, counted and logged
print_r(FerryAI\Metrics::report()); // counters + timing histograms per operation
print_r(FerryAI\Profiler::report());// per-operation count / avg / min / max ms
```

`AI::warmup([...])` preloads models into a memory-bounded LRU `ModelPool`; `classify()` / `moderate()` / `predict()` / `chat()` reuse pooled instances. Opt into cross-worker weight sharing via `ext-shmop`. Downloads retry transient failures. [`examples/22-observability.php`](examples/22-observability.php).

---

Install
-------

[](#install)

```
composer require ferry-ai/php-inference
```

Base requirements: **PHP 8.3+**, `ext-ffi`, `ext-json`, `ext-hash`, `ext-fileinfo`.

**After install — run the diagnostic** to see what's available:

```
vendor/bin/ferry-ai check              # PHP, extensions, backends, cache — full report
vendor/bin/ferry-ai check --json       # machine-readable

# Download models from HuggingFace and start using them immediately
vendor/bin/ferry-ai models:download sentence-transformers/all-MiniLM-L6-v2
vendor/bin/ferry-ai chat "Explain FFI in one sentence."
vendor/bin/ferry-ai chat "Hello" --stream --max=100
```

Everything else is **optional and on-demand** — install only what a feature needs. FerryAI degrades gracefully (pure-PHP fallback or a clear "not available" message) when a native library or model is missing.

---

Dependencies
------------

[](#dependencies)

What you need for each capability. Full source list with versions: [`docs/SOURCES.md`](docs/SOURCES.md).

CapabilityPHP sideNative artifactConfigONNX (embeddings, classification)`ext-ffi`ONNX Runtime lib`FERRY_AI_MODEL_DIR` or `backends.embedding.model_path`LLM chat / streaming`ext-ffi`llama.cpp + `ferry_llama` wrapper`FERRY_AI_LLAMA_DIR` / `FERRY_AI_LLAMA_LIB`GPU (ONNX CUDA / llama.cpp)—CUDA Toolkit + cuDNN for ONNX`device: 'cuda'` + GPU-enabled buildVector store (SQLite)`ext-pdo_sqlite` (bundled)—works out of the boxVector ANN (sqlite-vec)`ext-pdo_sqlite``vec0.{dll,so,dylib}``FERRY_AI_VEC_EXTENSION_LIB`Vector store (PostgreSQL)`ext-pdo_pgsql`PostgreSQL + pgvector`FERRY_AI_VECTOR_DRIVER=pgsql`Model Hub / HuggingFace`ext-curl`, `ext-zip`, `ext-sodium`—`FERRY_AI_MODEL_CACHE`CPU tabular ML (RubixML)`rubix/ml` (isolated)`.rbm` estimator`FERRY_AI_RUBIXML_AUTOLOAD`Native tokenizer (optional)`ext-ffi`tokenizers-cpp lib`FERRY_AI_TOKENIZERS_LIB`Shared weights (workers)`ext-shmop`—`model_pool.shared_memory=true`GPU setup guide (CUDA/cuDNN/curand/cufft + llama.cpp build): [`docs/DOCUMENTATION.md`](docs/DOCUMENTATION.md) → *Quick Start → GPU setup*. GPU→CPU fallback is automatic and silent.

---

Capabilities
------------

[](#capabilities)

### Inference

[](#inference)

CapabilityDescription`AI::embed()` / `AI::similarity()`Text → vector, cosine similarity. 4 pooling strategies (mean, cls, eos, max). Batch embedding.`AI::chat()` / `AI::stream()`LLM chat & token-by-token streaming. Samplers: greedy, top-k, top-p, GBNF grammar.`AI::streamResponse()`PSR-7 SSE/NDJSON streaming HTTP response for web apps.`AI::classify()`Run classification `.onnx` models (or CPU-native fallback).`AI::moderate()`Content moderation with per-category scores and a `flagged` boolean.`AI::predict()`CPU-native tabular prediction via pure-PHP tensor ops or RubixML `.rbm` models.### Structured generation

[](#structured-generation)

CapabilityDescriptionGBNF grammarConstrain LLM output to a formal grammar. Guaranteed valid JSON, enum values, DSLs.JSON Schema → GBNFPass a JSON Schema object as `grammar` — auto-converted to GBNF. No prompt engineering needed.### Vector store

[](#vector-store-1)

CapabilityDescriptionSQLite storeCRUD, brute-force KNN, metadata filtering. Optional sqlite-vec (vec0) native ANN.PostgreSQL + pgvectorNative `` / `` / ``, HNSW/IVFFlat indexes. Metadata filtering.`AI::pipeline()`Composable Generator-based pipeline with 8 built-in stages: chunk, tokenize, embed, classify, normalize, filter, store, transform.### Model management

[](#model-management)

CapabilityDescriptionModel HubDownload from HuggingFace with progress. SHA-256 + Ed25519 signature verification. LRU cache with size limits.Model poolMemory-bounded LRU eviction. Opt-in cross-worker weight sharing via `ext-shmop`.`AI::warmup()`Preload models into the pool so first inference is instant.Auto GPU→CPU fallbackONNX silently retries on CPU when GPU providers are missing (incomplete CUDA installs).### Concurrency

[](#concurrency)

CapabilityDescriptionFiberPipelinePipeline with cooperative concurrency and wall-clock timeout support.AsyncInference`runAsync()` / `runParallel()` — run multiple inferences concurrently via PHP Fibers.### Developer experience

[](#developer-experience)

CapabilityDescription`ferry-ai check`Full environment diagnostic: PHP, extensions, backends, cache — with `--json` mode.`ferry-ai models:download` / `chat`CLI model management and single-turn chat.Pure-PHP tokenizersBPE and WordPiece tokenizers with zero native dependencies.Pure-PHP tensor mathmatmul, transpose, reshape, slice — always available.Framework adaptersThin Laravel ServiceProvider and Symfony Bundle included.### Platform

[](#platform)

**Windows**✅ Unit + integration (ONNX, llama.cpp, SQLite, PostgreSQL)**Linux**✅ Unit + integration (all backends including CUDA)**macOS**✅ Supported (CI-targeted, not yet in active integration matrix)---

Packages
--------

[](#packages)

```
packages/
├── core/          Contracts, enums, value objects, exceptions, AIConfig
├── tensor/        ArrayTensor (pure PHP), BackedTensor, TensorFactory
├── onnx-backend/  ONNX Runtime via ankane/onnxruntime FFI
├── llama-backend/ llama.cpp FFI, samplers (greedy/top-k/top-p/grammar),
│                  GBNF grammar, JSON Schema→GBNF, ChatFormatter (5 templates)
├── tokenizer/     Pure PHP BPE + WordPiece (round-tripping, chunking)
├── embedding/     Mean/CLS/EOS/Max pooling, 4 built-in models
├── vector/        SQLite + PostgreSQL/pgvector store, brute-force & native ANN, metadata filtering
├── model-hub/     HF download, LRU cache, SHA-256+Ed25519, format detection
├── pipeline/      Generator-based stages (8 types)
├── cpu-backend/   Pure-PHP tensor math + optional RubixML (.rbm) tabular inference
├── dataframe/     Tabular data: typed columns, CSV/JSON I/O, Tensor conversion
├── ai/            Facade (AI::), backend registry, model pool, metrics, profiler
├── laravel/       Service provider + facade (env-based config)
└── symfony/       Bundle + DI extension

```

---

Testing
-------

[](#testing)

```
composer test                # Unit tests — 793 pure-PHP tests
composer test-integration    # Integration — needs ONNX Runtime / llama.cpp / PostgreSQL
composer check               # Lint (CS + PHPStan lvl8 + Psalm lvl3) + unit tests — gate
```

---

Examples
--------

[](#examples)

[`examples/`](examples/) — 26 standalone scripts covering every capability: embedding, tokenizer, chat, streaming, RAG, pipeline, SQLite + sqlite-vec & PostgreSQL/pgvector, grammar-constrained generation, model hub, profiling, async fibers, model pool, observability, retry, CPU tensor math + RubixML, benchmarks, Laravel, Symfony.

```
set FERRY_AI_MODEL_DIR=C:\models\all-MiniLM-L6-v2-onnx
php examples/01-hello-embedding.php
```

---

Documentation
-------------

[](#documentation)

Start here: **[`docs/DOCUMENTATION.md`](docs/DOCUMENTATION.md)** — definitive single-file reference (architecture, facade API, contracts, GPU setup).

Guides: [getting-started](docs/getting-started.md) · [configuration](docs/configuration.md) · [ONNX](docs/backends/onnx.md) / [llama.cpp](docs/backends/llama.md) · [embedding](docs/embedding.md) · [vector store](docs/vector-store.md) · [pipeline](docs/pipeline.md) · [model hub](docs/model-hub.md) · [safetensors → GGUF](docs/safetensors-conversion.md) · [tokenizer](docs/tokenizer.md) · [streaming](docs/streaming.md) · [security](docs/security.md) · [deployment](docs/deployment.md) · [Laravel](docs/laravel.md) / [Symfony](docs/symfony.md) · [troubleshooting](docs/troubleshooting.md) · [API reference](docs/api-reference.md) · [CHANGELOG](CHANGELOG.md)

DocumentPurpose[`docs/TECHNICAL_SPECIFICATION.md`](docs/TECHNICAL_SPECIFICATION.md)Architecture[`docs/FILE_TREE.md`](docs/FILE_TREE.md)Complete file map[`docs/INTERFACE_CONTRACTS.md`](docs/INTERFACE_CONTRACTS.md)Interface signatures[`docs/SOURCES.md`](docs/SOURCES.md)External stack reference[`docs/README.md`](docs/README.md)Full navigator---

Contributing & license
--------------------------

[](#contributing--license)

- **Contributing:** guidelines and workflow in [`.github/CONTRIBUTING.md`](.github/CONTRIBUTING.md).
- **Security:** report vulnerabilities via [`.github/SECURITY.md`](.github/SECURITY.md) — please do not open public issues for them.
- **Code of Conduct:** [`.github/CODE_OF_CONDUCT.md`](.github/CODE_OF_CONDUCT.md).
- **License:** MIT — see [LICENSE.md](LICENSE.md).

###  Health Score

47

—

FairBetter than 93% of packages

Maintenance95

Actively maintained with recent releases

Popularity27

Limited adoption so far

Community19

Small or concentrated contributor base

Maturity40

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 60% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~1 days

Total

2

Last Release

49d ago

PHP version history (2 changes)v0.1.0PHP >=8.5

v0.1.1PHP >=8.3

### Community

Maintainers

![](https://www.gravatar.com/avatar/285d32ac89643c66426e75f97a13c5eee2d8e8b809efd2d6fdde4665837b7b63?d=identicon)[Yevhen Leonidov](/maintainers/Yevhen%20Leonidov)

---

Top Contributors

[![MADEVAL](https://avatars.githubusercontent.com/u/10908537?v=4)](https://github.com/MADEVAL "MADEVAL (3 commits)")[![dependabot[bot]](https://avatars.githubusercontent.com/in/29110?v=4)](https://github.com/dependabot[bot] "dependabot[bot] (2 commits)")

---

Tags

aiembeddingsffigenerative-aiggufinferencellama-cppllmllm-inferencelocal-llmmachine-learningnlponnxonnxruntimephpphp8ragsemantic-searchtransformersvector-databaseaimlllmllamaembeddingragvector storeonnx

###  Code Quality

TestsPHPUnit

Static AnalysisPHPStan, Psalm

Type Coverage Yes

### Embed Badge

![Health badge](/badges/ferry-ai-php-inference/health.svg)

```
[![Health](https://phpackages.com/badges/ferry-ai-php-inference/health.svg)](https://phpackages.com/packages/ferry-ai-php-inference)
```

###  Alternatives

[guzzlehttp/psr7

PSR-7 message implementation that also provides common utility methods

7.9k1.1B4.5k](/packages/guzzlehttp-psr7)[tempest/framework

The PHP framework that gets out of your way.

2.3k37.6k21](/packages/tempest-framework)[flow-php/flow

PHP ETL - Extract Transform Load - Data processing framework

86337.5k](/packages/flow-php-flow)[getbrevo/brevo-php

Official PHP SDK for the Brevo API.

1004.1M59](/packages/getbrevo-brevo-php)[wordpress/php-ai-client

A provider agnostic PHP AI client SDK to communicate with any generative AI models of various capabilities using a uniform API.

26554.3k27](/packages/wordpress-php-ai-client)[shopware/core

Shopware platform is the core for all Shopware ecommerce products.

595.8M685](/packages/shopware-core)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)