PHPackages                             displace/ext-infer - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. displace/ext-infer

ActivePhp-ext[Utility & Helpers](/categories/utility)

displace/ext-infer
==================

PHP 8.3+ native, in-process LLM inference and embeddings via llama.cpp.

v0.2.0(1mo ago)516MITRustPHP ^8.3CI passing

Since Jun 7Pushed 1mo agoCompare

[ Source](https://github.com/DisplaceTech/ext-infer)[ Packagist](https://packagist.org/packages/displace/ext-infer)[ Docs](https://github.com/DisplaceTech/ext-infer)[ RSS](/packages/displace-ext-infer/feed)WikiDiscussions main Synced 1w ago

READMEChangelog (3)DependenciesVersions (5)Used By (0)

ext-infer
=========

[](#ext-infer)

 **Local LLM inference for PHP, in-process.**
 Chat, embeddings, and reasoning models — no Python sidecar, no remote API.

 [![CI](https://github.com/DisplaceTech/ext-infer/actions/workflows/ci.yml/badge.svg)](https://github.com/DisplaceTech/ext-infer/actions/workflows/ci.yml) [![Latest release](https://camo.githubusercontent.com/c4282f732a283c0cff9bc3634e9bba6ed54c35bf0122ec6e37b5ece11afe0050/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f72656c656173652f446973706c616365546563682f6578742d696e6665723f736f72743d73656d76657226696e636c7564655f70726572656c6561736573)](https://github.com/DisplaceTech/ext-infer/releases/latest) [![PHP 8.3 / 8.4 / 8.5](https://camo.githubusercontent.com/f8c66ec3458748baee370b7844203c334e62735196a3c31b11c504ab609f4be7/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5048502d382e33253230253743253230382e34253230253743253230382e352d3737374242343f6c6f676f3d706870266c6f676f436f6c6f723d7768697465)](https://camo.githubusercontent.com/f8c66ec3458748baee370b7844203c334e62735196a3c31b11c504ab609f4be7/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5048502d382e33253230253743253230382e34253230253743253230382e352d3737374242343f6c6f676f3d706870266c6f676f436f6c6f723d7768697465) [![Pre-release](https://camo.githubusercontent.com/9f096097aa41a5a065f1cfb6503ce800e794f0734b4ab8dc18231d463db099c5/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5374617475732d7072652d2d72656c656173652d6f72616e6765)](https://camo.githubusercontent.com/9f096097aa41a5a065f1cfb6503ce800e794f0734b4ab8dc18231d463db099c5/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5374617475732d7072652d2d72656c656173652d6f72616e6765) [![MIT License](https://camo.githubusercontent.com/5caa455d8debc46fb23abbadb45a733a937f3910a73fc875c2f7820468e1bb54/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4c6963656e73652d4d49542d677265656e)](LICENSE) [![Documentation](https://camo.githubusercontent.com/d7ff0ccc850ac07004b8cf286a75c5b9ee6aa79804b7d6dde430a424355471a5/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f646f63732d696e6665722e646973706c6163652e746563682d626c7565)](https://infer.displace.tech)

---

What is ext-infer?
------------------

[](#what-is-ext-infer)

`ext-infer` is a PHP 8.3+ extension that loads a GGUF model and runs inference *in the PHP process* via [llama.cpp](https://github.com/ggerganov/llama.cpp). PHP-native semantic search, RAG pipelines, and CLI/worker inference work without shelling out to Python or hitting a remote API.

Written in Rust on top of [`ext-php-rs`](https://github.com/davidcole1340/ext-php-rs)and the [`llama-cpp-2`](https://crates.io/crates/llama-cpp-2) bindings. The public PHP surface is fluent and role-aware — building a chat prompt looks like `Prompt::system(...)->withUser(...)`, not a string of ``tokens.

- 💬 **Chat completions** via an immutable `Prompt` builder that renders through the model's embedded template — no manual `` plumbing.
- 🧱 **Structured output** — pass a JSON Schema (or raw GBNF grammar) and sampling is constrained so malformed output is *impossible*, not retried. A 0.6B model becomes a dependable extractor.
- 🧠 **Reasoning-model aware** — `Response::answer()` and `Response::reasoning()` split Qwen3 / R1-style `…` output automatically.
- 📊 **Embeddings** — `Model::embed()` returns an `Embedding` with `dimensions()`, `normalize()`, `cosineSimilarity()`, and `packed()` (zero-copy handoff to vector indexes) built in.
- 🎯 **Reranking** — `RerankModel` scores (query, document) pairs through Qwen3-Reranker's calibrated yes/no judgment; completes the embed → rerank two-stage retrieval pipeline.
- ⚡ **In-process** — no subprocess fork, no IPC, no daemon. Latency is whatever the model takes to decode.
- 🛠️ **Apple Metal** acceleration is opt-in (`make release FEATURES=metal`); CPU is the portable default.
- 🧵 **Thread-safe** — `LlamaBackend` is a `Sync`-guarded singleton and each call builds its own context, so ZTS PHP + `parallel` works by design.

Quick start
-----------

[](#quick-start)

```
mkdir -p models
curl -L -o models/Qwen3-0.6B-Q8_0.gguf \
    https://huggingface.co/Qwen/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B-Q8_0.gguf
```

```