PHPackages ykachala/evals - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. [Testing & Quality](/categories/testing) 4. / 5. ykachala/evals ActiveLibrary[Testing & Quality](/categories/testing) ykachala/evals ============== Prompt and LLM-output regression testing for PHP. Define golden datasets, assert on model output (schema, semantic similarity, LLM-as-judge), and gate CI on quality scores. 00PHP Since Jun 2Pushed 1mo agoCompare [ Source](https://github.com/ykachala/evals)[ Packagist](https://packagist.org/packages/ykachala/evals)[ RSS](/packages/ykachala-evals/feed)WikiDiscussions main Synced 1w ago READMEChangelogDependenciesVersions (1)Used By (0) Ykachala Evals ============== [](#ykachala-evals) > **Regression testing for prompts and LLM output**, built for PHP CI. Define golden datasets, score model responses with deterministic *and* LLM-as-judge assertions, and fail the build when quality drops below a threshold. [![PHP Version](https://camo.githubusercontent.com/5a2eb3f6217f798b7ca58d59764e8fd3d779752c0cef2b8634034d9e7e4ba92d/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f7068702d253345253344382e332d373737626234)](https://www.php.net/)[![License](https://camo.githubusercontent.com/f8df3091bbe1149f398a5369b2c39e896766f9f6efba3477c63e9b4aa940ef14/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d677265656e)](LICENSE) --- Why this exists (the 2026 gap) ------------------------------ [](#why-this-exists-the-2026-gap) A prompt change is a code change with **no test suite**. Swap a model, tweak a system prompt, or upgrade your RAG retriever, and you have no idea whether you just made answers better or quietly broke 20% of them. Teams find out in production. Python has a rich eval stack — **promptfoo**, **DeepEval**, **Ragas**, Langfuse evals. PHP, even with the 2026 surge in Prism / Neuron / Laravel AI SDK adoption, has **no native eval harness**. You can call a model from PHP, but you can't *assert it's still good* in CI. Ykachala Evals is that missing test layer — it feels like PHPUnit/Pest, not a separate Python toolchain. What it does ------------ [](#what-it-does) - **Datasets** — golden cases (`input`, optional `expected`, metadata) from PHP arrays, JSON/CSV, or a generator. - **Assertions / scorers**, mixed freely per case: - Deterministic: `equals`, `contains`, `regex`, `jsonSchema`, `validJson`, `latencyUnder`, `costUnder` - Semantic: `similarTo` (embedding cosine ≥ threshold) - **LLM-as-judge**: `judge('Is the answer faithful to the context? Score 0–1.')` - **Runners** — drive it from Pest/PHPUnit *or* the `vendor/bin/evals` CLI. - **CI gates** — set pass thresholds (e.g. "≥ 0.9 of cases score ≥ 0.8"); non-zero exit on failure. - **Reports** — JSON, JUnit XML (for CI annotations), and an HTML diff vs. the last snapshot. Install ------- [](#install) ``` composer require --dev ykachala/evals ``` Quick start ----------- [](#quick-start) ``` use Ykachala\Evals\Suite; use Ykachala\Evals\Dataset; use function Ykachala\Evals\{contains, jsonSchema, judge, similarTo}; $suite = Suite::make('support-bot') ->using(fn (string $input) => $myAgent->reply($input)) // the system under test ->dataset(Dataset::fromJson(__DIR__.'/golden/support.json')) ->assert( contains('refund', caseInsensitive: true), similarTo('We will process your refund within 5 days', threshold: 0.82), judge('Is the reply polite and does it resolve the request?', pass: 0.7), ) ->gate(passRate: 0.9, minScore: 0.8); $report = $suite->run(); echo $report->summary(); // 47/50 passed · mean 0.91 · 2 regressions $report->writeJunit('build/evals.xml'); exit($report->passed() ? 0 : 1); ``` ### In Pest [](#in-pest) ``` it('keeps the support bot faithful', function () { $report = Suite::make('support-bot')->/* ... */->run(); expect($report)->toPassGate(); }); ``` ### From CI [](#from-ci) ``` vendor/bin/evals run evals/ --gate=0.9 --junit=build/evals.xml ``` LLM-as-judge, made affordable ----------------------------- [](#llm-as-judge-made-affordable) Judge and `similarTo` assertions cost tokens. Ykachala Evals integrates with **`ykachala/semantic-cache`** to cache judge verdicts and embeddings, so re-running the suite on an unchanged dataset is nearly free — and with **`ykachala/meter`** you see the exact token cost of each eval run. Architecture ------------ [](#architecture) ``` src/ ├── Suite.php # fluent builder: using() / dataset() / assert() / gate() / run() ├── Dataset.php # case loading (array, JSON, CSV, generator) ├── EvalCase.php # one eval case: input, expected, metadata (`Case` is reserved) ├── Assertion/ # deterministic + semantic + judge assertions ├── Judge/ # LLM-as-judge interface + adapters ├── Report.php # scores, regressions, JSON/JUnit/HTML output └── Gate.php # pass thresholds + CI exit semantics bin/evals # CLI runner ``` Roadmap ------- [](#roadmap) - EvalCase + Dataset loaders (array/JSON/CSV/generator) - Deterministic assertions (equals/contains/regex/jsonSchema/validJson/latency/cost) - Suite builder + Report (scores, regressions vs. snapshot) - Semantic `similarTo` + LLM-as-judge assertions - Gate + CLI runner + JUnit/HTML reports - semantic-cache + meter integration See [`CLAUDE.md`](CLAUDE.md) for the full phase plan and conventions. License ------- [](#license) MIT ### Health Score 19 — LowBetter than 9% of packages Maintenance59 Moderate activity, may be stable Popularity0 Limited adoption so far Community6 Small or concentrated contributor base Maturity11 Early-stage or recently created project Bus Factor1 Top contributor holds 100% of commits — single point of failure How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window. **Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores. **Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement. **Maturity (30%)** — Project age, version count, PHP version support, and release stability. ### Community Maintainers ![](https://www.gravatar.com/avatar/5c5892a06b5cc3a1f6adaa3fa3d0cb5e9f77c6fc613425df614e2c4778b2e0a7?d=identicon)[joel767443](/maintainers/joel767443) --- Top Contributors [![ykachala](https://avatars.githubusercontent.com/u/985991?v=4)](https://github.com/ykachala "ykachala (9 commits)") --- Tags aicievalsllmllm-as-judgepestphpunitprompt-engineeringqualityregressiontesting ### Embed Badge ![Health badge](/badges/ykachala-evals/health.svg) ``` [![Health](https://phpackages.com/badges/ykachala-evals/health.svg)](https://phpackages.com/packages/ykachala-evals) ``` PHPackages © 2026 [Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)