PHPackages sandstorm/fuzzer - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. [Testing & Quality](/categories/testing) 4. / 5. sandstorm/fuzzer AbandonedTypo3-flow-package[Testing & Quality](/categories/testing) sandstorm/fuzzer ================ Improve your Functional and Unit Tests by Fuzzing: A technique to find missing edge cases in tests. 2.0.0(13y ago)14244GPLv3 Since Apr 29Compare [ Source](https://github.com/sandstorm/Fuzzer)[ Packagist](https://packagist.org/packages/sandstorm/fuzzer)[ RSS](/packages/sandstorm-fuzzer/feed)WikiDiscussions Synced today READMEChangelogDependencies (2)Versions (4)Used By (0) Fuzzer -- Validating Test Coverage ================================== [](#fuzzer----validating-test-coverage) \-- Testing your Test Coverage -- (c) Sebastian Kurfürst, Sandstorm Media UG (haftungsbeschränkt) NEW in 2.0 ---------- [](#new-in-20) **Unit and Functional Testing** In version 1.\*, Fuzzer only worked with Unit Tests. In the new version, it works with Unit and Functional tests, expanding its use-cases greatly. **PHPUnit through Composer** Now also supports a PHPUnit installation through composer, with fallback to the PEAR-installed version **increased robustness** It is now checked whether the needed command-line tools are available. **Code Coverage Helpers for Functional Tests** Collect and analyze code coverage also for your Functional Tests. See below for usage instructions. NEW in 2.0: Code Coverage Helpers --------------------------------- [](#new-in-20-code-coverage-helpers) Besides fuzzing, this package implements several helpers to work with functional tests: ``` # run unit and functional tests, both creating code coverages bin/phpunit -c Build/BuildEssentials/UnitTests.xml --coverage-php Build/Reports/RawUnitTestCoverage.php bin/phpunit -c Build/BuildEssentials/FunctionalTests.xml --coverage-php Build/Reports/RawFunctionalTestCoverage.php # post-process the functional test coverage (conversion from cache directories to packages) ./flow codecoverage:convert Build/Reports/RawFunctionalTestCoverage.php Build/Reports/FunctionalTestCoverage.php --packages Your.Package,Your.OtherPackage # Merge the reports for unit and functional test coverage ./flow codecoverage:merge Build/Reports/RawUnitTestCoverage.php Build/Reports/FunctionalTestCoverage.php Build/Reports/TestCoverage.php # render the final report as HTML: ./flow codecoverage:render Build/Reports/TestCoverage.php Build/Reports/TestCoverage # OR: render the final report as clover file ./flow codecoverage:render Build/Reports/TestCoverage.php Build/Reports/TestCoverage.xml --format clover ``` Fuzzing ------- [](#fuzzing) Do you use automated unit/functional tests to check your software? Are you monitoring the Code Coverage of your Unit Tests? Yeah -- then this software is for you! It helps to find missing edge cases in tests. It implements a technique called *Fuzzing*. Our Notion Of Coverage ---------------------- [](#our-notion-of-coverage) Normal "code coverage" metrics (as measured by PHPUnit for example) only counts how often a code line is executed during the test runs. So, while PHPUnit Code Coverage checks that you *execute* all covered lines, it does not prove that you *valdiate* the functionality executed on the particular line. An example shall illustrate this. Imagine you have the following class: ``` class Foo { protected $someInternalState = 0; public function someMethod() { $this->someInternalState++; return 42; } public function getInternalState() { return $this->someInternalState; } } ``` ... and the following testcases: ``` /** * @test */ public function someTest() { $myObject = new Foo(); $this->assertSame(42, $myObject->someMethod()); } /** * @test */ public function someOtherTest() { $myObject = new Foo(); $this->assertSame(0, $myObject->someInternalState()); } ``` This example has a *Code Coverage* of **100 %**, so you might say "yeah, great, nothing to improve here". However, if you look closely into the code, you will see that commenting out the line `$this->someInternalState++` will **not break the unit tests**, despite of broken functionality. Here Comes The Fuzzer --------------------- [](#here-comes-the-fuzzer) The fuzzer automatically **modifies your source code**, checks that the resulting file has a valid syntax, and then runs the unit/functional tests. In a perfect world, the tests would fail after every modification, as we modified the source code, and deliberately broke some functionality. The fuzzer detects cases where **the source has been modified, but the tests still run through successfully**, giving indication which other test cases you need to write. Installation ------------ [](#installation) **The system has been tested on Mac OS, and should also run on Linux. It will probably not run on Windows.** Simply run `composer require --dev sandstorm/fuzzer:2.*` You additionally need the following command-line tools installed: - git - php (should not be problematic, as you have a working TYPO3 Flow installation) - xdebug (as we must be able to run phpunit with code coverage reports) - phpunit (can also be installed using Composer) - timeout (on Mac OS, you can install it using MacPorts with "port install timeout"; installed by default in many linux distributions) Usage ----- [](#usage) First, make sure you have a reasonably **high code coverage**, as the fuzzer will only work on code which is covered by Unit Tests (to reduce the number of false positives). ``` ./flow3 fuzzer:fuzz ``` The package must have **its own Git Repository** and must **not have any uncommitted changes**, else the tool will not run. Example Output -------------- [](#example-output) ``` Unmodified unit tests took 1 seconds, setting timeout to 3 seconds (to be safe). Generating and Testing Mutations: _.__.._..._.___....__.E._._..._..._.__.E._...._..E__.__.._.._.._E_.___._.___...E_.._._._..._.T._..._ Undetected Mutations -------------------- Classes/Domain/Model/FooBar.php Package: TYPO3.FooBar Diff follows below diff --git a/Classes/Domain/Model/FooBar.php b/Classes/Domain/Model/FooBar.php index 8fde46c..bee58d7 100644 --- a/Classes/Domain/Model/FooBar.php +++ b/Classes/Domain/Model/FooBar.php @@ -107,7 +107,7 public function registerIfPossible() { - parent::registerIfPossible(); +# parent::registerIfPossible(); foreach ($this->elements as $element) { $element->doStuff(); } ... output for all other undetected mutations ... Fuzzing Statistics ------------------ Total Mutations: 253 Mutations with Broken Syntax: 120 Detected Mutations: 127 Undetected mutations: 6 (see above for details) Total Runtime: 61 s ``` In the above example, you see that commenting out the "parent::..." call did not make the unit tests fail -- so you can now write this additional unit test. For every run mutation, a progress indicator is shown like it is done by PHPUnit. The characters mean the following: - `_` the mutation contains PHP syntax errors - `.` the unit tests could be ran, and they failed correctly - `T` the unit tests ran into a timeout (which is also expected behavior) - `E` the unit tests ran through SUCCESSFULLY; meaning a mutation was not detected. **These are the cases we are looking for.** Timeouts -------- [](#timeouts) As we modify source code, it could easily be that the modified source code does not terminate anymore. Thus, we first check how long the unit tests take, add some offset to it and abort the tests if they run longer than expected. If this happens, we handle it the same way like a test failure. Internals --------- [](#internals) There are different fuzzers available in the system which generate source code mutations: - *SingleLineFuzzer*: Comments out a single source code line at a time. Only works on lines which are executed at least once during the unit tests - (more to come here later) License ------- [](#license) All the code is licensed under the GPL license. ### Health Score 31 — LowBetter than 66% of packages Maintenance20 Infrequent updates — may be unmaintained Popularity18 Limited adoption so far Community6 Small or concentrated contributor base Maturity65 Established project with proven stability Bus Factor1 Top contributor holds 100% of commits — single point of failure How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window. **Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores. **Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement. **Maturity (30%)** — Project age, version count, PHP version support, and release stability. ### Release Activity Cadence Every ~21 days Total 3 Last Release 4768d ago Major Versions 1.0.1 → 2.0.02013-06-11 ### Community Maintainers ![](https://www.gravatar.com/avatar/2ced0d63cfdae881c32128c7f66451a013d3e24d9eed210d6a846b6d8e95fa3b?d=identicon)[sandstorm](/maintainers/sandstorm) --- Top Contributors [![skurfuerst](https://avatars.githubusercontent.com/u/190777?v=4)](https://github.com/skurfuerst "skurfuerst (15 commits)") ### Embed Badge ![Health badge](/badges/sandstorm-fuzzer/health.svg) ``` [![Health](https://phpackages.com/badges/sandstorm-fuzzer/health.svg)](https://phpackages.com/packages/sandstorm-fuzzer) ``` ### Alternatives [phpunit/phpunit The PHP Unit Testing framework. 20.0k940.0M150.5k](/packages/phpunit-phpunit)[brianium/paratest Parallel testing for PHP 2.5k129.9M920](/packages/brianium-paratest)[codeception/codeception All-in-one PHP Testing Framework 4.9k88.9M3.2k](/packages/codeception-codeception)[phpunit/phpcov CLI frontend for php-code-coverage 23722.2M643](/packages/phpunit-phpcov)[facile-it/paraunit paraunit 145802.1k15](/packages/facile-it-paraunit)[innmind/black-box Test library 44190.0k112](/packages/innmind-black-box) PHPackages © 2026 [Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)