PHPackages mbaynton/batch-framework - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. [Queues & Workers](/categories/queues) 4. / 5. mbaynton/batch-framework ActiveLibrary[Queues & Workers](/categories/queues) mbaynton/batch-framework ======================== An API and foundational algorithms for efficient processing of long-running jobs that can be divided into small work units. 1.0.0(7y ago)181MITPHPPHP >=5.4.0 Since Feb 26Pushed 7y agoCompare [ Source](https://github.com/mbaynton/batch-framework)[ Packagist](https://packagist.org/packages/mbaynton/batch-framework)[ RSS](/packages/mbaynton-batch-framework/feed)WikiDiscussions master Synced 3d ago READMEChangelogDependencies (4)Versions (4)Used By (0) Batch Processing Framework ========================== [](#batch-processing-framework) [![Build Status](https://camo.githubusercontent.com/5be4b2af07746c2ff808e97047258968ff69e77237327fa76082805c9a15325f/68747470733a2f2f7472617669732d63692e6f72672f6d6261796e746f6e2f62617463682d6672616d65776f726b2e7376673f6272616e63683d6d6173746572)](https://travis-ci.org/mbaynton/batch-framework)[![Coverage Status](https://camo.githubusercontent.com/ee1e16e3e3ca5c73e87a373181e20ac83e291143f64d4e18987b9b87a811a1b9/68747470733a2f2f636f766572616c6c732e696f2f7265706f732f6769746875622f6d6261796e746f6e2f62617463682d6672616d65776f726b2f62616467652e7376673f6272616e63683d6d6173746572)](https://coveralls.io/github/mbaynton/batch-framework?branch=master) This library offers foundational algorithms and structures to enable scenarios where long-running tasks that can be divided into small work units get processed progressively by successive calls to a PHP script on a webserver. This avoids exceeding script execution time and network timeout limitations often found in web execution environments. It emphasizes minimal overhead of the framework itself so that jobs complete as quickly as possible. Features include: - Support for processing the batch of work units across the lifespan of many requests when being run in a web environment. This prevents individual responses and webserver processes from running longer than is desirable. - Efficient determination of when to stop running more work units based on past work units' runtimes so that requests complete around a target duration. - Attention to minimizing the amount of state data and number of trips to a backing store that are involved with handing off between reqeusts. - Support for parallel execution of embarrasingly parallelizable problems, e.g. those where individual work units do not need to communicate or coordinate between each other during their execution. See parallelization for details. - No requirement to use a particular PHP framework, but with an awareness of controller and service design patterns. As this is a library, it offers no functionality "out of the box." Dependencies ------------ [](#dependencies) - PHP 5.4+ - `Psr\Http\Message\ResponseInterface` available via Composer, and any implementation of this interface. Documentation / Examples ------------------------ [](#documentation--examples) The docs here will help start you up writing code that's meant to work with this framework. If you encounter gaps or questions about the info here, you might want to refer to the [Curator application on GitHub](http://github.com/curator-wik/curator), which uses and was written alongside this framework. Documentation is accurate for `v1.0.0`. ### Terms and their definitions [](#terms-and-their-definitions) - **Runnable**: One of the user-implemented classes that models a long-running task. An instance of a Runnable models and provides the implementation for a single unit of work. It is its `run()`method whose body does the actual work/computation to further the Task's progress. - **Runnable Iterator**: A PHP `\Iterator` (please extend `AbstractRunnableIterator`) that produces `Runnables`appropriate to the segment of the overall task that should be performed, given as input the `Runner rank` and number of `Runnables` already performed on prior incarnations of the `Runner`. - **Runner**: The server-side code that runs the show. The Runner pumps the Runnable iterator for new Runnables, launches them, monitors the time runnables are taking and the time remaining to decide when to stop, dispatches Runnable and Task execution events to Task and Controller callbacks, and initiates Runnable and Task intermediate result aggregation. - **Runner id**: An integer uniquely identifying a given logical `Runner`. Clients are expected to create as many corresponding `Runner` requests as the framework's current `Task instance state` supports, initially assigning a unique integer id that the client has not used before to each of these requests. - **Runner incarnation**: Logically, the framework tries to create the illusion of `n` `Runnable` units of work that are executed by`x` `Runners` (concurrently if `x > 1`.) However, in order to prevent the HTTP request that started the `Runnable` from remaining incomplete for longer than desired, the framework may stop launching new `Runnables`, let the `Runner` stop doing work early, and signal the client to make a successive request with the same `Runner id`. Each HTTP request that's handled by starting a `Runner` bearing the same `Runner id` is called an *incarnation* of the runner with that id. All incarnations of a `Runner` also will share the same `Runner rank`. - **Runner rank**: A number uniquely identifying a given `Runner` within a Task. If your Task only supports one concurrent `Runner`, this will always be `0`. If your `Task` declares support for `n` concurrent `Runner`s, this will range from `0` to `n-1`. Differs from `Runner id` in that its range is always `0` to `n-1`. - **Task**: One of the user-implemented classes that models a long-running task. The `Task`serves as a factory for `Runnable Iterator`s, tells the framework what to do with results of `Runnable`s, may intervene in the event a `Runnable` experiences a throwable error or exception, provides methods to reduce multiple `Runnable` results to simpler intermediate results, and provides a method to translate the complete `Runnable` results to a `Psr\Http\Message\ResponseInterface`. - **Task instance state**: One of the user-implemented classes that models a long-running task. Task instance state captures the variable properties of a given task execution, such as where to find inputs to operate on, who (in terms of PHP session id) is currently running this `Task`, how large the `Task` is estimated to be (in terms of `Runnable`s), and how many concurrent `Runners` the `Task` supports. Typically, one can extend the `TaskInstanceState` class, which handles most everything but your task's unique inputs. Note that this class is not intended to be used to capture `Runnable` output. This framework primarily provides an implementation of the `Runner` in the class `AbstractRunner`. A complete system leveraging this library will typically include a concrete extension of `AbstractRunner` to interface with your application's persistence layer (e.g., database), and a controller or other script making use of the `HttpRunnerControllerTrait`to handle incoming requests and interface with your application's session layer. Coding a long-running task typically involves setting up the following components: - An implementation of `TaskInterface`. - An extension of `AbstractRunnableIterator` to serve `Runnables`. - An implementation of `RunnableInterface` to do the work units. - An extension of `TaskInstanceState` to provide input properties specific to the job. ### Parallelization: using multiple runners [](#parallelization-using-multiple-runners) Strictly speaking, this framework supports concurrent execution of more than one runnable from the same Task at a time. But, in order to do concurrent runnables, lots of other code must support this, too: - Your extension of `AbstractRunner` must implement its methods in a concurrency-safe manner, especially `AbstractRunner::retrieveRunnerState()` and `AbstractRunner::finalizeRunner()` should read and write to their underlying storage in a way that does not cause corruption or lost writes should several instances for the same Task instance be run simultaneously. - Your client must be programmed to send multiple concurrent batch runner requests. - The work you want to do must be [embarrasingly parallelizable](https://en.wikipedia.org/wiki/Embarrassingly_parallel). Each runnable can produce output, but runnables cannot take other runnables' output from the `Task` as input or otherwise interfere with each other if they access a shared resource. - Your `Task instance state`'s `getNumRunners()` must return more than 1 to declare concurrent support for more than 1 `Runner`. - The `Runnable iterator` constructed by your `Task` must take the `Runner rank` into account and be able to assign a portion of the total `Runnable`s to each `Runner rank`, as evenly as possible, with each `Runnable` unit of work being given out to one of the `Runner`s exactly once. - Your overall application (request controller, etc.) must not be impacted by several simultaneous requests from the same user, and must not be holding the [PHP session lock](http://php.net/manual/en/function.session-write-close.php)when the runnables are executing. ### Why is the Task's final result always an HTTP response? [](#why-is-the-tasks-final-result-always-an-http-response) Packaging the batch run's overall result in a standard HTTP response format enables applications to receive requests and decide whether or not to defer them to a batch task. In either case, the HTTP response that the client is expecting is ultimately generated. This works well when clients are implemented using libraries that support request middleware and the Promise pattern. The request middleware watches for raw responses that indicate a batch task is necessary, and rather than resolving the client application code's Promise with this incomplete raw response, launches `Runner` requests until it obtains the result HTTP response, which it finally resolves the original Promise with. License ------- [](#license) MIT ### Health Score 26 — LowBetter than 43% of packages Maintenance20 Infrequent updates — may be unmaintained Popularity7 Limited adoption so far Community7 Small or concentrated contributor base Maturity60 Established project with proven stability Bus Factor1 Top contributor holds 100% of commits — single point of failure How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window. **Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores. **Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement. **Maturity (30%)** — Project age, version count, PHP version support, and release stability. ### Release Activity Cadence Unknown Total 1 Last Release 2702d ago ### Community Maintainers ![](https://www.gravatar.com/avatar/b476fed6591946a9eeb4af8c9f9870051123e62da0920d2ff280d9600407717a?d=identicon)[mbaynton](/maintainers/mbaynton) --- Top Contributors [![mbaynton](https://avatars.githubusercontent.com/u/3026002?v=4)](https://github.com/mbaynton "mbaynton (40 commits)") --- Tags batchbatch-frameworkparallelization ### Code Quality TestsPHPUnit ### Embed Badge ![Health badge](/badges/mbaynton-batch-framework/health.svg) ``` [![Health](https://phpackages.com/badges/mbaynton-batch-framework/health.svg)](https://phpackages.com/packages/mbaynton-batch-framework) ``` ### Alternatives [aws/aws-php-sns-message-validator Amazon SNS message validation for PHP 21421.5M91](/packages/aws-aws-php-sns-message-validator)[koco/messenger-kafka Symfony Messenger Kafka Transport 931.1M1](/packages/koco-messenger-kafka)[easyswoole/wechat Coroutine safety WeChat library 8140.8k4](/packages/easyswoole-wechat)[aedart/athenaeum Athenaeum is a mono repository; a collection of various PHP packages 245.2k](/packages/aedart-athenaeum) PHPackages © 2026 [Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)