PHPackages                             trkisf2/remote-collection-stream - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. trkisf2/remote-collection-stream

ActiveLibrary[Utility & Helpers](/categories/utility)

trkisf2/remote-collection-stream
================================

Continuously streams remote collection in user defined chunks and yields them using Generators.

v0.2(8y ago)710.2k↓33.3%PHPPHP >=7.1

Since Jun 21Pushed 8y agoCompare

[ Source](https://github.com/TrkiSF2/remote-collection-stream)[ Packagist](https://packagist.org/packages/trkisf2/remote-collection-stream)[ RSS](/packages/trkisf2-remote-collection-stream/feed)WikiDiscussions master Synced 1mo ago

READMEChangelog (2)Dependencies (1)Versions (3)Used By (0)

Description
===========

[](#description)

Continuously streams remote collection in user defined chunks and yields them using Generators.

Usage
=====

[](#usage)

```
$stream = new Stream();

$campaignsCollectionGenerator = $stream->stream(
    new StreamConfiguration(self::BATCH_FETCH_SIZE_CAMPAIGNS),
    function ($offset, $limit) use ($job) {
        return $this->campaigns->getAll($job, $offset, $limit);
    }
);
```

Use case, implementation example
================================

[](#use-case-implementation-example)

Let's say we want to fully stream a bigger MySQL table (few million rows) into a messanging queue in chunks without crushing due to memory limits.

```
/** @var Chunk $campaignChunk */
foreach ($this->chunksProvider->streamCampaigns($job) as $campaignChunk) {
    try {
        echo $campaignChunk->key() . "\n";

        if ($this->chunksStorage->storeChunk($campaignChunk)) {
            $this->campaignQueue->insert($campaignChunk->serializedValue());
        }
    } catch (StoreChunkException $exception) {
        // Retry the process later
    }
}
```

ChunksProvider looks like:

```
/**
 * Continuously yields Campaign Chunk objects.
 *
 * @param Job $job
 *
 * @return \Generator
 */
public function streamCampaigns(Job $job): \Generator
{
    $stream = new Stream();

    $campaignsCollectionGenerator = $stream->stream(
        new StreamConfiguration(self::BATCH_FETCH_SIZE_CAMPAIGNS),
        function ($offset, $limit) use ($job) {
            return $this->campaigns->getAll($job, $offset, $limit);
        }
    );

    foreach ($campaignsCollectionGenerator as $collection) {
        yield new Chunk($collection);
    }
}
```

And $this->campaigns is a simple Repository fetching data from MySQL using $offset, $limit. Chunk object is just a custom DTO.

```
/**
 * @param Job $job
 * @param int $offset
 * @param int $limit
 *
 * @return LegacyCampaignsCollection
 */
public function getAll(Job $job, int $offset, int $limit) : LegacyCampaignsCollection;
```

Or it could be a dummy in memory collection

```
class DummyCollectionRepository
{
    private $allElements = [];

    /**
     * @param int $elementsCount
     */
    public function __construct(int $elementsCount)
    {
        for ($i = 1; $i allElements[] = $i;
        }
    }

    /**
     * @param int $offset
     * @param int $limit
     *
     * @return DummyCollection
     */
    public function getAll(int $offset, int $limit): DummyCollection
    {
        $collection      = new DummyCollection();
        $collectionChunk = array_slice($this->allElements, $offset, $limit);

        foreach ($collectionChunk as $element) {
            $collection->addElement($element);
        }

        return $collection;
    }
}
```

Few last words about the implementation
=======================================

[](#few-last-words-about-the-implementation)

The Stream objects fetches new collections using **callable** which is a bit loose as I can't enforce the callable arguments with an interface and even though it makes me feel as a JS developer, it gives the implementation side a nice advantage + there is a validation in place verifying the number of arguments passed.

The ability to combine the call with own parameters:

```
function ($offset, $limit) use ($job) {
    return $this->campaigns->getAll($job, $offset, $limit);
}
```

If Stream object would require some kind of "CollectionRepository" in the constructor then you would have to use setters, nullable attributes and other evil things in the repository in order to use additional arguments like filters in the query etc.

**Tests included.**

###  Health Score

29

—

LowBetter than 60% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity29

Limited adoption so far

Community2

Small or concentrated contributor base

Maturity49

Maturing project, gaining track record

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~0 days

Total

2

Last Release

3243d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/34d7e487d8a1f6ed43051780467ff6c1d34efdf8c69a46b29e0e4696ece0533e?d=identicon)[TrkiSF2](/maintainers/TrkiSF2)

---

Tags

generatorsyieldMySQL IteratorRemote collection stream

###  Code Quality

TestsPHPUnit

### Embed Badge

![Health badge](/badges/trkisf2-remote-collection-stream/health.svg)

```
[![Health](https://phpackages.com/badges/trkisf2-remote-collection-stream/health.svg)](https://phpackages.com/packages/trkisf2-remote-collection-stream)
```

###  Alternatives

[laracasts/generators

Advanced Laravel generators, that include schema information.

2.4k6.4M44](/packages/laracasts-generators)[loophp/collection

A (memory) friendly, easy, lazy and modular collection class.

745663.8k13](/packages/loophp-collection)[athari/yalinqo

YaLinqo, a LINQ-to-objects library for PHP

4561.2M5](/packages/athari-yalinqo)[laracademy/generators

This package will generate a Laravel Model based on your database table itself, filling in the required fields automatically.

355346.4k4](/packages/laracademy-generators)[laralib/l5scaffold

Extend Laravel 5's generators scaffold.

31474.1k](/packages/laralib-l5scaffold)[summerblue/generator

Extend Laravel's generators scaffold.

34139.9k](/packages/summerblue-generator)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)