PHPackages                             slayerbirden/dataflow - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. slayerbirden/dataflow

ActiveLibrary[Utility & Helpers](/categories/utility)

slayerbirden/dataflow
=====================

A framework to pipe your data from one source to another.

1.0.3(6y ago)41.5kMITPHPPHP ^7.1

Since Mar 1Pushed 6y ago1 watchersCompare

[ Source](https://github.com/SlayerBirden/dataflow)[ Packagist](https://packagist.org/packages/slayerbirden/dataflow)[ RSS](/packages/slayerbirden-dataflow/feed)WikiDiscussions master Synced 3d ago

READMEChangelog (4)Dependencies (5)Versions (5)Used By (0)

DataFlow
========

[](#dataflow)

[![Build Status](https://camo.githubusercontent.com/18215cb3f4128397e4f98b20e96cf2ec32f0def8350a7d738e090de8f6cd4a04/68747470733a2f2f7472617669732d63692e6f72672f536c6179657242697264656e2f64617461666c6f772e7376673f6272616e63683d6d6173746572)](https://travis-ci.org/SlayerBirden/dataflow)[![Scrutinizer Code Quality](https://camo.githubusercontent.com/c1bd500d49b3845fc7fad9a02a2522b52b81ded5f23aa2ab8295a93c3eda29af/68747470733a2f2f7363727574696e697a65722d63692e636f6d2f672f536c6179657242697264656e2f64617461666c6f772f6261646765732f7175616c6974792d73636f72652e706e673f623d6d6173746572)](https://scrutinizer-ci.com/g/SlayerBirden/dataflow/?branch=master)[![Code Coverage](https://camo.githubusercontent.com/93f5e027aa48a485736765f60a5e82f0ba7178daecbba7ed8496a19c45430b9f/68747470733a2f2f7363727574696e697a65722d63692e636f6d2f672f536c6179657242697264656e2f64617461666c6f772f6261646765732f636f7665726167652e706e673f623d6d6173746572)](https://scrutinizer-ci.com/g/SlayerBirden/dataflow/?branch=master)

Build a pipeline to pour your data through!

About
-----

[](#about)

This is a low level lib that helps you with building a data migration process. It requires a little bootstrapping to float, but it can be easily done if you just need a minimal process or looking to test things out.

The main idea is that you're building a pipeline where each section can mutate the data that's flowing through. It allows process to be extremely flexible, since each new step can potentially get (**completely**) transformed data. It also can be very dangerous, since mistake can be costly, hence we crafted awesome error handling and reporting mechanisms.

Examples
--------

[](#examples)

### Build a simple cli import script.

[](#build-a-simple-cli-import-script)

You have a csv file you need to import into database? Let's do that!

1. For the sake of example we'll be importing users into table "users" with columns:

    id (int)name (string)email (string, unique)age (int)
2. Our csv file looks like this

    firstlastemailageArthurDayne23GeroldHightower43EddardStark36JaimeLannister34Aegon ITargaryen64
3. Let's use PipeLine builder to set up some known steps. We need a little bootstrapping to get DBAL writer and empty emitter.

```
use Doctrine\DBAL\DriverManager;
use SlayerBirden\DataFlow\Emitter\BlackHole;
use SlayerBirden\DataFlow\PipelineBuilder;
use SlayerBirden\DataFlow\Writer\Dbal\UpdateStrategy\UniqueIndexStrategy;
use SlayerBirden\DataFlow\Writer\Dbal\Write;
use SlayerBirden\DataFlow\Writer\Dbal\WriterUtility;

# bootstrap
$connection = DriverManager::getConnection([
    'url' => 'mysql://test-user:testpwd@localhost:4486/foo?charset=UTF8',
]);
// this is just a utility class to "cache" schema info
$utility = new WriterUtility($connection);
$dbWrite = new Write(
    'users_write', // pipe ID for reporting
    $connection, // DBAL connection
    'users', // db table name
    $utility, // utility class
    new UniqueIndexStrategy('users', $utility), // update or insert will depend on unique fields in the table
    $this->emitter
);
$emitter = new BlackHole();

# pipeline
$pipeline = (new PipelineBuilder($emitter))
    ->addSection($dbWrite)
    ->build();
```

4. Now initiate the Plumber and pour.

```
use SlayerBirden\DataFlow\Plumber;
use SlayerBirden\DataFlow\Provider\Csv;
...
$file = new \SplFileObject(__DIR__ . '/users.csv');
$file->setFlags(\SplFileObject::READ_CSV | \SplFileObject::READ_AHEAD | \SplFileObject::SKIP_EMPTY);
(new Plumber(new Csv('users_file', $file), $pipeline, $emitter))->pour();
```

5. We want some reporting to know what's going on. Let's implement basic stdOut emitter.

```
$emitter = new class implements \SlayerBirden\DataFlow\EmitterInterface
{
    public function emit(string $event, ...$args): void
    {
        echo $event, ' ==> ', implode(', ', $args), PHP_EOL;
    }
};
```

6. There are something else we need to do: concat firstName and lastName and assign to "name"

```
use SlayerBirden\DataFlow\DataBagInterface;

$pipeline = (new PipelineBuilder($emitter))
    ->map('name', new class implements MapperCallbackInterface
    {
        public function __invoke($value, DataBagInterface $dataBag)
        {
            return $dataBag['first'] . ' ' . $dataBag['last'];
        }
    })
    ->addSection($dbWrite)
    ->build();
```

7. Full file can be found under `examples/example1-import-cli/import.php`.
8. Logged results: [![results](examples/example1-import-cli/2-cli-runs.png)](examples/example1-import-cli/2-cli-runs.png)

Goals
-----

[](#goals)

- **Easy to maintain**. Strong reporting shows what exactly went wrong.
- **Flexible**. Extremely flexible workflow allows to work with different import/export/migration scenarios.
    - one file -> multiple tables
    - multiple files -> one table
    - supports dynamic DB connection
    - supports different source types and destination types
- **Fast**. Low level operations produce maximum flow speed.
- **Stable**. Strong test coverage guarantees bug-free groundwork.

Influences
----------

[](#influences)

- [Ddeboer Data Import library](https://github.com/ddeboer/data-import)

###  Health Score

30

—

LowBetter than 64% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity19

Limited adoption so far

Community7

Small or concentrated contributor base

Maturity61

Established project with proven stability

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~189 days

Total

4

Last Release

2430d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/bc8f58e80607cbca09398f2fedfa3e3b4979dbba6108998708bee9237b129560?d=identicon)[Oleg](/maintainers/Oleg)

---

Top Contributors

[![SlayerBirden](https://avatars.githubusercontent.com/u/1726160?v=4)](https://github.com/SlayerBirden "SlayerBirden (23 commits)")

---

Tags

data-flowdataflowexportimportmigrationpipeline

###  Code Quality

TestsPHPUnit

### Embed Badge

![Health badge](/badges/slayerbirden-dataflow/health.svg)

```
[![Health](https://phpackages.com/badges/slayerbirden-dataflow/health.svg)](https://phpackages.com/packages/slayerbirden-dataflow)
```

###  Alternatives

[codefog/contao-haste

haste extension for Contao Open Source CMS

42650.8k139](/packages/codefog-contao-haste)[eliashaeussler/typo3-form-consent

Extension for TYPO3 CMS that adds double opt-in functionality to EXT:form

1481.0k](/packages/eliashaeussler-typo3-form-consent)[ronasit/laravel-helpers

Provided helpers function and some helper class.

1475.7k13](/packages/ronasit-laravel-helpers)[inspiredminds/contao-fieldset-duplication

Contao extension to allow the duplication of form fieldsets in the front end by the user for additional input fields.

158.2k1](/packages/inspiredminds-contao-fieldset-duplication)[numero2/contao-storelocator

Contao Plugin for managing stores (or in common address data) and providing a frontend-search based on geo data

121.5k](/packages/numero2-contao-storelocator)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)