PHPackages                             maketok/datamigration - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Database & ORM](/categories/database)
4. /
5. maketok/datamigration

ActiveLibrary[Database & ORM](/categories/database)

maketok/datamigration
=====================

Maketok Data-Migration engine

0.3.1(7y ago)429.0k↓78.6%3[1 issues](https://github.com/SlayerBirden/datamigration/issues)MITPHPPHP >=5.4.0

Since Jul 28Pushed 7y ago2 watchersCompare

[ Source](https://github.com/SlayerBirden/datamigration)[ Packagist](https://packagist.org/packages/maketok/datamigration)[ Docs](https://github.com/SlayerBirden/datamigration)[ RSS](/packages/maketok-datamigration/feed)WikiDiscussions master Synced 2d ago

READMEChangelog (10)Dependencies (6)Versions (17)Used By (0)

Data-Migration
==============

[](#data-migration)

[![Build Status](https://camo.githubusercontent.com/1ac1e1749b54a2da773874cf2ff1d7bfdf1c2c0a46e4fd4aa2ca742acfdef111/68747470733a2f2f7472617669732d63692e6f72672f536c6179657242697264656e2f646174616d6967726174696f6e2e7376673f6272616e63683d6d6173746572)](https://travis-ci.org/SlayerBirden/datamigration)[![Scrutinizer Code Quality](https://camo.githubusercontent.com/cf002a61191bd81ccbdf903820847e782ee5f260f7063a26bd7f2f7e6ac22aec/68747470733a2f2f7363727574696e697a65722d63692e636f6d2f672f536c6179657242697264656e2f646174616d6967726174696f6e2f6261646765732f7175616c6974792d73636f72652e706e673f623d6d6173746572)](https://scrutinizer-ci.com/g/SlayerBirden/datamigration/?branch=master)[![Code Coverage](https://camo.githubusercontent.com/6b531945a56bf8b42c61793302a7da545a23a0f5fd951f582d20fc662ab1da6e/68747470733a2f2f7363727574696e697a65722d63692e636f6d2f672f536c6179657242697264656e2f646174616d6967726174696f6e2f6261646765732f636f7665726167652e706e673f623d6d6173746572)](https://scrutinizer-ci.com/g/SlayerBirden/datamigration/?branch=master)

This package is aimed at migrating large chunks of data across different resources.

Some of the influences:

- [mysqlimport](https://dev.mysql.com/doc/refman/5.0/en/mysqlimport.html)
- [Ddeboer Data Import library](https://github.com/ddeboer/data-import)

It's perfect to plan and execute a complex import or export. It also can be used as a transport tool to migrate your data from one structure to another. The unit (workers) structure allows it to parse a complex import file and extract a multi-level data structure from it.

The same's true for reverse process as well: the multi-level data structure can be inserted into a single file.

Installation
------------

[](#installation)

Use composer to include it in your project:

```
composer require maketok/datamigration

```

Examples
--------

[](#examples)

There are few examples in `tests/integration/QueueWorkflowTest` integration test. Here's the typical uses.

### Import

[](#import)

Import customers and addresses from flat CSV file.

emailnameagestreetcityzipBart Robbinson3220 Chestnut TerraceNew York07003Kale D15123 MansonLA90023111 Dale str.Chicago60333```
use Maketok\DataMigration\MapInterface;
use Maketok\DataMigration\QueueWorkflow;
use Maketok\DataMigration\Unit\SimpleBag;
use Maketok\DataMigration\Unit\Type\Unit;
use Maketok\DataMigration\Storage\Db\ResourceHelperInterface;
use Maketok\DataMigration\Action\Type\CreateTmpFiles;
use Maketok\DataMigration\Action\Type\Load;
use Maketok\DataMigration\Action\Type\Move;
use Maketok\DataMigration\Input\Shaper\Processor\Nulls;

$customerUnit = new Unit('customers');
$customerUnit->setTable('customers');
$customerUnit->setMapping([
    'id' => 'map.customer_id'
    // ExpressionLanguage is used to interpret string expressions
    'email' => 'mail.email',
    // closure or any other callable is also acceptable
    'name' => function (MapInterface $map) {
        return $map['name'];
    },
    'age' => 'map.age',
]);
/*
 * the is_entity condition resolves whether
 *  unit should consider current row as the entity
 * some utility functions are available in
 *  Maketok\DataMigration\Expression\HelperExpressionsProvider
 */
$customerUnit->setIsEntityCondition("trim(map.email) is not empty");
/*
 * the contributions is the way for unit to
 *  add some data into general pool for every other unit to use
 *
 * This is the logic for assigning customer_id
 * First it checks if it exists in the pre-compiled Hashmap
 * If it does not, it's calling for frozen increment for "new_customer_id" key
 *  and assign the last increment id if it's non existent
 * The frozenIncr is different from incr in that it's incremented only once
 *  is_entity condition resolves for current row
 * So it's perfect for incrementing "parent" entities
 */
$customerUnit->addContribution(function (
    MapInterface $map,
    ResourceHelperInterface $resource,
    array $hashmaps
    ) {
        if (isset($hashmaps['email-id'][trim($map->email)])) {
            $map['customer_id'] = $hashmaps['email-id'][trim($map->email)];
        } else {
            $map['customer_id'] = $map->frozenIncr(
                'new_customer_id',
                 $resource->getLastIncrementId('customers')
             )
        }
    });

$addressUnit = new Unit('addresses');
$addressUnit->setTable('addresses');
$addressUnit->setMapping([
    'id' => 'map.incr('address_id', resource.getLastIncrementId('addresses'))'
    'street' => 'map.street',
    'city' => 'map.city',
    'zip' => 'map.zip',
    'parent_id' => 'map.customer_id',
]);
$addressUnit->setParent($customerUnit);
$bag = new SimpleBag();
$bag->addSet([$customerUnit, $addressUnit]);

/*
 * Last but not least, since we're using CSV file, we need a Shaper
 * instance to shape up our flat file before feeding it to CreateTmpFiles action
 */
$input = new Csv($fname, 'r', new Nulls($bag, new ArrayMap(), $this->getLanguageAdapter()));

$workflow = new QueueWorkflow($config, $result);
$workflow->add(new CreateTmpFiles($bag, $config, $languageAdapter,
    $input, new ArrayMap(), $helperResource));
$workflow->add(new Load($bag, $config, $resource));
$workflow->add(new Move($bag, $config, $resource));
$workflow->execute();
```

### Export

[](#export)

We have 3 DB tables for customers and their addresses.

customer

idemailage132215customer\_data

idparent\_idfirstnamelastname11BartRobinson22KaleDageraddress

idcustomer\_idstreetcityzip1120 Chestnut TerraceNew York0700321123 MansonLA9002332111 Dale str.Chicago60333We want to get next output:

customers.csv

emailnameagestreetcityzipBart Robbinson3220 Chestnut TerraceNew York07003Kale D15123 MansonLA90023111 Dale str.Chicago60333```
use Maketok\DataMigration\MapInterface;
use Maketok\DataMigration\QueueWorkflow;
use Maketok\DataMigration\Unit\SimpleBag;
use Maketok\DataMigration\Unit\Type\Unit;
use Maketok\DataMigration\Storage\Db\ResourceHelperInterface;
use Maketok\DataMigration\Action\Type\AssembleInput;
use Maketok\DataMigration\Action\Type\Load;
use Maketok\DataMigration\Action\Type\Move;
use Maketok\DataMigration\Input\Shaper\Processor\Nulls;

$customerUnit = new Unit('customers');
$customerUnit->setTable('customer');
$customerUnit->setMapping([
    'id' => 'map.customer_id'
    'email' => 'mail.email',
    'age' => 'map.age',
]);
$customerUnit->setIsEntityCondition("trim(map.email) is not empty");
$customerUnit->addContribution("map.offsetSet(
    'customer_id',
    (isset(hashmaps['email-id'][trim(map.email)]) ?
        hashmaps['email-id'][trim(map.email)] :
        map.frozenIncr(
            'new_customer_id',
            resource.getLastIncrementId('customer')
        )
    )
)");
$customerUnit->setReversedConnection([
  'customer_id' => 'id',
]);
$customerUnit->setReversedMapping([
    'email' => 'map.email',
    'age' => 'map.age',
]);

$customerDataUnit = new Unit('customer_data');
$customerDataUnit->setTable('customer_data');
$customerDataUnit->setMapping([
    'id' => 'map.incr('customer_data_id', resource.getLastIncrementId('customer_data'))'
    'parent_id' => 'map.customer_id'
    'firstname' => 'map.firstname',
    'lastname' => 'map.lastname',
]);
$customerDataUnit->addContribution("map.offsetSet(
    'complexName',
    explode(' ', map.name)
)");
$customerDataUnit->addContribution("map.offsetSet(
    'firstname',
    (count(map.complexName) >= 2 && isset(map.complexName[0]) ? map.complexName[0] : map.name)
)");
$customerDataUnit->addContribution("map.offsetSet(
    'lastname',
    (count(map.complexName) >= 2 && isset(map.complexName[1]) ? map.complexName[1] : '')
)");
$customerDataUnit->setReversedConnection([
    'customer_id' => 'parent_id',
]);
$customerDataUnit->setReversedMapping([
    'name' => 'map.firstname ~ " " ~ map.lastname',
]);
$customerUnit->addSibling($customerDataUnit);

$addressUnit = new Unit('addresses');
$addressUnit->setTable('address');
$addressUnit->setMapping([
    'id' => 'map.incr('address_id', resource.getLastIncrementId('address'))'
    'customer_id' => 'map.customer_id',
    'street' => 'map.street',
    'city' => 'map.city',
    'zip' => 'map.zip',
]);
$addressUnit->setReversedConnection([
    'customer_id' => 'customer_id',
]);
$addressUnit->setReversedMapping([
    'street' => 'map.street',
    'city' => 'map.city',
    'zip' => 'map.zip',
]);
$addressUnit->setParent($customerUnit);
$bag = new SimpleBag();
$bag->addSet([$customerUnit, $customerDataUnit, $addressUnit]);

$input = new Csv($fname, 'w', new Nulls($bag, new ArrayMap(), $this->getLanguageAdapter()));

$result = new Result();
$workflow = new QueueWorkflow($this->config, $result);
$workflow->add(new ReverseMove($bag, $config, $resource));
$workflow->add(new Dump($bag, $config, $resource));
$workflow->add(new AssembleInput($bag, $config, $languageAdapter, $input, new ArrayMap()));
$workflow->execute();
```

FAQ
---

[](#faq)

###  Health Score

32

—

LowBetter than 69% of packages

Maintenance15

Infrequent updates — may be unmaintained

Popularity31

Limited adoption so far

Community13

Small or concentrated contributor base

Maturity56

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 97.8% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~71 days

Recently: every ~278 days

Total

17

Last Release

2857d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/bc8f58e80607cbca09398f2fedfa3e3b4979dbba6108998708bee9237b129560?d=identicon)[Oleg](/maintainers/Oleg)

---

Top Contributors

[![SlayerBirden](https://avatars.githubusercontent.com/u/1726160?v=4)](https://github.com/SlayerBirden "SlayerBirden (135 commits)")[![Adfir](https://avatars.githubusercontent.com/u/12199430?v=4)](https://github.com/Adfir "Adfir (2 commits)")[![bsoroka](https://avatars.githubusercontent.com/u/14123911?v=4)](https://github.com/bsoroka "bsoroka (1 commits)")

---

Tags

migrationdataexportimportgenerationmaketokdatamigration

###  Code Quality

TestsPHPUnit

### Embed Badge

![Health badge](/badges/maketok-datamigration/health.svg)

```
[![Health](https://phpackages.com/badges/maketok-datamigration/health.svg)](https://phpackages.com/packages/maketok-datamigration)
```

###  Alternatives

[ddeboer/data-import

Import data from, and export data to, a range of file formats and media

5594.3M9](/packages/ddeboer-data-import)[rcsofttech/audit-trail-bundle

Enterprise-grade, high-performance Symfony audit trail bundle. Automatically track Doctrine entity changes with split-phase architecture, multiple transports (HTTP, Queue, Doctrine), and sensitive data masking.

1189.8k](/packages/rcsofttech-audit-trail-bundle)[portphp/portphp

Data import/export workflow

2873.0M25](/packages/portphp-portphp)[contao-community-alliance/dc-general

Universal data container for Contao

1680.8k92](/packages/contao-community-alliance-dc-general)[open-dxp/opendxp

Content & Product Management Framework (CMS/PIM)

9421.6k61](/packages/open-dxp-opendxp)[code-rhapsodie/dataflow-bundle

Data processing framework inspired by PortPHP

1854.9k5](/packages/code-rhapsodie-dataflow-bundle)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)