PHPackages                             yoosuf/laravel-dataflow - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Database & ORM](/categories/database)
4. /
5. yoosuf/laravel-dataflow

ActiveLibrary[Database & ORM](/categories/database)

yoosuf/laravel-dataflow
=======================

Streaming-first Laravel data pipeline package for filter/search/sort/import/export workflows at any scale with queue-native execution.

0.9.3(1mo ago)12↓66.7%MITPHPPHP ^8.3CI failing

Since Jul 18Pushed 1mo agoCompare

[ Source](https://github.com/yoosuf/laravel-dataflow)[ Packagist](https://packagist.org/packages/yoosuf/laravel-dataflow)[ RSS](/packages/yoosuf-laravel-dataflow/feed)WikiDiscussions main Synced 2w ago

READMEChangelogDependencies (19)Versions (4)Used By (0)

yoosuf/laravel-dataflow
=======================

[](#yoosuflaravel-dataflow)

Build production-grade data pipelines in Laravel without custom ETL glue.

`laravel-dataflow` is a streaming-first package for filtering, searching, sorting, importing, and exporting large datasets with queue-ready execution and predictable memory use.

[![Latest Version on Packagist](https://camo.githubusercontent.com/78f709c5ba087804b2847b48429b1b48ff65c765fac4f0e7122abb9a4aa86ee7/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f796f6f7375662f6c61726176656c2d64617461666c6f772e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/yoosuf/laravel-dataflow)[![Total Downloads](https://camo.githubusercontent.com/6acf3f1681f6041239fd571564aba4fa766c37e32f54c7e5a28d3afdb6801bff/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f64742f796f6f7375662f6c61726176656c2d64617461666c6f772e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/yoosuf/laravel-dataflow)[![Tests](https://camo.githubusercontent.com/b0e3fda7d34c4fc49d0c34459c2c83286a4fbd3d80100b81d49ac99435ae6127/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f616374696f6e732f776f726b666c6f772f7374617475732f796f6f7375662f6c61726176656c2d64617461666c6f772f63692e796d6c3f6272616e63683d6d61696e266c6162656c3d7465737473267374796c653d666c61742d737175617265)](https://github.com/yoosuf/laravel-dataflow/actions)[![License](https://camo.githubusercontent.com/dd58b3ba6a7f89c2ef51df34e11b040056d05f9ebad436604350eb9be054af23/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f6c2f796f6f7375662f6c61726176656c2d64617461666c6f772e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/yoosuf/laravel-dataflow)

Why Laravel Teams Use It
------------------------

[](#why-laravel-teams-use-it)

- Ship CSV/XLSX/JSON/NDJSON/PDF/Parquet flows from one fluent API.
- Keep memory stable with stream-based processing and chunk coordination.
- Run sync for fast tasks, queue for heavy jobs, with progress snapshots.
- Keep query safety with allowlisted filters, search, and sorting.
- Integrate with existing Eloquent builders and complex query constraints.

Quick Pitch
-----------

[](#quick-pitch)

If your app needs admin exports, BI feeds, audit extracts, or bulk imports, this package gives you a single Laravel-native pipeline instead of ad-hoc jobs and one-off scripts.

Open Source Project Docs
------------------------

[](#open-source-project-docs)

- Contributing guide: CONTRIBUTING.md
- Code of Conduct: CODE\_OF\_CONDUCT.md
- Security policy: SECURITY.md
- Support guide: SUPPORT.md
- Changelog: CHANGELOG.md
- Upgrade notes: UPGRADE.md

30-Second Demo
--------------

[](#30-second-demo)

 ```
flowchart LR
  A[Developer runs queued export command in terminal] --> B[DataFlow for User with filter search and CSV export]
  B --> C[Queue job dispatched]
  C --> D[Queue worker picks up chunk jobs]
  D --> E[Chunk processing and merge]
  E --> F[CSV written to storage]
  F --> G[Downloadable output file for user]
```

      Loading Copy-Paste Recipes
------------------

[](#copy-paste-recipes)

### 1) Admin Panel Export (Queued CSV)

[](#1-admin-panel-export-queued-csv)

```
use Yoosuf\LaravelDataFlow\DataFlow;
use App\Models\User;

$runId = DataFlow::for(User::class)
  ->allowedFilters(['status', 'country'])
  ->allowedSearch(['name', 'email'])
  ->allowedSorts(['created_at'])
  ->filter(['status' => 'active'])
  ->search('gmail.com')
  ->sort('-created_at')
  ->export('csv')
  ->to('exports', 'active-users.csv')
  ->queue();
```

### 2) BI Feed (Nightly NDJSON)

[](#2-bi-feed-nightly-ndjson)

```
use Yoosuf\LaravelDataFlow\DataFlow;
use App\Models\Order;

DataFlow::forQuery(
  Order::query()->whereDate('created_at', now()->subDay()->toDateString())
)
  ->export('ndjson')
  ->to('feeds', 'orders-nightly.ndjson')
  ->sync();
```

### 3) Bulk Import (Chunked)

[](#3-bulk-import-chunked)

```
use Yoosuf\LaravelDataFlow\DataFlow;
use App\Models\Product;

DataFlow::for(Product::class)
  ->import('csv')
  ->from('imports', 'products.csv')
  ->map([
    'sku' => 'sku',
    'name' => 'name',
    'price' => 'price_cents',
  ])
  ->upsertBy(['sku'])
  ->queue();
```

### Import Examples

[](#import-examples)

#### A) CSV Upsert By Natural Key

[](#a-csv-upsert-by-natural-key)

```
use Yoosuf\LaravelDataFlow\DataFlow;
use App\Models\Customer;

DataFlow::for(Customer::class)
  ->import('csv')
  ->from('imports', 'customers.csv')
  ->map([
    'email' => 'email',
    'name' => 'name',
    'phone' => 'phone',
    'country' => 'country_code',
  ])
  ->upsertBy(['email'])
  ->queue();
```

#### B) JSON Import With Column Remap

[](#b-json-import-with-column-remap)

```
use Yoosuf\LaravelDataFlow\DataFlow;
use App\Models\Product;

DataFlow::for(Product::class)
  ->import('json')
  ->from('imports', 'catalog.json')
  ->map([
    'sku' => 'sku',
    'title' => 'name',
    'price' => 'price_cents',
    'is_active' => 'status',
  ])
  ->upsertBy(['sku'])
  ->queue();
```

#### C) NDJSON Sync Import For Small Batches

[](#c-ndjson-sync-import-for-small-batches)

```
use Yoosuf\LaravelDataFlow\DataFlow;
use App\Models\Lead;

DataFlow::for(Lead::class)
  ->import('ndjson')
  ->from('imports', 'leads.ndjson')
  ->map([
    'external_id' => 'external_id',
    'email' => 'email',
    'source' => 'source',
  ])
  ->upsertBy(['external_id'])
  ->sync();
```

Copy For GitHub Repo Settings
-----------------------------

[](#copy-for-github-repo-settings)

Use this as your repository description:

> Streaming-first Laravel package for filtering, search, sorting, import, and export at any scale, with queue-native execution and low memory usage.

Use these GitHub topics:

`laravel`, `laravel-package`, `eloquent`, `data-pipeline`, `dataflow`, `import`, `export`, `csv`, `xlsx`, `ndjson`, `parquet`, `etl`, `query-builder`, `queue`, `large-datasets`

Installation (Path Repository)
------------------------------

[](#installation-path-repository)

Add to your root composer repositories:

```
{
  "type": "path",
  "url": "packages/yoosuf/laravel-dataflow",
  "options": { "symlink": true }
}
```

Then require:

```
composer require yoosuf/laravel-dataflow:*
```

Publish config:

```
php artisan vendor:publish --tag=dataflow-config
```

Configuration
-------------

[](#configuration)

Default configuration is in `config/dataflow.php` after publishing.

### Exporter Fallback Support (Opt-In)

[](#exporter-fallback-support-opt-in)

For production resilience, exporter resolution supports optional fallback formats when a requested exporter is not registered or unavailable.

- `dataflow.exports.fallback.enabled` (default: `false`)
- `dataflow.exports.fallback.default_format` (default: `csv`)
- `dataflow.exports.fallback.format_map` (per-format overrides, e.g. `xlsx => csv`)

Example:

```
'exports' => [
  'fallback' => [
    'enabled' => true,
    'default_format' => 'csv',
    'format_map' => [
      'xlsx' => 'csv',
      'pdf' => 'csv',
    ],
  ],
],
```

When disabled, the package remains strict and throws an exception for unsupported formats.

Development
-----------

[](#development)

```
composer install
composer lint
composer analyse
composer test
```

Benchmark Results (Docker)
--------------------------

[](#benchmark-results-docker)

All enterprise join export benchmark results are consolidated below.

Common shape per profile:

- orders per user: `6`
- items per order: `3`
- workload: `users -> orders -> order_items` join + aggregate export

ProfileEngineUsersOrdersOrder ItemsResult RowsBatch SizeSchema (s)Seed (s)Export (s)Total (s)Rows/sPeak Mem (MB)1MMySQL 8.41,000,0006,000,00018,000,000517,92650,0000.5578643.963770.5498715.07127,341.2945.911MPostgreSQL 161,000,0006,000,00018,000,000517,92650,0000.02721,039.839224.67541,064.541920,989.562.001MMariaDB 111,000,0006,000,00018,000,000517,92650,00016.07341,664.5361162.07781,842.68733,195.5445.91100kOracle Free 23c100,000600,0001,800,00051,32610,0000.2354206.17102.7220209.128418,856.142.00100kSQL Server 2022100,000600,0001,800,00051,32610,0000.0748335.98292.6955338.753219,041.182.00Throughput ratios (`rows_per_second`) by profile:

- 1M profile: PostgreSQL / MySQL `2.86x`, PostgreSQL / MariaDB `6.57x`, MySQL / MariaDB `2.30x`
- 100k profile: SQL Server / Oracle `1.01x` (Oracle / SQL Server `0.99x`)

Notes:

- Measurements are single-run Docker comparisons.
- Absolute timings vary by host resources; compare engines primarily within the same profile.
- Oracle runs used Docker Oracle Free with runtime-installed `oci8` + `pdo_oci` in the PHP benchmark container.

Complex Query Support
---------------------

[](#complex-query-support)

Use `DataFlow::forQuery($builder)` when your export/import source is a prebuilt Eloquent query with scopes, nested conditions, relation constraints, or subqueries.

```
use Yoosuf\LaravelDataFlow\DataFlow;
use App\Models\User;

$runId = DataFlow::forQuery(
  User::query()->where('status', 'active')->whereHas('posts')
)
  ->export('csv')
  ->to('exports', 'active-users.csv')
  ->sync();
```

Builder-based query sources are supported in `sync()` mode. Queued `queue()` runs are supported via an internal serialized query specification that reconstructs the query in worker jobs.

Roadmap
-------

[](#roadmap)

See `docs/PHASE_PLAN.md` for the phase-by-phase task breakdown.

License
-------

[](#license)

MIT

###  Health Score

37

—

LowBetter than 81% of packages

Maintenance90

Actively maintained with recent releases

Popularity4

Limited adoption so far

Community6

Small or concentrated contributor base

Maturity41

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~0 days

Total

3

Last Release

45d ago

### Community

Maintainers

![](https://avatars.githubusercontent.com/u/64164?v=4)[Yoosuf Mo](/maintainers/yoosuf)[@yoosuf](https://github.com/yoosuf)

---

Top Contributors

[![yoosuf](https://avatars.githubusercontent.com/u/64164?v=4)](https://github.com/yoosuf "yoosuf (8 commits)")

---

Tags

csvdata-pipelinedatafloweloquentetlexportimportlaravellaravel-packagelarge-datasetsndjsonparquetquery-builderqueuexlsxsearchlaravelexportstreamingeloquentxlsxcsvlaravel-packageNDJSONqueuereportingfilterimportsortquery builderetlparquetdataflowlarge-datasets

###  Code Quality

TestsPest

Static AnalysisPHPStan, Rector

Code StyleLaravel Pint

Type Coverage Yes

### Embed Badge

![Health badge](/badges/yoosuf-laravel-dataflow/health.svg)

```
[![Health](https://phpackages.com/badges/yoosuf-laravel-dataflow/health.svg)](https://phpackages.com/packages/yoosuf-laravel-dataflow)
```

###  Alternatives

[laravel/scout

Laravel Scout provides a driver based solution to searching your Eloquent models.

1.7k59.5M721](/packages/laravel-scout)[psalm/plugin-laravel

Psalm plugin for Laravel

3365.5M359](/packages/psalm-plugin-laravel)[laravel/pulse

Laravel Pulse is a real-time application performance monitoring tool and dashboard for your Laravel application.

1.7k17.6M172](/packages/laravel-pulse)[laravel/ai

The official AI SDK for Laravel.

1.1k6.4M363](/packages/laravel-ai)[pressbooks/pressbooks

Pressbooks is an open source book publishing tool built on a WordPress multisite platform. Pressbooks outputs books in multiple formats, including PDF, EPUB, web, and a variety of XML flavours, using a theming/templating system, driven by CSS.

45945.2k1](/packages/pressbooks-pressbooks)[illuminate/notifications

The Illuminate Notifications package.

513.2M1.3k](/packages/illuminate-notifications)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)