PHPackages zappzerapp/laravel-ingest - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. [Queues & Workers](/categories/queues) 4. / 5. zappzerapp/laravel-ingest ActiveLibrary[Queues & Workers](/categories/queues) zappzerapp/laravel-ingest ========================= A robust, configuration-driven ETL and data import framework for Laravel. Handles CSV/Excel streaming, queues, validation, and relationships. v0.5.2(3mo ago)852064[2 PRs](https://github.com/zappzerapp/laravel-ingest/pulls)MITPHPPHP ^8.3CI passing Since Dec 3Pushed 2mo agoCompare [ Source](https://github.com/zappzerapp/laravel-ingest)[ Packagist](https://packagist.org/packages/zappzerapp/laravel-ingest)[ Docs](https://github.com/zappzerapp/laravel-ingest)[ Fund](https://www.buymeacoffee.com/wbbl)[ Patreon](https://www.patreon.com/zappzerapp)[ RSS](/packages/zappzerapp-laravel-ingest/feed)WikiDiscussions main Synced 1mo ago READMEChangelog (10)Dependencies (17)Versions (12)Used By (0) [![Laravel Ingest Banner](https://raw.githubusercontent.com/zappzerapp/laravel-ingest/refs/heads/main/.github/header.png)](https://raw.githubusercontent.com/zappzerapp/laravel-ingest/refs/heads/main/.github/header.png) [![Latest Version](https://camo.githubusercontent.com/d8ec9f1e798034890aea9684816198bca03a42870fff15c4d8321d07eb45112a/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f7a6170707a65726170702f6c61726176656c2d696e676573742e7376673f7374796c653d666f722d7468652d6261646765)](https://packagist.org/packages/zappzerapp/laravel-ingest) [![Total Downloads](https://camo.githubusercontent.com/f62ae807c7a0cc30764dd011e7edfb03fe73480f705cb9656991efee471f735a/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f64742f7a6170707a65726170702f6c61726176656c2d696e676573742e7376673f7374796c653d666f722d7468652d6261646765)](https://packagist.org/packages/zappzerapp/laravel-ingest) [![Build Status](https://camo.githubusercontent.com/bba46eb6098d2feaab099a3e88fc30f3d771705620bef7fcb99e83996e5f4c58/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f616374696f6e732f776f726b666c6f772f7374617475732f7a6170707a65726170702f6c61726176656c2d696e676573742f6d61696e2d706970656c696e652e796d6c3f7374796c653d666f722d7468652d6261646765)](https://github.com/zappzerapp/laravel-ingest/actions) [![Documentation](https://camo.githubusercontent.com/fbf3a5532b60106aa665b84500453063c6732575c2961694c6dcc5c6bd2d8949/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f646f63732d6f6e6c696e652d626c75652e7376673f7374796c653d666f722d7468652d6261646765)](https://zappzerapp.github.io/laravel-ingest/) [![License](https://camo.githubusercontent.com/b1baaa23ebe5e1abc465cf1ab88abd02e04c616cf59083904d68f812e92be5e3/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f6c2f7a6170707a65726170702f6c61726176656c2d696e676573742e7376673f7374796c653d666f722d7468652d6261646765)](LICENSE) --- **Stop writing spaghetti code for imports.** **Laravel Ingest** is a robust, configuration-driven ETL (Extract, Transform, Load) framework for Laravel. It replaces fragile, procedural import scripts with elegant, declarative configuration classes. Whether you are importing **100 rows** or **10 million**, Laravel Ingest handles the heavy lifting: streaming, chunking, queueing, validation, relationships, and error reporting. ⚑ Why use this? --------------- [](#-why-use-this) Most import implementations suffer from the same issues: memory leaks, timeouts, lack of validation, and messy controllers. Laravel Ingest solves this by treating imports as a first-class citizen: - **♾️ Infinite Scalability:** Uses Generators and Queues to process files of *any* size with flat memory usage. - **πŸ“ Declarative Syntax:** Define *what* to import, not *how* to loop over it. - **πŸ§ͺ Dry Runs:** Simulate imports to find validation errors without touching the database. - **πŸ”— Auto-Relations:** Automatically resolves `BelongsTo` and `BelongsToMany` relationships (e.g., finding IDs by names). - **πŸ›‘οΈ Robust Error Handling:** Tracks every failed row and allows you to download a CSV of *only* the failures to fix and retry. - **πŸ”Œ API & CLI Ready:** Comes with auto-generated API endpoints and Artisan commands. --- πŸ“š Documentation --------------- [](#-documentation) Full documentation is available at **[zappzerapp.github.io/laravel-ingest](https://zappzerapp.github.io/laravel-ingest/)**. --- πŸš€ Quick Start ------------- [](#-quick-start) ### 1. Installation [](#1-installation) ``` composer require zappzerapp/laravel-ingest # Publish config & migrations php artisan vendor:publish --provider="LaravelIngest\IngestServiceProvider" # Create tables php artisan migrate ``` ### 2. Define an Importer [](#2-define-an-importer) Create a class implementing `IngestDefinition`. This is the only code you need to write. ``` namespace App\Ingest; use App\Models\User; use LaravelIngest\Contracts\IngestDefinition; use LaravelIngest\IngestConfig; use LaravelIngest\Enums\SourceType; use LaravelIngest\Enums\DuplicateStrategy; class UserImporter implements IngestDefinition { public function getConfig(): IngestConfig { return IngestConfig::for(User::class) ->fromSource(SourceType::UPLOAD) ->keyedBy('email') // Identify records by email ->onDuplicate(DuplicateStrategy::UPDATE) // Update if exists // Map CSV columns to DB attributes ->map('Full Name', 'name') ->map(['E-Mail', 'Email Address'], 'email') // Supports aliases // Handle Relationships automatically ->relate('Role', 'role', Role::class, 'slug', createIfMissing: true) // Validate rows before processing ->validate([ 'email' => 'required|email', 'Full Name' => 'required|string|min:3' ]); } } ``` ### 3. Register it [](#3-register-it) In `App\Providers\AppServiceProvider`: ``` use LaravelIngest\IngestServiceProvider; public function register(): void { $this->app->tag([UserImporter::class], IngestServiceProvider::INGEST_DEFINITION_TAG); } ``` ### 4. Run it! [](#4-run-it) You can now trigger the import via CLI or API. **Via Artisan (Backend / Cron):** ``` php artisan ingest:run user-importer --file=users.csv ``` **Via API (Frontend / Upload):** ``` curl -X POST \ -H "Authorization: Bearer " \ -F "file=@users.csv" \ https://your-app.com/api/v1/ingest/upload/user-importer ``` --- πŸ’‘ Demo Project -------------- [](#-demo-project) Want to see Laravel Ingest in action? Check out our **[Laravel Ingest Demo](https://github.com/zappzerapp/Laravel-Ingest-Demo)** repository for a complete working example. ``` # Clone the demo git clone https://github.com/zappzerapp/Laravel-Ingest-Demo.git cd Laravel-Ingest-Demo # Start and benchmark docker compose up -d docker compose exec app php artisan benchmark:ingest ``` --- πŸ›  Features in Depth ------------------- [](#-features-in-depth) ### Monitoring & Management [](#monitoring--management) Ingest runs happen in the background. You can monitor and manage them easily: CommandDescription`ingest:list`Show all registered importers.`ingest:status {id}`Show progress bar, stats, and errors for a run.`ingest:cancel {id}`Stop a running import gracefully.`ingest:retry {id}`Create a **new run** containing only the rows that failed previously.### API Endpoints [](#api-endpoints) The package automatically exposes endpoints for building UI integrations (e.g., React/Vue progress bars). - `GET /api/v1/ingest` - List recent runs. - `GET /api/v1/ingest/{id}` - Get status and progress. - `GET /api/v1/ingest/{id}/errors/summary` - Get aggregated error stats (e.g., "50x Email invalid"). - `GET /api/v1/ingest/{id}/failed-rows/download` - Download a CSV of failed rows to fix & re-upload. ### Events [](#events) Hook into the lifecycle to send notifications (e.g., Slack) or trigger downstream logic. - `LaravelIngest\Events\IngestRunStarted` - `LaravelIngest\Events\ChunkProcessed` - `LaravelIngest\Events\RowProcessed` - `LaravelIngest\Events\IngestRunCompleted` - `LaravelIngest\Events\IngestRunFailed` ### Pruning [](#pruning) To keep your database clean, logs are prunable. Add this to your scheduler: ``` $schedule->command('model:prune', [ '--model' => [LaravelIngest\Models\IngestRow::class], ])->daily(); ``` --- 🧩 Configuration Reference ------------------------- [](#-configuration-reference) The `IngestConfig` fluent API handles complex scenarios with ease. ``` IngestConfig::for(Product::class) // Sources: UPLOAD, FILESYSTEM, URL, FTP, SFTP ->fromSource(SourceType::FTP, ['disk' => 'erp', 'path' => 'daily.csv']) // Performance ->setChunkSize(1000) ->atomic() // Wrap chunks in transactions // Logic ->keyedBy('sku') ->onDuplicate(DuplicateStrategy::UPDATE_IF_NEWER) ->compareTimestamp('last_modified_at', 'updated_at') // Transformation ->mapAndTransform('price_cents', 'price', fn($val) => $val / 100) ->resolveModelUsing(fn($row) => $row['type'] === 'digital' ? DigitalProduct::class : Product::class); ``` See the [Documentation](https://zappzerapp.github.io/laravel-ingest/) for all available methods. --- πŸ§ͺ Testing --------- [](#-testing) We provide a Docker-based test environment to ensure consistency. ``` # Start Docker composer docker:up # Run Tests composer docker:test # Check Coverage composer docker:coverage ``` --- 🀝 Contributing -------------- [](#-contributing) We welcome contributions! Please see [CONTRIBUTING.md](.github/CONTRIBUTING.md) for details. πŸ“„ License --------- [](#-license) The MIT License (MIT). Please see [License File](LICENSE) for more information. ### Health Score 45 β€” FairBetter than 92% of packages Maintenance83 Actively maintained with recent releases Popularity30 Limited adoption so far Community8 Small or concentrated contributor base Maturity46 Maturing project, gaining track record Bus Factor1 Top contributor holds 100% of commits β€” single point of failure How is this calculated?**Maintenance (25%)** β€” Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window. **Popularity (30%)** β€” Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores. **Community (15%)** β€” Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement. **Maturity (30%)** β€” Project age, version count, PHP version support, and release stability. ### Release Activity Cadence Every ~7 days Total 10 Last Release 96d ago ### Community Maintainers ![](https://www.gravatar.com/avatar/c613761fee7765238fb31a2c359c380a3237b87042fd22ae76d10bc89b2d967c?d=identicon)[w0bble](/maintainers/w0bble) --- Top Contributors [![zappzerapp](https://avatars.githubusercontent.com/u/9962724?v=4)](https://github.com/zappzerapp "zappzerapp (59 commits)") --- Tags csvdata-ingestionetlexcelimportimporterlaravelqueuestreaminglaravelstreamingexcelcsvqueueimportimporteretldata-ingestion ### Code Quality TestsPest Static AnalysisPHPStan Code StyleLaravel Pint ### Embed Badge ![Health badge](/badges/zappzerapp-laravel-ingest/health.svg) ``` [![Health](https://phpackages.com/badges/zappzerapp-laravel-ingest/health.svg)](https://phpackages.com/packages/zappzerapp-laravel-ingest) ``` ### Alternatives [roots/acorn Framework for Roots WordPress projects built with Laravel components. 9682.1M97](/packages/roots-acorn)[laravel/scout Laravel Scout provides a driver based solution to searching your Eloquent models. 1.7k49.4M479](/packages/laravel-scout)[laravel/pulse Laravel Pulse is a real-time application performance monitoring tool and dashboard for your Laravel application. 1.7k12.1M99](/packages/laravel-pulse)[illuminate/queue The Illuminate Queue package. 20331.4M1.2k](/packages/illuminate-queue)[aedart/athenaeum Athenaeum is a mono repository; a collection of various PHP packages 255.2k](/packages/aedart-athenaeum)[alajusticia/laravel-logins Session management in Laravel apps, user notifications on new access, support for multiple separate remember tokens, IP geolocation, User-Agent parser 2011.0k](/packages/alajusticia-laravel-logins) PHPackages Β© 2026 [Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)