PHPackages turbostream/export-engine - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. [Database & ORM](/categories/database) 4. / 5. turbostream/export-engine ActiveLaravel-package[Database & ORM](/categories/database) turbostream/export-engine ========================= High-performance Laravel package for exporting 300M+ records with 9000+ page PDF support. Features include subtotals, grand totals, colspan, rowspan. Supports CSV, Excel, PDF, DOCX, and SQL formats. 1.0.0(2w ago)05↑2900%MITPHPPHP ^8.1 Since Mar 19Pushed todayCompare [ Source](https://github.com/moshiur1412/export-engine)[ Packagist](https://packagist.org/packages/turbostream/export-engine)[ Docs](https://github.com/moshiur1412/export-engine)[ RSS](/packages/turbostream-export-engine/feed)WikiDiscussions main Synced today READMEChangelogDependencies (10)Versions (1)Used By (0) TurboStream Export Engine ========================= [](#turbostream-export-engine) A high-performance Laravel package for exporting 300M+ records with professional PDF support (9000+ pages without breaking). Supports CSV, XLSX, PDF, DOCX, and SQL formats. [![Latest Version](https://camo.githubusercontent.com/10b7aa8d2faf8456578c964840ec578e254c5bd3de522743db43eb078bc32ec4/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f747572626f73747265616d2f6578706f72742d656e67696e652e737667)](https://packagist.org/packages/turbostream/export-engine)[![Total Downloads](https://camo.githubusercontent.com/45583490172d1e588dc65eee6e463131b21e748e485fb9e6015e0151478b410c/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f64742f747572626f73747265616d2f6578706f72742d656e67696e652e737667)](https://packagist.org/packages/turbostream/export-engine)[![License](https://camo.githubusercontent.com/0fb9c04f7e781d26f09eb3c0ebc3fb3a52c99633046e518be47dfcc9a53b8134/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f6c2f747572626f73747265616d2f6578706f72742d656e67696e652e737667)](https://packagist.org/packages/turbostream/export-engine)[![PHP Version](https://camo.githubusercontent.com/c0ae56290e0ad90ae226e4aa78eb120f716aeee0d8da9663dc00071a1aa76249/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f7068702d762f747572626f73747265616d2f6578706f72742d656e67696e652e737667)](https://packagist.org/packages/turbostream/export-engine) Overview -------- [](#overview) TurboStream Export Engine is designed for Laravel applications that need to export massive datasets (300M+ records) efficiently without memory issues. It uses chunked queries to process data in batches and leverages Laravel Queues with Redis for background processing. ### Key Features [](#key-features) - **300M+ Records Support**: Optimized chunk sizes for massive datasets (10K-20K per chunk) - **9000+ Page PDFs**: Professional PDF generation without breaking - **Memory Efficient**: Uses `cursor()` instead of `chunk()` to stream records without loading all into memory - **5 Export Formats**: CSV, XLSX, PDF, DOCX, SQL - **PDF Full Features**: Subtotals, grand totals, colspan, rowspan, custom headers - **Async Processing**: Background jobs via Laravel Queues with Redis - **Real-time Progress**: Track export progress via Redis cache - **Filter Names in Filename**: Downloaded files include applied filters in filename - **Multiple Queue Drivers**: Redis (recommended), Database, or Sync - **Auto Chunk Sizing**: Automatically adjusts chunk size based on data volume - **Laravel Native**: Integrates seamlessly with Laravel 9, 10, 11, 12, and 13 Requirements ------------ [](#requirements) - PHP 8.1+ - Laravel 9.0, 10.0, 11.0, 12.0, or 13.0 - Redis (recommended) or Database queue driver - ext-json, ext-mbstring ### Dependencies (All Included) [](#dependencies-all-included) FormatPackageDescriptionCSVLeague CSVBest for 300M+ recordsXLSXPhpSpreadsheetReports & Excel filesPDFTCPDF9000+ page PDFs with all featuresDOCXPhpWordWord documentsSQLBuilt-inDatabase backup/migrationInstallation ------------ [](#installation) ### 1. Install via Composer [](#1-install-via-composer) ``` composer require turbostream/export-engine ``` ### 2. Publish Configuration [](#2-publish-configuration) ``` php artisan vendor:publish --tag=turbo-export ``` ### 3. Configure Environment [](#3-configure-environment) Add Redis configuration to your `.env` file: ``` CACHE_DRIVER=redis QUEUE_CONNECTION=redis REDIS_HOST=127.0.0.1 REDIS_PORT=6379 ``` ### 4. Run Migrations (optional - for database queue) [](#4-run-migrations-optional---for-database-queue) ``` php artisan migrate ``` Quick Start ----------- [](#quick-start) ### Basic Usage [](#basic-usage) ``` use TurboStreamExport\Facades\ExportFacade; use App\Models\User; $exportId = ExportFacade::createExport([ 'model' => User::class, 'columns' => ['id', 'name', 'email', 'created_at'], 'format' => 'csv', 'filename' => 'users_export', ]); ``` ### Using Filters [](#using-filters) ``` $exportId = ExportFacade::createExport([ 'model' => \App\Models\User::class, 'columns' => ['id', 'name', 'email', 'status'], 'format' => 'csv', 'filters' => [ ['status', '=', 'active'], ['created_at', '>=', '2026-01-01'], ], ]); ``` **Filename with filters:**When filters are applied, the downloaded file includes filter details: ``` users_export_filtered_status=_active_created_at=_>=_2026-01-01.csv ``` ### Check Progress [](#check-progress) ``` $progress = ExportFacade::getProgress($exportId); echo $progress['progress']; // 75 echo $progress['status']; // 'processing' or 'completed' echo $progress['filters']; // Array of applied filters echo $progress['filter_summary']; // 'status=_active' ``` ### Download File [](#download-file) ``` $downloadUrl = ExportFacade::getDownloadUrl($exportId); // Returns signed URL valid for 1 hour ``` Export Formats -------------- [](#export-formats) This package supports **5 export formats** for different use cases: ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ 5 EXPORT FORMATS SUPPORTED │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ │ │ CSV │ │ XLSX │ │ PDF │ │ DOCX │ │ SQL │ │ │ ├──────────┤ ├──────────┤ ├──────────┤ ├──────────┤ ├──────────┤ │ │ │ 📄 │ │ 📊 │ │ 📑 │ │ 📝 │ │ 🗄️ │ │ │ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │ │ │ │ │ │ │ │ │ ▼ ▼ ▼ ▼ ▼ │ │ Streaming Memory- TCPDF Memory- Streaming │ │ (300M+ OK) based (Full Features)based (300M+ OK) │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ ``` ### CSV (Recommended for Large Data) [](#csv-recommended-for-large-data) ``` ┌────────────────────────────────────────────────────────┐ │ Format: comma-separated-values.csv │ │ Speed: ⚡⚡⚡⚡⚡ (FASTEST) │ │ Memory: ~1GB (constant - streaming) │ │ Best for: 300M+ records, data backup, API export │ │ Install: ✅ Included (League CSV) │ └────────────────────────────────────────────────────────┘ ``` ### XLSX (Excel) [](#xlsx-excel) ``` ┌────────────────────────────────────────────────────────┐ │ Format: spreadsheet.xlsx │ │ Speed: ⚡⚡⚡ (moderate) │ │ Memory: ~2GB (grows with records) │ │ Best for: Reports, sharing, printing │ │ Install: ✅ Included (PhpSpreadsheet) │ └────────────────────────────────────────────────────────┘ ``` ### PDF (Professional Documents) [](#pdf-professional-documents) ``` ┌────────────────────────────────────────────────────────┐ │ Format: document.pdf │ │ Speed: ⚡⚡ (slower) │ │ Memory: ~2-4GB (grows with records) │ │ Best for: Invoices, reports, 9000+ page documents │ │ │ │ Features (all included): │ │ ✅ Subtotals per group ✅ Grand Total │ │ ✅ Colspan & Rowspan ✅ Custom Headers/Footers │ │ ✅ 9000+ pages ✅ Professional layouts │ │ Install: ✅ TCPDF included (no extra package) │ └────────────────────────────────────────────────────────┘ ``` ### DOCX (Word) [](#docx-word) ``` ┌────────────────────────────────────────────────────────┐ │ Format: document.docx │ │ Speed: ⚡⚡ (slower) │ │ Memory: ~2GB (grows with records) │ │ Best for: Documentation, letters, contracts │ │ Install: ✅ Included (PhpWord) │ └────────────────────────────────────────────────────────┘ ``` ### SQL (Database) [](#sql-database) ``` ┌────────────────────────────────────────────────────────┐ │ Format: dump.sql │ │ Speed: ⚡⚡⚡⚡⚡ (fast) │ │ Memory: ~1GB (constant - streaming) │ │ Best for: DB backup, migration, re-import │ │ Install: ✅ Built-in (no extra package) │ └────────────────────────────────────────────────────────┘ ``` ### Quick Decision Guide [](#quick-decision-guide) ``` ┌────────────────────────────────────────────────────────────────────┐ │ WHICH FORMAT TO CHOOSE? │ ├────────────────────────────────────────────────────────────────────┤ │ │ │ How many records? │ │ │ │ ┌─────────────┐ │ │ │ 100M+ │ ──────▶ CSV (streaming, memory efficient) │ │ └─────────────┘ │ │ │ │ ┌─────────────┐ │ │ │ 1M - 100M │ ──────▶ CSV (recommended) │ │ └─────────────┘ or SQL │ │ │ │ ┌─────────────┐ │ │ │ < 1M │ ──────▶ Any format (CSV/XLSX/PDF/DOCX/SQL) │ │ └─────────────┘ │ │ │ │ ───────────────────────────────────────────────────────── │ │ │ │ What is the purpose? │ │ │ │ • Data backup/migration ──▶ CSV or SQL │ │ • Excel report ──────────▶ XLSX │ │ • Print document ────────▶ PDF │ │ • Word document ────────▶ DOCX │ │ • Share with non-tech ──▶ XLSX │ │ │ └────────────────────────────────────────────────────────────────────┘ ``` API Reference ------------- [](#api-reference) ### Create Export [](#create-export) ``` ExportFacade::createExport(array $config); ``` **Configuration Options:** ParameterTypeRequiredDescriptionmodelstringYesFull model class pathcolumnsarrayYesColumns to exportformatstringNoExport format (default: 'csv')filenamestringNoCustom filename (without extension)filtersarrayNoQuery where clauseschunk\_sizeintegerNoRecords per chunk (auto-calculated)**Example:** ``` $exportId = ExportFacade::createExport([ 'model' => \App\Models\Transaction::class, 'columns' => ['id', 'amount', 'status', 'created_at'], 'format' => 'xlsx', 'filename' => 'transactions_report', 'filters' => [ ['status', '=', 'completed'], ['amount', '>', 100], ['created_at', '>=', '2026-01-01'], ['category_id', 'IN', [1, 2, 3]], ], ]); ``` ### Get Progress [](#get-progress) ``` ExportFacade::getProgress(string $exportId): array; ``` **Returns:** ``` { "export_id": "550e8400-e29b-41d4-a716-446655440000", "progress": 75, "total": 100000000, "status": "processing", "file_path": null, "filters": [ ["status", "=", "completed"], ["created_at", ">=", "2026-01-01"] ], "filter_summary": "status=_completed_created_at=_>=_2026-01-01", "updated_at": "2026-03-19T10:30:00Z" } ``` When completed: ``` { "export_id": "550e8400-e29b-41d4-a716-446655440000", "progress": 100, "total": 100000000, "status": "completed", "file_path": "exports/transactions_report_filtered_status=_completed.csv", "filters": [["status", "=", "completed"]], "filter_summary": "status=_completed", "updated_at": "2026-03-19T12:30:00Z" } ``` ### Get Download URL [](#get-download-url) ``` ExportFacade::getDownloadUrl(string $exportId, int $minutes = 60): string; ``` Returns a signed URL valid for the specified duration. ### List Exports [](#list-exports) ``` ExportFacade::listExports(int $limit = 10): array; ``` ### Delete Export [](#delete-export) ``` ExportFacade::deleteExport(string $exportId): bool; ``` PDF Export (300M+ Records, 9000+ Pages) --------------------------------------- [](#pdf-export-300m-records-9000-pages) This package uses **TCPDF** with advanced features for professional PDF generation: ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ PDF EXPORT CAPABILITIES │ ├─────────────────────────────────────────────────────────────────────────┤ │ │ │ ✅ 300M+ Records Support │ ✅ Colspan & Rowspan │ │ ✅ 9000+ Pages Without Break│ ✅ Auto Subtotals │ │ ✅ Grand Total │ ✅ Custom Headers/Footers │ │ ✅ Memory Optimized │ ✅ Professional Layouts │ │ │ │ No additional packages needed - TCPDF included by default! │ │ │ └─────────────────────────────────────────────────────────────────────────┘ ``` ### When to Use PDF [](#when-to-use-pdf) Use CaseRecordsPagesRecommendedInvoice generation<50K<100PDFFinancial reports<100K<500PDFLarge data export300M+9000+CSV (recommended)Database backup300M+-SQL### PDF Features [](#pdf-features) ``` use TurboStreamExport\Contracts\Drivers\PdfExportDriver; $driver = new PdfExportDriver(); // Set report information $driver->setReportInfo('Annual Financial Report 2021-2026', [ 'company' => 'ABC Corporation', 'start_date' => '2021-01-01', 'end_date' => '2026-03-31', ]); // Enable automatic subtotals by department $driver->setGroupBy('department'); // Define columns with numeric formatting $columns = ['id', 'employee', 'department', 'year', 'basic', 'allowances', 'gross', 'tax', 'net']; $driver->setNumericColumns(['basic', 'allowances', 'gross', 'tax', 'net']); // Write header $driver->writeHeader($columns); // Add custom row with colspan for year header $driver->addCustomRow([ 0 => [ 'TEXT' => '═══ YEAR 2026 ═══', 'COLSPAN' => 9, 'STYLE' => 'subtotal', 'FONT_WEIGHT' => 'B', 'TEXT_ALIGN' => 'C', 'BACKGROUND_COLOR' => [52, 152, 219], ] ]); // Write data rows foreach ($employees as $employee) { $driver->writeRow([...]); } // Grand total added automatically at end $driver->finalize($filePath); ``` ### PDF Methods Reference [](#pdf-methods-reference) MethodDescription`setReportInfo($name, $filters)`Set report title and filter info`setGroupBy($column)`Enable auto-subtotals when column changes`setNumericColumns($columns)`Format numeric columns with commas`writeHeader($columns)`Write table header`writeRow($data)`Write single data row`addCustomRow($cells)`Add custom row with colspan/rowspan`addColspanRow($data, $colspan, $text, $style)`Add row with merged cells`addEmptyRow()`Add blank row for spacing`finalize($filePath)`Save PDF to file### Real-World PDF Examples [](#real-world-pdf-examples) #### 1. Employee Salary Report with Subtotals by Department [](#1-employee-salary-report-with-subtotals-by-department) ``` $driver = new PdfExportDriver(); $driver->setReportInfo('Employee Salary Report', [ 'department' => 'All', 'month' => 'March 2026', ]); $driver->setGroupBy('department'); $driver->setNumericColumns(['basic_salary', 'house_rent', 'medical', 'gross', 'net']); $columns = ['id', 'name', 'department', 'basic_salary', 'house_rent', 'medical', 'gross', 'net']; $driver->writeHeader($columns); foreach ($employees as $emp) { $driver->writeRow([ $emp->id, $emp->name, $emp->department, $emp->basic_salary, $emp->house_rent, $emp->medical, $emp->gross_salary, $emp->net_salary, ]); } $driver->finalize(storage_path('app/exports/salary_report.pdf')); // Output: Department subtotals + Grand total at end ``` #### 2. Sales Report with Monthly Section Headers (Colspan) [](#2-sales-report-with-monthly-section-headers-colspan) ``` $driver = new PdfExportDriver(); $driver->setReportInfo('Quarterly Sales Report Q1 2026', [ 'quarter' => 'Q1', 'year' => 2026, ]); $driver->setGroupBy('month'); $columns = ['product', 'jan', 'feb', 'mar', 'total']; $driver->setNumericColumns(['jan', 'feb', 'mar', 'total']); $driver->writeHeader($columns); foreach ($months as $month) { // Add month header with colspan $driver->addCustomRow([ 0 => [ 'TEXT' => "━━━ $month ━━━", 'COLSPAN' => 5, 'STYLE' => 'header', 'TEXT_ALIGN' => 'C', 'BACKGROUND_COLOR' => [41, 128, 185], ] ]); foreach ($products as $product) { $driver->writeRow([...]); } } $driver->finalize($filePath); ``` #### 3. Inventory Report with Rowspan for Categories [](#3-inventory-report-with-rowspan-for-categories) ``` $driver = new PdfExportDriver(); $driver->setReportInfo('Inventory Status Report'); $columns = ['category', 'product', 'quantity', 'value']; $driver->writeHeader($columns); $currentCategory = ''; foreach ($inventory as $item) { if ($item->category !== $currentCategory) { // First item of category - use rowspan $driver->addCustomRow([ 0 => [ 'TEXT' => $item->category, 'ROWSPAN' => $categoryCount[$item->category], 'STYLE' => 'header', 'BACKGROUND_COLOR' => [149, 165, 166], ], 1 => ['TEXT' => $item->name], 2 => ['TEXT' => $item->quantity, 'TEXT_ALIGN' => 'R'], 3 => ['TEXT' => number_format($item->value), 'TEXT_ALIGN' => 'R'], ]); $currentCategory = $item->category; } else { $driver->writeRow(['', $item->name, $item->quantity, $item->value]); } } $driver->finalize($filePath); ``` ### Memory Optimization for Large PDFs [](#memory-optimization-for-large-pdfs) For 9000+ page PDFs with 300M+ records: ``` // In config/turbo-export.php return [ 'memory_limit_pdf' => '4G', // Increase for large PDFs // For very large exports, process in chunks 'large_data_chunk_size' => 20000, 'log_progress_interval' => 500000, ]; ``` ### Cell Configuration Options [](#cell-configuration-options) OptionTypeDescriptionTEXTstringCell textCOLSPANintegerColumns to spanROWSPANintegerRows to spanSTYLEstringheader, body, subtotal, grandtotalTEXT\_ALIGNstringL, R, CFONT\_WEIGHTstringB for boldFONT\_SIZEintegerFont size in pointsTEXT\_COLORarrayRGB \[R, G, B\]BACKGROUND\_COLORarrayRGB \[R, G, B\]BORDER\_SIZEfloatBorder widthPADDINGintegerCell padding// Write data rows foreach ($employees as $employee) { $driver->writeRow(\[ $employee->id, $employee->name, $employee->department, $employee->basic\_salary, $employee->house\_rent, $employee->medical, $employee->gross\_salary, $employee->deductions, $employee->net\_salary, \]); } // Add custom section header with colspan $driver->addCustomRow(\[ 0 => \[ 'TEXT' => '★ Report Summary - Last 5 Years Financial Data ★', 'COLSPAN' => 9, 'STYLE' => 'subtotal', 'FONT\_WEIGHT' => 'B', 'TEXT\_ALIGN' => 'C', 'BACKGROUND\_COLOR' => \[68, 114, 196\], \] \]); // Grand total added automatically at end // Finalize and save $filePath = storage\_path('app/exports/salary\_report.pdf'); $driver->finalize($filePath); ``` ### Methods Reference #### setReportInfo(string $name, array $filters = []) Set the report title and filters to be displayed. ```php $driver->setReportInfo('Monthly Sales Report', [ 'start_date' => '2026-01-01', 'end_date' => '2026-03-31', 'region' => 'Dhaka' ]); ``` #### setGroupBy(string $column) [](#setgroupbystring-column) Enable automatic subtotals when a column value changes. ``` // Subtotals will be added automatically when department changes $driver->setGroupBy('department'); // Can also group by year, category, region, etc. $driver->setGroupBy('year'); ``` #### setNumericColumns(array $columns) [](#setnumericcolumnsarray-columns) Define which columns contain numeric values (for formatting). ``` $driver->setNumericColumns([ 'basic_salary', 'house_rent', 'gross_salary', 'net_salary' ]); ``` #### addCustomRow(array $cellData) [](#addcustomrowarray-celldata) Add a custom row with full control over each cell. ``` // Section header spanning all columns $driver->addCustomRow([ 0 => [ 'TEXT' => 'Quarterly Summary - Q1 2026', 'COLSPAN' => 8, 'STYLE' => 'subtotal', 'FONT_WEIGHT' => 'B', 'TEXT_ALIGN' => 'C', ] ]); // Custom row with specific cell values $driver->addCustomRow([ 0 => ['TEXT' => 'Section A', 'STYLE' => 'header'], 1 => ['TEXT' => '', 'STYLE' => 'header'], 2 => ['TEXT' => 'Total Amount', 'STYLE' => 'header', 'TEXT_ALIGN' => 'R'], 3 => ['TEXT' => '1,234,567', 'STYLE' => 'subtotal', 'TEXT_ALIGN' => 'R'], 4 => ['TEXT' => '', 'STYLE' => 'header'], ]); ``` #### addColspanRow(array $data, int $colspan, string $text, string $style = 'subtotal') [](#addcolspanrowarray-data-int-colspan-string-text-string-style--subtotal) Add a row with a cell spanning multiple columns. ``` // Create a row where first 3 columns are merged $driver->addColspanRow( ['', '', '', 'Value 1', 'Value 2'], // Cell data 3, // Span 3 columns 'Merged Header Text', // Text for merged cell 'subtotal' // Style ); ``` #### addEmptyRow() [](#addemptyrow) Add a blank row for visual separation. ``` $driver->addEmptyRow(); ``` ### Cell Configuration Options [](#cell-configuration-options-1) Each cell in `addCustomRow()` supports these options: OptionTypeDescriptionTEXTstringCell text contentCOLSPANintegerNumber of columns to spanROWSPANintegerNumber of rows to spanSTYLEstringStyle name (header, body, subtotal, grandtotal)TEXT\_ALIGNstringAlignment: L, R, CVERTICAL\_ALIGNstringVertical: T, M, BFONT\_WEIGHTstring'B' for boldFONT\_SIZEintegerFont size in pointsTEXT\_COLORarrayRGB \[R, G, B\]BACKGROUND\_COLORarrayRGB \[R, G, B\]BORDER\_SIZEfloatBorder widthPADDING\_TOPintegerTop paddingPADDING\_BOTTOMintegerBottom padding### Example: Financial Report with All Features [](#example-financial-report-with-all-features) ``` use TurboStreamExport\Contracts\Drivers\PdfExportDriver; $driver = new PdfExportDriver(); $driver->setReportInfo('Annual Financial Report 2021-2026', [ 'start_date' => '2021-01-01', 'end_date' => '2026-03-31', 'company' => 'ABC Corporation' ]); // Group by department for subtotals $driver->setGroupBy('department'); // Define columns with numeric ones $columns = ['id', 'employee', 'department', 'year', 'basic', 'allowances', 'gross', 'tax', 'net']; $driver->setNumericColumns(['basic', 'allowances', 'gross', 'tax', 'net']); // Write header $driver->writeHeader($columns); // Data by year and department $years = [2021, 2022, 2023, 2024, 2025, 2026]; $departments = ['HR', 'IT', 'Finance', 'Operations', 'Marketing']; foreach ($years as $year) { // Year header with colspan $driver->addCustomRow([ 0 => [ 'TEXT' => "═══ YEAR $year ═══", 'COLSPAN' => 9, 'STYLE' => 'subtotal', 'FONT_WEIGHT' => 'B', 'TEXT_ALIGN' => 'C', 'BACKGROUND_COLOR' => [52, 152, 219], ] ]); foreach ($departments as $dept) { // Write employee rows (subtotals added automatically when department changes) foreach ($employees as $emp) { if ($emp->department === $dept && $emp->year === $year) { $driver->writeRow([ $emp->id, $emp->name, $emp->department, $emp->year, $emp->basic, $emp->allowances, $emp->gross, $emp->tax, $emp->net, ]); } } } } // Grand total row added automatically at the end $filePath = storage_path('app/exports/financial_report.pdf'); $driver->finalize($filePath); ``` Configuration ------------- [](#configuration) Publish the configuration file: ``` php artisan vendor:publish --tag=turbo-export --force ``` ### config/turbo-export.php [](#configturbo-exportphp) ``` return [ 'disk' => env('EXPORT_DISK', 'local'), 'chunk_size' => env('EXPORT_CHUNK_SIZE', 5000), 'large_data_chunk_size' => env('EXPORT_LARGE_DATA_CHUNK_SIZE', 10000), 'queue' => env('EXPORT_QUEUE', 'exports'), 'retention_hours' => env('EXPORT_RETENTION_HOURS', 24), 'max_records' => env('EXPORT_MAX_RECORDS', 100000000), 'formats' => [ 'csv', 'xlsx', 'pdf', 'docx', 'sql', ], 'default_format' => env('EXPORT_DEFAULT_FORMAT', 'csv'), // Default memory limit for CSV/SQL (streaming exports) 'memory_limit' => env('EXPORT_MEMORY_LIMIT', '1G'), // Format-specific memory limits for memory-based exports 'memory_limit_xlsx' => env('EXPORT_MEMORY_LIMIT_XLSX', '2G'), 'memory_limit_pdf' => env('EXPORT_MEMORY_LIMIT_PDF', '2G'), 'memory_limit_docx' => env('EXPORT_MEMORY_LIMIT_DOCX', '2G'), 'batch_commit_size' => env('EXPORT_BATCH_COMMIT_SIZE', 50000), 'include_filter_in_filename' => env('EXPORT_INCLUDE_FILTER_IN_FILENAME', true), 'download_expiry_minutes' => env('EXPORT_DOWNLOAD_EXPIRY_MINUTES', 60), 'drivers' => [ 'csv' => \TurboStreamExport\Contracts\Drivers\CsvExportDriver::class, 'xlsx' => \TurboStreamExport\Contracts\Drivers\XlsxExportDriver::class, 'pdf' => \TurboStreamExport\Contracts\Drivers\PdfExportDriver::class, 'docx' => \TurboStreamExport\Contracts\Drivers\DocxExportDriver::class, 'sql' => \TurboStreamExport\Contracts\Drivers\SqlExportDriver::class, ], 'large_data_threshold' => env('EXPORT_LARGE_DATA_THRESHOLD', 1000000), 'enable_progress_logging' => env('EXPORT_PROGRESS_LOGGING', true), 'log_progress_interval' => env('EXPORT_LOG_PROGRESS_INTERVAL', 100000), ]; ``` ### Memory Management [](#memory-management) The package uses `cursor()` instead of `chunk()` to stream database records without loading all into memory: FormatExport TypeDefault MemoryNotesCSVStreaming1GBBest for 100M+ recordsSQLStreaming1GBBest for database backupXLSXMemory-based2GBUses running totalsPDFMemory-based2GBSlow for 50K+ recordsDOCXMemory-based2GBWorks well with 47K+ recordsTesting Package --------------- [](#testing-package) For a complete Laravel application demonstrating the export engine in action, see the testing package: **Testing App**: This testing package includes: - Full Laravel application setup - Example models and controllers - Export API routes and views - Queue worker configuration - Sample data generators Testing ------- [](#testing) ``` # Run all tests composer test # Run with coverage composer test:coverage # Run unit tests only composer test:unit # Run feature tests only composer test:feature # Run large data tests (memory/performance) composer test:large # Or using Pest directly ./vendor/bin/pest ./vendor/bin/pest --coverage ./vendor/bin/pest tests/Unit ./vendor/bin/pest tests/LargeData ``` ### Test Coverage [](#test-coverage) SuiteDescriptionFilesUnitDriver tests, Service tests, Job testsExportDriverTest, ExportServiceTest, ProcessExportJobTestFeatureIntegration tests, Filter testsExportServiceFeatureTestLargeDataPerformance tests, Memory testsLargeDataExportTest### Writing Tests [](#writing-tests) ``` use TurboStreamExport\Contracts\Drivers\CsvExportDriver; use TurboStreamExport\Services\ExportService; class ExportServiceTest extends TestCase { public function test_csv_driver_returns_correct_format(): void { $driver = new CsvExportDriver(); $this->assertEquals('csv', $driver->getFormat()); $this->assertEquals('text/csv', $driver->getContentType()); $this->assertEquals('csv', $driver->getFileExtension()); } public function test_filter_summary_generation(): void { $service = new ExportService('local', [new CsvExportDriver()]); $filters = [ ['status', '=', 'active'], ['category_id', '=', '5'], ]; // Access private method via reflection $reflection = new \ReflectionClass($service); $method = $reflection->getMethod('buildFilterSummary'); $method->setAccessible(true); $summary = $method->invoke($service, $filters); $this->assertStringContainsString('status', $summary); $this->assertStringContainsString('active', $summary); } } ``` ### Large Data Testing [](#large-data-testing) ``` public function test_csv_driver_handles_large_batch(): void { $driver = new CsvExportDriver(); $filePath = $this->tempDir . '/large_export.csv'; $handle = fopen($filePath, 'w'); $columns = ['id', 'name', 'email', 'status', 'created_at']; $driver->writeHeader($columns, $handle); $recordCount = 100000; $startTime = microtime(true); for ($i = 1; $i writeRow([ $i, "User Name $i", "user$i@example.com", $i % 2 === 0 ? 'active' : 'inactive', '2026-01-15 10:30:00', ], $handle); } $driver->finalize($handle, $filePath); $duration = microtime(true) - $startTime; $this->assertFileExists($filePath); $this->assertLessThan(30, $duration, 'Large batch export took too long'); } ``` Architecture ------------ [](#architecture) ### Export Process Flow [](#export-process-flow) ``` ┌─────────────────────────────────────────────────────────────────────┐ │ USER REQUEST │ │ ExportFacade::createExport([ │ │ 'model' => User::class, │ │ 'format' => 'csv', │ │ 'filters' => [['status', '=', 'active']] │ │ ]) │ └────────────────────────────────┬────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ VALIDATION │ │ • Validate model exists │ │ • Check columns exist │ │ • Validate format (csv/xlsx/pdf/docx/sql) │ └────────────────────────────────┬────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ DISPATCH JOB │ │ ProcessExportJob dispatched to 'exports' queue │ │ Job ID = exportId (UUID) │ └────────────────────────────────┬────────────────────────────────────┘ │ async ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ QUEUE WORKER (Redis) │ │ │ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ Worker 1 │ │ Worker 2 │ │ Worker N │ │ │ │ ProcessExport│ │ ProcessExport│ │ ProcessExport│ │ │ └──────────────┘ └──────────────┘ └──────────────┘ │ └────────────────────────────────┬────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ CHUNK PROCESSING │ │ │ │ Auto Chunk Sizing: │ │ ┌─────────────┬─────────────┬─────────────┬─────────────┐ │ │ │ < 1M │ 1M - 10M │ 10M - 100M │ 100M+ │ │ │ │ 5,000 rec │ 10,000 rec │ 15,000 rec │ 20,000 rec │ │ │ └─────────────┴─────────────┴─────────────┴─────────────┘ │ │ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ cursor() │──▶│ Write │──▶│ Progress │ │ │ │ (stream) │ │ Batch │ │ Update │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │ │ │ │ │ ▼ ▼ ▼ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ Memory │ │ File │ │ Redis │ │ │ │ Efficient │ │ Output │ │ Cache │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ │ └────────────────────────────────┬────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ FILE GENERATED │ │ │ │ exports/users_export_filtered_status=_active_2026-01-01.csv │ │ │ └────────────────────────────────┬────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────────────┐ │ DOWNLOAD READY │ │ │ │ ExportFacade::getDownloadUrl($exportId) │ │ → Signed URL (valid 1 hour) │ │ │ └─────────────────────────────────────────────────────────────────────┘ ``` ### 5 Export Formats - When to Use Which [](#5-export-formats---when-to-use-which) ``` ┌────────────────────────────────────────────────────────────────────────────┐ │ CHOOSE YOUR FORMAT │ └────────────────────────────────┬───────────────────────────────────────────┘ │ ┌────────────────────────┼────────────────────────────────┐ │ │ │ ▼ ▼ ▼ ┌───────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ CSV │ │ XLSX │ │ SQL │ │ (FASTEST) │ │ (Excel) │ │ (Database) │ ├───────────────┤ ├─────────────────┤ ├─────────────────┤ │ Best for: │ │ Best for: │ │ Best for: │ │ • 100M+ rec │ │ • Reports │ │ • DB Migration │ │ • Data backup │ │ • Sharing │ │ • Backup │ │ • API export │ │ • Printing │ │ • Re-import │ ├───────────────┤ ├─────────────────┤ ├─────────────────┤ │ Memory: 1GB │ │ Memory: 2GB │ │ Memory: 1GB │ │ Stream: YES │ │ Stream: NO │ │ Stream: YES │ └───────────────┘ └─────────────────┘ └─────────────────┘ │ │ │ └────────────────────────┼────────────────────────────────┘ │ ▼ ┌─────────────────┐ ┌─────────────────┐ │ PDF │ │ DOCX │ │ (Professional) │ │ (Word) │ ├─────────────────┤ ├─────────────────┤ │ Best for: │ │ Best for: │ │ • 9000+ pages │ │ • Documentation │ │ • Invoices │ │ • Letters │ │ • Reports │ │ • Contracts │ ├─────────────────┤ ├─────────────────┤ │ TCPDF: Full │ │ Memory: 2GB │ │ Memory: 2-4GB │ │ Stream: NO │ │ Stream: NO │ └─────────────────┘ └─────────────────┘ ``` ### Memory Usage by Format [](#memory-usage-by-format) ``` ┌─────────────────────────────────────────────────────────────────────────┐ │ MEMORY EFFICIENCY │ ├─────────────────────────────────────────────────────────────────────────┤ │ │ │ CSV / SQL ████████████████░░░░░░░░░░░ ~1GB (Streaming) │ │ │ │ XLSX █████████████████████████░░ ~2GB │ │ │ │ PDF (TCPDF) █████████████████████████░░ ~2-4GB │ │ │ │ DOCX █████████████████████████░░ ~2GB │ │ │ ├─────────────────────────────────────────────────────────────────────────┤ │ KEY INSIGHT: │ │ • CSV/SQL use STREAMING - constant memory (best for 100M+) │ │ • XLSX/PDF/DOCX load ALL data - memory grows with records │ │ • For 100M+ records, ALWAYS use CSV! │ └─────────────────────────────────────────────────────────────────────────┘ ``` ### Chunk Size Strategy [](#chunk-size-strategy) RecordsChunk SizeMemory< 1M5,000~50MB1M - 10M10,000~100MB10M - 100M15,000~200MB100M+20,000~512MB### Directory Structure [](#directory-structure) ``` turbostream/export-engine/ ├── src/ │ ├── Contracts/ │ │ ├── ExportDriverInterface.php │ │ ├── ExportableInterface.php │ │ └── Drivers/ │ │ ├── CsvExportDriver.php │ │ ├── XlsxExportDriver.php │ │ ├── PdfExportDriver.php # Unified: simple, subtotals, colspan, 100M+ records │ │ ├── DocxExportDriver.php │ │ └── SqlExportDriver.php │ ├── Facades/ │ │ └── ExportFacade.php │ ├── Http/Controllers/ │ │ └── ExportController.php │ ├── Jobs/ │ │ └── ProcessExportJob.php │ ├── Providers/ │ │ └── TurboStreamExportServiceProvider.php │ └── Services/ │ └── ExportService.php ├── config/ │ └── turbo-export.php ├── tests/ │ ├── Unit/ │ │ ├── ExportDriverTest.php │ │ ├── ExportServiceTest.php │ │ └── ProcessExportJobTest.php │ ├── Feature/ │ │ └── ExportServiceFeatureTest.php │ └── LargeData/ │ └── LargeDataExportTest.php ├── routes/ │ └── api.php ├── composer.json ├── phpunit.xml └── README.md ``` Queue Workers ------------- [](#queue-workers) ### Start Queue Worker [](#start-queue-worker) ``` php artisan queue:work redis --queue=exports ``` ### High Priority Queue [](#high-priority-queue) For urgent exports: ``` php artisan queue:work redis --queue=exports-high,exports ``` ### Production Setup (Supervisor) [](#production-setup-supervisor) ``` [program:export-worker] process_name=%(program_name)s_%(process_num)02d command=php /path/to/artisan queue:work redis --queue=exports --sleep=3 --tries=3 autostart=true autorestart=true stopasgroup=true killasgroup=true user=www-data numprocs=4 redirect_stderr=true stdout_logfile=/var/log/export-worker.log ``` Extending the Package --------------------- [](#extending-the-package) ### Custom Export Driver [](#custom-export-driver) Create your own driver by implementing `ExportDriverInterface`: ```