PHPackages                             w3spi5/bigdump - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Database & ORM](/categories/database)
4. /
5. w3spi5/bigdump

ActiveProject[Database & ORM](/categories/database)

w3spi5/bigdump
==============

Staggered MySQL Dump Importer - Import large MySQL dumps on servers with execution time limits

v2.29(2mo ago)2922MITPHPPHP >=8.1CI passing

Since Dec 6Pushed 2mo ago1 watchersCompare

[ Source](https://github.com/w3spi5/bigdump)[ Packagist](https://packagist.org/packages/w3spi5/bigdump)[ Docs](https://github.com/w3spi5/bigdump)[ RSS](/packages/w3spi5-bigdump/feed)WikiDiscussions main Synced 1mo ago

READMEChangelog (10)Dependencies (2)Versions (30)Used By (0)

BigDump 2.29 - Staggered MySQL Dump Importer
============================================

[](#bigdump-229---staggered-mysql-dump-importer)

[![PHP Version](https://camo.githubusercontent.com/186342cb44bb168eff2b2a27896ee1b2a19384ec82c1468429a37a9db847bc18/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f7068702d382e312b2d79656c6c6f772e737667)](https://php.net/)[![License](https://camo.githubusercontent.com/8bb50fd2278f18fc326bf71f6e88ca8f884f72f179d3e555e20ed30157190d0d/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d677265656e2e737667)](LICENSE)[![Package Version](https://camo.githubusercontent.com/420f3957dec3fc7a77bd25caf873cc2f2b44ad761ac82f2311cc01dbda10fcff/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f76657273696f6e2d322e32392d626c75652e737667)](https://php.net/)[![Build Assets](https://camo.githubusercontent.com/2687b88f32daaded7ce742f41a4274272a472b67bf9689a607e955b329dff3d3/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6275696c642d4769744875625f416374696f6e732d3230383846462e737667)](https://github.com/w3spi5/bigdump/actions)[![PHAR](https://camo.githubusercontent.com/cb6bbbf43ab0e3b198e3601f1fba1adae37b4f75479f3d48ba2ee8cadade9a3f/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f504841522d73696e676c652d2d66696c652d707572706c652e737667)](https://github.com/w3spi5/bigdump/releases)

 [![BigDump Logo](assets/img/logo.png)](assets/img/logo.png)

BigDump is a PHP tool for importing large MySQL dumps on web servers with strict execution time limits. Originally created by Alexey Ozerov in 2003, this major version 2 is a complete refactoring using object-oriented MVC architecture.

See [CHANGELOG.md](CHANGELOG.md) for detailed version history.

Features
--------

[](#features)

- **Single-File PHAR**: Download one file, upload to server, done — like Adminer
- **Staggered Import**: Imports dumps in sessions to bypass timeout limits
- **Multi-format Support**: `.sql`, `.gz` (gzip), `.bz2` (bzip2), and `.csv` files
- **CLI Optimizer**: Standalone command-line tool to optimize SQL dumps with INSERT batching
- **SSE Streaming**: Real-time progress with Server-Sent Events and elapsed timer
- **SQL Preview**: Preview file contents and queries before importing
- **Import History**: Track all import operations with statistics
- **Session Persistence**: Resume imports after browser refresh or server restart
- **Modern Interface**: Tailwind CSS with dark mode, drag & drop upload, smooth animations
- **Zero-CDN**: Self-hosted purged assets (~47KB total vs ~454KB CDN)
- **Auto-Tuning**: Dynamic batch size based on available RAM (up to 1.5M lines/batch)
- **Enhanced Security**: Protection against path traversal, XSS, and other vulnerabilities
- **UTF-8 Support**: Proper handling of multi-byte characters and BOM

Performance Optimizations (v2.16)
---------------------------------

[](#performance-optimizations-v216)

BigDump 2.16 includes several performance optimizations that significantly reduce import time:

### MySQL Pre-queries (Enabled by Default)

[](#mysql-pre-queries-enabled-by-default)

```
'pre_queries' => [
    'SET autocommit=0',        // Batch commits instead of per-INSERT
    'SET unique_checks=0',     // Skip unique index verification
    'SET foreign_key_checks=0', // Skip FK constraint checks
    'SET sql_log_bin=0',       // Disable binary logging
],
```

**Impact:** 5-10x faster imports by eliminating per-row overhead.

### Optimized SQL Parsing

[](#optimized-sql-parsing)

- **Quote Analysis**: Uses `strpos()` jumps instead of character-by-character iteration
- **INSERT Detection**: String functions replace complex regex patterns
- **Buffered Reading**: 64KB read buffer reduces system calls

### Performance Comparison

[](#performance-comparison)

OptimizationBeforeAfterImprovementMySQL autocommitPer-INSERT commitBatch commit~10xQuote parsingO(n) per charO(1) jumps~3xINSERT detectionComplex regexString functions~2xFile I/O16KB fgets64KB buffer~2xPerformance Profiles (v2.19)
----------------------------

[](#performance-profiles-v219)

BigDump 2.19 introduces a **performance profile system** allowing you to choose between optimized configurations:

### Conservative Mode (Default)

[](#conservative-mode-default)

Best for shared hosting environments with limited memory (64MB).

```
'performance_profile' => 'conservative',
```

### Aggressive Mode

[](#aggressive-mode)

For dedicated servers with 128MB+ memory, providing +20-30% throughput improvement.

```
'performance_profile' => 'aggressive',
```

### Profile Comparison

[](#profile-comparison)

SettingConservativeAggressive`insert_batch_size`2,0005,000`file_buffer_size`64KB128KB`max_batch_bytes`16MB32MB`commit_frequency`Every batchEvery 3 batchesMemory limit<64MB<128MB**Note:** Aggressive mode automatically falls back to conservative if PHP `memory_limit` is below 128MB.

Continue On Error Mode (v2.28)
------------------------------

[](#continue-on-error-mode-v228)

BigDump 2.27 introduces a **continue on error** option that allows imports to complete even when some SQL statements fail. This is useful for:

- Large imports where some statements may fail (duplicate keys, missing tables)
- Recovering as much data as possible from partially corrupted dumps
- Imports where you expect certain errors (e.g., re-importing existing data)

### Enable Continue On Error

[](#enable-continue-on-error)

```
'continue_on_error' => true,  // Default: false (stop on first error)
```

### How It Works

[](#how-it-works)

When enabled:

1. SQL errors are **collected as warnings** instead of stopping the import
2. Import continues processing remaining statements
3. After completion, a **success message with warning section** is displayed
4. Warning section shows line numbers and error messages (collapsible)
5. Maximum **100 warnings** stored to prevent memory issues

### Example Output

[](#example-output)

```
✅ Import completed successfully!
   Queries: 15,432 | Lines: 89,234 | Duration: 2m 34s

⚠️ 3 warning(s) during import [Show details]
   - Line 45,231: Duplicate entry '123' for key 'PRIMARY'
   - Line 67,892: Table 'temp_backup' doesn't exist
   - Line 78,001: Data truncated for column 'description'

```

**Important:** Default is `false` (stop on first error) for safety. Only enable when you understand the implications of ignoring SQL errors.

CLI SQL Optimizer (v2.22)
-------------------------

[](#cli-sql-optimizer-v222)

BigDump includes a **standalone CLI tool** for optimizing SQL dump files without requiring a database connection. It rewrites dumps with INSERT batching for faster imports.

### Basic Usage

[](#basic-usage)

```
php cli.php  --output  [options]
```

### Examples

[](#examples)

```
# Basic optimization
php cli.php dump.sql -o optimized.sql

# With gzip compressed input
php cli.php dump.sql.gz --output optimized.sql --batch-size=5000

# Aggressive profile with force overwrite
php cli.php backup.sql.bz2 -o backup_batched.sql --profile=aggressive -f
```

### CLI Options

[](#cli-options)

OptionDescription`-o, --output `Output file path (required)`--batch-size=`INSERT batch size (default: profile-based)`--profile=`Performance profile: `conservative` or `aggressive``-f, --force`Overwrite output file if it exists`-h, --help`Display help message### CLI Profile Defaults

[](#cli-profile-defaults)

ProfileBatch SizeMax Batch Bytesconservative2,00016MBaggressive5,00032MB### What It Does

[](#what-it-does)

The CLI optimizer transforms individual INSERT statements into batched multi-value INSERTs:

**Before:**

```
INSERT INTO users VALUES (1, 'Alice');
INSERT INTO users VALUES (2, 'Bob');
INSERT INTO users VALUES (3, 'Charlie');
```

**After:**

```
INSERT INTO users VALUES (1, 'Alice'), (2, 'Bob'), (3, 'Charlie');
```

This can provide **10-50x speedup** when importing the optimized file.

### Supported Input Formats

[](#supported-input-formats)

- `.sql` - Plain SQL files
- `.sql.gz` - Gzip compressed SQL
- `.sql.bz2` - Bzip2 compressed SQL

PHAR Distribution (v2.25) — Easiest Install
-------------------------------------------

[](#phar-distribution-v225--easiest-install)

Download `bigdump.phar` from [Releases](https://github.com/w3spi5/bigdump/releases) and upload to your server. That's it!

### Quick Start

[](#quick-start)

1. **Download** `bigdump.phar` and `bigdump-config.example.php` from releases
2. **Upload** both files to your server
3. **Rename** `bigdump-config.example.php` → `bigdump-config.php`
4. **Edit** `bigdump-config.php` with your database credentials
5. **Access** `https://yoursite.com/bigdump.phar` in your browser

### PHAR Features

[](#phar-features)

- **Zero installation**: Single file contains everything (PHP + CSS + JS + icons)
- **Web + CLI**: Same file works in browser and command line
- **External config**: Edit `bigdump-config.php` next to the PHAR
- **~478 KB**: Compact, fast to upload

### PHAR CLI Mode

[](#phar-cli-mode)

```
# Check version
php bigdump.phar --version

# Optimize SQL dump
php bigdump.phar dump.sql -o optimized.sql

# With options
php bigdump.phar dump.sql.gz -o optimized.sql --profile=aggressive
```

### Build PHAR Locally

[](#build-phar-locally)

```
php -d phar.readonly=0 build/build-phar.php
# Output: dist/bigdump.phar
```

---

Requirements
------------

[](#requirements)

- PHP 8.1 or higher
- MySQLi extension
- MySQL/MariaDB server
- Write permissions on the `uploads/` directory

Installation (Traditional)
--------------------------

[](#installation-traditional)

1. **Download** the project to your web server:

    ```
    git clone https://github.com/w3spi5/bigdump.git
    ```
2. **Configure** the database:

    ```
    cp config/config.example.php config/config.php
    ```

    Then edit `config/config.php` with your database credentials:

    ```
    return [
        'db_server' => 'localhost',
        'db_name' => 'your_database',
        'db_username' => 'your_username',
        'db_password' => 'your_password',
        'db_connection_charset' => 'utf8mb4',
    ];
    ```
3. **Set permissions**:

    ```
    chmod 755 uploads/
    ```
4. **Access** BigDump via your browser:

    ```
    http://your-site.com/bigdump/

    ```

Configuration
-------------

[](#configuration)

### Auto-Tuning (RAM-based, NVMe-optimized)

[](#auto-tuning-ram-based-nvme-optimized)

```
return [
    'auto_tuning' => true,        // Enable dynamic batch sizing
    'min_batch_size' => 10000,    // Safety floor
    'max_batch_size' => 1500000,  // NVMe ceiling
    'force_batch_size' => 0,      // Force specific size (0 = auto)
];
```

Available RAMBatch Size< 1 GB80,000< 2 GB150,000< 4 GB300,000< 8 GB620,000< 12 GB940,000< 16 GB1,260,000> 16 GB1,500,000### INSERT Batching (x10-50 speedup)

[](#insert-batching-x10-50-speedup)

For dumps with simple INSERT statements, BigDump can group them into multi-value INSERTs:

```
return [
    'insert_batch_size' => 10000,  // Group 10000 INSERTs into 1 query (16MB max)
];
```

This transforms:

```
INSERT INTO t VALUES (1);
INSERT INTO t VALUES (2);
-- ... 1000 more
```

Into:

```
INSERT INTO t VALUES (1), (2), ... ;  -- Single query
```

### Windows Optimization

[](#windows-optimization)

For accurate RAM detection on Windows, enable the COM extension in `php.ini`:

```
extension=com_dotnet
```

### Import Options

[](#import-options)

```
return [
    'linespersession' => 50000, // Lines per session (if auto_tuning disabled)
    'delaypersession' => 0,     // Delay between sessions (ms)
    'ajax' => true,             // AJAX/SSE mode (recommended)
    'test_mode' => false,       // Parse without executing
];
```

### CSV Import

[](#csv-import)

```
return [
    'csv_insert_table' => 'my_table',
    'csv_preempty_table' => false,
    'csv_delimiter' => ',',
    'csv_enclosure' => '"',
];
```

### Pre/Post-queries (Recommended for large imports)

[](#prepost-queries-recommended-for-large-imports)

```
return [
    'pre_queries' => [
        'SET autocommit = 0',
        'SET unique_checks = 0',
        'SET foreign_key_checks = 0',
        'SET sql_log_bin = 0',  // Disable binary logging
    ],
    'post_queries' => [
        'COMMIT',
        'SET autocommit = 1',
        'SET unique_checks = 1',
        'SET foreign_key_checks = 1',
    ],
];
```

Pre-queries disable constraints for speed; post-queries restore them automatically after import.

Project Structure
-----------------

[](#project-structure)

```
bigdump/
├── config/
│   └── config.php
├── index.php              # Web entry point
├── cli.php                # CLI optimizer entry point
├── build/                 # PHAR build scripts
│   ├── build-phar.php     # Main build script
│   └── stubs/             # Entry point stubs
├── dist/                  # Build output (gitignored)
│   ├── bigdump.phar
│   └── bigdump-config.example.php
├── assets/
│   ├── dist/              # Compiled assets (auto-generated)
│   │   ├── app.min.css
│   │   └── *.min.js
│   ├── src/               # Source files
│   │   ├── css/tailwind.css
│   │   └── js/*.js
│   ├── icons.svg          # SVG icon sprite
│   ├── img/
│   │   └── logo.png
│   └── docs/
│       └── demov2.2.png   # Screenshot
├── src/
│   ├── Config/Config.php
│   ├── Controllers/BigDumpController.php
│   ├── Core/
│   │   ├── Application.php
│   │   ├── PharContext.php    # PHAR detection utilities
│   │   ├── Request.php
│   │   ├── Response.php
│   │   ├── Router.php
│   │   └── View.php
│   ├── Models/
│   │   ├── Database.php
│   │   ├── FileHandler.php
│   │   ├── ImportSession.php
│   │   └── SqlParser.php
│   └── Services/
│       ├── AjaxService.php
│       ├── AutoTunerService.php
│       ├── CliFileReader.php      # CLI file reading
│       ├── CliOptimizerService.php # CLI orchestration
│       ├── CliSqlParser.php       # CLI SQL parsing
│       ├── ImportService.php
│       ├── InsertBatcherService.php
│       └── SseService.php
├── templates/
│   ├── error.php
│   ├── error_bootstrap.php
│   ├── home.php
│   ├── import.php
│   ├── layout.php
│   └── layout_phar.php        # PHAR layout with inlined assets
├── tests/                 # Test suite
│   ├── Cli*.php           # CLI tests (7 files)
│   ├── Phar*.php          # PHAR tests (4 files)
│   └── *.php              # Other tests
├── uploads/
├── scripts/
│   └── generate-icons.php # SVG sprite generator
├── .github/
│   └── workflows/
│       ├── build-assets.yml  # CI asset pipeline
│       └── build-phar.yml    # PHAR build & release
├── CHANGELOG.md
├── LICENSE
└── README.md

```

How It Works
------------

[](#how-it-works-1)

### Staggered Import (Progress in Steps)

[](#staggered-import-progress-in-steps)

BigDump uses a **staggered import** approach - you'll notice the progress counters increment in steps (every ~5 seconds) rather than continuously. **This is by design:**

- **Avoids PHP timeouts**: Each batch completes within `max_execution_time`
- **Server breathing room**: Prevents overloading shared hosting environments
- **Shared hosting compatible**: Works on hosts with strict execution limits
- **Resume capability**: If interrupted, import can resume from the last batch

The batch size is automatically tuned based on your server's available RAM (see Auto-Tuning section).

### Real-time Progress with SSE

[](#real-time-progress-with-sse)

BigDump uses **Server-Sent Events (SSE)** for real-time progress updates:

- Single persistent HTTP connection (no polling overhead)
- Progress updates sent after each batch completes
- Elapsed time counter updates every second
- Automatic reconnection if connection drops

Troubleshooting
---------------

[](#troubleshooting)

### SSE "Connecting..." Modal Stuck

[](#sse-connecting-modal-stuck)

If the progress modal stays on "Connecting..." indefinitely but the import actually works (data appears in database), your server is buffering SSE responses.

**Solutions by server type:**

ServerConfiguration FileFix**Apache + mod\_fcgid**`conf/extra/httpd-fcgid.conf`Add `FcgidOutputBufferSize 0`**Apache + mod\_proxy\_fcgi**VirtualHost configAdd `flushpackets=on` to ProxyPass**nginx + PHP-FPM**`nginx.conf`Add `proxy_buffering off;` and `fastcgi_buffering off;`**Laragon (Windows)**Uses mod\_fcgidEdit `laragon/bin/apache/httpd-2.4.x/conf/extra/httpd-fcgid.conf`**Quick diagnostic**: Test with PHP's built-in server:

```
cd /path/to/bigdump
php -S localhost:8000
```

If the built-in server works but Apache/nginx doesn't, it's definitely a server buffering issue.

### Upload Errors (HTTP 500 for Large Files)

[](#upload-errors-http-500-for-large-files)

If uploading large files (>500MB) fails with HTTP 500 error but smaller files work:

ServerConfigurationFix**Apache + mod\_fcgid**`httpd-fcgid.conf`Add `FcgidMaxRequestLen 2147483648`**Laragon (Windows)**`laragon/etc/apache2/fcgid.conf`Set `FcgidMaxRequestLen 2147483648`**nginx**`nginx.conf`Set `client_max_body_size 2G;`> **Note**: `FcgidMaxRequestLen` limits requests BEFORE PHP starts. Even if `upload_max_filesize=2G` in php.ini, mod\_fcgid may reject the request first.

**Alternative for very large files**: Upload via FTP/SCP directly to the `uploads/` directory.

### Import Errors

[](#import-errors)

- **"Table already exists"**: Use the "Drop & Restart" button to drop tables and restart
- **"No active import session"**: Refresh the page and try again (timing issue, auto-retries)
- **Timeout errors**: Reduce `linespersession` in config or enable `auto_tuning`

Security
--------

[](#security)

- **NEVER** leave BigDump and your dump files on a production server after use
- Dump files may contain sensitive data
- The `uploads/` directory is protected by `.htaccess`
- Delete the application as soon as the import is complete

License
-------

[](#license)

[MIT](LICENSE)

Credits
-------

[](#credits)

- **Original**: Alexey Ozerov () — Created in 2003
- **MVC Refactoring**: Version 2 by [w3spi5](https://github.com/w3spi5) — 2025

> 🔗 This fork is officially linked from the [original BigDump page](https://www.ozerov.de/bigdump).

---

Screenshots
-----------

[](#screenshots)

 [![BigDump Screenshot](assets/docs/demov2.2.png)](assets/docs/demov2.2.png)

---

Acknowledgments
---------------

[](#acknowledgments)

This project is maintained by [w3spi5](https://github.com/w3spi5) with AI-assisted development using Claude.

AI assists with:

- Code implementation and help to refactoring
- Documentation drafting
- Test coverage

Human-driven:

- Refactoring
- Architecture decisions
- Feature prioritization
- Code review and quality control

###  Health Score

43

—

FairBetter than 91% of packages

Maintenance85

Actively maintained with recent releases

Popularity14

Limited adoption so far

Community9

Small or concentrated contributor base

Maturity54

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~3 days

Recently: every ~13 days

Total

29

Last Release

78d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/e4ff8a61fb93cab60f87112af5f040144ec19a28574935f83c9ae999bab4a3d5?d=identicon)[w3spi5](/maintainers/w3spi5)

---

Top Contributors

[![w3spi5](https://avatars.githubusercontent.com/u/8407711?v=4)](https://github.com/w3spi5 "w3spi5 (134 commits)")

---

Tags

migrationdatabasedumpmysqlsqlimportbigdump

###  Code Quality

TestsPHPUnit

### Embed Badge

![Health badge](/badges/w3spi5-bigdump/health.svg)

```
[![Health](https://phpackages.com/badges/w3spi5-bigdump/health.svg)](https://phpackages.com/packages/w3spi5-bigdump)
```

###  Alternatives

[rah/danpu

Zero-dependency MySQL dump library for easily exporting and importing databases

64401.8k10](/packages/rah-danpu)[phelium/mysql-backup

Backup easily your MySQL database !

278.2k2](/packages/phelium-mysql-backup)[davmixcool/php-dbcloud

Easily backup PostgreSql or MySql database to the cloud

111.5k](/packages/davmixcool-php-dbcloud)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)