PHPackages memeddev/mysql-to-google-bigquery - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. [Database & ORM](/categories/database) 4. / 5. memeddev/mysql-to-google-bigquery ActiveLibrary[Database & ORM](/categories/database) memeddev/mysql-to-google-bigquery ================================= MySQL to Google BigQuery Sync Tool v0.3(5y ago)6279626[4 issues](https://github.com/MemedDev/mysql-to-google-bigquery/issues)[2 PRs](https://github.com/MemedDev/mysql-to-google-bigquery/pulls)MITPHP Since Oct 25Pushed 3y ago7 watchersCompare [ Source](https://github.com/MemedDev/mysql-to-google-bigquery)[ Packagist](https://packagist.org/packages/memeddev/mysql-to-google-bigquery)[ Docs](http://github.com/memeddev/mysql-to-google-bigquery)[ RSS](/packages/memeddev-mysql-to-google-bigquery/feed)WikiDiscussions master Synced 4w ago READMEChangelog (3)Dependencies (6)Versions (6)Used By (0) [![MySQL to Google BigQuery Logo](https://cloud.githubusercontent.com/assets/2197005/19776979/f4abd1be-9c54-11e6-9842-212f26e765a5.png)](https://cloud.githubusercontent.com/assets/2197005/19776979/f4abd1be-9c54-11e6-9842-212f26e765a5.png) MySQL to Google BigQuery Sync Tool ================================== [](#mysql-to-google-bigquery-sync-tool) Table of Contents ----------------- [](#table-of-contents) - [How it works](#how-it-works) - [Requirements](#requirements) - [Usage](#usage) - [Credits](#credits) - [License](#license) How it works ------------ [](#how-it-works) Steps when no order column has been supplied: - Count MySQL table rows - Count BigQuery table rows - MySQL rows > BigQuery rows? - Get the rows diff, split in batches of XXXXX rows/batch Steps when order column has been supplied: - Get max value for order column from MySQL table - Get max value for order column from BigQuery table - Max value MySQL > Max value BigQuery? - Delete all rows with order column value = max value BigQuery to make sure no duplicate records are being created in BigQuery - Get max value for order column from BigQuery table - Get the rows diff based on new max value BigQuery, split in batches of XXXXX rows/batch Final three steps: - Dump MySQL rows to a JSON - Send JSON to BigQuery - Repeat until all batches are sent Tip: Create a cron job for keep syncing the tables using an interval like 15 minutes (respect the Load Jobs [quota policy](https://cloud.google.com/bigquery/quota-policy)) Requirements ------------ [](#requirements) The following PHP versions are supported: - PHP 7 - HHVM - PDO Extension with MySQL driver Usage ----- [](#usage) Download the library using [composer](https://packagist.org/packages/memeddev/mysql-to-google-bigquery): ``` $ composer require memeddev/mysql-to-google-bigquery ``` Now, define some environment variables or create a `.env` file on the root of the project, replacing the values: ``` BQ_PROJECT_ID=bigquery-project-id BQ_KEY_FILE=google-service-account-json-key-file.json BQ_DATASET=bigquery-dataset-name DB_DATABASE_NAME=mysql-database-name DB_USERNAME=mysql_username DB_PASSWORD=mysql_password DB_HOST=mysql-host DB_PORT=3306 IGNORE_COLUMNS=password,hidden_column,another_column ``` PS: To create the `Google Service Account JSON Key File`, access Run: ``` vendor/bin/console sync table-name ``` If you want to auto create the table on BigQuery: ``` vendor/bin/console sync table-name --create-table ``` If you want to delete (and create) the table on BigQuery for a full dump: ``` vendor/bin/console sync table-name --delete-table ``` Credits ------- [](#credits) ❤️ Memed SA ([memed.com.br](https://memed.com.br)) License ------- [](#license) MIT license, see [LICENSE](LICENSE) ### Health Score 34 — LowBetter than 75% of packages Maintenance19 Infrequent updates — may be unmaintained Popularity29 Limited adoption so far Community20 Small or concentrated contributor base Maturity57 Maturing project, gaining track record Bus Factor1 Top contributor holds 60% of commits — single point of failure How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window. **Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores. **Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement. **Maturity (30%)** — Project age, version count, PHP version support, and release stability. ### Release Activity Cadence Every ~698 days Total 3 Last Release 2139d ago ### Community Maintainers ![](https://www.gravatar.com/avatar/0e904e2d00a1391f421ba18af4e9e2826787ff02a93345828809472596f5fef9?d=identicon)[gabrielrcouto](/maintainers/gabrielrcouto) --- Top Contributors [![gabrielrcouto](https://avatars.githubusercontent.com/u/2197005?v=4)](https://github.com/gabrielrcouto "gabrielrcouto (18 commits)")[![vidaxl](https://avatars.githubusercontent.com/u/13419108?v=4)](https://github.com/vidaxl "vidaxl (9 commits)")[![bviolier](https://avatars.githubusercontent.com/u/1247610?v=4)](https://github.com/bviolier "bviolier (1 commits)")[![Mmann099](https://avatars.githubusercontent.com/u/50181057?v=4)](https://github.com/Mmann099 "Mmann099 (1 commits)")[![Pionell](https://avatars.githubusercontent.com/u/24505446?v=4)](https://github.com/Pionell "Pionell (1 commits)") --- Tags googledatamysqltoolsendsyncloadbigquery ### Embed Badge ![Health badge](/badges/memeddev-mysql-to-google-bigquery/health.svg) ``` [![Health](https://phpackages.com/badges/memeddev-mysql-to-google-bigquery/health.svg)](https://phpackages.com/packages/memeddev-mysql-to-google-bigquery) ``` ### Alternatives [matomo/matomo Matomo is the leading Free/Libre open analytics platform 21.6k38.2k](/packages/matomo-matomo)[flow-php/flow PHP ETL - Extract Transform Load - Data processing framework 84735.1k](/packages/flow-php-flow)[awssat/laravel-sync-migration Laravel tool helps to sync migrations without refreshing the database 10923.3k](/packages/awssat-laravel-sync-migration)[duxweb/dux-lite The lightweight framework based on slim php 161.0k9](/packages/duxweb-dux-lite) PHPackages © 2026 [Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)