PHPackages torqnorth/data-importer-extensions-bundle - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. torqnorth/data-importer-extensions-bundle ActivePimcore-bundle torqnorth/data-importer-extensions-bundle ========================================= v5.3.0(2mo ago)07↓100%proprietaryPHP Since Mar 21Pushed 1mo agoCompare [ Source](https://github.com/Torq-North/data-importer-extensions-bundle)[ Packagist](https://packagist.org/packages/torqnorth/data-importer-extensions-bundle)[ RSS](/packages/torqnorth-data-importer-extensions-bundle/feed)WikiDiscussions master Synced 1mo ago READMEChangelog (2)Dependencies (9)Versions (47)Used By (0) title Data Importer Torq North Data Importer Extensions =================================== [](#torq-north-data-importer-extensions) This extension adds a number of additional features to the [Pimcore Data Importer](https://github.com/pimcore/data-importer) bundle. Installation ------------ [](#installation) ``` composer require torqnorth/data-importer-extensions-bundle ``` Path Syntax ----------- [](#path-syntax) A number of our extensions make use of the **Path** syntax that allows for Paths to be created based on values in the import. Take for example an Excel File: YearMakeModelColor2015GMCSierraWhite2001ChevroletSilveradoBlueTo build the Path `/Products/Cars/GMC/Sierra/2015` using Path Syntax would be `/Products/Cars/$[1]/$[2]/$[0]`. The numerical values correspond to the indexes of the values in the Excel file (starting at 0). For an XML file: ``` GMC Sierra 2015 White Chevrolet Silverado 2001 Blue ``` the **Path Syntax** would use the Attribute names instead `/Products/Cars/$[Make]/$[Model]/$[Year]` Data Interpreters ----------------- [](#data-interpreters) Data Interpreters are the supported "File Formats" that the Data Importer bundle can use. We've added a few of our own. ### Advanced XLSX Interpreter [](#advanced-xlsx-interpreter) The Advanced XLSX interpreter makes a few improvements over the default XLSX interpreter. This interpreter uses `openspout` as the Excel parser. Open Spout XLSX parsing uses **much** less memory than the default XLSX parses which makes use of `PHPOffice`. We've seen files that required >4GB RAM on PHPOffice use less than 50MB with openspout. We've also detected a memory leak in some cases with the PHPOffice implementation where RAM gets allocated on the server and never released. Configuration OptionDescriptionUnique Column IndexesAccepts a comma separated list of column indexes to treat as unique values. Used to filter the rows in an excel file. For example an excel file with the headers `Brand,Model,SubModel` and you want to import a unique `Brand` object for each new `Brand` you encounter. In this case, use value `0` to only take unique values from the first column in the Excel file. If you want to create a data object for each `Brand` and `Model` use `0,1` as the value.Row FilterThis accepts a [Symfony Expression](https://symfony.com/doc/current/reference/formats/expression_language.html) to be applied to the rows of the Excel file. Each row in the Excel file get's handed to the expression evaluator as a variable named `row`. The expression `row[0] == 'Apple'` would only process rows where the value of the first column is Apple.### Bulk XLSX Interpreter [](#bulk-xlsx-interpreter) This Bulk XLSX Interpreter has the same options as the Advanced XLSX Interpreter. The main difference is that the Excel file gets converted to a CSV and loaded into the Data Importer queue using `LOAD LOCAL INFILE`. This **VERY DRASTICALLY** improves the performance of loading the queue table. We've seen 200K rows loaded in <5s. Our experience with a 16GB RAM server shows that Excel files over 30K rows often are not imported successfully by the default XLSX Interpreter. **This Feature Requires the Database Server to be configured to permit local infile / infile permissions!** See [MySQL Documentation](https://dev.mysql.com/doc/refman/8.0/en/load-data-local-security.html#load-data-local-configuration) regarding `LOCAL INFILE`. Also in your database connection you'll need to add the Bulk option (1001:true) in example: ``` doctrine: dbal: connections: default: host: "%env(string:DATABASE_HOST)%" port: 3306 user: "%env(string:DATABASE_USER)%" password: "%env(string:DATABASE_PASSWORD)%" dbname: "%env(string:DATABASE_NAME)%" mapping_types: { enum: string, bit: boolean } server_version: "5.5.5-10.4.22-MariaDB-1:10.4.22+maria~focal" options: 1001: true ``` ### Bulk CSV Interpreter [](#bulk-csv-interpreter) The Bulk CSV Interpreter has the same options as the regular CSV interpreter but like the Bulk XLSX Interpreter it uses `LOAD LOCAL INFILE` to queue data rows. Please see the [Bulk XLSX Interpreter Section](#bulk-xlsx-interpreter) for limitions and requirments. ### Bulk SQL Interpreter [](#bulk-sql-interpreter) This Interpreter is to be used when using the [Bulk SQL Data Loader](#bulk-sql-data-loader). Behind the scenes this uses the Bulk CSV Interpreter as it is very fast. If you run into errors please see the [Bulk XLSX Interpreter Section](#bulk-xlsx-interpreter) for limitions and requirments. ### XML Schema Based Preview Interpreter [](#xml-schema-based-preview-interpreter) This Interpreter is an expansion upon the default XML based interpreter that will load all fields defined by the provided Xsd file for use in the preview screen. Data Loaders ------------ [](#data-loaders) ### Bulk SQL Data Loader [](#bulk-sql-data-loader) As of 4.0 (and pimcore data-importer 1.10) This is an extended version of the default SQL data loader which uses the Bulk CSV implementation rather than the json implementation for better performance. Note that the pimcore JSON implementation will save mapping fields by name, which the Bulk SQL saves them by index so switching back and forth may break mappings. The SQL Data Loader uses [DBAL](https://www.doctrine-project.org/projects/dbal.html) to allow data to be loaded from a SQL source. Connections to any database supported by DBAL will work provided they are configured correctly inside of `database.yaml`. (Database configuration can be placed in any valid Symfony config file, provided its in the correct format as can be seen in `database.yaml`). To set up a Bulk SQL loader 1. Create a new connection in `database.yaml` or if using the Pimcore database skip this step. [![SQL Loader Configuration](docs/img/sql_loader_config.png)](docs/img/sql_loader_config.png) 2. Select the correct connection from the **Connection Name** dropdown 3. Provide a valid query using the Select, where, from, Group By, and Limit fields. 4. Ensure to select **Bulk SQL** under File Format! This loader produces a CSV file as part of loading the SQL. Data Targets ------------ [](#data-targets) Data Targets control where data flows as its being mapped to Data Objects. ### Advanced Classification Store [](#advanced-classification-store) This is the same as the [Classification Store](https://pimcore.com/docs/platform/Data_Importer/Configuration/Mapping_Configuration/Data_Target/#classification-store) Data Target except it adds the `Overwrite` options as seen on the `Direct` Data Target. ### Image Gallery Appender [](#image-gallery-appender) This can be used to add an image into an Image Gallery field. ### Property [](#property) This is used to set a property on a Data Object. ### Tags [](#tags) This is used to add tags on a Data Object. Operators --------- [](#operators) ### As Country Code [](#as-country-code) Operating on an input string that is expected to be either a 2- or 3- character Country Code, filters this input to the valid 2-character Country Code (or blank if none is found). ### Constants [](#constants) This operator simply returns a constant string. Useful if wanting to control `OBJECT_TYPE` object or variant. ### SafeKey [](#safekey) This ensure that a value is cleaned to be a valid Key value. ### Import Asset Advanced [](#import-asset-advanced) This allows two additional pieces of functionality when importing an asset: **Path** Uses the **Path Syntax** described above to store the asset in a specified folder. **URL Property** Specifies the name of the property on the asset to store the source URL the asset was captured from. ### Arithmetic [](#arithmetic) This allows you to apply addition, subtraction, multiplication, or division with a defined constant and your value. ### Regex Replace [](#regex-replace) This allows you to do string replaces using [preg\_replace](https://www.php.net/manual/en/function.preg-replace.php) Element Loading --------------- [](#element-loading) ### Advanced Path Strategy [](#advanced-path-strategy) This allows loading objects using the **Path** syntax described earlier in this ReadMe. Using the example Excel file in the **Path** section you could load the Data Object at `/Products/Cars/GMC/Sierra/2015` using Path Syntax `/Products/Cars/$[1]/$[2]/$[0]`. ### Property [](#property-1) This allows a data object to be loaded based on the value of a property stored on it. **This assumes that the property value is unique**. If a non-unique value exists, it'll be a random object returned that matches the criteria. Element Creation ---------------- [](#element-creation) ### Advanced Parent Strategy [](#advanced-parent-strategy) This allows locating objects using the **Path** syntax described earlier in this ReadMe. Using the example Excel file in the **Path** section you could create a Data Object with parent `/Products/Cars/GMC/Sierra/2015` using Path Syntax `/Products/Cars/$[1]/$[2]/$[0]`. License ======= [](#license) This bundle is licensed under the Pimcore Open Core License (POCL) and is intended for use with Pimcore Platform 2025.1 and newer. See LICENSE.md for full license text. ### Health Score 45 — FairBetter than 92% of packages Maintenance94 Actively maintained with recent releases Popularity6 Limited adoption so far Community14 Small or concentrated contributor base Maturity57 Maturing project, gaining track record Bus Factor3 3 contributors hold 50%+ of commits How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window. **Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores. **Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement. **Maturity (30%)** — Project age, version count, PHP version support, and release stability. ### Release Activity Cadence Every ~41 days Recently: every ~6 days Total 27 Last Release 59d ago Major Versions v1.4.0 → v2.0.02023-10-11 v2.0.0 → v3.0.02024-05-08 v3.1.1 → v4.0.02025-01-27 v4.5.0 → v5.0.02026-01-29 ### Community Maintainers ![](https://www.gravatar.com/avatar/85e9ffa5092017101650f30f5fedeb12da6d48344373f33d56e02fa9f1c2d92b?d=identicon)[torqnorth](/maintainers/torqnorth) --- Top Contributors [![cameronfromtorq](https://avatars.githubusercontent.com/u/105230931?v=4)](https://github.com/cameronfromtorq "cameronfromtorq (42 commits)")[![IronSean](https://avatars.githubusercontent.com/u/1960190?v=4)](https://github.com/IronSean "IronSean (32 commits)")[![jeremygnoonan](https://avatars.githubusercontent.com/u/33028237?v=4)](https://github.com/jeremygnoonan "jeremygnoonan (26 commits)")[![lukemacausland](https://avatars.githubusercontent.com/u/58705994?v=4)](https://github.com/lukemacausland "lukemacausland (13 commits)")[![rjjackson22](https://avatars.githubusercontent.com/u/38430762?v=4)](https://github.com/rjjackson22 "rjjackson22 (12 commits)")[![arrandavidfraser](https://avatars.githubusercontent.com/u/206713298?v=4)](https://github.com/arrandavidfraser "arrandavidfraser (10 commits)")[![Jonathon-Meney-Torq](https://avatars.githubusercontent.com/u/253076990?v=4)](https://github.com/Jonathon-Meney-Torq "Jonathon-Meney-Torq (8 commits)")[![msoroka](https://avatars.githubusercontent.com/u/25011263?v=4)](https://github.com/msoroka "msoroka (4 commits)")[![LoganMacKinnon](https://avatars.githubusercontent.com/u/61194412?v=4)](https://github.com/LoganMacKinnon "LoganMacKinnon (2 commits)")[![evanjamesjackson](https://avatars.githubusercontent.com/u/5866481?v=4)](https://github.com/evanjamesjackson "evanjamesjackson (1 commits)") ### Code Quality Static AnalysisPHPStan Code StylePHP CS Fixer Type Coverage Yes ### Embed Badge ![Health badge](/badges/torqnorth-data-importer-extensions-bundle/health.svg) ``` [![Health](https://phpackages.com/badges/torqnorth-data-importer-extensions-bundle/health.svg)](https://phpackages.com/packages/torqnorth-data-importer-extensions-bundle) ``` ### Alternatives [rap2hpoutre/fast-excel Fast Excel import/export for Laravel 2.3k24.9M47](/packages/rap2hpoutre-fast-excel)[pimcore/data-hub Pimcore Datahub 1351.3M9](/packages/pimcore-data-hub)[pimcore/customer-management-framework-bundle 82489.9k2](/packages/pimcore-customer-management-framework-bundle)[coreshop/core-shop CoreShop - Pimcore eCommerce 289197.5k10](/packages/coreshop-core-shop)[filament/actions Easily add beautiful action modals to any Livewire component. 1220.8M36](/packages/filament-actions)[yajra/laravel-datatables-export Laravel DataTables Queued Export Plugin. 341.9M3](/packages/yajra-laravel-datatables-export) PHPackages © 2026 [Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)