PHPackages blue-tomato/elasticsearch-feeder - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. [Search & Filtering](/categories/search) 4. / 5. blue-tomato/elasticsearch-feeder ActiveProcesswire-module[Search & Filtering](/categories/search) blue-tomato/elasticsearch-feeder ================================ Schema-flexible ProcessWire module for getting your page into ElasticSearch 2.5.0(6mo ago)101.9k↓100%1MITPHPPHP >=7.0 Since May 21Pushed 6mo ago3 watchersCompare [ Source](https://github.com/blue-tomato/ElasticsearchFeeder)[ Packagist](https://packagist.org/packages/blue-tomato/elasticsearch-feeder)[ Docs](https://github.com/blue-tomato/ElasticsearchFeeder)[ RSS](/packages/blue-tomato-elasticsearch-feeder/feed)WikiDiscussions master Synced 1mo ago READMEChangelog (10)Dependencies (1)Versions (26)Used By (0) ElasticsearchFeeder =================== [](#elasticsearchfeeder) [![License: MIT](https://camo.githubusercontent.com/08cef40a9105b6526ca22088bc514fbfdbc9aac1ddbf8d4e6c750e3a88a44dca/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4c6963656e73652d4d49542d626c75652e737667)](https://opensource.org/licenses/MIT)[![ProcessWire 3](https://camo.githubusercontent.com/34449e2237c61f5c40a0cbf06273d9f22ee4b9ba45442c77b6554c874927b1ea/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f50726f63657373576972652d332e782d6f72616e67652e737667)](https://github.com/processwire/processwire) This ElasticSearch module for [ProcessWire CMS/CMF](http://processwire.com/) will sync your page content to an ElasticSearch index, which provides you a convenient way to search it. Table of Contents ----------------- [](#table-of-contents) - [Features](#features) - [Prerequisites](#prerequisites) - [Installation](#installation) - [Usage](#usage) - [Module Configuration](#configuration) - [Schema](#schema) - [Support](#support) - [Contributing](#contributing) - [License](#license) Features -------- [](#features) - Batch add / updates pages in ElasticSearch - Add / update page in ElasticSearch after page save/publish - Remove page from ElasticSearch after trash, delete or unpublish page Prerequisites ------------- [](#prerequisites) Before you'll start using this module, make sure it's compatible with your technical ElasticSearch setup and that it's fulfills your content indexing requires. We've created this module to support a `bonsai.io` (alternatives: [AWS ES](https://aws.amazon.com/de/elasticsearch-service/), [Elastic Cloud](https://www.elastic.co/cloud), [Google Cloud](https://console.cloud.google.com/marketplace/details/google/elasticsearch), etc.) hosted ElasticSearch SaaS instance. It should also work with own-hosted ElasticSearch instances or within Docker-Containers. Usually ES SaaS providers inform about this in a setup/configuration section in their backend. E.g. for bonsai.io: `https://{ES_ACCESS_KEY}:{ES_ACCESS_SECRET}@{ES_INSTANCE_URL}.bonsaisearch.net` E.g. for local ElasticSearch: `http://localhost:9200` Installation ------------ [](#installation) 1. Execute the following command in the root directory of your ProcessWire installation: ``` composer require blue-tomato/elasticsearch-feeder ``` 2. ProcessWire will detect the module and list it in the backend's `Modules` > `Site` > `ElasticsearchFeeder` section. Navigate there and install it. Usage ----- [](#usage) To use this module you'll have to setup some module configurations and schema instructions. *NOTE: This module will save time of the last update for each page and the document ID in a meta field called `elasticsearchfeeder_meta`. You can access this value with `$page->meta('elasticsearchfeeder_meta')`* ### Configuration [](#configuration) Configure the module in your ProcessWire module backend (which will be available after the module was installed). The module configuration enables to do the following via a ProcessWire-driven backend form: - define ES backend protocol (http or https) - insert `ES_ACCESS_HOST` - insert `ES_ACCESS_KEY` (optional) - insert `ES_ACCESS_SECRET` (optional) - insert the path to your schema (see [Schema](#schema) to see how those work) - optionally insert a prefix string that'll be used when hashing your ES ids - insert template configurations (see [Schema](#schema) to see how those work) - (re)index all pages matching our module configuration by clicking the "Index All Pages" button. *NOTE: using this option can take quite a long time (primarily depending on how much ProcessWire pages you're going to send to the ElasticSearch index).* - or use the batchSync.php script (description below) ### Schema [](#schema) Setup a schema or multiple schemas to define which content(s) will be shipped to your ElasticSearch instance. Consider to place your schema files with a `.schema.php` file ending in the directory path declared when configuring your module in the ProcessWire backend. Basically said: for each ElasticSearch document type, there must be a PHP function returning the contents to be indexed in your ElasticSearch instance. The naming convention of this function has to be the `camelCased` document type name you declare in the ProcessWire backend module configuration. So i.e.: a document type named **news-details-page** in the ProcessWire backend requires schema function named **newsDetailsPage**. The filename itself has to be the same name as the template name. I.e.: **news-details-page.php** should be **news-details-page.schema.php** #### Page Filtering in Schema [](#page-filtering-in-schema) If a Schema Function returns `false` as value, the page will not be sent to ElasticSearch. You can use this for filtering your pages and sending only specific pages from this template to ElasticSearch. #### Schema Function (i.e. news-details-page.schema.php) [](#schema-function-ie-news-details-pageschemaphp) This module passes the following arguments to your schema function. - @arg1 `$page` the ProcessWire page WireArray ``` function newsDetailsPage($page) { // don't send page to Elasticsearch in case we don't want to if($page->property->value == "xyz") { return false; } // now start building the $document array you want to ship to Elasicsearch $document = []; $document['type'] = 'news-detailspage'; $document['name'] = $page->title; // if you need to generate an ElasticSeach ID withtin a schema file $ElasticsearchFeeder = wire('modules')->get('ElasticsearchFeeder'); $indexPrefix = $ElasticsearchFeeder->getIndexPrefix(); $document['some_other_page'] = $ElasticsearchFeeder->createElasticSearchDocumentHashedId($page->get('some_other_page')->id, $indexPrefix), // send $document back to ElasticsearchFeeder module return $document; } ``` #### Schema Function Debugging [](#schema-function-debugging) This module provides a convenient way to debug your schema and thus see what's being sent to ElasticSearch. You can see this after saving a page in the ProcessWire admin message bar as JSON output. To enable module debugging: - go to the module configuration page in `Modules` > `Site` > `ElasticsearchFeeder` - find the debug config area like shown below [![](docs/images/debugModule.png)](docs/images/debugModule.png) ### Batch Sync via CLI or Cron [](#batch-sync-via-cli-or-cron) You can send your pages through the "Index all Pages" button in the module configuration page in `Modules` > `Site` > `ElasticsearchFeeder`. If you have many pages, this can run very long and it can cause server timeouts. For this reason you can use the `batchSync.php` script in the module path via command line. You can also set up a repeating cronjob to ensure a full sync every *x* times. The `batchSync.php` script send all pages to ElasticSearch. I.e: ``` php site/modules/ElasticsearchFeeder/batchSync.php ``` To check if in ElasticSearch existing also still exists in ProcessWire and are public available use this script: ``` php site/modules/ElasticsearchFeeder/batchCleanup.php ``` ### Request to ElasticSearch through a proxy server [](#request-to-elasticsearch-through-a-proxy-server) If you have your Server behind a proxy, you can add to your `config.php` file following properties: - `$config->httpProxy = "your-http-proxy-server.xyz:8888";` - `$config->httpsProxy = "your-https-proxy-server.xyz:5394";` ### Override ElasticSearch Connection through config.php [](#override-elasticsearch-connection-through-configphp) You can override your Connection to ElasticSearch through `$config->elasticsearchFeederConnectionOverride`. E.g. if you want to work local or on stage Servers with your local ElasticSearch Server but let the Database values untouched. ``` $config->elasticsearchFeederConnectionOverride = [ "es_protocol" => "http", "es_host" => "localhost:9200", "es_access_key" => "", "es_access_secret" => "" ]; ``` ### Deactivate ElasticSearchFeeder through config.php [](#deactivate-elasticsearchfeeder-through-configphp) If you want to prevent to send pages to ElasticSearch from your development or staging server but don't want to deactivate the module in the database, you can add `$config->elasticsearchFeederDisabled = true` to your `config.php` or `config-dev.php` file. This will prevent of adding the necessary hooks for the indexation. ### Show ElasticSearch Meta Information / Status in ProcessWire Admin Panel [](#show-elasticsearch-meta-information--status-in-processwire-admin-panel) Since we save the document-ID and the last time when the page is sent to the index in the `$page-meta()`, you can show them also in the admin panel with [MarkupRuntime Module](https://modules.processwire.com/modules/fieldtype-runtime-markup/). Here is an example configuration for MarkupRuntime: [![](docs/images/markupRuntimeConfig.png)](docs/images/markupRuntimeConfig.png)[![](docs/images/markupRuntimeField.png)](docs/images/markupRuntimeField.png) ElasticSearch Version and Document-Type --------------------------------------- [](#elasticsearch-version-and-document-type) Please consider that ElasticSearch 6.0.0 removed the support for multiple document-types in one and the same index. We support both variants with this module. You can define for each ProcessWire template seperate index and document-type names. I.e. if you use ElasticSearch => 6.0.0, you can use the same name for index and document-type. Support ------- [](#support) Please [open an issue](https://github.com/blue-tomato/ElasticsearchFeeder/issues/new) for support. Contributing ------------ [](#contributing) Create a branch on your fork, add commits to your fork, and open a pull request from your fork to this repository. To get better insights and onboard you on module implementation details just open a support issue. We'll get back to you asap. Credits ------- [](#credits) This module is made by people from Blue Tomato. If you want to read more about our work, follow us on License ------- [](#license) Find all information about this module's license in the LICENCE.txt file. ### Health Score 46 — FairBetter than 93% of packages Maintenance69 Regular maintenance activity Popularity26 Limited adoption so far Community9 Small or concentrated contributor base Maturity64 Established project with proven stability Bus Factor1 Top contributor holds 100% of commits — single point of failure How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window. **Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores. **Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement. **Maturity (30%)** — Project age, version count, PHP version support, and release stability. ### Release Activity Cadence Every ~102 days Recently: every ~399 days Total 24 Last Release 183d ago Major Versions 1.5.0 → 2.0.02022-06-07 ### Community Maintainers ![](https://avatars.githubusercontent.com/u/41725719?v=4)[Blue Tomato](/maintainers/blue-tomato)[@blue-tomato](https://github.com/blue-tomato) --- Top Contributors [![tiefenb](https://avatars.githubusercontent.com/u/488163?v=4)](https://github.com/tiefenb "tiefenb (154 commits)") --- Tags elasticsearchprocesswireelasticsearchprocesswireelasticbonsai ### Embed Badge ![Health badge](/badges/blue-tomato-elasticsearch-feeder/health.svg) ``` [![Health](https://phpackages.com/badges/blue-tomato-elasticsearch-feeder/health.svg)](https://phpackages.com/packages/blue-tomato-elasticsearch-feeder) ``` ### Alternatives [elasticsearch/elasticsearch PHP Client for Elasticsearch 5.3k178.3M943](/packages/elasticsearch-elasticsearch)[10up/elasticpress Supercharge WordPress with Elasticsearch. 1.3k374.3k6](/packages/10up-elasticpress)[babenkoivan/elastic-scout-driver Elasticsearch driver for Laravel Scout 2773.8M5](/packages/babenkoivan-elastic-scout-driver)[babenkoivan/elastic-scout-driver-plus Extension for Elastic Scout Driver 2862.8M1](/packages/babenkoivan-elastic-scout-driver-plus)[jeroen-g/explorer Next-gen Elasticsearch driver for Laravel Scout. 397612.3k](/packages/jeroen-g-explorer)[babenkoivan/elastic-migrations Elasticsearch migrations for Laravel 1962.5M6](/packages/babenkoivan-elastic-migrations) PHPackages © 2026 [Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)