PHPackages                             r4j4h/php-druid-ingest - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Database & ORM](/categories/database)
4. /
5. r4j4h/php-druid-ingest

ActiveLibrary[Database & ORM](/categories/database)

r4j4h/php-druid-ingest
======================

PHP Classes to wrap ingestion of data into Druid

0.1.0(11y ago)4623[1 PRs](https://github.com/r4j4h/php-druid-ingest/pulls)MITPHPPHP >=5.3.0

Since Dec 5Pushed 9y ago2 watchersCompare

[ Source](https://github.com/r4j4h/php-druid-ingest)[ Packagist](https://packagist.org/packages/r4j4h/php-druid-ingest)[ RSS](/packages/r4j4h-php-druid-ingest/feed)WikiDiscussions master Synced 1mo ago

READMEChangelogDependencies (8)Versions (4)Used By (0)

php-druid-ingest
================

[](#php-druid-ingest)

Experimental PHP wrapper around ingesting data from a variety of data sources into [Druid](http://druid.io) as a data source.

Overview
--------

[](#overview)

The wrapper lives in the namespace `PhpDruidIngest`.

Classes provide for the various tasks related to the extraction, transformation, and loading of data from other sources into druid. This involves:

1. the extraction, transformation, and loading of that data
2. preparing it in a place and format ready for Druid
3. the generation of a compatible Druid indexing task
4. the execution of the indexing task
5. usage of the returned job id for monitoring of the indexing task job, and
6. removal of prepared ingestion data after Druid has finished ingesting the file.

When executed, this will need to live on the Druid node that will ingest the data, using `LocalFilePreparer`. Otherwise it will need a way to move or stream the file from itself to the destination Node (say via `scp`). `RemoteSCPPreparer` is an initial stab at this.

Design
------

[](#design)

1. Instantiate a `IFetcher`, configured to fetch the desired records for the desired time periods.
2. Instantiate a `IPreparer` to record the results in memory or in file, tranfser results to destination, returning the destination path
3. Instantiate a `IDruidQueryParameters`, configured with parameters. <-- was dimension definition, now index task params + path from IFetcher
4. Instantiate a `IDruidQueryExecutor`, configured to hit a Druid endpoint.
5. Instantiate a `IDruidQueryGenerator`. <-- index task generator
6. Instantiate a `IDruidQueryResponseHandler`. <-- gets the task id
7. Run the `IDruidQueryExecutor`'s `executeQuery` function with the IDruidQuery, getting the result.
8. Hand the resulting task id to `IDruidJobWatcher` who polls until task succeeds or finishes
9. `IPreparer` then cleans up left over ingestion file

Fetchers are the most interesting element in play here. By adding new Fetchers we can support new input sources. Initially we are using `mysqli` to handle fetching from MySQL databases. Fetching from HTTP endpoints, or a log, or running map/reduce or storm and getting the results results are all good ideas for other fetchers.

Please refer to this diagram for an overview of how this works underneath the hood:

[![Process Flow](docs/process-flow.png)](docs/process-flow.png)

(From this [Dynamic LucidChart Source URL](https://www.lucidchart.com/publicSegments/view/542c8edd-b500-4dc7-b37f-7a010a0048fd/image.png))

How to Test
-----------

[](#how-to-test)

From the root directory, in a command terminal run: `php vendor/bin/phpunit tests` or more preferably `php vendor/bin/phing`.

Generate Documentation
----------------------

[](#generate-documentation)

From the root directory, in a command terminal run: `php vendor/bin/phing docs`.

How to Install
--------------

[](#how-to-install)

Right now, there is no tagged version. To be ready for it when it comes, branch-aliases are in place.

- Stable branch: `~1.0@dev`
- Stable branch w/ PHP 5.3 Compatibility Support: `dev-php-53-compat`
- Cutting edge: `~1.1@dev`

To install, it is suggested to use [Composer](http://getcomposer.org). If you have it installed, then the following instructions in a composer.json should be all you need to get started:

If you are using PHP 5.3, there [is](https://bugs.php.net/bug.php?id=66818) [a](http://php.net/archive/2014.php#id2014-08-14-1) [bug](https://bugs.php.net/bug.php?id=43200) and you will need to use an alternative branch.

Up to date PHP:

```
{
    "repositories": [
        {
            "type": "vcs",
            "url": "git@github.com:r4j4h/php-druid-ingest"
        }
    ],
    "require": {
        "r4j4h/php-druid-ingest": "~1.0@dev"
    }
}
```

PHP 5.3 Compatibility:

```
{
    "repositories": [
        {
            "type": "vcs",
            "url": "git@github.com:r4j4h/php-druid-ingest"
        }
    ],
    "require": {
        "r4j4h/php-druid-ingest": "dev-php-53-compat"
    }
}
```

Once that is in, `composer install` and `composer update` should work.

Once those are run, require Composer's autoloader and you are off to the races:

1. `require 'vendor/autoload.php';`

References
----------

[](#references)

- [php-druid-query](https://github.com/r4j4h/php-druid-query)
- [Druid](http://druid.io)
- [Composer](http://getcomposer.org)
- [Guzzle](http://guzzle.readthedocs.org)

###  Health Score

26

—

LowBetter than 43% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity15

Limited adoption so far

Community10

Small or concentrated contributor base

Maturity50

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Unknown

Total

1

Last Release

4182d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/84c480751333fa4e6bb8de34b6254ed66d421c576bc3f5b50795df692917b31a?d=identicon)[r4j4h](/maintainers/r4j4h)

---

Top Contributors

[![r4j4h](https://avatars.githubusercontent.com/u/703007?v=4)](https://github.com/r4j4h "r4j4h (84 commits)")

---

Tags

indexingdruidingestionoverlord

###  Code Quality

TestsPHPUnit

### Embed Badge

![Health badge](/badges/r4j4h-php-druid-ingest/health.svg)

```
[![Health](https://phpackages.com/badges/r4j4h-php-druid-ingest/health.svg)](https://phpackages.com/packages/r4j4h-php-druid-ingest)
```

###  Alternatives

[kimai/kimai

Kimai - Time Tracking

4.6k7.4k1](/packages/kimai-kimai)[guikingone/scheduler-bundle

A Symfony bundle that allows to schedule and create repetitive tasks

114217.4k](/packages/guikingone-scheduler-bundle)[basemkhirat/elasticsearch

Laravel, Lumen and Native php elasticseach query builder to build complex queries using an elegant syntax

402312.0k](/packages/basemkhirat-elasticsearch)[bartlett/php-compatinfo-db

Reference Database of all functions, constants, classes, interfaces on PHP standard distribution and about 110 extensions

1183.0k1](/packages/bartlett-php-compatinfo-db)[perplorm/perpl

Perpl is an improved and still maintained fork of Propel2, an open-source Object-Relational Mapping (ORM) for PHP.

203.7k](/packages/perplorm-perpl)[matchory/elasticsearch

The missing elasticsearch ORM for Laravel!

3059.0k](/packages/matchory-elasticsearch)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)