PHPackages                             starfruit/crawler-bundle - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. starfruit/crawler-bundle

ActiveLibrary

starfruit/crawler-bundle
========================

Starfruit Crawler Bundle

0.0.10(6mo ago)0181MITPHPCI passing

Since Sep 18Pushed 6mo agoCompare

[ Source](https://github.com/starfruit-php/crawler-bundle)[ Packagist](https://packagist.org/packages/starfruit/crawler-bundle)[ Docs](https://github.com/starfruit-php/crawler-bundle)[ RSS](/packages/starfruit-crawler-bundle/feed)WikiDiscussions master Synced 1mo ago

READMEChangelog (10)Dependencies (2)Versions (11)Used By (0)

Starfruit Crawler Bundle

[![Crawler UI](public/images/ui.png "Crawler UI")](public/images/ui.png)

Requirements
============

[](#requirements)

Google Cloud
------------

[](#google-cloud)

1. Create a new project then enable below libraries:
    - [Google Drive API](https://console.cloud.google.com/apis/library/drive.googleapis.com "Google Drive API")
    - [ Google Docs API](https://console.cloud.google.com/apis/library/docs.googleapis.com " Google Docs API")
2. Create a service account and download JSON credentials file

[![Google Cloud screenshort](public/images/googlecloud.png "Google Cloud screenshort")](public/images/googlecloud.png)

Installation
============

[](#installation)

```
    composer require starfruit/crawler-bundle
```

OR

```
    composer require starfruit/crawler-bundle --ignore-platform-req=ext-amqp
```

- Update `config/bundles.php` file:

```
    return [
        ....
        Starfruit\CrawlerBundle\StarfruitCrawlerBundle::class => ['all' => true],
    ];
```

Setup
=====

[](#setup)

- Create a new variable in `.env` file:

```
# path to file Google Cloud JSON, example:
CRAWLER_BUNDLE_GOOGLE_JSON=/root/project/public/crawler-google-credential.json
```

- Update `config/config.yaml` file:

```
imports:
        - { resource: 'local/' }

pimcore:
    ...
    ...

# config for crawler bundle
starfruit_crawler:
    target:
        class_object: # list of classname as key, and fields
            News: # name of class
                content_field: 'content' # field to paste crawled content
                last_version_field: 'importUrl' # field to store last version, can be null

            Event: # name of class
                content_field: 'mainContent'

    # custom asset path in Admin to store images, media
    asset_store_path: '/default-crawler-media/image'

    # custom format for html after crawling
    content_format:
        heading:
            # all default config to mapping headling value to html tag
            default: 'p' # default tag
            HEADING_1: 'h1'
            HEADING_2: 'h2'
            HEADING_3: 'h3'
            HEADING_4: 'h4'
```

###  Health Score

30

—

LowBetter than 65% of packages

Maintenance66

Regular maintenance activity

Popularity13

Limited adoption so far

Community2

Small or concentrated contributor base

Maturity32

Early-stage or recently created project

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~2 days

Total

10

Last Release

208d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/1d696716a6c3042e3c7243d1ddfa35c9a46cfe1602f1d6f3afe51e382f92047c?d=identicon)[Starfruit](/maintainers/Starfruit)

---

Tags

crawlerstarfruit

### Embed Badge

![Health badge](/badges/starfruit-crawler-bundle/health.svg)

```
[![Health](https://phpackages.com/badges/starfruit-crawler-bundle/health.svg)](https://phpackages.com/packages/starfruit-crawler-bundle)
```

###  Alternatives

[jaybizzle/crawler-detect

CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

2.3k96.7M215](/packages/jaybizzle-crawler-detect)[spatie/crawler

Crawl all internal links found on a website

2.8k16.3M52](/packages/spatie-crawler)[wa72/htmlpagedom

jQuery-inspired DOM manipulation extension for Symfony's Crawler

3383.9M34](/packages/wa72-htmlpagedom)[vdb/php-spider

A configurable and extensible PHP web spider

1.4k181.0k7](/packages/vdb-php-spider)[jaybizzle/laravel-crawler-detect

A Laravel package to detect web crawlers via the user agent

3232.6M17](/packages/jaybizzle-laravel-crawler-detect)[crwlr/crawler

Web crawling and scraping library.

37214.8k2](/packages/crwlr-crawler)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)