PHPackages                             dezento/crawlify - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. dezento/crawlify

ActiveLibrary[Utility & Helpers](/categories/utility)

dezento/crawlify
================

Fast Concurrent Crawler

1.0(4y ago)724MITPHPPHP ^8.0

Since May 31Pushed 3y ago1 watchersCompare

[ Source](https://github.com/Dezento/crawlify)[ Packagist](https://packagist.org/packages/dezento/crawlify)[ RSS](/packages/dezento-crawlify/feed)WikiDiscussions main Synced 1mo ago

READMEChangelog (1)Dependencies (6)Versions (2)Used By (0)

Crawlify
--------

[](#crawlify)

### Installation

[](#installation)

```
composer require dezento/crawlify

```

### Overview

[](#overview)

Crawlify is a lightweight crawler for manipulating HTML,XML and JSON using [DomCrawler](https://symfony.com/doc/current/components/dom_crawler.html). It uses [GuzzleHttp\\Pool](https://docs.guzzlephp.org/en/stable/quickstart.html#concurrent-requests) to make concurrent request and that means you can use all [Request Options](https://docs.guzzlephp.org/en/stable/request-options.html) available.
The result it gives back is wrapped with [Laravel Collections](https://laravel.com/docs/8.x/collections).

### Examples

[](#examples)

##### CRAWL JSON

[](#crawl-json)

```
use Dezento\Crawlify;

$links = [];
for ($i = 1; $i settings([
  'type' => 'JSON'  //this is Crawlify Option
])
->fetch()
->get('fulfilled')
->map(fn ($p) => collect(json_decode($p->response)))
->dd();

```

##### CRAWL XML

[](#crawl-xml)

For traversing XML refer to [DomCrawler](https://symfony.com/doc/current/components/dom_crawler.html) documentation.

```
$xml = (new Crawlify([
    'https://www.nytimes.com/svc/collections/v1/publish/https://www.nytimes.com/section/world/rss.xml',
]))
->fetch()
->get('fulfilled')
->map(fn ($item) =>
  collect($item->response->filter('item')->children())
  ->map(fn ($data) => $data->textContent)
)->dd();

```

##### CRAWL HTML

[](#crawl-html)

For traversing HTML refer to [DomCrawler](https://symfony.com/doc/current/components/dom_crawler.html) documentation.

```
$html = (new Crawlify([
  'https://en.wikipedia.org/wiki/Category:Lists_of_spider_species_by_family'
]))
->settings([
  #'proxy' => 'http://username:password@192.168.16.1:10',
  'concurrency' => 5,
  'delay' => 0
])
->fetch()
->get('fulfilled')
->map(fn ($item) =>
  collect($item->response->filter('a')->links())
  ->map(fn($el) => $el->getUri())
)
->reject(fn($a) => $a->isEmpty())
->dd();

```

##### OPTIONS

[](#options)

```
->settings([
  'proxy' => 'http://username:password@192.168.16.1:10',
  'concurrency' => 5,
  'delay' => 0,
  ....
])

```

For options you can refer to [Request Options](https://docs.guzzlephp.org/en/stable/request-options.html) documentation. The only Crawlify custom options is ` 'type' => 'JSON'`

#### Note

[](#note)

Before using `dd()` helper you must install it.

` composer require symfony/var-dumper`

###  Health Score

26

—

LowBetter than 43% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity12

Limited adoption so far

Community7

Small or concentrated contributor base

Maturity56

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Unknown

Total

1

Last Release

1812d ago

### Community

Maintainers

![](https://avatars.githubusercontent.com/u/101515547?v=4)[comidev](/maintainers/comiDev)[@comidev](https://github.com/comidev)

---

Top Contributors

[![Dezento](https://avatars.githubusercontent.com/u/15669109?v=4)](https://github.com/Dezento "Dezento (11 commits)")

---

Tags

crawlerhacktoberfestscraping

### Embed Badge

![Health badge](/badges/dezento-crawlify/health.svg)

```
[![Health](https://phpackages.com/badges/dezento-crawlify/health.svg)](https://phpackages.com/packages/dezento-crawlify)
```

###  Alternatives

[craftcms/cms

Craft CMS

3.6k3.6M2.6k](/packages/craftcms-cms)[spatie/laravel-pjax

A pjax middleware for Laravel 5

513371.8k11](/packages/spatie-laravel-pjax)[ashallendesign/favicon-fetcher

A Laravel package for fetching website's favicons.

190272.4k3](/packages/ashallendesign-favicon-fetcher)[glhd/conveyor-belt

14797.0k](/packages/glhd-conveyor-belt)[aedart/athenaeum

Athenaeum is a mono repository; a collection of various PHP packages

245.2k](/packages/aedart-athenaeum)[visuellverstehen/statamic-classify

A useful helper to add CSS classes to all HTML tags generated by the bard editor.

20116.8k](/packages/visuellverstehen-statamic-classify)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)