PHPackages                             mehrabx/web-crawler - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. mehrabx/web-crawler

ActiveLibrary[Utility & Helpers](/categories/utility)

mehrabx/web-crawler
===================

A web crawler package

v1.0(4y ago)411PHP

Since Dec 1Pushed 4y ago1 watchersCompare

[ Source](https://github.com/mehrabx/php-crawler)[ Packagist](https://packagist.org/packages/mehrabx/web-crawler)[ RSS](/packages/mehrabx-web-crawler/feed)WikiDiscussions master Synced 3w ago

READMEChangelogDependencies (1)Versions (2)Used By (0)

 [ ![Laravel Toman](./resources/imgs/logo.png?raw=true) ](https://mehrabx.github.io/web-crwaler/)

PHP Web Crawler
===============

[](#php-web-crawler)

This library is a php web crawler which takes collection of URLs and DOM selects to crawl through the webpages and executing customized analyzers on each page.

Installation
------------

[](#installation)

Install this library using composer :

```
composer require mehrabx/web-crawler
```

Usage
-----

[](#usage)

In current version use [xpath expressions](https://www.w3schools.com/xml/xpath_intro.asp) to select element

```
//set list of URLs and selects DOM elements of each URL page
$urls = [
    'https://test.exp/?page=1' => ["//img[@class='type1']","//a[@class='type1']"],
    'https://test.exp/?page=2' => ["//img[@class='type2'"],
    'https://test.exp/?page=3' => "//img[@class='type3']",
];

//return array of results
return \Crawler\Facades\CrawlFacade::make($urls)->start() ;
```

options
-------

[](#options)

### sleep

[](#sleep)

To avoid being blocked by the target url you can set sleep time between crawling each url :

```
$urls = [
    'https://test.exp/?page=1' => ["//img[@class='type1']","//a[@class='type1']"],
    'https://test.exp/?page=2' => ["//img[@class='type2'"],
];

//set 5 seconds sleep time
return \Crawler\Facades\CrawlFacade::make($urls)->sleep(10)->start() ;
```

### defualt select

[](#defualt-select)

You can set default select. URLs that have no selects can use it :

```
$urls = [
    'https://test.exp/?page=1', //this url has not select
    'https://test.exp/?page=2' => ["//img[@class='type2'"],
];

return \Crawler\Facades\CrawlFacade::make($urls)
                                    ->defaultSelect("//img[@class='type1']")
                                    ->start() ;
```

###  Health Score

24

—

LowBetter than 31% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity9

Limited adoption so far

Community7

Small or concentrated contributor base

Maturity50

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Unknown

Total

1

Last Release

1676d ago

### Community

Maintainers

![](https://avatars.githubusercontent.com/u/83074328?v=4)[Mehrab MZ](/maintainers/mehrabx)[@mehrabx](https://github.com/mehrabx)

---

Top Contributors

[![mehrabx](https://avatars.githubusercontent.com/u/83074328?v=4)](https://github.com/mehrabx "mehrabx (14 commits)")

### Embed Badge

![Health badge](/badges/mehrabx-web-crawler/health.svg)

```
[![Health](https://phpackages.com/badges/mehrabx-web-crawler/health.svg)](https://phpackages.com/packages/mehrabx-web-crawler)
```

###  Alternatives

[aws/aws-sdk-php

AWS SDK for PHP - Use Amazon Web Services in your PHP project

6.3k543.5M2.6k](/packages/aws-aws-sdk-php)[neuron-core/neuron-ai

The PHP Agentic Framework.

2.0k656.1k38](/packages/neuron-core-neuron-ai)[tencentcloud/tencentcloud-sdk-php

TencentCloudApi php sdk

3741.3M46](/packages/tencentcloud-tencentcloud-sdk-php)[civicrm/civicrm-core

Open source constituent relationship management for non-profits, NGOs and advocacy organizations.

751291.4k43](/packages/civicrm-civicrm-core)[spatie/laravel-export

Create a static site bundle from a Laravel app

674146.0k6](/packages/spatie-laravel-export)[oat-sa/tao-core

TAO core extension

66143.7k124](/packages/oat-sa-tao-core)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)