PHPackages                             travy/web-crawler - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. travy/web-crawler

ActiveLibrary[Utility & Helpers](/categories/utility)

travy/web-crawler
=================

A web crawler which will traverse links found in all parent and child web pages running and performs tasks on each page.

07PHP

Since Sep 26Pushed 8y ago1 watchersCompare

[ Source](https://github.com/travy/web-crawler)[ Packagist](https://packagist.org/packages/travy/web-crawler)[ RSS](/packages/travy-web-crawler/feed)WikiDiscussions master Synced 3d ago

READMEChangelogDependenciesVersions (2)Used By (0)

Web Crawler
===========

[](#web-crawler)

Description
-----------

[](#description)

*This is a work in progress and is not yet complete*

Web Crawler is an open source technology which will enable users to crawl through the a collection of webpages and executing customized analyzers on each page.

Installation
------------

[](#installation)

Add the library to your PHP project using composer.

```
composer require travy/web-crawler
```

Use Case
--------

[](#use-case)

The Crawler will automatically pull all URL addresses listed under an HTML anchor tag on the root URL. Each page that is visited will be run through a collection of Analyzers. These Analyzers can perform various tasks needed for the use of the application such as pruning the markup in order to build a search engine, or almost anything else that can be analyzed.

### Custom Analyzer

[](#custom-analyzer)

Analyzers can be created by extending the `AbstractAnalyzer` class

```
class MyAnalyzer extends AbstractAnalyzer
{
    public function analyze($url, $html, Dom $parser)
    {
        //  perform tasks
    }
}
```

### Analyzer Registry

[](#analyzer-registry)

The `AnalyzerRegistry` will contain a list of all Analyzers that should be used while crawling the web. Each analyzer will be assigned a unique key so that fields can be manipulated if needed.

```
$analyzer = new MyAnalyzer();

$analyzerRegistry = new AnalyzerRegistry();
$analyzerRegistry->registrer($analyzer, 'add-to-database');

$crawler = new Crawler('https://google.com', $analyzerRegsitry);
$crawler->crawl();
```

###  Health Score

20

—

LowBetter than 14% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity4

Limited adoption so far

Community7

Small or concentrated contributor base

Maturity43

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

### Community

Maintainers

![](https://www.gravatar.com/avatar/e2e17c26cd91563daf8cfe8e9910f2179aea067ccc7e814958a3b818a34433d3?d=identicon)[travy](/maintainers/travy)

---

Top Contributors

[![travy](https://avatars.githubusercontent.com/u/4945102?v=4)](https://github.com/travy "travy (8 commits)")

### Embed Badge

![Health badge](/badges/travy-web-crawler/health.svg)

```
[![Health](https://phpackages.com/badges/travy-web-crawler/health.svg)](https://phpackages.com/packages/travy-web-crawler)
```

###  Alternatives

[spatie/laravel-feed

Generate rss feeds

9743.6M28](/packages/spatie-laravel-feed)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)