PHPackages                             webhubworks/site-crawler - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Framework](/categories/framework)
4. /
5. webhubworks/site-crawler

ActiveProject[Framework](/categories/framework)

webhubworks/site-crawler
========================

A straightforward site crawler

3.2.3(1mo ago)148MITPHPPHP ^8.5

Since Nov 12Pushed 1mo ago1 watchersCompare

[ Source](https://github.com/webhubworks/site-crawler)[ Packagist](https://packagist.org/packages/webhubworks/site-crawler)[ Docs](https://webhub.de)[ RSS](/packages/webhubworks-site-crawler/feed)WikiDiscussions main Synced 1w ago

READMEChangelog (3)Dependencies (35)Versions (35)Used By (0)

Site Crawler
============

[](#site-crawler)

Use this site crawler as a quick way to crawl any website. This is useful to detect any slow pages or pages with HTTP errors.

Please use this crawler responsibly. Do not use it to crawl websites that you do not own or have permission to crawl.

Installation
------------

[](#installation)

Run `composer global require webhubworks/site-crawler -W` in your terminal. After that, running `site-crawler` should output the version and command list.

Development
-----------

[](#development)

- To run the crawler locally (instead of using the globally installed version): `php site-crawler app:crawl URL`
- To build the standalone app, run `php site-crawler app:build site-crawler` and specify the next version.

Usage
-----

[](#usage)

Run `site-crawler` to get a list of all available crawling commands.

Example: `site-crawler crawl:url https://example.com --limit=50 --basic-auth=user:pass --exclude=action,imprint`

Roadmap
-------

[](#roadmap)

- Add support for websites containing links in JS generated markup
- Run requests in parallel

###  Health Score

49

—

FairBetter than 94% of packages

Maintenance93

Actively maintained with recent releases

Popularity12

Limited adoption so far

Community10

Small or concentrated contributor base

Maturity69

Established project with proven stability

 Bus Factor1

Top contributor holds 74.5% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~18 days

Recently: every ~2 days

Total

34

Last Release

32d ago

Major Versions

1.1.1 → 2.1.02025-10-13

1.2.0 → 2.1.12025-10-13

1.2.1 → 2.2.02026-01-29

1.3.0 → 2.3.02026-01-30

2.3.1 → 3.0.02026-03-30

PHP version history (3 changes)1.0.0PHP ^8.2.0

2.0.0PHP ^8.4

3.0.0PHP ^8.5

### Community

Maintainers

![](https://avatars.githubusercontent.com/u/53560154?v=4)[webhub](/maintainers/webhubworks)[@webhubworks](https://github.com/webhubworks)

---

Top Contributors

[![MariusKnirk](https://avatars.githubusercontent.com/u/111346131?v=4)](https://github.com/MariusKnirk "MariusKnirk (38 commits)")[![marventhieme](https://avatars.githubusercontent.com/u/53627227?v=4)](https://github.com/marventhieme "marventhieme (8 commits)")[![rostockahoi](https://avatars.githubusercontent.com/u/16897125?v=4)](https://github.com/rostockahoi "rostockahoi (5 commits)")

---

Tags

cliconsoleframeworklaravellaravel-zero

###  Code Quality

TestsPest

Code StyleLaravel Pint

### Embed Badge

![Health badge](/badges/webhubworks-site-crawler/health.svg)

```
[![Health](https://phpackages.com/badges/webhubworks-site-crawler/health.svg)](https://phpackages.com/packages/webhubworks-site-crawler)
```

###  Alternatives

[laravel/mcp

Rapidly build MCP servers for your Laravel applications.

79227.1M228](/packages/laravel-mcp)[mehrancodes/laravel-harbor

A CLI tool to Quickly create On-Demand preview environment for your apps.

99100.2k](/packages/mehrancodes-laravel-harbor)[laravel-zero/laravel-zero

The Laravel Zero Framework.

4.0k44.6k3](/packages/laravel-zero-laravel-zero)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)