PHPackages                             webhubworks/site-crawler - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Framework](/categories/framework)
4. /
5. webhubworks/site-crawler

ActiveProject[Framework](/categories/framework)

webhubworks/site-crawler
========================

A straightforward site crawler

3.1.3(2mo ago)140MITPHPPHP ^8.5

Since Nov 12Pushed 2mo ago1 watchersCompare

[ Source](https://github.com/webhubworks/site-crawler)[ Packagist](https://packagist.org/packages/webhubworks/site-crawler)[ Docs](https://webhub.de)[ RSS](/packages/webhubworks-site-crawler/feed)WikiDiscussions main Synced today

READMEChangelog (3)Dependencies (21)Versions (30)Used By (0)

Site Crawler
============

[](#site-crawler)

Use this site crawler as a quick way to crawl any website. This is useful to detect any slow pages or pages with HTTP errors.

Please use this crawler responsibly. Do not use it to crawl websites that you do not own or have permission to crawl.

Installation
------------

[](#installation)

`composer global require webhubworks/site-crawler`

Development
-----------

[](#development)

- To run the crawler locally (instead of using the globally installed version): `php site-crawler app:crawl URL`
- To build the standalone app, run `php site-crawler app:build site-crawler` and specify the next version.

Usage
-----

[](#usage)

Use the help: `site-crawler --help`

Example: `site-crawler https://example.com --limit=50 --basic-auth=user:pass --exclude=action,imprint`

Roadmap
-------

[](#roadmap)

- Add support for websites containing links in JS generated markup
- Run requests in parallel

###  Health Score

46

—

FairBetter than 92% of packages

Maintenance85

Actively maintained with recent releases

Popularity9

Limited adoption so far

Community10

Small or concentrated contributor base

Maturity68

Established project with proven stability

 Bus Factor1

Top contributor holds 74% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~18 days

Recently: every ~4 days

Total

29

Last Release

78d ago

Major Versions

1.1.1 → 2.1.02025-10-13

1.2.0 → 2.1.12025-10-13

1.2.1 → 2.2.02026-01-29

1.3.0 → 2.3.02026-01-30

2.x-dev → 3.0.02026-03-30

PHP version history (3 changes)1.0.0PHP ^8.2.0

2.0.0PHP ^8.4

3.0.0PHP ^8.5

### Community

Maintainers

![](https://avatars.githubusercontent.com/u/53560154?v=4)[webhub](/maintainers/webhubworks)[@webhubworks](https://github.com/webhubworks)

---

Top Contributors

[![MariusKnirk](https://avatars.githubusercontent.com/u/111346131?v=4)](https://github.com/MariusKnirk "MariusKnirk (37 commits)")[![marventhieme](https://avatars.githubusercontent.com/u/53627227?v=4)](https://github.com/marventhieme "marventhieme (8 commits)")[![rostockahoi](https://avatars.githubusercontent.com/u/16897125?v=4)](https://github.com/rostockahoi "rostockahoi (5 commits)")

---

Tags

cliconsoleframeworklaravellaravel-zero

###  Code Quality

TestsPest

Code StyleLaravel Pint

### Embed Badge

![Health badge](/badges/webhubworks-site-crawler/health.svg)

```
[![Health](https://phpackages.com/badges/webhubworks-site-crawler/health.svg)](https://phpackages.com/packages/webhubworks-site-crawler)
```

###  Alternatives

[laravel/mcp

Rapidly build MCP servers for your Laravel applications.

77022.3M151](/packages/laravel-mcp)[mehrancodes/laravel-harbor

A CLI tool to Quickly create On-Demand preview environment for your apps.

10097.5k](/packages/mehrancodes-laravel-harbor)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)