PHPackages                             l1n6yun/crawler - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Framework](/categories/framework)
4. /
5. l1n6yun/crawler

ActiveLibrary[Framework](/categories/framework)

l1n6yun/crawler
===============

swoole crawler frame

v0.0.3(1y ago)02MITPHP

Since Dec 3Pushed 7mo ago1 watchersCompare

[ Source](https://github.com/l1n6yun/crawler)[ Packagist](https://packagist.org/packages/l1n6yun/crawler)[ Docs](https://github.com/l1n6yun/crawler)[ RSS](/packages/l1n6yun-crawler/feed)WikiDiscussions master Synced 1mo ago

READMEChangelogDependencies (2)Versions (4)Used By (0)

Crawler
=======

[](#crawler)

一个基于 Swoole 协程的高效爬虫框架，旨在快速实现网络爬虫任务。

特性
--

[](#特性)

- **协程支持**：利用 Swoole 的协程特性，提高爬虫效率。
- **灵活配置**：支持自定义线程数、起始页和结束页等参数。
- **数据库集成**：支持 MongoDB 数据库存储，方便数据管理。
- **易于扩展**：提供清晰的钩子函数，方便用户自定义抓取逻辑。

安装
--

[](#安装)

请确保您的环境已经安装了 PHP 和 Swoole 扩展。然后，通过 Composer 安装本项目：

```
composer require l1n6yun/crawler
```

### 使用

[](#使用)

以下是如何使用本爬虫框架的基本示例：

```
require_once __DIR__ . '/vendor/autoload.php';

use Crawler\Crawler;

// 设置线程数
$threads = 10;
// 设置起始页和结束页
$startPage = 1;
$endPage = 100;

// MongoDB 数据库连接信息
$mongoDbUri = 'mongodb://localhost:27017/';
$mongoDbName = 'blog';
$mongoDbCollection = 'article';

// 创建 Crawler 实例
$crawler = new Crawler($mongoDbUri, $mongoDbName, $mongoDbCollection);

// 抓取列表
$crawler->list($threads, function ($page) {
    // 在这里实现抓取列表的逻辑...
    // 例如，访问一个 URL 并解析页面，然后返回数据数组
    return [
        [
            'title' => '页面标题', // 页面标题
            'link' => '页面链接', // 页面链接
            '_id' => md5('页面链接'), // 使用链接的 MD5 值作为唯一标识
        ],
    ];
}, $endPage, $startPage);

// 抓取详情
$crawler->detail($threads, function ($doc) {
    // 在这里实现抓取详情的逻辑...
    // 例如，访问详情页并解析内容，然后返回数据数组
    return [
        'content' => '页面内容', // 页面内容
    ];
});

// 查询爬虫状态
$status = $crawler->status();
```

### 许可证

[](#许可证)

本项目采用 MIT 许可证。

###  Health Score

24

—

LowBetter than 32% of packages

Maintenance55

Moderate activity, may be stable

Popularity2

Limited adoption so far

Community7

Small or concentrated contributor base

Maturity30

Early-stage or recently created project

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~0 days

Total

3

Last Release

522d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/5b5c8903e1260404645fe86c614e50f6f87d9e75a2ba27dec9a3623c813cf97c?d=identicon)[l1n6yun](/maintainers/l1n6yun)

---

Top Contributors

[![l1n6yun](https://avatars.githubusercontent.com/u/19780706?v=4)](https://github.com/l1n6yun "l1n6yun (4 commits)")

---

Tags

crawlerswoolecrawler frame

### Embed Badge

![Health badge](/badges/l1n6yun-crawler/health.svg)

```
[![Health](https://phpackages.com/badges/l1n6yun-crawler/health.svg)](https://phpackages.com/packages/l1n6yun-crawler)
```

###  Alternatives

[laravel/octane

Supercharge your Laravel application's performance.

4.0k21.5M156](/packages/laravel-octane)[blink/blink

A high performance web framework and application server in PHP

83441.9k3](/packages/blink-blink)[hyperf/hyperf-skeleton

A coroutine framework that focuses on hyperspeed and flexible, specifically use for build microservices and middlewares.

301187.4k](/packages/hyperf-hyperf-skeleton)[fastd/fastd

The FastD High Performance API Framework

42015.2k17](/packages/fastd-fastd)[swoft/framework

swoft framework component

64181.4k85](/packages/swoft-framework)[easyswoole/component

easyswoole component

12199.3k68](/packages/easyswoole-component)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)