PHPackages                             thewinterwind/arachnid - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Search & Filtering](/categories/search)
4. /
5. thewinterwind/arachnid

ActiveLibrary[Search & Filtering](/categories/search)

thewinterwind/arachnid
======================

A crawler to find all unique internal pages on a given website

1.1(9y ago)010MITPHPPHP >=5.6.0

Since Jan 6Pushed 9y ago1 watchersCompare

[ Source](https://github.com/thewinterwind/arachnid)[ Packagist](https://packagist.org/packages/thewinterwind/arachnid)[ Docs](http://github.com/codeguy/arachnid)[ RSS](/packages/thewinterwind-arachnid/feed)WikiDiscussions master Synced 4w ago

READMEChangelogDependencies (5)Versions (7)Used By (0)

Arachnid Web Crawler
====================

[](#arachnid-web-crawler)

This library will crawl all unique internal links found on a given website up to a specified maximum page depth.

This library is based on the original blog post by Zeid Rashwani here:

Josh Lockhart adapted the original blog post's code (with permission) for Composer and Packagist and updated the syntax to conform with the PSR-2 coding standard.

[![SensioLabsInsight](https://camo.githubusercontent.com/48843ba16475df2ea619b81d75e19c0f5bd2137ba5627a175ca5f55566b2a71a/68747470733a2f2f696e73696768742e73656e73696f6c6162732e636f6d2f70726f6a656374732f38666631653462322d643863382d343436352d623965612d6635646236396333633264302f6d696e692e706e67)](https://insight.sensiolabs.com/projects/8ff1e4b2-d8c8-4465-b9ea-f5db69c3c2d0)[![Build Status](https://camo.githubusercontent.com/c03a2f92f2acb744d62da6df4568fbac19bc7963afd6c113e27c1fc33ba38561/68747470733a2f2f7472617669732d63692e6f72672f7a7261736877616e692f61726163686e69642e7376673f6272616e63683d6d6173746572)](https://travis-ci.org/zrashwani/arachnid)[![codecov](https://camo.githubusercontent.com/ea57350b660cbb5a7796d70dd3f997719c40a442cc60850c52855f64bc2d5dda/68747470733a2f2f636f6465636f762e696f2f67682f7a7261736877616e692f61726163686e69642f6272616e63682f6d61737465722f67726170682f62616467652e737667)](https://codecov.io/gh/zrashwani/arachnid)

How to Install
--------------

[](#how-to-install)

You can install this library with [Composer](http://getcomposer.org/). Drop this into your `composer.json`manifest file:

```
{
    "require": {
        "zrashwani/arachnid": "dev-master"
    }
}

```

Then run `composer install`.

Getting Started
---------------

[](#getting-started)

Here's a quick demo to crawl a website:

```

```

You can set crawler to visit only pages with specific criteria by specifying callback closure using `filterLinks` method:

```