PHPackages                             piopi/behatcrawler - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Testing & Quality](/categories/testing)
4. /
5. piopi/behatcrawler

ActiveBehat-extension[Testing & Quality](/categories/testing)

piopi/behatcrawler
==================

A Behat extension that crawls links on a website and executes user-defined function on each one of them.

122[1 issues](https://github.com/piopi/BehatCrawler/issues)PHP

Since Oct 1Pushed 5y ago1 watchersCompare

[ Source](https://github.com/piopi/BehatCrawler)[ Packagist](https://packagist.org/packages/piopi/behatcrawler)[ RSS](/packages/piopi-behatcrawler/feed)WikiDiscussions master Synced 1mo ago

READMEChangelogDependenciesVersions (1)Used By (0)

BehatCrawler
============

[](#behatcrawler)

[![PHP Composer](https://github.com/piopi/BehatCrawler/workflows/PHP%20Composer/badge.svg)](https://github.com/piopi/BehatCrawler/workflows/PHP%20Composer/badge.svg)

The BehatCrawler is a [Behat](https://github.com/Behat/Behat), [MinkExtension](https://github.com/Behat/MinkExtension) and [Selenium2Driver](https://github.com/minkphp/MinkSelenium2Driver) extension that crawls a given URL and executes user-defined functions in each crawled page.

Multiple options for crawling are available, see [available options](#available).

Installation
------------

[](#installation)

```
composer require piopi/behatcrawler
```

Usage
-----

[](#usage)

Start by importing the extension, to your Feature Context (or any of your Context):

```
use Behat\Crawler\Crawler;
```

Create your Crawler object with the default configuration:

**The crawler is only compatible at this time with [Selenium2Driver](https://github.com/minkphp/MinkSelenium2Driver)**

```
//$crawler=New Crawler(BehatSession);
$crawler= New Crawler($this->getSession());
```

For custom settings (passed as an array), see the following table for all the [available options](#available).

```
$crawler= New Crawler($this->getSession(),["internalLinksOnly"=>true,"HTMLOnly"=>true,'MaxCrawl'=>20]);
```

#### Available options: (More functionalities coming soon)

[](#available-options-more-functionalities-coming-soon-)

OptionDescriptionDefault ValueDepthMaximum depth that can be crawled from URL0 (unlimited)MaxCrawlMaximum number of crawls0 (unlimited)HTMLOnlyWill only crawl HTML/xHTML pagestrueinternalLinksOnlyWill crawl internal links only (links with same Domaine name as the initial URL)truewaitForCrawlWill wait for the crawler to finish crawling before throwing any exception originating from the user defined functions. (Compile a list of all exceptions found with their respective location)false**Option can either be set in the constructor or with the appropriate getters/setters:**

```
 $crawler= New Crawler($this->getSession(),["MaxCrawl"=>10]);
//or
$crawler->setMaximumCrawl(10);
```

#### Start Crawling

[](#start-crawling)

After creating and setting up the crawler, you can start crawling by passing your function as an argument:

Please refer to the PHP [Callables documentation](https://www.php.net/manual/en/language.types.callable.php) for more details.

**Examples**:

> Closure::fromCallable is used to pass by parameter private function

```
//function 1 is a private function
$crawler->startCrawling(Closure::fromCallable([$this, 'function1']));
//function 2 is a public class function
$crawler->startCrawling([$this, 'function1']);
```

For functions with one or more arguments, they can be passed as the following:

```
$crawler->startCrawling(Closure::fromCallable([$this, 'function3']),[arg1]);
$crawler->startCrawling(Closure::fromCallable([$this, 'function4']),[arg1,arg2]);
```

### Usage Example

[](#usage-example)

```
use Behat\Crawler\Crawler;
//Crawler with different settings
$crawler= New Crawler($this->getSession(),["internalLinksOnly"=>true,"HTMLOnly"=>true,'MaxCrawl'=>20,"waitForCrawl"=>true]);
//Function without arguments
$crawler->startCrawling(Closure::fromCallable([$this, 'function1'])); //Will start crawling
//Function with one or more argument
$crawler->startCrawling(Closure::fromCallable([$this, 'function2']),[arg1,arg2]);
```

**In a Behat step function:**

```
   /**
     * @Given /^I crawl the website with a maximum of (\d+) level$/
     */
    public function iCrawlTheWebsiteWithAMaximumOfLevel($arg1)
    {
        $crawler= New Crawler($this->getSession(),["Depth"=>$arg1]);
        $crawler->startCrawling([$this, 'test']);
    }
```

### Copyright

[](#copyright)

Copyright (c) 2020 Mostapha El Sabah

Maintainers
-----------

[](#maintainers)

Mostapha El Sabah [Piopi](https://github.com/piopi)

###  Health Score

13

—

LowBetter than 1% of packages

Maintenance0

Infrequent updates — may be unmaintained

Popularity8

Limited adoption so far

Community7

Small or concentrated contributor base

Maturity32

Early-stage or recently created project

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

### Community

Maintainers

![](https://www.gravatar.com/avatar/e2a3af2d6ba381317a72e840529c975b00bcef0511c7ffffa56232cb0933254b?d=identicon)[Piopi](/maintainers/Piopi)

---

Top Contributors

[![piopi](https://avatars.githubusercontent.com/u/23510615?v=4)](https://github.com/piopi "piopi (9 commits)")

---

Tags

behatbehat-extensioncrawlerphpselenium-webdriver

### Embed Badge

![Health badge](/badges/piopi-behatcrawler/health.svg)

```
[![Health](https://phpackages.com/badges/piopi-behatcrawler/health.svg)](https://phpackages.com/packages/piopi-behatcrawler)
```

###  Alternatives

[phpspec/prophecy

Highly opinionated mocking framework for PHP 5.3+

8.5k551.7M682](/packages/phpspec-prophecy)[vimeo/psalm

A static analysis tool for finding errors in PHP applications

5.8k77.5M6.7k](/packages/vimeo-psalm)[brianium/paratest

Parallel testing for PHP

2.5k118.8M754](/packages/brianium-paratest)[beberlei/assert

Thin assertion library for input validation in business models.

2.4k96.9M570](/packages/beberlei-assert)[mikey179/vfsstream

Virtual file system to mock the real file system in unit tests.

1.4k108.0M2.7k](/packages/mikey179-vfsstream)[orchestra/testbench

Laravel Testing Helper for Packages Development

2.2k39.1M32.1k](/packages/orchestra-testbench)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)