PHPackages                             terminal42/escargot - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Search & Filtering](/categories/search)
4. /
5. terminal42/escargot

ActiveLibrary[Search & Filtering](/categories/search)

terminal42/escargot
===================

A web crawler or spider library based on Symfony components

1.7.0(5mo ago)581.4M↓39.8%7[1 issues](https://github.com/terminal42/escargot/issues)2MITPHPPHP ^8.1CI failing

Since Nov 25Pushed 5mo ago5 watchersCompare

[ Source](https://github.com/terminal42/escargot)[ Packagist](https://packagist.org/packages/terminal42/escargot)[ Docs](https://www.terminal42.ch)[ RSS](/packages/terminal42-escargot/feed)WikiDiscussions main Synced 1mo ago

READMEChangelog (10)Dependencies (13)Versions (41)Used By (2)

Escargot - a Symfony HttpClient based Crawler framework
=======================================================

[](#escargot---a-symfony-httpclient-based-crawler-framework)

   ![The Escargot logo which consists of an illustration of a snail shell](assets/logo/logo.svg)---

A library that provides everything you need to crawl anything based on HTTP and process the responses in whatever way you prefer based on Symfony components.

### Why yet another crawler?

[](#why-yet-another-crawler)

There are so many different implementations in so many programming languages, right? Well, the ones I found in PHP did not really live up to my personal quality standards and also I wanted something that's built on top of the [Symfony HttpClient](https://symfony.com/doc/current/components/http_client.html) component and is not bound to crawl websites (HTML) only but can be used as the foundation for anything you may want to crawl. Hence, yet another library.

### What about that name «Escargot»?

[](#what-about-that-name-escargot)

When I created this library I didn't want to name it «crawler» or «spider» or anything similar that's been used hundreds of times before. So I started to think about things that actually crawl and one thing that came to my mind immediately were snails. But «snail» doesn't really sound super beautiful and so I just went with the French translation for it which is «escargot». There you go! Also French is a beautiful language anyway and in case you didn't know: tons of libraries in the PHP ecosystem were invented and are still maintained by French people so it's also some kind of tribute to the French PHP community (and Symfony one for that matter).

By the way: Thanks to the Symfony HttpClient `Escargot` is actually not slow at all ;-)

### Installation

[](#installation)

```
composer require terminal42/escargot
```

### Usage

[](#usage)

Everything in `Escargot` is assigned to a job ID. The reason for this design is that crawling huge amounts of URIs can take very long and chances that you'll want to stop at some point and pick up where you left are pretty high. For that matter, every `Escargot` instance also needs a queue plus a base URI collection as to where to start crawling.

#### Instantiating Escargot

[](#instantiating-escargot)

The factory method when you do not have a job ID yet has to be used as follows:

```