PHPackages                             reliqarts/laravel-scavenger - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [API Development](/categories/api)
4. /
5. reliqarts/laravel-scavenger

ActiveLibrary[API Development](/categories/api)

reliqarts/laravel-scavenger
===========================

The most integrated web scraper package for Laravel.

v3.5.2(3y ago)1571.5k9MITPHPPHP ^7.4 || ^8.0CI failing

Since Aug 16Pushed 2y ago2 watchersCompare

[ Source](https://github.com/reliqarts/laravel-scavenger)[ Packagist](https://packagist.org/packages/reliqarts/laravel-scavenger)[ GitHub Sponsors](https://github.com/reliq)[ RSS](/packages/reliqarts-laravel-scavenger/feed)WikiDiscussions main Synced 2d ago

READMEChangelog (3)Dependencies (8)Versions (26)Used By (0)

Laravel Scavenger
=================

[](#laravel-scavenger)

[![Laravel Scavenger](./docs/images/inline-preview.png "Laravel Scavenger")](./docs/images/inline-preview.png)

The most integrated web scraping package for Laravel.

[![Unit Tests](https://github.com/reliqarts/laravel-scavenger/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/reliqarts/laravel-scavenger/actions/workflows/unit-tests.yml)[![Codecov](https://camo.githubusercontent.com/018024906013ed1c853360b0606c792618b320f7f5864cf3a55653882754573b/68747470733a2f2f696d672e736869656c64732e696f2f636f6465636f762f632f6769746875622f72656c6971617274732f6c61726176656c2d73636176656e6765722e7376673f7374796c653d666c61742d737175617265)](https://codecov.io/gh/reliqarts/laravel-scavenger)[![Scrutinizer](https://camo.githubusercontent.com/4fad1549404f3cd6783a4630bcfb90d6a447d36d7da37e6cd846bd74c4526bf7/68747470733a2f2f696d672e736869656c64732e696f2f7363727574696e697a65722f672f72656c6971617274732f6c61726176656c2d73636176656e6765722e7376673f7374796c653d666c61742d737175617265)](https://scrutinizer-ci.com/g/reliqarts/laravel-scavenger/)[![License](https://camo.githubusercontent.com/a1c8b35414f2d2dcabaf5891e4f9f3aaf2ef9782690ba7a6bf5161ee358810e1/68747470733a2f2f706f7365722e707567782e6f72672f72656c6971617274732f6c61726176656c2d73636176656e6765722f6c6963656e73653f666f726d61743d666c61742d737175617265)](https://packagist.org/packages/reliqarts/laravel-scavenger)[![Latest Stable Version](https://camo.githubusercontent.com/2fcce90389b15aadaf312556dc54b914cb45c1cf7ab4c0d841a5793dcfdd3310/68747470733a2f2f706f7365722e707567782e6f72672f72656c6971617274732f6c61726176656c2d73636176656e6765722f76657273696f6e3f666f726d61743d666c61742d737175617265)](https://packagist.org/packages/reliqarts/laravel-scavenger)[![Latest Unstable Version](https://camo.githubusercontent.com/e1b79a7e9bcb257a6d55b7c88a37ed42e56b0e83444e40782b268f8a400c2e2f/68747470733a2f2f706f7365722e707567782e6f72672f72656c6971617274732f6c61726176656c2d73636176656e6765722f762f756e737461626c653f666f726d61743d666c61742d737175617265)](//packagist.org/packages/reliqarts/laravel-scavenger)[![Maintainability](https://camo.githubusercontent.com/89562c7900da28dc212223c6cd17b8a623ee59343c5501ae55ab71b3206fa00f/68747470733a2f2f6170692e636f6465636c696d6174652e636f6d2f76312f6261646765732f31306164326561396330313638373663373862632f6d61696e7461696e6162696c697479)](https://codeclimate.com/github/reliqarts/laravel-scavenger/maintainability)

Top Features
------------

[](#top-features)

Scavenger provides the following features and more out-the-box.

- Ease of use
    - Scavenger is super-easy to configure. Simply publish the config file and set your targets.
- Scrape data from multiple sources at once.
- Convert scraped data into usable Laravel model objects.
    - eg. You may scrape an article, have it converted into an object of your choice, and save it in your database. Immediately available to your viewers.
- You can easily perform one or more operations, on each property, of any scraped entity.
    - eg. You may call a paraphrase service from a model or package of your choice on data attributes before saving them to your database.
- Data integrity constraints
    - Scavenger uses a hashing algorithm of your choice to maintain data integrity. This hash is used to ensure that one scrap (source article) is not converted to multiple output objects (model duplicates).
- Console Command
    - Once Scavenger is configured, a simple artisan command launches the seeker. Since this is a console command it is more efficient and timeouts are less likely to occur.
    - Artisan command: `php artisan scavenger:seek`
- Schedule ready
    - Scavenger can easily be set to scrape on a schedule. Hence, creating a somewhat autonomous website is super easy!
- SERP
    - Scavenger can be used to flexibly scrape Search Engine Result Pages.

Installation
------------

[](#installation)

1. Install via composer; in your terminal:

    ```
    composer require reliqarts/laravel-scavenger

    ```

    or require in *composer.json*:

    ```
    {
        "require": {
            "reliqarts/laravel-scavenger": "^3.1"
        }
    }
    ```

    then run `composer update` in your terminal to pull it in.
2. (Optional) Publish package resources and configuration:

```
php artisan vendor:publish --provider="ReliqArts\Scavenger\ServiceProvider"

```

You may opt to publish only configuration by using the `scavenger-config` tag:

```
php artisan vendor:publish --provider="ReliqArts\Scavenger\ServiceProvider" --tag="scavenger-config"

```

or only the migrations via the `scavenger-migrations` tag:

```
php artisan vendor:publish --provider="ReliqArts\Scavenger\ServiceProvider" --tag="scavenger-migrations"

```

Configuration
-------------

[](#configuration)

Scavenger is highly configurable. Once configured, the settings will be used for every scrape.

### Structure

[](#structure)

Below is an example of a typical config file structure, with comments explaining each setting.

```