PHPackages                             andrew-svirin/resource-crawler-bundle - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. andrew-svirin/resource-crawler-bundle

ActiveSymfony-bundle[Utility & Helpers](/categories/utility)

andrew-svirin/resource-crawler-bundle
=====================================

Crawler for any sort of resources.

355PHP

Since Mar 5Pushed 3y ago1 watchersCompare

[ Source](https://github.com/andrew-svirin/resource-crawler-bundle)[ Packagist](https://packagist.org/packages/andrew-svirin/resource-crawler-bundle)[ RSS](/packages/andrew-svirin-resource-crawler-bundle/feed)WikiDiscussions main Synced 1mo ago

READMEChangelogDependenciesVersions (1)Used By (0)

resource-crawler-bundle
=======================

[](#resource-crawler-bundle)

Symfony bundle for crawling disk/web resource.
Spider bot can navigate other disk or web resources.
Internet bot can be customized by path mask and other options.
Crawler scan HTML-document extract hyperlinks and push them to the index pool of next iteration.

Install
-------

[](#install)

`composer require andrew-svirin/resource-crawler-bundle:dev-main`

Add to `doctrine.yaml` to avoid table to be associated with entities.

```
doctrine:
    dbal:
        schema_filter: ~^(?!resource_crawler_)~

```

Add to `resource_crawler.yaml` to avoid table to be associated with entities.

```
resource_crawler:
  process:
    is_lockable: true
    store: 'resource_crawler.process_db_store'
#    store: 'resource_crawler.process_file_store'
    file_store:
      dir: "%kernel.project_dir%/storage/saver"

```

Add migration `Version20230101010000.php`:

```