PHPackages                             shammaa/laravel-smart-scraper - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Logging & Monitoring](/categories/logging)
4. /
5. shammaa/laravel-smart-scraper

ActiveLibrary[Logging & Monitoring](/categories/logging)

shammaa/laravel-smart-scraper
=============================

Advanced intelligent web scraper for Laravel with caching, rate limiting, middleware, monitoring, and much more. Built on Puppeteer with smart features.

v1.0.0(4mo ago)12MITPHPPHP ^8.1

Since Dec 16Pushed 4mo agoCompare

[ Source](https://github.com/shammaa/laravel-smart-scraper)[ Packagist](https://packagist.org/packages/shammaa/laravel-smart-scraper)[ RSS](/packages/shammaa-laravel-smart-scraper/feed)WikiDiscussions main Synced 1mo ago

READMEChangelogDependencies (7)Versions (2)Used By (0)

Laravel Smart Scraper
=====================

[](#laravel-smart-scraper)

[![Latest Version](https://camo.githubusercontent.com/282512b8d8a86b8b81ce201b95d0e23f0fa81fdbf15e82f5bf33780f83a23888/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f7368616d6d61612f6c61726176656c2d736d6172742d736372617065722e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/shammaa/laravel-smart-scraper)[![Total Downloads](https://camo.githubusercontent.com/476430515c577e493ef3e7f183e537c423bf035ed998a48d95c907e92277c214/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f64742f7368616d6d61612f6c61726176656c2d736d6172742d736372617065722e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/shammaa/laravel-smart-scraper)[![License](https://camo.githubusercontent.com/e48a2077d4bf8bc127194a710214d2144ef38dcde0d04c5c00b5be3e8940751e/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f6c2f7368616d6d61612f6c61726176656c2d736d6172742d736372617065722e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/shammaa/laravel-smart-scraper)

**Advanced intelligent web scraper for Laravel** with caching, rate limiting, middleware, monitoring, and much more. Built on Puppeteer with smart features that make web scraping professional, efficient, and reliable.

🚀 Key Features
--------------

[](#-key-features)

- ✅ **Intelligent Caching** - Automatic caching to avoid redundant requests
- ✅ **Rate Limiting** - Prevent overwhelming target websites
- ✅ **User-Agent Rotation** - Rotate user agents automatically to avoid detection
- ✅ **Middleware System** - Extensible middleware for request manipulation
- ✅ **Automatic Retry** - Exponential backoff retry logic for failed requests
- ✅ **Screenshot & PDF** - Capture screenshots and generate PDFs
- ✅ **Proxy Support** - Full proxy support with authentication
- ✅ **Monitoring & Logging** - Comprehensive monitoring and logging
- ✅ **Schema Validation** - Validate extracted data against schemas
- ✅ **Concurrent Scraping** - Scrape multiple URLs concurrently
- ✅ **Queue Support** - Process scraping jobs in background
- ✅ **Error Handling** - Robust error handling and recovery
- ✅ **Smart Site Detection** - Automatically detect site type and use appropriate selectors
- ✅ **Multi-Site Support** - Handle multiple websites with different HTML structures intelligently

📦 Installation
--------------

[](#-installation)

Install the package via Composer:

```
composer require shammaa/laravel-smart-scraper
```

Publish the configuration file:

```
php artisan vendor:publish --tag=smart-scraper-config
```

Install Node.js dependencies (required for Puppeteer):

```
npm install puppeteer puppeteer-extra puppeteer-extra-plugin-stealth
```

**Note:** Make sure Node.js is installed and available in your PATH. If using NVM, see the [NVM Configuration](#nvm-configuration) section below.

🎯 Quick Start
-------------

[](#-quick-start)

### 1. Create a Scraper

[](#1-create-a-scraper)

Generate a scraper class using the Artisan command:

```
php artisan make:scraper ProductScraper
```

This creates a file at `app/Scrapers/ProductScraper.php`:

```