PHPackages johndetomal/browser-channel - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. [Utility & Helpers](/categories/utility) 4. / 5. johndetomal/browser-channel ActiveLibrary[Utility & Helpers](/categories/utility) johndetomal/browser-channel =========================== PHP library for Browser-based scraping 11PHP Since Apr 24Pushed 1mo agoCompare [ Source](https://github.com/John-detomal/BrowserDriverEngine)[ Packagist](https://packagist.org/packages/johndetomal/browser-channel)[ RSS](/packages/johndetomal-browser-channel/feed)WikiDiscussions main Synced 1w ago READMEChangelogDependenciesVersions (1)Used By (0) [![License](https://camo.githubusercontent.com/f8df3091bbe1149f398a5369b2c39e896766f9f6efba3477c63e9b4aa940ef14/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d677265656e)](https://camo.githubusercontent.com/f8df3091bbe1149f398a5369b2c39e896766f9f6efba3477c63e9b4aa940ef14/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4d49542d677265656e)[![PHP](https://camo.githubusercontent.com/8e616b32ea6d2562125d6120fddeedd8bddd16548f82d100ebe6a50980445733/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5048502d382532422d626c7565)](https://camo.githubusercontent.com/8e616b32ea6d2562125d6120fddeedd8bddd16548f82d100ebe6a50980445733/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f5048502d382532422d626c7565)[![Node](https://camo.githubusercontent.com/a743cd9773b6bc8f98161c1a223170087432a587c263a2ab489d2784c621edf7/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4e6f64652e6a732d72657175697265642d6f72616e6765)](https://camo.githubusercontent.com/a743cd9773b6bc8f98161c1a223170087432a587c263a2ab489d2784c621edf7/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4e6f64652e6a732d72657175697265642d6f72616e6765)[![Status](https://camo.githubusercontent.com/b411594136144442e4c5e6e2e2d63ee7461762fbe5e2eb0e5abbbec3fd812845/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f7374617475732d6163746976652d73756363657373)](https://camo.githubusercontent.com/b411594136144442e4c5e6e2e2d63ee7461762fbe5e2eb0e5abbbec3fd812845/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f7374617475732d6163746976652d73756363657373) ๐ŸŽฏ Purpose & Practical Use ----------------------------- [](#-purpose--practical-use) This project was created primarily as a learning exercise to understand how scalable web content retrieval systems are designed. It explores real-world concepts such as: - Multi-driver request handling (HTTP, cURL, headless browser) - Proxy-aware routing systems - Driver pooling and lifecycle management - Fallback execution strategies - Basic response classification and reliability handling While the project is not a production scraping product, it is structured in a way that it can serve as a **foundation or base engine** for more advanced scraping or automation systems. ๐Ÿ’ก Why This Project ------------------ [](#-why-this-project) Most scraping solutions fail when websites become dynamic or protected. This engine is designed to improve reliability by combining multiple strategies: - Lightweight requests for speed - Browser automation for complex pages - Intelligent fallback between methods The goal is simple: **maximize success rate while keeping performance efficient.** ๐Ÿ’ผ Real-World Value ------------------ [](#-real-world-value) This project can serve as the foundation for: - Data collection systems - Monitoring tools - Automation pipelines - Custom scraping APIs It focuses on reliable data acquisition โ€” the most critical layer in any scraping workflow. ๐Ÿš€ Browser Scraper Engine ======================== [](#-browser-scraper-engine) A scalable, driver-based web scraping engine designed to reliably retrieve web page content using multiple strategies such as HTTP, cURL, and headless browser automation. Built to handle dynamic websites, fallback failures, proxy rotation, and caching in a structured and extensible architecture. --- โœจ Key Features -------------- [](#-key-features) - ๐Ÿง  Multi-driver system (Curl / HTTP / Browser automation) - ๐Ÿ” Automatic fallback between scraping strategies - ๐ŸŒ Proxy rotation support (improves success rate) - โšก File-based caching system - ๐Ÿงฉ Modular architecture (easy to extend) - ๐Ÿ“Š Driver-level success/failure tracking - ๐Ÿงช Debug mode for monitoring --- ๐Ÿ— Architecture Overview ----------------------- [](#-architecture-overview) The system uses a driver-based approach to fetch web pages: If one method fails, the system automatically retries using alternative drivers. This improves reliability across different website types and protection levels. --- ๐Ÿ“ฆ Installation -------------- [](#-installation) ### 1. Install PHP dependencies [](#1-install-php-dependencies) ``` composer require johndetomal/browser-channel ``` ### 2. Install Node.js dependencies (for browser driver) [](#2-install-nodejs-dependencies-for-browser-driver) ``` cd node npm install ``` ๐Ÿš€ Running the Browser Engine ---------------------------- [](#-running-the-browser-engine) Start the Puppeteer service: ``` node server.js ``` Default endpoint: โš™๏ธ Basic Usage (Quick Start) ---------------------------- [](#๏ธ-basic-usage-quick-start) ``` use Browser\Services\Browser\BrowserService; use Browser\Services\Browser\Enum\BrowserDriver; $scraper = new BrowserService([ 'settings' => [ 'driver' => BrowserDriver::Curl, ] ]); $response = $scraper->openPage("https://example.com"); echo $response['content']; ``` ๐Ÿ“Š Response Format ----------------- [](#-response-format) ``` [ 'content' => '...', 'status_code' => 200, 'retries' => 1, 'process_start_time' => 458252, 'process_end_time' => 458828252, 'message' => 'success', 'reason' => $reason, 'driver' => $driverType, ] ``` ๐Ÿ’พ Caching System ---------------- [](#-caching-system) ``` 'settings' => [ 'cache' => true, ] ``` ๐Ÿงช Debug Mode ------------ [](#-debug-mode) ``` $this->isDebugMode = true; ``` Debug output includes: - Driver used - Proxy used - Request status - Success/failure tracking - Response message ๐ŸŒ Proxy Configuration --------------------- [](#-proxy-configuration) ``` $scraper->proxies([ ['ip' => '127.0.0.1', 'port' => '8080'], ['ip' => '127.0.0.2', 'port' => '8080'], ]); ``` ๐Ÿ” Fallback System ----------------- [](#-fallback-system) If the primary method fails, the engine automatically switches between: - Primary configured driver - HTTP driver - cURL driver - Browser automation driver This improves reliability across different website structures. ๐Ÿงฉ Driver Strategy ----------------- [](#-driver-strategy) Each driver serves a different purpose: Curl / HTTP โ†’ fast, lightweight requests Browser (Puppeteer) โ†’ full rendering for JavaScript-heavy sites ๐Ÿ“ˆ Scalability & Architecture -------------------------------- [](#-scalability--architecture) This project is designed for scalability. It uses a modular architecture that allows extension without modifying core logic. You can extend the system by adding: - New drivers (e.g. Playwright) - Custom proxy strategies - Advanced caching layers - Enhanced response handling ๐Ÿ“Œ Use Cases ----------- [](#-use-cases) This engine acts as a data acquisition layer and can be used for: - Web page content collection - Data extraction pipelines (with custom parsers) - Website monitoring and change detection - Automation workflows - Research and large-scale data collection โš ๏ธ Notes - Puppeteer requires a working Chrome/Chromium environment - Some Linux servers may require additional dependencies - Curl/HTTP drivers work without Node.js ๐Ÿค Contributions This project is open to contributions and improvements. Developers are welcome to: - Add new scraping drivers - Improve proxy rotation and scoring logic - Enhance caching mechanisms - Optimize performance and reliability - Suggest architectural improvements All constructive feedback is appreciated. โš ๏ธ Limitations This system is optimized for public and moderately protected websites. Performance depends on: - Website protection level (anti-bot systems) - Proxy quality - Request patterns and concurrency Some heavily protected websites may require additional strategies. ### Health Score 20 โ€” LowBetter than 13% of packages Maintenance60 Regular maintenance activity Popularity4 Limited adoption so far Community6 Small or concentrated contributor base Maturity11 Early-stage or recently created project Bus Factor1 Top contributor holds 100% of commits โ€” single point of failure How is this calculated?**Maintenance (25%)** โ€” Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window. **Popularity (30%)** โ€” Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores. **Community (15%)** โ€” Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement. **Maturity (30%)** โ€” Project age, version count, PHP version support, and release stability. ### Community Maintainers ![](https://www.gravatar.com/avatar/e2b8888a2cd6b379a6cb582d7bffde3d51d9e29b03ac4a52b9937fabdffb75cd?d=identicon)[John-detomal](/maintainers/John-detomal) --- Top Contributors [![John-detomal](https://avatars.githubusercontent.com/u/71999846?v=4)](https://github.com/John-detomal "John-detomal (2 commits)") --- Tags crawlerdistributed-systemsheadless-browserpackagephp ### Embed Badge ![Health badge](/badges/johndetomal-browser-channel/health.svg) ``` [![Health](https://phpackages.com/badges/johndetomal-browser-channel/health.svg)](https://phpackages.com/packages/johndetomal-browser-channel) ``` ### Alternatives [trentrichardson/jquery-timepicker-addon Adds a timepicker to jQueryUI Datepicker. 2.6k70.5k](/packages/trentrichardson-jquery-timepicker-addon)[slatiusa/yii2-nestable Yii 2.0 implementation of nested set behavior using jquery.nestable plugin. 1718.5k2](/packages/slatiusa-yii2-nestable) PHPackages ยฉ 2026 [Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)