PHPackages abdelrahmanmedhat/blogsscraper - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. abdelrahmanmedhat/blogsscraper ActiveLibrary abdelrahmanmedhat/blogsscraper ============================== This package will help you sync posts from popular blogs on the Internet. 110[1 PRs](https://github.com/Abdelrahman-Medhat/BlogsScraper/pulls)PHP Since Dec 27Pushed 4y ago1 watchersCompare [ Source](https://github.com/Abdelrahman-Medhat/BlogsScraper)[ Packagist](https://packagist.org/packages/abdelrahmanmedhat/blogsscraper)[ RSS](/packages/abdelrahmanmedhat-blogsscraper/feed)WikiDiscussions master Synced 2d ago READMEChangelogDependenciesVersions (2)Used By (0) Blogs Scraper ============= [](#blogs-scraper) This package will help you sync posts from popular blogs on the Internet. ­ [![Logo](https://camo.githubusercontent.com/96a4af1cdc79b08d9b6f3f31223c71ad5a107f57bd3fbbfb7e614a9de5e70dc3/68747470733a2f2f69302e77702e636f6d2f6d6968616973706c6163652e626c6f672f77702d636f6e74656e742f75706c6f6164732f323032312f31302f6175746f736372617065722e706e673f6669743d3531322532433334312673736c3d31)](https://camo.githubusercontent.com/96a4af1cdc79b08d9b6f3f31223c71ad5a107f57bd3fbbfb7e614a9de5e70dc3/68747470733a2f2f69302e77702e636f6d2f6d6968616973706c6163652e626c6f672f77702d636f6e74656e742f75706c6f6164732f323032312f31302f6175746f736372617065722e706e673f6669743d3531322532433334312673736c3d31) Authors ------- [](#authors) - [Abdelrahman Medhat](https://github.com/Abdelrahman-Medhat) 1 ) Installation ---------------- [](#1--installation) #### **1) Install package** [](#1-install-package) Run bellow command in terminal for base path of laravel project ``` composer require abdelrahmanmedhat/blogsscraper ``` #### **2) Add Service Provider** [](#2-add-service-provider) Open config/app.php then add below line into providers array ``` AbdelrahmanMedhat\BlogsScraper\BlogsScraperServiceProvider::class ``` #### **3) Publish package in project** [](#3-publish-package-in-project) Run bellow command in terminal for base path of laravel project ``` php artisan vendor:publish --provider="AbdelrahmanMedhat\BlogsScraper\BlogsScraperServiceProvider" ``` #### **4) Connect to database** [](#4-connect-to-database) Make sure you are connect laravel project with database #### **5) Publish package tables to database** [](#5-publish-package-tables-to-database) Run bellow command in terminal for base path of project ``` php artisan migrate ``` 2 ) Parser ---------- [](#2--parser) #### **1) Parsers folder** [](#1-parsers-folder) In this folder `app/Parsers` we will put any parser here that you will create from artisan command. #### **2) Create your first parser** [](#2-create-your-first-parser) • Open terminal on base path of project. • Copy bellow command and paste it on terminal. • Replace : `blog-name` from command with blog name you will scrape data from it. • Run bellow command. • You will see new parser in Parsers folder at this path `app/Parsers`. ``` php artisan create:parser blog-name ``` #### **3) Parser architecture** [](#3-parser-architecture) • Go to this path : `app/Parsers` . • Open this parser `scitechdailyParser.php` . **In this variable you need define blog name.** ``` public static $blogName = 'SciTechDaily'; ``` ­ --- ­ **In this variable you need define home blog url.** ``` public static $blogUrl = 'https://scitechdaily.com'; ``` ­ --- ­ **In this variable you need define blog logo.** ``` public static $blogLogo = 'https://scitechdaily.com/images/cropped-scitechdaily-amp60.png'; ``` --- ­ **In bellow variable you need define blog query that we will scrape data from it** **And you need to put : `{{tag}}` instead of tag name and put : `{{page}}` instead of page number.** **Before : `https://scitechdaily.com/news/technology/amp/page/2/`** **After : `https://scitechdaily.com/news/{{tag}}/amp/page/{{page}}/`** ``` public static $blogQuery = 'https://scitechdaily.com/news/{{tag}}/amp/page/{{page}}/'; ``` ­ --- ­ **In this function you need to define all posts html nodes from class of post html div .** ``` public function posts($html){ return $html->filter('.listing-item'); } ``` | ***Notice*** | => `$html` this will return html of posts page. | ***Notice*** | => you can use `->filter('selector here')` to extract another nodes or text or ...etc from `$html` or from `$node`. ­ --- ­ **In this function you need to define post link .** ``` public function postLink($node){ return $node->filter('.post-title a')->attr('href'); } ``` | ***Notice*** | => `$node` this will return html of post node in posts page. | ***Notice*** | => you can use `->attr('attribute here')` to get value from attribute like we do above on href. ­ --- ­ **In this function you need to define post excerpt .** ``` public function postExcerpt($node){ return $node->filter('.post-excerpt p')->text(); } ``` | ***Notice*** | => you can use `->text()` to get text from html element like we do above. ­ --- ­ **In this function you need to define post title .** ``` public function postTitle($node){ return $node->filter('.post-title a')->text(); } ``` ­ --- ­ **In this function you need to define post image .** ``` public function postImage($node){ return $node->filter('.post-thumbnail amp-img')->attr('src'); } ``` ­ --- ­ **In this function you need to define post category .** ``` public function postCategory($node){ return $node->filter('.post-categories li a')->text(); } ``` ­ --- ­ **In this function you need to define post author .** ``` public function postAuthor($post){ return $post->filter('.post-author')->text(); } ``` | ***Notice*** | => `$post` this will return html of post inner page. ­ --- ­ **In this function you need to define post content .** we replace any `amp-img` with `img` in post content because blog we scrape from it , replace img tag with amp-img and if we use it with amp-img the image will not load on your blog we need to change it to img tag to browser can load it . ``` public function postContent($post){ return str_replace('amp-img', 'img', $post->filter('.post-content')->html()); } ``` | ***Notice*** | => you can use `->html()` to get html value of node. ­ --- ­ **In this function you need to define post tags .** ``` public function postTags($post){ $GLOBALS['postTags'] = []; $post->filter('.tags a')->each(function ($tagsNodes) { $GLOBALS['postTags'] []= $tagsNodes->text() ; }); return $GLOBALS['postTags']; } ``` | ***Notice*** | => you can use below method to loop on all element : `` inside element have class : `.tags` like what we do on above function ``` $nodes->each(function ($node) { // use $node here }); ``` ­| ***Notice*** | => you need return all tags in array. --- ­ ­ 3 ) Scraping ------------ [](#3--scraping) #### Sync all posts from specific blog. [](#sync-all-posts-from-specific-blog) ``` use AbdelrahmanMedhat\BlogsScraper\BlogsScraper; $blog_name = 'scitechdaily'; $tag_name = 'technology'; $pages = [1,2,3]; $BlogsScraper = new BlogsScraper; $BlogsScraper->sync($blog_name,$tag_name,$pages); ``` ­| ***Notice*** | => you also can use this script to corn job on server With a specific schedule. --- ­ 4 ) Database Relationships -------------------------- [](#4--database-relationships) ­ [![Logo](https://camo.githubusercontent.com/b386acd3a6abe5613ada535ad89b2489d10eb86c196b5d28e7571b8c429af372/68747470733a2f2f692e6962622e636f2f39675a663742682f626c6f67732d536372617065722d444264657369676e2e706e67)](https://camo.githubusercontent.com/b386acd3a6abe5613ada535ad89b2489d10eb86c196b5d28e7571b8c429af372/68747470733a2f2f692e6962622e636f2f39675a663742682f626c6f67732d536372617065722d444264657369676e2e706e67)­ #### Get all posts with relationships. [](#get-all-posts-with-relationships) ``` use AbdelrahmanMedhat\BlogsScraper\Models\Post; $posts = Post::with(['category','blog','author','tags']) ->simplePaginate(15)->toArray(); ``` ­ - [](#) ­ #### Get all blogs with relationships. [](#get-all-blogs-with-relationships) ``` use AbdelrahmanMedhat\BlogsScraper\Models\Blog; $blogs = Blog::with(['posts']) ->simplePaginate(15)->toArray(); ``` ­ - [](#-1) ­ #### Get all authors with relationships. [](#get-all-authors-with-relationships) ``` use AbdelrahmanMedhat\BlogsScraper\Models\Author; $authors = Author::with(['posts']) ->simplePaginate(15)->toArray(); ``` ­ - [](#-2) ­ #### Get all categories with relationships. [](#get-all-categories-with-relationships) ``` use AbdelrahmanMedhat\BlogsScraper\Models\Category; $categories = Category::with(['posts']) ->simplePaginate(15)->toArray(); ``` ­ - [](#-3) ­ #### Get all tags with relationships. [](#get-all-tags-with-relationships) ``` use AbdelrahmanMedhat\BlogsScraper\Models\Tag; $tags = Tag::with(['posts']) ->simplePaginate(15)->toArray(); ``` ­ - [](#-4) ­ ### Health Score 17 — LowBetter than 6% of packages Maintenance20 Infrequent updates — may be unmaintained Popularity7 Limited adoption so far Community7 Small or concentrated contributor base Maturity29 Early-stage or recently created project Bus Factor1 Top contributor holds 100% of commits — single point of failure How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window. **Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores. **Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement. **Maturity (30%)** — Project age, version count, PHP version support, and release stability. ### Community Maintainers ![](https://www.gravatar.com/avatar/15844b792ea7729a5380bf5b863d56005d1e38c534b4baaa623050d80d6c9883?d=identicon)[Abdelrahman-Medhat](/maintainers/Abdelrahman-Medhat) --- Top Contributors [![Abdelrahman-Medhat](https://avatars.githubusercontent.com/u/92066623?v=4)](https://github.com/Abdelrahman-Medhat "Abdelrahman-Medhat (42 commits)") ### Embed Badge ![Health badge](/badges/abdelrahmanmedhat-blogsscraper/health.svg) ``` [![Health](https://phpackages.com/badges/abdelrahmanmedhat-blogsscraper/health.svg)](https://phpackages.com/packages/abdelrahmanmedhat-blogsscraper) ``` PHPackages © 2026 [Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)