PHPackages                             cswannauger/pwa-spider - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. cswannauger/pwa-spider

ActiveLibrary[Utility & Helpers](/categories/utility)

cswannauger/pwa-spider
======================

A web spider to scrape Progressive Web Apps

04PHP

Since Apr 6Pushed 5y ago1 watchersCompare

[ Source](https://github.com/ChesterSA/spider-library)[ Packagist](https://packagist.org/packages/cswannauger/pwa-spider)[ RSS](/packages/cswannauger-pwa-spider/feed)WikiDiscussions master Synced yesterday

READMEChangelogDependenciesVersions (1)Used By (0)

PWA Spider
==========

[](#pwa-spider)

This is a simple PHP package to install a web spider to find and return examples of Progressive Web Apps from the internet, all it needs is a collection of sites to start from and the rest is done for you.

### Installation

[](#installation)

The package is hosted on composer so can be installed with the command:

```
composer require cswannauger/pwa-spider
```

### Using the Spider

[](#using-the-spider)

Once it has been installed, to begin the Spider simply use PWASpider\\Spider and then call

```
$sites = Spider::start(array $links, int $limit = 1000000);
```

The `$links` array is an array of URLs you would like the scraper to start with, and the optional variable `$limit` is set to the maximum amount of sites you would like returned. If not used it defaults to 1 million which could take days to run.

The method returns a nested array containing details of every web app. The array for each site is in the following format

```
 [
    'url' => 'https://twitter.com',
    'title' => 'Twitter',
    'img' => 'https://abs.twimg.com/responsive-web/client-web-legacy/icon-default-large.8e027b65.png',
    'description' => 'Twitter is what’s happening and what people are talking about right now.'
 ]
```

You also have access to the `Spider::checkPWA($doc)` method to confirm if any sites you have access to are Progressive Web Apps. This takes an duzun/hQuery document and processes it to check for PWA validity. If it is a PWA then it will return the location of the site's manifest.json file, else it will return `false`

### How it works

[](#how-it-works)

The check is a fairly naive one at this point but from testing it is still accurate. It iterates through the `` tags in the header of the site to see if any of them are a link to the site's manifest file (usually stored at `/manifest.json`). This file is a necessity to create well-formed progressive web apps, it is where the information about the application itself (name, icon, theme color, etc) is all stored.

###  Health Score

16

—

LowBetter than 5% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity3

Limited adoption so far

Community7

Small or concentrated contributor base

Maturity30

Early-stage or recently created project

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

### Community

Maintainers

![](https://www.gravatar.com/avatar/8b09d13295902554b7c472c92447b87c9d5c4ba9e9e35f5f8d021836b077613e?d=identicon)[ChesterSA](/maintainers/ChesterSA)

---

Top Contributors

[![ChesterSA](https://avatars.githubusercontent.com/u/23403613?v=4)](https://github.com/ChesterSA "ChesterSA (13 commits)")

### Embed Badge

![Health badge](/badges/cswannauger-pwa-spider/health.svg)

```
[![Health](https://phpackages.com/badges/cswannauger-pwa-spider/health.svg)](https://phpackages.com/packages/cswannauger-pwa-spider)
```

###  Alternatives

[stefanbauer/laravel-favicon-extractor

A favicon extractor for Laravel

10125.3k](/packages/stefanbauer-laravel-favicon-extractor)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)