PHPackages                             hugsbrugs/php-robots-txt - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. hugsbrugs/php-robots-txt

ActiveLibrary[Utility & Helpers](/categories/utility)

hugsbrugs/php-robots-txt
========================

PHP Robots.txt Utilities

0351[3 issues](https://github.com/hugsbrugs/php-robots-txt/issues)PHP

Since Apr 19Pushed 7y ago1 watchersCompare

[ Source](https://github.com/hugsbrugs/php-robots-txt)[ Packagist](https://packagist.org/packages/hugsbrugs/php-robots-txt)[ RSS](/packages/hugsbrugs-php-robots-txt/feed)WikiDiscussions master Synced 2mo ago

READMEChangelogDependenciesVersions (1)Used By (0)

php-robots-txt
==============

[](#php-robots-txt)

This librairy provides utilities function to ease robots.txt manipulation. If you want to check if URLs respect robots.txt policy with optional cache then it's your lucky day ;)

[![Build Status](https://camo.githubusercontent.com/42225c5d20d05ae1fdd9b6234195181ab174dfd2cea8722f58177a2b99512e7b/68747470733a2f2f7472617669732d63692e6f72672f6875677362727567732f7068702d726f626f74732d7478742e7376673f6272616e63683d6d6173746572)](https://travis-ci.org/hugsbrugs/php-robots-txt)[![Coverage Status](https://camo.githubusercontent.com/b510bebdcd63b18d687977d1b96b71c2ae2288db68169712c08239d1b216cdf7/68747470733a2f2f636f766572616c6c732e696f2f7265706f732f6769746875622f6875677362727567732f7068702d726f626f74732d7478742f62616467652e7376673f6272616e63683d6d6173746572)](https://coveralls.io/github/hugsbrugs/php-robots-txt?branch=master)

Install
-------

[](#install)

Install package with composer

```
composer require hugsbrugs/php-robots-txt

```

In your PHP code, load library

```
require_once __DIR__ . '/../vendor/autoload.php';
use Hug\Robots\Robots as Robots;
```

Usage
-----

[](#usage)

Returns if a page is accessible by respecting robots.txt policy. Optionaly pass a user agent to also check against UA policy.

```
Robots::is_allowed($url, $user_agent = null);
```

With this simple method a call to remote robots.txt will be fired on each request. To enable a cache define following variables

```
define('HUG_ROBOTS_CACHE_PATH', '/path/to/robots-cache/');
define('HUG_ROBOTS_CACHE_DURATION', 7*86400);
```

Cache in seconds (86400: 1 day) Don't forget to make your path writable by webserver user robots.txt files are gzcompressed to save disk space

You Should not need following methods unless you want to play with code and tweak it !

```
Robots::download_robots($url, $user_agent);
Robots::get_robots($url, $user_agent);
Robots::is_cache_obsolete($file);
Robots::empty_cache();
```

Unit Tests
----------

[](#unit-tests)

```
phpunit --bootstrap vendor/autoload.php tests

```

Author
------

[](#author)

Hugo Maugey [visit my website ;)](https://hugo.maugey.fr)

###  Health Score

20

—

LowBetter than 14% of packages

Maintenance20

Infrequent updates — may be unmaintained

Popularity8

Limited adoption so far

Community8

Small or concentrated contributor base

Maturity37

Early-stage or recently created project

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

### Community

Maintainers

![](https://www.gravatar.com/avatar/cc04033020c3e5219ac30edf7d7362952e88ea64a0de2f41b89559f849f09901?d=identicon)[hugsbrugs](/maintainers/hugsbrugs)

---

Top Contributors

[![hugsbrugs](https://avatars.githubusercontent.com/u/4385090?v=4)](https://github.com/hugsbrugs "hugsbrugs (8 commits)")

### Embed Badge

![Health badge](/badges/hugsbrugs-php-robots-txt/health.svg)

```
[![Health](https://phpackages.com/badges/hugsbrugs-php-robots-txt/health.svg)](https://phpackages.com/packages/hugsbrugs-php-robots-txt)
```

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)