PHPackages                             innmind/robots-txt - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Parsing & Serialization](/categories/parsing)
4. /
5. innmind/robots-txt

ActiveLibrary[Parsing & Serialization](/categories/parsing)

innmind/robots-txt
==================

Robots.txt parser

7.0.0(2mo ago)04221MITPHPPHP ~8.4CI passing

Since Feb 5Pushed 2mo ago1 watchersCompare

[ Source](https://github.com/Innmind/Robots.txt)[ Packagist](https://packagist.org/packages/innmind/robots-txt)[ RSS](/packages/innmind-robots-txt/feed)WikiDiscussions develop Synced 2mo ago

READMEChangelog (4)Dependencies (4)Versions (14)Used By (1)

Robots.txt
==========

[](#robotstxt)

[![CI](https://github.com/Innmind/Robots.txt/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/Innmind/Robots.txt/actions/workflows/ci.yml)[![codecov](https://camo.githubusercontent.com/4472ce4e2b1dffb8d08eddd9792f39c551598002ab84b9f71377a3fbe6ff0b22/68747470733a2f2f636f6465636f762e696f2f67682f496e6e6d696e642f526f626f74732e7478742f6272616e63682f646576656c6f702f67726170682f62616467652e737667)](https://codecov.io/gh/Innmind/Robots.txt)[![Type Coverage](https://camo.githubusercontent.com/20fea98a2186bf247ce88a91e7a6a2820e1f02e2908b88f2e6f1c039c8fcb99c/68747470733a2f2f73686570686572642e6465762f6769746875622f496e6e6d696e642f526f626f74732e7478742f636f7665726167652e737667)](https://shepherd.dev/github/Innmind/Robots.txt)

Robots.txt parser

Installation
------------

[](#installation)

```
composer require innmind/robots-txt
```

Usage
-----

[](#usage)

```
use Innmind\RobotsTxt\Parser;
use Innmind\OperatingSystem\Factory;
use Innmind\Url\Url;

$os = Factory::build();
$parse = Parser::of(
    $os->remote()->http(),
    'My user agent',
);
$robots = $parse(Url::of('https://github.com/robots.txt'))->match(
    static fn($robots) => $robots,
    static fn() => throw new \RuntimeException('robots.txt not found'),
);
$robots->disallows('My user agent', Url::of('/humans.txt')); //false
$robots->disallows('My user agent', Url::of('/any/other/url')); //true
```

Note

Here only the path `/humans.txt` is allowed because by default github disallows any user agent to crawl there website except for this file.

###  Health Score

52

—

FairBetter than 96% of packages

Maintenance83

Actively maintained with recent releases

Popularity12

Limited adoption so far

Community9

Small or concentrated contributor base

Maturity88

Battle-tested with a long release history

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~329 days

Recently: every ~260 days

Total

11

Last Release

85d ago

Major Versions

2.0.0 → 3.0.02017-10-02

3.0.0 → 4.0.02019-01-05

4.0.0 → 5.0.02020-02-05

5.1.0 → 6.0.02023-04-10

6.3.0 → 7.0.02026-02-15

PHP version history (7 changes)1.0.0PHP ~7.1

4.0.0PHP ~7.2

5.0.0PHP ~7.4

5.1.0PHP ~7.4|~8.0

6.0.0PHP ~8.1

6.2.0PHP ~8.2

7.0.0PHP ~8.4

### Community

Maintainers

![](https://avatars.githubusercontent.com/u/851425?v=4)[Baptiste Langlade](/maintainers/Baptouuuu)[@Baptouuuu](https://github.com/Baptouuuu)

---

Top Contributors

[![Baptouuuu](https://avatars.githubusercontent.com/u/851425?v=4)](https://github.com/Baptouuuu "Baptouuuu (122 commits)")

---

Tags

parserrobots-txt

### Embed Badge

![Health badge](/badges/innmind-robots-txt/health.svg)

```
[![Health](https://phpackages.com/badges/innmind-robots-txt/health.svg)](https://phpackages.com/packages/innmind-robots-txt)
```

###  Alternatives

[mtdowling/jmespath.php

Declaratively specify how to extract elements from a JSON document

2.0k472.8M135](/packages/mtdowling-jmespathphp)[opis/closure

A library that can be used to serialize closures (anonymous functions) and arbitrary data.

2.6k230.0M284](/packages/opis-closure)[masterminds/html5

An HTML5 parser and serializer.

1.8k242.8M229](/packages/masterminds-html5)[sabberworm/php-css-parser

Parser for CSS Files written in PHP

1.8k191.2M65](/packages/sabberworm-php-css-parser)[michelf/php-markdown

PHP Markdown

3.5k52.4M345](/packages/michelf-php-markdown)[jms/metadata

Class/method/property metadata management in PHP

1.8k152.8M88](/packages/jms-metadata)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)