PHPackages                             nexxes/tokenmatcher - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Parsing & Serialization](/categories/parsing)
4. /
5. nexxes/tokenmatcher

ActiveLibrary[Parsing & Serialization](/categories/parsing)

nexxes/tokenmatcher
===================

PHP library to match tokenstreams (arrays of tokens) for later parsing.

09PHP

Since Sep 8Pushed 11y ago1 watchersCompare

[ Source](https://github.com/nexxes/php-tokenmatcher)[ Packagist](https://packagist.org/packages/nexxes/tokenmatcher)[ RSS](/packages/nexxes-tokenmatcher/feed)WikiDiscussions master Synced today

READMEChangelogDependenciesVersions (1)Used By (0)

Package nexxes/tokenmatcher
===========================

[](#package-nexxestokenmatcher)

[![Build Status](https://camo.githubusercontent.com/a14978d2cec0cdb42f905c035199c48508b8218eb5ff011d5c6541f61e1331bb/68747470733a2f2f7472617669732d63692e6f72672f6e65787865732f7068702d746f6b656e6d6174636865722e7376673f6272616e63683d6d6173746572)](https://travis-ci.org/nexxes/php-tokenmatcher)

PHP library to match tokenstreams (arrays of tokens) for later parsing.

The tokenmatcher library depends on the tokenizer package. To use, you have to tokenize a string or construct a token stream yourself.

Installation
------------

[](#installation)

Use composer:

```
"require": {
	"nexxes/tokenmatcher": "dev-master"
}
```

The library is currently under development and the API not stable yet. The development is based on features required to create a working and standard compliant [CommonMark](http://commonmark.org "CommonMark") parser. See our [nexxes/commonmark](https://github.com/nexxes/commonmark "nexxes/commonmark") package.

Match a token
-------------

[](#match-a-token)

```
// Produces two tokens: a Token::WHITESPACE token and a Token::MINUS token
$tokenizer = new Tokenizer("   ---");
$tokens = $tokenizer->getTokens();

$whitespaceMatcher = new Matches(Token::WHITESPACE);
// stores 1, the number of tokens matched
$matched = $whitespaceMatcher->match($tokens);
```

The `Matches` class is the simplest matcher available and matches just one token of the specified type.

Match a token of a choice of types
----------------------------------

[](#match-a-token-of-a-choice-of-types)

The `Choice` type just works like the `Matches` type but accepts as many token types as you want:

```
$tokenizer = new Tokenizer('{{[[=', [ Token::NEWLINE, Token::WHITESPACE ]
);
```

According to the compliance test of [CommonMark](http://commonmark.org "CommonMark") this even works!

The two Matchers `Optional` and `Either` have not been mentioned before. `Optional` is quite obvious: it matches the contained matcher if it can or not. But it will always succeed. The `Either` matcher would have been the `Or` matcher if `Or` wasn't a reserved work in PHP. It succeeds as long as one matcher succeeds.

Debugging patterns
------------------

[](#debugging-patterns)

The tokenmatcher package was designed with easy debugging in mind. If you want to know was the matcher actually did, just print it!

The above matcher executed on the tokenized text `***` would

```
$tokenizer = new Tokenizer("***\n");
$tokens = $tokenizer->getTokens();

$matched = $horizontalRulerMatcher->match($tokens);
echo $matched . PHP_EOL;
```

would output:

```
Length checked 3 >= 3 ignored NEWLINE, WHITESPACE has status "Token type matched."
  Sequence has status "Token type matched."
    Length checked 0