PHPackages                             mgribov/apiscrape - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [HTTP & Networking](/categories/http)
4. /
5. mgribov/apiscrape

ActiveLibrary[HTTP & Networking](/categories/http)

mgribov/apiscrape
=================

Generic HTTP REST API scraper with local cache

053PHP

Since Jan 22Pushed 9y ago1 watchersCompare

[ Source](https://github.com/mgribov/apiscrape)[ Packagist](https://packagist.org/packages/mgribov/apiscrape)[ RSS](/packages/mgribov-apiscrape/feed)WikiDiscussions master Synced 4d ago

READMEChangelogDependenciesVersions (1)Used By (0)

[![Code Climate](https://camo.githubusercontent.com/36ef7fb335074ddd01af517964b67e4a1b0c9de85065aad0d72edaee230ee7be/68747470733a2f2f636f6465636c696d6174652e636f6d2f6769746875622f6d677269626f762f6170697363726170652e706e67)](https://codeclimate.com/github/mgribov/apiscrape)

JSON API scraper with local cache
=================================

[](#json-api-scraper-with-local-cache)

Motivation - a transparent GET cache
------------------------------------

[](#motivation---a-transparent-get-cache)

I've encountered several APIs which were not as complex as Twitter or Facebook, but nice enough to utilize JSON and HTTP caching (Etag and Cache-Control) for GET requests.

I needed to scrape them periodically and store their results somewhere for later exploration/use.

[apiscrape](https://github.com/mgribov/apiscrape) does just that - takes a URL with optional (but primitive for now) authentication, and stores the GET responses locally for later usage.
This library can also be used for real-time requests as well of course, not just scraping, and still provide local cache benefits.

It uses [MongoDB](http://www.php.net/manual/en/book.mongo.php) as the cache layer, so JSON is stored exactly as it is returned.
It wraps [cURL](http://www.php.net/manual/en/book.curl.php), so it supports all other HTTP verbs as well, but will cache responses only for GETs.

Limitations - no OAuth
----------------------

[](#limitations---no-oauth)

Currently, only HTTP Basic and URL authentication is supported.
HTTP Basic authentication adds "Authorization: Basic" HTTP header.
URL authentication simply appends a secret to each request URL. (ex: ?secret=1234)
See [src/Scrape/Auth](https://github.com/mgribov/apiscrape/tree/master/src/Scrape/Auth) for details

Install via Composer into existing project
------------------------------------------

[](#install-via-composer-into-existing-project)

```
curl -sS https://getcomposer.org/installer | php # if composer is not installed
./composer.phar require mgribov/apiscrape

```

Examples
--------

[](#examples)

For a real-world complex example which inspired this library see [Triptelligent PHP API wrapper](https://github.com/mgribov/php-triptelligent)

See [examples/](https://github.com/mgribov/apiscrape/tree/master/examples) folder for all use cases
Make sure you install composer and run "./composer.phar dump-autoload" first to use the examples if you just want to play around with it

```