PHPackages                             digitaldreams/local-batch-api - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [API Development](/categories/api)
4. /
5. digitaldreams/local-batch-api

ActiveLibrary[API Development](/categories/api)

digitaldreams/local-batch-api
=============================

Drop-in replacement for Anthropic and OpenAI batch APIs, self-hosted against Ollama or LM Studio.

v1.0.1(4w ago)08MITPHPPHP ^8.4CI passing

Since Jun 26Pushed 4w agoCompare

[ Source](https://github.com/digitaldreams/local-batch-api)[ Packagist](https://packagist.org/packages/digitaldreams/local-batch-api)[ RSS](/packages/digitaldreams-local-batch-api/feed)WikiDiscussions main Synced 1w ago

READMEChangelogDependencies (3)Versions (3)Used By (0)

local-batch-api
===============

[](#local-batch-api)

Drop-in replacement for the **Anthropic** and **OpenAI** batch APIs — self-hosted, running against local [Ollama](https://ollama.ai) or [LM Studio](https://lmstudio.ai). No API keys, no cloud costs, no rate limits.

---

Requirements
------------

[](#requirements)

- PHP 8.4+
- Laravel 13+
- Running [Ollama](https://ollama.ai/download) or [LM Studio](https://lmstudio.ai) instance
- A queue worker (`php artisan queue:work`)

---

Installation
------------

[](#installation)

### Step 1 — Install the package

[](#step-1--install-the-package)

```
composer require digitaldreams/local-batch-api
```

### Step 2 — Run migrations

[](#step-2--run-migrations)

```
php artisan migrate
```

This creates two tables: `batches` and `batch_files`.

### Step 3 — Configure your inference backend

[](#step-3--configure-your-inference-backend)

Add to your `.env`:

```
# 'ollama' (default) or 'lmstudio'
INFERENCE_PROVIDER=ollama

# Base URL of your local inference server
INFERENCE_URL=http://localhost:11434   # Ollama default
# INFERENCE_URL=http://localhost:1234  # LM Studio default

# Default model (can be overridden per request)
INFERENCE_MODEL=llama3.2

# Seconds before a single request times out
INFERENCE_TIMEOUT=120

# Parallel requests per batch chunk — keep at 1 for CPU, raise to 3-5 for GPU
INFERENCE_CONCURRENCY=1
```

### Step 4 — Start a queue worker

[](#step-4--start-a-queue-worker)

Batch jobs run asynchronously. The worker must be running:

```
php artisan queue:work
```

---

Two Ways to Use This Package
----------------------------

[](#two-ways-to-use-this-package)

This package supports two independent usage patterns:

Event-basedREST API**Who calls it**Your own Laravel codeAny HTTP client (SDK, curl, external app)**Auth**Laravel's existing authSanctum token (or your middleware)**Routes needed**NoYes**Best for**Internal pipelines, jobs, commandsReplacing Anthropic/OpenAI SDK endpoints---

Approach 1 — Event-based (Internal Usage)
-----------------------------------------

[](#approach-1--event-based-internal-usage)

Use this when your own Laravel application needs to submit and process batches. No HTTP routes required.

### Submitting an Anthropic-format batch

[](#submitting-an-anthropic-format-batch)

Fire a `SubmitAnthropicBatchEvent` event. The package listener picks it up and dispatches the processing job automatically.

```
use BatchApi\Events\SubmitAnthropicBatchEvent;
use BatchApi\Data\Input\AnthropicBatchItemDto;

$items = [
    new AnthropicBatchItemDto(
        customId: 'req-1',
        maxTokens: 512,
        messages: [
            ['role' => 'user', 'content' => 'Summarise this article in one paragraph.'],
        ],
    ),
    new AnthropicBatchItemDto(
        customId: 'req-2',
        maxTokens: 256,
        messages: [
            ['role' => 'user', 'content' => 'What is the capital of France?'],
        ],
        system: 'You are a geography expert.',
    ),
];

event(new SubmitAnthropicBatchEvent($items));
```

### Submitting an OpenAI-format batch

[](#submitting-an-openai-format-batch)

The OpenAI flow requires a file ID. Upload first using the `BatchService`, then fire the event.

```
use BatchApi\BatchService;
use BatchApi\Events\SubmitOpenAiBatchEvent;
use BatchApi\Data\Input\OpenAiBatchItemDto;

$service = app(BatchService::class);

// Build items from raw JSONL or manually
$items = [
    new OpenAiBatchItemDto(
        customId: 'req-1',
        messages: [['role' => 'user', 'content' => 'Hello']],
        maxTokens: 512,
    ),
];

// Create a file record (mirrors OpenAI's file upload step)
$file = $service->uploadFile(
    collect($items)->map(fn ($item) => json_encode([
        'custom_id' => $item->customId,
        'method' => 'POST',
        'url' => '/v1/chat/completions',
        'body' => ['messages' => $item->messages, 'max_tokens' => $item->maxTokens],
    ]))->implode("\n")
);

event(new SubmitOpenAiBatchEvent($file->id, $items));
```

### Listening for results

[](#listening-for-results)

Listen to `BatchCompletedEvent` to act on results when processing finishes:

```
// app/Listeners/HandleBatchCompletedListener.php

use BatchApi\Events\BatchCompletedEvent;
use BatchApi\Data\BatchResultDto;

class HandleBatchCompletedListener
{
    public function handle(BatchCompletedEvent $event): void
    {
        $batch = $event->batch;

        foreach ($event->results as $result) {
            /** @var BatchResultDto $result */
            if ($result->succeeded) {
                // $result->customId   — matches your request's custom_id
                // $result->content    — the model's response text
                // $result->model      — model used
                // $result->inputTokens / $result->outputTokens
            } else {
                // $result->error — failure message
            }
        }
    }
}
```

Register it in `AppServiceProvider::boot()`:

```
// app/Providers/AppServiceProvider.php

use BatchApi\Events\BatchCompletedEvent;
use App\Listeners\HandleBatchCompletedListener;
use Illuminate\Support\Facades\Event;

public function boot(): void
{
    Event::listen(BatchCompletedEvent::class, HandleBatchCompletedListener::class);
}
```

### All available events

[](#all-available-events)

EventPropertiesFired when`BatchCreatedEvent``$batch`, `$items`, `$provider`Batch record saved, job dispatched`BatchProcessingEvent``$batch`Queue worker picks up the job`BatchItemStartedEvent``$batch`, `$dto`Single request about to fire`BatchItemCompletedEvent``$batch`, `$result`Single request finished`BatchCompletedEvent``$batch`, `$results`All requests done`BatchFailedEvent``$batch`, `$exception`Job threw an unrecoverable error`BatchCancelledEvent``$batch`Batch cancelled---

Approach 2 — REST API (External HTTP Clients)
---------------------------------------------

[](#approach-2--rest-api-external-http-clients)

Use this when you want to **point an existing Anthropic or OpenAI SDK** at your local server instead of the cloud. The API surface is identical to the real APIs.

### Step 1 — Register routes with authentication

[](#step-1--register-routes-with-authentication)

Do **not** set `BATCH_API_EXPOSE_ROUTES=true`. Instead, register routes manually inside a protected middleware group so you control authentication.

Install Sanctum if you haven't already:

```
composer require laravel/sanctum
php artisan install:api
```

In your `routes/api.php` (or a service provider), wrap `BatchApi::routes()` with Sanctum middleware:

```
use BatchApi\Facades\BatchApi;

Route::middleware('auth:sanctum')->group(function () {
    BatchApi::routes();
});
```

This registers all 11 endpoints, each requiring a valid Sanctum token.

> **Note:** `BatchApi::routes()` also applies the `api` middleware internally. Wrapping it with `auth:sanctum` stacks both, so your routes have `api` + `auth:sanctum`.

### Step 2 — Issue a token

[](#step-2--issue-a-token)

```
// In a controller or seeder
$token = $user->createToken('batch-api-client')->plainTextToken;
// Pass this token to the HTTP client
```

### Step 3 — Call the API

[](#step-3--call-the-api)

All requests need the token in the `Authorization` header:

```
Authorization: Bearer

```

---

### Anthropic Batch API — Step by Step

[](#anthropic-batch-api--step-by-step)

#### 1. Submit a batch

[](#1-submit-a-batch)

```
POST /api/anthropic/v1/messages/batches
Content-Type: application/json
Authorization: Bearer
```

```
{
  "requests": [
    {
      "custom_id": "req-1",
      "params": {
        "model": "llama3.2",
        "max_tokens": 512,
        "messages": [
          { "role": "user", "content": "Say hello in one sentence." }
        ]
      }
    },
    {
      "custom_id": "req-2",
      "params": {
        "model": "llama3.2",
        "max_tokens": 512,
        "system": "You are a pirate. Always respond like a pirate.",
        "messages": [
          { "role": "user", "content": "What is the capital of France?" }
        ]
      }
    }
  ]
}
```

Response `202 Accepted`:

```
{
  "id": "550e8400-e29b-41d4-a716-446655440000",
  "type": "message_batch",
  "processing_status": "in_progress",
  "request_counts": { "processing": 2, "succeeded": 0, "errored": 0, "canceled": 0, "expired": 0 },
  "created_at": "2026-05-25T10:00:00+00:00",
  "expires_at": "2026-05-26T10:00:00+00:00",
  "ended_at": null,
  "cancel_initiated_at": null,
  "results_url": null
}
```

#### 2. Poll until done

[](#2-poll-until-done)

```
GET /api/anthropic/v1/messages/batches/{id}
Authorization: Bearer
```

Keep polling until `processing_status` is `"ended"`.

#### 3. Fetch results (NDJSON)

[](#3-fetch-results-ndjson)

```
GET /api/anthropic/v1/messages/batches/{id}/results
Accept: application/x-ndjson
Authorization: Bearer
```

Returns `204 No Content` if still processing. When ready, streams one JSON object per line:

```
{"custom_id":"req-1","result":{"type":"succeeded","message":{"id":"msg_abc","type":"message","role":"assistant","model":"llama3.2","content":[{"type":"text","text":"Hello! Great to meet you."}],"stop_reason":"end_turn","usage":{"input_tokens":12,"output_tokens":10}}}}
{"custom_id":"req-2","result":{"type":"errored","error":{"type":"server_error","message":"Ollama timeout"}}}
```

#### Other Anthropic endpoints

[](#other-anthropic-endpoints)

```
GET  /api/anthropic/v1/messages/batches              # list (supports ?limit=&before_id=&after_id=)
POST /api/anthropic/v1/messages/batches/{id}/cancel  # cancel
```

---

### OpenAI Batch API — Step by Step

[](#openai-batch-api--step-by-step)

#### 1. Upload a JSONL file

[](#1-upload-a-jsonl-file)

Create a `.jsonl` file (one request per line):

```
{"custom_id":"req-1","method":"POST","url":"/v1/chat/completions","body":{"model":"llama3.2","messages":[{"role":"user","content":"Hello"}],"max_tokens":512}}
{"custom_id":"req-2","method":"POST","url":"/v1/chat/completions","body":{"model":"llama3.2","messages":[{"role":"user","content":"What is 2+2?"}],"max_tokens":256}}
```

Upload it:

```
POST /api/openai/v1/files
Content-Type: multipart/form-data
Authorization: Bearer

file=@requests.jsonl
purpose=batch
```

Response `201 Created`:

```
{
  "id": "file-abc123",
  "object": "file",
  "purpose": "batch",
  "created_at": 1716631200
}
```

#### 2. Submit the batch

[](#2-submit-the-batch)

```
POST /api/openai/v1/batches
Content-Type: application/json
Authorization: Bearer
```

```
{
  "input_file_id": "file-abc123",
  "endpoint": "/v1/chat/completions",
  "completion_window": "24h"
}
```

Response `201 Created`:

```
{
  "id": "550e8400-e29b-41d4-a716-446655440001",
  "object": "batch",
  "status": "validating",
  "input_file_id": "file-abc123",
  "output_file_id": null,
  "request_counts": { "total": 2, "completed": 0, "failed": 0 }
}
```

#### 3. Poll until completed

[](#3-poll-until-completed)

```
GET /api/openai/v1/batches/{id}
Authorization: Bearer
```

Poll until `status` is `"completed"`. Note the `output_file_id` in the response.

#### 4. Download results

[](#4-download-results)

```
GET /api/openai/v1/files/{output_file_id}/content
Authorization: Bearer
```

Returns JSONL, one result per line:

```
{"id":"batch_req_abc","custom_id":"req-1","response":{"status_code":200,"body":{"id":"chatcmpl-123","object":"chat.completion","model":"llama3.2","choices":[{"index":0,"message":{"role":"assistant","content":"Hello! How can I help?"},"finish_reason":"stop"}],"usage":{"prompt_tokens":10,"completion_tokens":8,"total_tokens":18}}},"error":null}
```

#### Other OpenAI endpoints

[](#other-openai-endpoints)

```
GET  /api/openai/v1/batches                  # list (supports ?limit=&after=)
POST /api/openai/v1/batches/{id}/cancel      # cancel
```

---

### Pointing an existing SDK at this server

[](#pointing-an-existing-sdk-at-this-server)

**Python (Anthropic SDK):**

```
import anthropic

client = anthropic.Anthropic(
    api_key="any-value",           # required by SDK but not validated here
    base_url="http://localhost:8000/api/anthropic",
    default_headers={"Authorization": "Bearer "},
)
```

**Python (OpenAI SDK):**

```
from openai import OpenAI

client = OpenAI(
    api_key="any-value",
    base_url="http://localhost:8000/api/openai",
    default_headers={"Authorization": "Bearer "},
)
```

---

Batch Status Lifecycle
----------------------

[](#batch-status-lifecycle)

```
pending → processing → completed
                     → failed
       → cancelling  → cancelled

```

InternalAnthropic `processing_status`OpenAI `status``pending``in_progress``validating``processing``in_progress``in_progress``completed``ended``completed``failed``ended``failed``cancelling``canceling``cancelling``cancelled``ended``cancelled`Batches expire after 24 hours.

---

Switching to LM Studio
----------------------

[](#switching-to-lm-studio)

1. Open LM Studio → start the local server (default port `1234`)
2. Load a model
3. Update `.env`:

```
INFERENCE_PROVIDER=lmstudio
INFERENCE_URL=http://localhost:1234
INFERENCE_MODEL=your-model-name
```

No other changes needed.

---

Concurrency Tuning
------------------

[](#concurrency-tuning)

`INFERENCE_CONCURRENCY` controls parallel requests per batch chunk.

HardwareValueCPU-only`1`GPU with spare VRAM`3`–`5`---

Postman Collection
------------------

[](#postman-collection)

Import `Local-Batch-API.postman_collection.json`. Set the `baseUrl` variable to your server URL. The collection auto-saves batch IDs and file IDs between requests so you can run folders top-to-bottom without manually copying values.

---

Troubleshooting
---------------

[](#troubleshooting)

**Batches stay `pending` forever** — Queue worker not running. Run `php artisan queue:work`.

**`Ollama timeout` in results** — Model is slow or `INFERENCE_TIMEOUT` too low. Raise to `300`.

**Routes return 404** — Routes not registered. Either set `BATCH_API_EXPOSE_ROUTES=true` (no auth) or call `BatchApi::routes()` manually in a middleware group.

**401 Unauthorized on API routes** — Sanctum token missing or invalid. Pass `Authorization: Bearer ` header.

**`cannot chdir` git error in submodule** — Run `git submodule update --init` in the parent repo.

---

License
-------

[](#license)

MIT

###  Health Score

42

—

FairBetter than 88% of packages

Maintenance94

Actively maintained with recent releases

Popularity7

Limited adoption so far

Community6

Small or concentrated contributor base

Maturity52

Maturing project, gaining track record

 Bus Factor1

Top contributor holds 100% of commits — single point of failure

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Every ~0 days

Total

2

Last Release

29d ago

### Community

Maintainers

![](https://avatars.githubusercontent.com/u/6059541?v=4)[Tuhin Bepari](/maintainers/digitaldreams)[@digitaldreams](https://github.com/digitaldreams)

---

Top Contributors

[![digitaldreams](https://avatars.githubusercontent.com/u/6059541?v=4)](https://github.com/digitaldreams "digitaldreams (8 commits)")

---

Tags

anthropic-batch-apibatch-apilaravellmstudioollamaopenai-batch

###  Code Quality

TestsPHPUnit

### Embed Badge

![Health badge](/badges/digitaldreams-local-batch-api/health.svg)

```
[![Health](https://phpackages.com/badges/digitaldreams-local-batch-api/health.svg)](https://phpackages.com/packages/digitaldreams-local-batch-api)
```

###  Alternatives

[statamic/cms

The Statamic CMS Core Package

4.8k3.6M1.0k](/packages/statamic-cms)[darkaonline/l5-swagger

OpenApi or Swagger integration to Laravel

2.9k37.6M141](/packages/darkaonline-l5-swagger)[knuckleswtf/scribe

Generate API documentation for humans from your Laravel codebase.✍

2.3k14.2M63](/packages/knuckleswtf-scribe)[duncanmcclean/statamic-cargo

Comprehensive e-commerce addon for Statamic. Build bespoke e-commerce sites without the complexity.

3518.3k](/packages/duncanmcclean-statamic-cargo)[scriptdevelop/whatsapp-manager

Paquete para manejo de WhatsApp Business API en Laravel

783.8k](/packages/scriptdevelop-whatsapp-manager)[weapnl/laravel-junction

Easily create a REST API with extended functionality, such as eager loading, searching, filtering, and more.

2214.3k](/packages/weapnl-laravel-junction)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)