PHPackages                             asifshoumik/kafka-laravel - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Queues & Workers](/categories/queues)
4. /
5. asifshoumik/kafka-laravel

ActiveLibrary[Queues & Workers](/categories/queues)

asifshoumik/kafka-laravel
=========================

A robust Kafka queue driver for Laravel microservices with support for delayed jobs, retry mechanisms, and comprehensive error handling.

v1.0.1(4mo ago)1178MITPHPPHP ^8.1

Since Jul 23Pushed 4mo agoCompare

[ Source](https://github.com/asifshoumik/kafka-laravel)[ Packagist](https://packagist.org/packages/asifshoumik/kafka-laravel)[ Docs](https://github.com/asifshoumik/kafka-laravel)[ RSS](/packages/asifshoumik-kafka-laravel/feed)WikiDiscussions main Synced 1mo ago

READMEChangelog (2)Dependencies (6)Versions (3)Used By (0)

Kafka Queue Driver for Laravel
==============================

[](#kafka-queue-driver-for-laravel)

A robust and feature-rich Kafka queue driver for Laravel microservices with comprehensive error handling, delayed jobs, retry mechanisms, and dead letter queue support.

[![Latest Version on Packagist](https://camo.githubusercontent.com/ba3861612059e89e231f1568d90b7f9bc09ae418676cacf569f7c7c0c9ad1259/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f762f6173696673686f756d696b2f6b61666b612d6c61726176656c2e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/asifshoumik/kafka-laravel)[![Total Downloads](https://camo.githubusercontent.com/09f6a84132478de2d8d57a8e722332ee50016692dc0551e5918beb8c3f7410bf/68747470733a2f2f696d672e736869656c64732e696f2f7061636b61676973742f64742f6173696673686f756d696b2f6b61666b612d6c61726176656c2e7376673f7374796c653d666c61742d737175617265)](https://packagist.org/packages/asifshoumik/kafka-laravel)

Features
--------

[](#features)

- ✅ **Full Laravel Queue Integration** - Works with Laravel's built-in queue system
- ✅ **Error Handling & Retries** - Comprehensive error handling with configurable retry mechanisms
- ✅ **Dead Letter Queue** - Failed jobs are automatically moved to a dead letter queue
- ✅ **Delayed Jobs** - Support for delayed job execution
- ✅ **Multiple Security Protocols** - PLAINTEXT, SSL/TLS, SASL authentication
- ✅ **Configurable Timeouts** - Fine-tuned timeout configurations for different scenarios
- ✅ **Batch Processing** - Optimized batch processing for better performance
- ✅ **Monitoring & Logging** - Comprehensive logging and monitoring capabilities
- ✅ **Console Commands** - Artisan commands for queue management
- ✅ **Topic Mapping** - Route different job types to different Kafka topics

Requirements
------------

[](#requirements)

- PHP 8.1 or higher
- Laravel 10.0 or higher
- rdkafka PHP extension
- Kafka 2.1.0 or higher

Installation
------------

[](#installation)

> 📋 **Important:** The rdkafka PHP extension is required. See [INSTALLATION.md](INSTALLATION.md) for detailed installation instructions.

### 1. Install rdkafka Extension First

[](#1-install-rdkafka-extension-first)

**Important:** The rdkafka PHP extension must be installed before installing this package.

**Ubuntu/Debian:**

```
sudo apt-get install librdkafka-dev
sudo pecl install rdkafka
# Add extension=rdkafka.so to your php.ini file
```

**macOS (with Homebrew):**

```
brew install librdkafka
pecl install rdkafka
# Add extension=rdkafka.so to your php.ini file
```

**Windows:**

1. Download the appropriate DLL from the [PECL rdkafka page](https://pecl.php.net/package/rdkafka)
2. Place the DLL in your PHP extensions directory
3. Add `extension=rdkafka` to your php.ini file
4. Restart your web server

**Verify Installation:**

```
php -m | grep rdkafka
```

### 2. Install the Package

[](#2-install-the-package)

```
composer require asifshoumik/kafka-laravel
```

### 3. Publish Configuration (Optional)

[](#3-publish-configuration-optional)

The package works out of the box, but you can publish the configuration file for customization:

```
php artisan vendor:publish --tag=kafka-queue-config
```

This creates `config/kafka-queue.php` with all available options.

### 4. Configure Environment Variables

[](#4-configure-environment-variables)

Add the following to your `.env` file:

```
# Queue Configuration
QUEUE_CONNECTION=kafka

# Kafka Configuration
KAFKA_BOOTSTRAP_SERVERS=localhost:9092
KAFKA_GROUP_ID=laravel-consumer-group
KAFKA_DEFAULT_TOPIC=laravel-jobs
KAFKA_DEAD_LETTER_QUEUE=laravel-failed-jobs

# Security (if needed)
KAFKA_SECURITY_PROTOCOL=PLAINTEXT
KAFKA_SASL_MECHANISMS=PLAIN
KAFKA_SASL_USERNAME=
KAFKA_SASL_PASSWORD=

# Performance Tuning
KAFKA_BATCH_SIZE=16384
KAFKA_LINGER_MS=5
KAFKA_COMPRESSION_TYPE=none
KAFKA_MAX_ATTEMPTS=3
```

That's it! The package automatically registers the Kafka queue connection with all configurations, including:

- **Main connection settings** (brokers, security, SSL, etc.)
- **Topic mapping** for routing different job types to different topics
- **Consumer configuration** for worker management
- **Monitoring and logging** settings

No manual configuration of `config/queue.php` is required.

Quick Start
-----------

[](#quick-start)

### 1. Dispatch Jobs

[](#1-dispatch-jobs)

Any existing Laravel job will work with Kafka:

```
use App\Jobs\ProcessOrder;

// Dispatch to default topic
dispatch(new ProcessOrder($order));

// Dispatch to specific topic
ProcessOrder::dispatch($order)->onQueue('orders');

// Dispatch with delay
ProcessOrder::dispatch($order)->delay(now()->addMinutes(10));
```

### 2. Start Consumer

[](#2-start-consumer)

```
php artisan kafka:work
```

### 3. Monitor Jobs

[](#3-monitor-jobs)

```
# Work specific queue/topic
php artisan kafka:work orders

# Work with options
php artisan kafka:work --timeout=60 --tries=3
```

Usage
-----

[](#usage)

### Basic Job Dispatching

[](#basic-job-dispatching)

```
use App\Jobs\ProcessPodcast;

// Dispatch to default topic
ProcessPodcast::dispatch($podcast);

// Dispatch to specific topic
ProcessPodcast::dispatch($podcast)->onQueue('emails');

// Dispatch with delay
ProcessPodcast::dispatch($podcast)->delay(now()->addMinutes(10));
```

### Creating Jobs

[](#creating-jobs)

```