PHPackages                             nacento/connector - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [File & Storage](/categories/file-storage)
4. /
5. nacento/connector

ActiveMagento2-module[File & Storage](/categories/file-storage)

nacento/connector
=================

R2/S3 Akeneo/Magento shared images connector

v0.1.1(2mo ago)01MITPHPPHP ^8.1 || ^8.2 || ^8.3

Since Feb 28Pushed 2mo agoCompare

[ Source](https://github.com/nacento-conn/nacento-connector)[ Packagist](https://packagist.org/packages/nacento/connector)[ RSS](/packages/nacento-connector/feed)WikiDiscussions main Synced 1mo ago

READMEChangelogDependencies (7)Versions (2)Used By (0)

Nacento Connector
=================

[](#nacento-connector)

*A Magento 2.4.8 module to synchronize product image galleries from a PIM, using S3/R2 as the source of truth and optimizing performance through ETag change detection and asynchronous bulk processing.*

> **ALPHA – HIGHLY EXPERIMENTAL**
> This project is **alpha-stage** and **not suitable for production**. Use at your own risk.
> If you have questions, please open an **Issue** – but **do not expect quick responses**.

---

What is this?
-------------

[](#what-is-this)

This is a custom Magento 2.4.8 module that exposes a Web API to **synchronize product image galleries** between a PIM (like Akeneo) and Magento, using an S3-compatible object storage (like Cloudflare R2) as the source of truth for the image files.

The primary goal is to **bypass Magento's native image processing and copying**, drastically reducing bandwidth, processing time, and accelerating gallery updates, especially for large catalogs.

Evolution and Key Concepts
--------------------------

[](#evolution-and-key-concepts)

This module has evolved from a simple single-SKU endpoint into a more comprehensive bulk processing system focused on performance and flexibility.

1. **Bulk Processing:** The module offers a bulk endpoint to process hundreds or thousands of SKUs in a single request.
2. **Asynchronous Processing:** Requests are queued via Magento's Message Queue for background processing, which is ideal for very large workloads.
3. **ETag Change Detection:** The module uses a lightweight S3 client to perform `HEAD` requests and retrieve the **ETag** of each image. This allows it to detect if a file's content has actually changed, avoiding unnecessary database writes and only updating metadata if the file itself is unchanged.

---

Status
------

[](#status)

- **Stage:** Alpha (breaking changes can happen anytime)
- **Target Magento:** 2.4.8 (PHP 8.1/8.2/8.3)
- **License:** MIT

---

Features
--------

[](#features)

- **One REST Web API Endpoint:**
    - **Asynchronous bulk** processing via Magento's Message Queue.
- **Recent Robustness Improvements (internal, payload unchanged):**
    - Shared payload normalization and validation before queueing.
    - Stable `operation_key` generation for safer `magento_operation` status updates.
    - Transactional per-SKU gallery writes (best effort atomicity).
    - Retry classification (retriable vs non-retriable queue failures).
    - Payload-owned managed roles (clear then reassign from payload).
- **Performance Optimization:**
    - Skips Magento’s image processing for improved speed.
    - Uses a dedicated **S3/R2 client** for `HEAD` requests to check **ETags** and skips no-op writes when nothing changed.
- **Complete Gallery Management:**
    - Assigns **roles** (`base`, `small_image`, `thumbnail`, `swatch_image`), **labels**, **position**, and **disabled** status.
- **External Source of Truth:**
    - Works directly with file paths in S3/R2 storage, with no binary uploads to Magento.

---

Compatibility (Tested)
----------------------

[](#compatibility-tested)

This module has been **only tested** under the following environment:

- **Magento Open Source:** 2.4.8
- **PHP:** 8.3-fpm
- **Web Server:** nginx 1.24
- **Message Queue:** RabbitMQ 4.1
- **Cache:** Valkey 8.1
- **Database:** MariaDB 11.4
- **Search Engine:** OpenSearch 2.12

> ⚠️ Other versions of Magento, PHP, or related services **have not been tested**.
> Use in different environments **at your own risk**, in fact, use it in general at your own risk.

---

Prerequisite: Configure Remote Storage (S3/R2/MinIO)
----------------------------------------------------

[](#prerequisite-configure-remote-storage-s3r2minio)

For this module to work in its **optimized mode** (ETag change detection + external source-of-truth workflow), Magento should be configured to use an S3-compatible remote storage driver in `app/etc/env.php`.

Without remote storage, asynchronous bulk processing still works, but ETag-based no-op optimization is skipped.

Example (S3/MinIO/R2 compatible):

```
'remote_storage' => [
    'driver' => 'aws-s3',
    'config' => [
        'bucket' => 'your-catalog-bucket',
        'region' => 'auto',
        'endpoint' => 'https://.r2.cloudflarestorage.com',
        'use_path_style_endpoint' => true,
        'bucket_endpoint' => false,
        'credentials' => [
            'key' => 'your-access-key',
            'secret' => 'your-secret-key'
        ]
    ]
],
```

---

S3 Bucket Layout & Paths (Required)
---------------------------------------

[](#s3-bucket-layout--paths-required)

Magento’s S3 storage driver treats the **bucket root as `pub/media`**. That means product images must live under:

```
/media/catalog/product/

```

This module will not fight against this established behaviour so if you would like to use it, you should adapt your S3 configuration so both Magento and your PIM access the same path.

Akeneo S3 adapter based on Symfony Framework can be configured by setting a **prefix**:

```
oneup_flysystem:
  adapters:
    catalog_storage_adapter:
      awss3v3:
        client: 'Aws\S3\S3Client'
        bucket: 'catalog'                 # your bucket name
        prefix: 'media/catalog/product'   # required for Magento compatibility
```

With this, Akeneo will write to:

```
/media/catalog/product/

```

---

Installation
------------

[](#installation)

Add the repository to your Magento project's `composer.json`:

```
{
  "repositories": [
    { "type": "vcs", "url": "git@github.com:xturavaina/nacento-connector.git" }
  ]
}
```

Then, install and enable the module:

```
composer require nacento/connector:*
bin/magento module:enable Nacento_Connector
bin/magento setup:upgrade
bin/magento cache:flush
```

> ⚠️ ***IMPORTANT***: For the bulk async operations to work you should setup a consumer on your cron, supervisord or whatever you use to process queues.

A potential minimalist entry for supervisord will look like:

```
[program:magento-consumer-nacento-gallery]
command=php bin/magento queue:consumers:start nacento.gallery.consumer
directory=/var/www/html
autostart=true
autorestart=true
stdout_logfile=/var/log/magento-consumers/consumer_nacento_gallery.log
stderr_logfile=/var/log/magento-consumers/consumer_nacento_gallery.err
```

However, your environment and settings may differ. Adapt accordingly.

Uninstallation
--------------

[](#uninstallation)

You can uninstall the module in two ways: removing only the code (leaving the database table intact) or performing a complete removal that also cleans up the database, the queue and the exchange in RabbitMQ

### Option 1: Remove Code Only (Standard Method)

[](#option-1-remove-code-only-standard-method)

This is the standard Magento process. It will remove the module's code, but the `nacento_media_gallery_meta` database table will be preserved in case you decide to reinstall the module later or you forgot to make a backup.

```
composer remove nacento/connector
bin/magento setup:upgrade
```

---

### Option 2: Complete Removal (Code + Database Table + RabbitMQ queue and Exchange)

[](#option-2-complete-removal-code--database-table--rabbitmq-queue-and-exchange)

> **Warning:** This process is irreversible and will permanently delete the `nacento_media_gallery_meta` table and all its data.

This repository currently **does not ship a custom uninstall script** for queue/exchange cleanup. You can still use Magento's `module:uninstall` command, but RabbitMQ queue/exchange cleanup should be treated as a **manual step** (see below).

```
bin/magento module:uninstall Nacento_Connector --remove-data
```

---

### Cleaning Up RabbitMQ (Manual Step)

[](#cleaning-up-rabbitmq-manual-step)

If queue was not deleted, you can manually remove queues or exchanges from your RabbitMQ server. This must be done to ensure a completely clean environment.

Follow these steps after uninstalling the module:

```
1.  Log in to the RabbitMQ Management UI (typically at `http://your-server:15672`).
2.  Navigate to the **Queues** tab.
3.  Find and click on the queue named `nacento.gallery.process`.
4.  Scroll to the bottom of the page and click the **Delete** button.
5.  Navigate to the **Exchanges** tab.
6.  Find and click on the exchange named `nacento.gallery.process`.
7.  Scroll to the bottom of the page and click the **Delete** button.

```

---

Message Queue & Consumers
-----------------------------

[](#message-queue--consumers)

This module uses a topic named **`nacento.gallery.process`** (publisher) and a consumer named **`nacento.gallery.consumer`** (listens to queue `nacento.gallery.process`).

Current module configuration uses Magento's default queue connection for the consumer, so backend selection (AMQP/DB) follows your Magento environment configuration.

Common commands:

see all consumers

```
bin/magento queue:consumers:list
```

start the connector consumer

```
bin/magento queue:consumers:start nacento.gallery.consumer -vvv
```

Publishing does not require a running consumer; messages will queue up and be processed when the consumer runs.

---

API Endpoints
-------------

[](#api-endpoints)

The module exposes a single endpoint for asynchronous bulk processing. Please check `etc/webapi.xml` for the definitive definitions.

### Bulk Processing (Asynchronous)

[](#bulk-processing-asynchronous)

Submits a batch to Magento's message queue for background processing. The response is immediate and contains a `bulk_uuid` for tracking. This is the best option for large batches.

- **Endpoint:** `POST /rest/V1/nacento-connector/products/media/bulk/async`
- **Sample Payload:**

```
{
  "request": {
    "request_id": "op-12345",
    "items": [
      {
        "sku": "SKU-001",
        "images": [{ "file_path": "...", "label": "...", "roles": ["base"] }]
      },
      {
        "sku": "SKU-002",
        "images": [{ "file_path": "...", "label": "...", "roles": ["base"] }]
      }
    ]
  }
}
```

- **Sample Response:**

```
{
    "bulk_uuid": "f8d3c1a3-5b8e-4a9f-8c7e-1a2b3c4d5e6f",
    "request_items": [
        { "id": 1, "data_hash": "...", "status": "accepted", "error_message": null },
        { "id": 2, "data_hash": "...", "status": "accepted", "error_message": null }
    ],
    "errors": false
}
```

> Note: `request_items` are acknowledgment entries, not a strict mirror of input order/cardinality. With SKU deduplication (last-wins) and validation rejection, the number of returned items may differ from the number submitted.

### Behavior Notes (Current Implementation)

[](#behavior-notes-current-implementation)

- **Payload is the public contract** and remains unchanged.
- **Bulk dedupe by SKU is last-wins** (only the last valid item for a SKU is queued).
- Invalid items (for example empty `sku`, empty `images`, or images without `file_path`) are **rejected in the async acknowledgment**, and `errors` may be `true`.
- `request_items[].data_hash` is a stable hash derived from SKU (or fallback item id when SKU is empty). It is intended for acknowledgment correlation, not as a full payload checksum/idempotency key.
- `request_id` is accepted and propagated for **logging/correlation** (not persisted idempotency yet).
- Gallery synchronization is **upsert-only**:
    - listed images are inserted/updated
    - images not listed are **not deleted**
- Incoming image `file_path` values are normalized to Magento gallery value format (leading slash), while legacy no-leading-slash rows are still recognized for lookup compatibility.
- Managed image roles are **payload-owned**:
    - when all payload images are valid, Magento role attributes (`image`, `small_image`, `thumbnail`, `swatch_image`) are cleared first
    - then reassigned from payload entries (last role assignment wins)
    - if any payload image is invalid for the SKU, managed role synchronization is skipped to avoid accidental role wipes
- Unknown image roles in payload are ignored.

---

Caveats & Limitations
-------------------------

[](#caveats--limitations)

- Assumes your frontend/media layer can resolve and serve the stored media paths correctly (CORS, CDN, permissions are your responsibility).
- Some Magento features or 3rd-party modules may **expect images to exist physically in `pub/media`**. Validate compatibility.
- This module intentionally writes gallery rows **directly** (bypassing Magento's native image import/process pipeline). Validate compatibility with modules expecting native side effects.
- Error handling and retries are improved, but this is still **alpha-stage** software and needs operational monitoring.

---

Roadmap (subject to change)
---------------------------

[](#roadmap-subject-to-change)

- Enhance the the bussiness logic, as of today, the bulk sync/async is invoking the single sku processing logic, LOL!
- Enhance the statistics returned in bulk processing results. (maybe improve integration with magento default uuid)
- Expand integration/E2E test coverage (queue + DB + S3 driver scenarios).

---

Troubleshooting
---------------

[](#troubleshooting)

- **“Data in topic must be of type OperationInterface”**
    Your topic is typed (Async/Bulk). The module publishes a valid `OperationInterface`, so this should only happen if custom topology overrides were installed. Re-run `bin/magento setup:upgrade`.
- **No messages seen in RabbitMQ logs**
    Magento validates message type & mapping **before** connecting to AMQP. Check your `etc/queue.xml`, `etc/queue_consumer.xml`, and `etc/queue_publisher.xml` configuration.
- **Messages are queued but `magento_operation` statuses do not change**
    Common causes:

    1. Consumer is not running (`nacento.gallery.consumer`).
    2. Consumer processed the message but failed before status update (check Magento logs for `GalleryConsumer` / `GalleryProcessor` errors).
    3. Status update lookup failed (this module now updates by `operation_key` first, then falls back to `id`).
    4. If the operation row no longer exists, the consumer logs the condition and drops the message without retry by design (even if the processing error was retriable).
- **`magento_bulk`, `magento_operation`, and `magento_acknowledged_bulk` look inconsistent**

    This is often an **async lifecycle visibility** issue, not necessarily a queue-vs-cron problem.

    Typical async bulk lifecycle (what Magento writes):

    1. `magento_bulk`: one row per bulk request (`bulk_uuid`, metadata, description/user context).
    2. `magento_operation`: one row per queued item/SKU (serialized payload + status transitions).
    3. `magento_acknowledged_bulk`: admin/user acknowledgment tracking for bulk notifications (not required for queue processing itself, and it may remain empty).

    Important notes:

    - **Queue vs cron:** RabbitMQ consumers do the actual processing. Cron is only one way to start/manage consumers (`consumers_runner`). Running the consumer under `supervisord` is valid.
    - If the consumer is down, `magento_bulk` and `magento_operation` rows may exist while RabbitMQ still has pending messages.
    - If RabbitMQ is empty but operations remain open, inspect Magento logs and status update behavior.
    - `magento_acknowledged_bulk` is not the source of truth for whether your SKU/gallery processing completed.
- **Quick DB checks for async bulk debugging**

    Replace `` with the UUID returned by the API:

    ```
    SELECT * FROM magento_bulk WHERE uuid = '';

    SELECT id, bulk_uuid, topic_name, operation_key, status, error_code, result_message
    FROM magento_operation
    WHERE bulk_uuid = ''
    ORDER BY id;

    SELECT * FROM magento_acknowledged_bulk WHERE bulk_uuid = '';
    ```

    Also verify:

    - RabbitMQ queue depth for `nacento.gallery.process`
    - consumer process health for `bin/magento queue:consumers:start nacento.gallery.consumer`

---

License
-------

[](#license)

**MIT © Nacento**

###  Health Score

34

—

LowBetter than 77% of packages

Maintenance86

Actively maintained with recent releases

Popularity1

Limited adoption so far

Community2

Small or concentrated contributor base

Maturity39

Early-stage or recently created project

How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window.

**Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores.

**Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement.

**Maturity (30%)** — Project age, version count, PHP version support, and release stability.

###  Release Activity

Cadence

Unknown

Total

1

Last Release

80d ago

### Community

Maintainers

![](https://www.gravatar.com/avatar/d70ab1934ec108f3f839aaa16ec5a0ec501367730d10b41d937d375a28c1b116?d=identicon)[nacento](/maintainers/nacento)

---

Tags

s3imagesconnectorPIMakeneomagento2r2

### Embed Badge

![Health badge](/badges/nacento-connector/health.svg)

```
[![Health](https://phpackages.com/badges/nacento-connector/health.svg)](https://phpackages.com/packages/nacento-connector)
```

###  Alternatives

[smile/elasticsuite

Magento 2 merchandising and search engine built on ElasticSearch

8044.5M33](/packages/smile-elasticsuite)[mollie/magento2

Mollie Payment Module for Magento 2

1121.6M10](/packages/mollie-magento2)[akeneo/magento-connector-bundle

Akeneo PIM bundle to export to Magento platform

582.2k](/packages/akeneo-magento-connector-bundle)[akeneo-labs/excel-connector-bundle

Akeneo PIM Excel connector bundle

166.4k](/packages/akeneo-labs-excel-connector-bundle)[opengento/module-category-import-export

This module add the capability to import and export the categories from the back-office.

119.1k](/packages/opengento-module-category-import-export)

PHPackages © 2026

[Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)