PHPackages                             resoul/data-reader - PHPackages - PHPackages  [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register)

1. [Directory](/)
2. /
3. [Utility & Helpers](/categories/utility)
4. /
5. resoul/data-reader

ActiveLibrary[Utility & Helpers](/categories/utility)

resoul/data-reader
==================

description

0.1.1(7mo ago)65proprietaryPHP

Since Dec 24Pushed 6mo ago1 watchersCompare

[ Source](https://github.com/resoul/data-reader)[ Packagist](https://packagist.org/packages/resoul/data-reader)[ RSS](/packages/resoul-data-reader/feed)WikiDiscussions main Synced 1mo ago

READMEChangelog (2)DependenciesVersions (4)Used By (0)

Advanced Usage
--------------

[](#advanced-usage)

### Data Validation and Filtering

[](#data-validation-and-filtering)

```
use DataReader\Config\BaseConfig;

class ValidatedUserConfig extends BaseConfig
{
    public function __construct()
    {
        // Multiple validators
        $this->addValidator(function($item) {
            return isset($item['email']) && filter_var($item['email'], FILTER_VALIDATE_EMAIL);
        });

        $this->addValidator(function($item) {
            return isset($item['age']) && $item['age'] >= 18;
        });

        // Field mapping
        $this->setFieldMapping([
            0 => 'name',
            1 => 'email',
            2 => 'age',
            3 => 'country'
        ]);
    }

    public function configureItem($item): ?array
    {
        $mapped = $this->mapFields($item);

        // Skip invalid items
        if (!$this->validateItem($mapped)) {
            return null;
        }

        return [
            'name' => ucwords(strtolower($mapped['name'])),
            'email' => strtolower($mapped['email']),
            'age' => (int)$mapped['age'],
            'country' => strtoupper($mapped['country']),
            'is_adult' => $mapped['age'] >= 18
        ];
    }

    public function configureFirstItem($item)
    {
        return false; // Skip headers
    }
}
```

### Chaining Multiple Transformations

[](#chaining-multiple-transformations)

```
class MultiStepConfig implements ConfigInterface
{
    private array $processors = [];

    public function addProcessor(callable $processor): self
    {
        $this->processors[] = $processor;
        return $this;
    }

    public function configureItem($item): array
    {
        $result = $item;

        foreach ($this->processors as $processor) {
            $result = $processor($result);
            if ($result === null) {
                break; // Skip this item
            }
        }

        return $result;
    }

    public function configureFirstItem($item)
    {
        return false;
    }
}

// Usage
$config = new MultiStepConfig();
$config->addProcessor(function($item) {
    // Step 1: Clean data
    return array_map('trim', $item);
})
->addProcessor(function($item) {
    // Step 2: Validate
    return filter_var($item[1], FILTER_VALIDATE_EMAIL) ? $item : null;
})
->addProcessor(function($item) {
    // Step 3: Transform
    return [
        'name' => $item[0],
        'email' => strtolower($item[1]),
        'created' => date('Y-m-d H:i:s')
    ];
});
```

### Working with Large Datasets

[](#working-with-large-datasets)

```
class StreamingConfig extends BaseConfig
{
    private int $processed = 0;
    private int $memoryLimit;

    public function __construct(int $memoryLimitMB = 128)
    {
        $this->memoryLimit = $memoryLimitMB * 1024 * 1024;
    }

    public function configureItem($item): array
    {
        $this->processed++;

        // Memory management
        if ($this->processed % 1000 === 0) {
            $usage = memory_get_usage(true);

            if ($usage > $this->memoryLimit) {
                gc_collect_cycles();
                error_log("Memory usage: " . round($usage / 1024 / 1024, 2) . "MB after processing {$this->processed} items");
            }
        }

        return $this->processItem($item);
    }

    private function processItem($item): array
    {
        // Your processing logic here
        return [
            'id' => $item[0],
            'data' => $item[1],
            'processed_at' => time()
        ];
    }
}
```

### Custom Resource with Pagination

[](#custom-resource-with-pagination)

```
use DataReader\Resource\Resource;
use DataReader\ResourceInterface;
use DataReader\ConfigInterface;

class PaginatedApiResource extends Resource implements ResourceInterface
{
    private string $baseUrl;
    private int $perPage;
    private array $headers;

    public function __construct(string $baseUrl, int $perPage = 100, array $headers = [])
    {
        $this->baseUrl = $baseUrl;
        $this->perPage = $perPage;
        $this->headers = $headers;
    }

    public function apply(ConfigInterface $config): array
    {
        $allItems = [];
        $page = 1;

        do {
            $url = $this->baseUrl . "?page={$page}&per_page={$this->perPage}";
            $response = $this->makeRequest($url);
            $data = json_decode($response, true);

            if (empty($data['items'])) {
                break;
            }

            foreach ($data['items'] as $item) {
                $processed = $config->configureItem($item);
                if ($processed !== null) {
                    $allItems[] = $processed;
                }
            }

            $page++;
        } while (count($data['items']) === $this->perPage);

        $this->setData($allItems);
        return $this->getData();
    }

    private function makeRequest(string $url): string
    {
        $context = stream_context_create([
            'http' => [
                'method' => 'GET',
                'header' => implode("\r\n", $this->headers)
            ]
        ]);

        $result = file_get_contents($url, false, $context);

        if ($result === false) {
            throw new ResourceException("Failed to fetch data from: {$url}");
        }

        return $result;
    }

    public function setData($data): void
    {
        $this->data = $data;
    }

    public function getData(): array
    {
        return $this->data ?? [];
    }
}
```

Testing
-------

[](#testing)

### Unit Testing Example

[](#unit-testing-example)

```
use PHPUnit\Framework\TestCase;
use DataReader\Reader;
use DataReader\Resource\ArrayData;
use DataReader\Output\Json;

class ReaderTest extends TestCase
{
    public function testBasicDataProcessing(): void
    {
        $data = [
            ['John', 'john@example.com', '30'],
            ['Jane', 'jane@example.com', '25']
        ];

        $config = new class implements ConfigInterface {
            public function configureItem($item): array {
                return [
                    'name' => $item[0],
                    'email' => $item[1],
                    'age' => (int)$item[2]
                ];
            }

            public function configureFirstItem($item) {
                return $this->configureItem($item);
            }
        };

        $reader = new Reader(
            new ArrayData($data),
            new Json(),
            $config
        );

        $result = $reader->run();
        $decoded = json_decode($result, true);

        $this->assertCount(2, $decoded);
        $this->assertEquals('John', $decoded[0]['name']);
        $this->assertEquals(30, $decoded[0]['age']);
    }
}
```

Troubleshooting
---------------

[](#troubleshooting)

### Common Issues

[](#common-issues)

**1. Memory Exhaustion with Large Files**

```
// Solution: Use chunked processing
ini_set('memory_limit', '512M');

class ChunkedFileProcessor
{
    public function processFile(string $filename, int $chunkSize = 1000): void
    {
        $handle = fopen($filename, 'r');
        $chunk = [];
        $count = 0;

        while (($line = fgetcsv($handle)) !== false) {
            $chunk[] = $line;
            $count++;

            if ($count >= $chunkSize) {
                $this->processChunk($chunk);
                $chunk = [];
                $count = 0;
                gc_collect_cycles();
            }
        }

        if (!empty($chunk)) {
            $this->processChunk($chunk);
        }

        fclose($handle);
    }

    private function processChunk(array $chunk): void
    {
        $reader = new Reader(
            new ArrayData($chunk),
            new Json(),
            new MyConfig()
        );

        echo $reader->run();
    }
}
```

**2. Character Encoding Issues**

```
class EncodingAwareConfig extends BaseConfig
{
    private string $encoding;

    public function __construct(string $encoding = 'UTF-8')
    {
        $this->encoding = $encoding;
    }

    public function configureItem($item): array
    {
        // Convert encoding
        foreach ($item as $key => $value) {
            if (is_string($value)) {
                $item[$key] = mb_convert_encoding($value, 'UTF-8', $this->encoding);
            }
        }

        return $this->processItem($item);
    }
}
```

**3. Invalid Data Handling**

```
class RobustConfig extends BaseConfig
{
    public function configureItem($item): ?array
    {
        try {
            // Validate required fields
            if (empty($item[0]) || empty($item[1])) {
                return null; // Skip invalid records
            }

            return [
                'name' => $this->sanitizeString($item[0]),
                'email' => $this->validateEmail($item[1]),
                'age' => $this->parseAge($item[2] ?? null)
            ];
        } catch (\Exception $e) {
            error_log("Error processing item: " . json_encode($item) . " - " . $e->getMessage());
            return null;
        }
    }

    private function sanitizeString(?string $value): string
    {
        return trim(strip_tags($value ?? ''));
    }

    private function validateEmail(?string $email): ?string
    {
        $clean = filter_var($email, FILTER_VALIDATE_EMAIL);
        return $clean ?: null;
    }

    private function parseAge($value): ?int
    {
        if ($value === null || $value === '') {
            return null;
        }

        $age = (int)$value;
        return ($age > 0 && $age < 150) ? $age : null;
    }
}
```

Roadmap
-------

[](#roadmap)

### Version 1.0 (In Progress)

[](#version-10-in-progress)

- Complete ArrayData implementation
- Add comprehensive error handling with custom exceptions
- Implement XML and CSV output formats
- Create BaseConfig class with field mapping and validation
- Add factory methods for common use cases
- Add unit tests with PHPUnit
- Implement streaming support for large files (&gt;100MB)

### Version 1.1 (Planned)

[](#version-11-planned)

- Add Excel file format support (.xlsx, .xls)
- Implement caching mechanisms for processed data
- Add batch processing capabilities
- Create CLI tool for command-line usage
- Add data transformation pipelines
- Implement async processing support

### Version 1.2 (Future)

[](#version-12-future)

- Add database resource connectors (MySQL, PostgreSQL, SQLite)
- Implement API resource connectors (REST, GraphQL)
- Add data validation rule system
- Create visual data mapping interface
- Add support for nested data structures
- Implement data diff and merge capabilities

### Long-term Goals

[](#long-term-goals)

- Plugin system for third-party extensions
- Web-based data transformation UI
- Integration with popular frameworks (Laravel, Symfony)
- Performance optimization for big data processing
- Machine learning integration for data analysis# Data Reader

A flexible and robust PHP library for reading, processing, and outputting data from various sources with configurable transformation pipelines, validation, and multiple output formats.

Features
--------

[](#features)

- **Multiple Data Sources**: Support for files (CSV, JSON, XML), arrays, and extensible resource types
- **Configurable Processing**: Transform and validate data during reading with custom configuration classes
- **Multiple Output Formats**: JSON, XML, CSV output with customizable options
- **Robust Error Handling**: Custom exceptions and comprehensive validation
- **Clean Architecture**: Interface-driven design following SOLID principles
- **Type Safety**: Full PHP 7.4+ type hints and strict typing
- **Easy Integration**: Simple fluent API for chaining operations
- **Factory Methods**: Quick setup for common use cases
- **Field Mapping &amp; Validation**: Built-in support for data transformation and validation

Installation
------------

[](#installation)

Install via Composer:

```
composer require resoul/data-reader
```

Quick Start
-----------

[](#quick-start)

```