PHPackages detain/php-duplication-samples - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. [Testing & Quality](/categories/testing) 4. / 5. detain/php-duplication-samples ActiveProject[Testing & Quality](/categories/testing) detain/php-duplication-samples ============================== Dev-only generator for the graduated duplication-detection test-set corpus (testsets/). Corpus files themselves are dependency-free; this dependency is used only by gen/. 00PHP Since Jul 18Pushed 1mo agoCompare [ Source](https://github.com/detain/php-duplication-samples)[ Packagist](https://packagist.org/packages/detain/php-duplication-samples)[ RSS](/packages/detain-php-duplication-samples/feed)WikiDiscussions master Synced 1w ago READMEChangelogDependenciesVersions (1)Used By (0) PHP Duplication Samples ======================= [](#php-duplication-samples) A comprehensive reference corpus and benchmark suite for PHP code duplication patterns. This project provides a structured collection of duplicated code examples across 54 distinct categories, along with refactored solutions and tooling to compare duplicate detection tools. Table of Contents ----------------- [](#table-of-contents) - [Overview](#overview) - [Features](#features) - [Corpus Explorer (Web UI)](#corpus-explorer-web-ui) - [Directory Structure](#directory-structure) - [Duplication Categories](#duplication-categories) - [Benchmark Tools](#benchmark-tools) - [Getting Started](#getting-started) - [Adding New Samples](#adding-new-samples) - [Contributing](#contributing) - [License](#license) - [References](#references) --- Overview -------- [](#overview) Code duplication is one of the most common sources of technical debt in software projects. This project serves as: 1. **Educational Resource** - Understanding the many forms code duplication can take 2. **Benchmark Corpus** - Testing and comparing duplicate detection tools 3. **Refactoring Reference** - Seeing how duplicated code can be deduplicated The corpus contains **54 distinct categories** of duplication patterns, ranging from obvious copy-paste clones to subtle semantic duplication that requires deep analysis to detect. --- Features -------- [](#features) - **54 Duplication Categories** - Comprehensive coverage of duplication types from exact clones to semantic equivalents - **Refactored Solutions** - Each category includes deduplicated implementations showing how to eliminate duplication - **Benchmark Harness** - End-to-end comparison framework for duplicate detection tools - **Multiple Detection Tools** - Support for phpcpd, jscpd, PMD-CPD, Simian, and phpdup - **Ground Truth Scoring** - Synthetic corpus with known duplication topology for precise measurement - **PHP 8.1+ Ready** - Modern PHP patterns and strict typing throughout - **Corpus Explorer** - Browser-based UI for browsing and filtering the test corpus --- Corpus Explorer (Web UI) ------------------------ [](#corpus-explorer-web-ui) A browser-based explorer for viewing and filtering the test corpus. ### Quick Start [](#quick-start) ``` # Open directly in browser (no server needed) open public_html/index.html # Or serve locally python3 -m http.server 8000 -d public_html # Then visit http://localhost:8000 ``` ### Features [](#features-1) - Filter by level, family, clone type, difficulty - Full-text search across titles, descriptions, seeds - Syntax-highlighted code snippets - Deep linking via URL (shareable filter views) ### Rebuilding Data [](#rebuilding-data) If you modify test sets, regenerate the data file: ``` php scripts/generate-corpus-data.php php scripts/extract-snippets.php ``` --- Directory Structure ------------------- [](#directory-structure) ``` php-duplication-samples/ ├── samples/ # 54 categories of duplicated code examples │ ├── algorithm/ # Algorithm duplication (20 samples) │ ├── architectural/ # Architectural patterns (10 samples) │ ├── behavioral/ # Behavioral duplication (20 samples) │ ├── build/ # Build/deployment duplication (9 samples) │ ├── caching/ # Caching patterns (10 samples) │ ├── clone_type_1/ # Type-1 clones (exact copies) (20 samples) │ ├── clone_type_2/ # Type-2 clones (renamed) (10 samples) │ ├── clone_type_3/ # Type-3 clones (modified) (15 samples) │ ├── clone_type_4/ # Type-4 clones (semantic) (10 samples) │ ├── configuration/ # Configuration duplication (20 samples) │ ├── copy_paste/ # Direct copy-paste examples (60 samples) │ ├── cross_service/ # Cross-service duplication (10 samples) │ ├── data/ # Data/constant duplication (20 samples) │ ├── dependency/ # Dependency injection duplication (10 samples) │ ├── documentation/ # Documentation duplication (10 samples) │ ├── error_handling/ # Error handling patterns (various) │ ├── event/ # Event handling duplication (10 samples) │ ├── functional/ # Functional duplication (10 samples) │ ├── knowledge/ # Business knowledge duplication (20 samples) │ ├── lexical/ # Lexical patterns (10 samples) │ ├── localization/ # i18n duplication (10 samples) │ ├── logic/ # Business logic duplication (10 samples) │ ├── mapping/ # Data mapping duplication (10 samples) │ ├── monitoring/ # Monitoring instrumentation (10 samples) │ ├── orm_query_duplication/ # ORM query patterns (10 samples) │ ├── permission/ # Authorization patterns (10 samples) │ ├── process/ # Process/workflow duplication (10 samples) │ ├── protocol/ # Protocol handling (50 samples) │ ├── query/ # Database query duplication (30 samples) │ ├── representation/ # Model duplication (20 samples) │ ├── schema/ # Schema duplication (10 samples) │ ├── semantic/ # Semantic duplication (20 samples) │ ├── serialization/ # Serialization patterns (10 samples) │ ├── structural/ # Structural duplication (20 samples) │ ├── syntactic/ # Syntactic patterns (10 samples) │ ├── temporal/ # Temporal patterns (20 samples) │ ├── test/ # Test duplication (20 samples) │ ├── textual/ # Textual duplication (10 samples) │ ├── type/ # Type-specific duplication (20 samples) │ ├── ui/ # UI duplication (20 samples) │ ├── validation_*/ # Validation patterns (email, password, phone) │ ├── workflow/ # Workflow patterns (10 samples) │ └── ... # Additional categories │ ├── refactored/ # Deduplicated refactorings │ ├── algorithm/ # Refactored algorithm solutions │ ├── behavioral/ # Refactored behavioral patterns │ ├── permission/ # Refactored authorization │ └── ... # One refactored solution per sample │ ├── bench/ # Benchmark harness │ ├── run.php # Execute all tools on all corpora │ ├── run-samples.php # Run detection on sample categories │ ├── score.php # Compute precision/recall/F1 │ ├── corpora.php # Download/manage test corpora │ ├── comparative.php # Quick one-shot comparison │ ├── run-all.sh # Full benchmark automation │ ├── feature-matrix.md # Tool capability comparison │ ├── tools/ # Auto-downloaded tools │ └── results/ # Benchmark results │ ├── samples.json # Master index of all categories and samples ├── code_duplication_types.md # Type taxonomy with examples ├── code_duplication_challenges.md # "Hidden" duplication patterns └── code_duplication_alternatives.md # Alternative tools reference ``` ### Sample File Structure [](#sample-file-structure) Each sample directory follows this pattern: ``` samples/// ├── block_a.php # First duplicated code block ├── block_b.php # Second duplicated code block ├── block_c.php # Third duplicated code block (optional) └── refactored/ └── code.php # Deduplicated refactored solution ``` --- Duplication Categories ---------------------- [](#duplication-categories) The project defines **54 distinct categories** of code duplication. Here is a summary: ### Clone Classification (Roy & Ossher Taxonomy) [](#clone-classification-roy--ossher-taxonomy) CategoryDescriptionSamples**clone\_type\_1**Exact copy except whitespace/comments20**clone\_type\_2**Renamed identifiers/literals10**clone\_type\_3**Statements added/removed/changed15**clone\_type\_4**Semantic equivalence only10### Semantic & Business Duplication [](#semantic--business-duplication) CategoryDescriptionSamples**semantic**Same business rule, different code20**knowledge**Business facts in multiple places20**behavioral**Same behavior, different implementation20**domain\_rule**Core invariants duplicatedvaries**policy**Business policies scatteredvaries### Structural Duplication [](#structural-duplication) CategoryDescriptionSamples**architectural**Same architecture, different modules10**structural**Same pipeline/workflow pattern20**syntactic**Same AST shape, different content10**template**Same template with different paramsvaries### Code Pattern Duplication [](#code-pattern-duplication) CategoryDescriptionSamples**algorithm**Same algorithmic pattern, different thresholds20**logic**Conditional business rules duplicated10**validation**Input validation repeatedvarious**error\_handling**Exception handling patternsvarious**caching**Cache management duplicated10**monitoring**Instrumentation repeated10### Data & Configuration Duplication [](#data--configuration-duplication) CategoryDescriptionSamples**data**Constants/literals repeated20**configuration**Config patterns duplicated20**schema**Schema definitions repeated10**representation**Multiple models for same entity20### Cross-Cutting Concerns [](#cross-cutting-concerns) CategoryDescriptionSamples**permission**Authorization checks repeated10**dependency**Dependency injection duplicated10**documentation**Docs scattered and inconsistent10**localization**i18n patterns repeated10**serialization**Serialization logic duplicated10### ORM & Database Patterns [](#orm--database-patterns) CategoryDescriptionSamples**query**SQL/ORM queries repeated30**orm\_query\_duplication**Same op, different ORM API10**nosql\_document\_duplication**Same op, different NoSQL API10**graph\_query\_duplication**Same op, different graph DB API5**timeseries\_cache\_duplication**Same op, different cache API5### Other Patterns [](#other-patterns) CategoryDescriptionSamples**copy\_paste**Direct copy-paste examples60**textual**Identical text strings10**lexical**Token-level similarity10**functional**Same functional outcome, different code10**process**Manual processes codified10**cross\_service**Duplication across microservices10**build**CI/CD configuration duplication9**workflow**Workflow patterns repeated10**ui**Interface patterns duplicated20**event**Event handling duplicated10**protocol**Protocol handling repeated50**test**Test setup/fixtures repeated20--- Benchmark Tools --------------- [](#benchmark-tools) The benchmark suite supports the following duplicate detection tools: ToolTypeDescription**phpdup**PrimaryCustom PHP duplicate detector (built into this repo)**phpcpd**ExternalSebastian Bergmann's PHP Copy/Paste Detector**jscpd**ExternalMulti-language CPD clone via JavaScript**pmd-cpd**ExternalPMD's CPD for PHP (via PHPMD)**simian**ExternalCommercial duplicate detector (Java-based)### Tool Comparison Matrix [](#tool-comparison-matrix) Featurephpdupphpcpdjscpdpmd-cpdsimianToken-based detection✓✓✓✓✓AST-based detection✓---✓Min tokens threshold✓✓✓✓✓Ignore annotations✓✓✓✓✓Language supportPHPPHP223+Java, PHP, etc.Java, C#, etc.JSON output✓XMLJSONXMLTextFree/Open Source✓✓✓✓---- Getting Started --------------- [](#getting-started) ### Requirements [](#requirements) - **PHP 8.1+** with CLI and required extensions - **Composer** (for some benchmark dependencies) - **Node.js 18+** (optional, for jscpd) - **Git** (for downloading corpora) ### Running Benchmarks [](#running-benchmarks) #### Full Benchmark Suite [](#full-benchmark-suite) ``` # Run the complete benchmark (downloads tools, clones corpora, runs all tests) ./bench/run-all.sh ``` This produces: - `bench/results/latest.md` - Wall time, RSS, cluster count per (tool, corpus) - `bench/results/detection-rate.md` - Precision/recall/F1 on synthetic corpus #### Individual Components [](#individual-components) ``` # Download/refresh test corpora php bench/corpora.php # Run only one corpus across all tools php bench/run.php --corpus=synthetic-fuzz --label=initial # Score a specific run php bench/score.php bench/results/initial.json # Quick comparative check php bench/comparative.php ``` #### Running on Sample Categories [](#running-on-sample-categories) ``` # Run detection on all samples php bench/run-samples.php # Run on specific category php bench/run-samples.php --category=permission --id=1 ``` ### Adding New Samples [](#adding-new-samples) #### Sample Structure [](#sample-structure) Each sample needs: 1. **Duplicated blocks** (`block_a.php`, `block_b.php`, `block_c.php`) 2. **Refactored solution** (`refactored/code.php`) 3. **Entry in `samples.json`** #### Example [](#example) ``` // samples/my_category/1/block_a.php