PHPackages crowdtruth/crowdtruth - PHPackages - PHPackages [Skip to content](#main-content)[PHPackages](/)[Directory](/)[Categories](/categories)[Trending](/trending)[Leaderboard](/leaderboard)[Changelog](/changelog)[Analyze](/analyze)[Collections](/collections)[Log in](/login)[Sign up](/register) 1. [Directory](/) 2. / 3. [Framework](/categories/framework) 4. / 5. crowdtruth/crowdtruth ActiveLibrary[Framework](/categories/framework) crowdtruth/crowdtruth ===================== The CrowdTruth Framework 20150619(11y ago)60613[31 issues](https://github.com/CrowdTruth/CrowdTruth/issues)[2 PRs](https://github.com/CrowdTruth/CrowdTruth/pulls)MITJavaScript Since Mar 7Pushed 7y ago18 watchersCompare [ Source](https://github.com/CrowdTruth/CrowdTruth)[ Packagist](https://packagist.org/packages/crowdtruth/crowdtruth)[ RSS](/packages/crowdtruth-crowdtruth/feed)WikiDiscussions dev Synced 3w ago READMEChangelog (10)Dependencies (10)Versions (31)Used By (0) *Notice:* This repository is no longer being maintained. Please see [CrowdTruth-core](https://github.com/CrowdTruth/CrowdTruth-core) for the latest release of the framework. ============================================================================================================================================================================= [](#notice-this-repository-is-no-longer-being-maintained-please-see-crowdtruth-core-for-the-latest-release-of-the-framework) [![Latest Stable Version](https://camo.githubusercontent.com/34d27f998b85f35328d1e0c10570166bc76fb5dfceb67b029d0fcad072a38031/687474703a2f2f696d672e736869656c64732e696f2f6769746875622f72656c656173652f63726f776474727574682f63726f776474727574682e737667)](https://packagist.org/packages/crowdtruth/crowdtruth) [![Build Status](https://camo.githubusercontent.com/99f88f23b6f35dcc11554bf2cc9a48126ade952e9823c8aeb2394786bcfdfc5d/68747470733a2f2f7472617669732d63692e6f72672f43726f776454727574682f43726f776454727574682e7376673f6272616e63683d6d6173746572)](https://travis-ci.org/CrowdTruth/CrowdTruth) [![Code Coverage](https://camo.githubusercontent.com/da8a8c8e1e38e5aa4e596356845c67a9466c1c5af702eb899661ade0e86dd4e8/68747470733a2f2f7363727574696e697a65722d63692e636f6d2f672f43726f776454727574682f43726f776454727574682f6261646765732f636f7665726167652e706e673f623d6d6173746572)](https://scrutinizer-ci.com/g/CrowdTruth/CrowdTruth/?branch=master) [![Scrutinizer Code Quality](https://camo.githubusercontent.com/39c4103bef231c687086c68f525523d528f5ec516c252fc361ddc23f15476b01/68747470733a2f2f7363727574696e697a65722d63692e636f6d2f672f43726f776454727574682f43726f776454727574682f6261646765732f7175616c6974792d73636f72652e706e673f623d6d6173746572)](https://scrutinizer-ci.com/g/CrowdTruth/CrowdTruth/?branch=master) The [CrowdTruth Framework](http://crowdtruth.org) implements an approach to machine-human computing for collecting annotation data on text, images, sounds and videos. The approach is focussed specifically on collecting gold standard data for training and evaluation of cognitive computing systems. The original framework was inspired by the [IBM Watson project](http://www.research.ibm.com/labs/watson/) for providing improved (multi-perspective) gold standard (medical) text annotation data for the training and evaluation of various IBM Watson components, such as Medical Relation Extraction, Medical Factor Extraction and Question-Answer passage alignment. [![](https://camo.githubusercontent.com/9dde85053b63571be2636c1d48b5c47a0c109a95f8ab659b85fed285fb824cbd/68747470733a2f2f75706c6f61642e77696b696d656469612e6f72672f77696b6970656469612f656e2f652f65312f43726f7764666c6f7765722d6c6f676f2e706e67)](https://camo.githubusercontent.com/9dde85053b63571be2636c1d48b5c47a0c109a95f8ab659b85fed285fb824cbd/68747470733a2f2f75706c6f61642e77696b696d656469612e6f72672f77696b6970656469612f656e2f652f65312f43726f7764666c6f7765722d6c6f676f2e706e67)[![](https://camo.githubusercontent.com/a0c692021974ef54662d43690d581576e628e8b4cd4d98d9f7116fec8e4365e4/687474703a2f2f746865636f6c6c656765696e766573746f72636f6d2e632e707265737363646e2e636f6d2f77702d636f6e74656e742f75706c6f6164732f323030392f31302f616d617a6f6e2d6d7475726b2d6c6f676f2e6a7067)](https://camo.githubusercontent.com/a0c692021974ef54662d43690d581576e628e8b4cd4d98d9f7116fec8e4365e4/687474703a2f2f746865636f6c6c656765696e766573746f72636f6d2e632e707265737363646e2e636f6d2f77702d636f6e74656e742f75706c6f6164732f323030392f31302f616d617a6f6e2d6d7475726b2d6c6f676f2e6a7067) The CrowdTruth framework supports the composition of CrowdTruth gathering workflows, where a sequence of micro-annotation tasks can be configured and sent out to a number of crowdsourcing platforms (e.g. [CrowdFlower](http://crowdflower.com) and [Amazon Mechanical Turk](https://www.mturk.com)) and applications (e.g. Expert annotation game [Dr. Detective](http://game.crowdtruth.org)). The CrowdTruth framework has a special focus on micro-tasks for knowledge extraction in medical text (e.g. medical documents, from various sources such as Wikipedia articles or patient case reports). The main steps involved in the CrowdTruth workflow are: 1. Exploring & processing of input data 2. Collecting of annotation data 3. Applying disagreement analytics on the results These steps are realised in an automatic end-to-end workflow, that can support a continuous collection of high quality gold standard data with feedback loop to all steps of the process. Have a look at our [presentations](http://crowdtruth.org/presentations/ "Presentations") and [papers](http://crowdtruth.org/papers/ "Papers") for more details on the research. Using CrowdTruth ================ [](#using-crowdtruth) Start using CrowdTruth right now, completely free, and explore all its possiblities. Follow the [installation guide](https://github.com/CrowdTruth/CrowdTruth/wiki/installation) to get started, or check out our [wiki](https://github.com/CrowdTruth/CrowdTruth/wiki) for all documentation of the platform. We have some [crowdsourcing templates](https://github.com/CrowdTruth/CrowdTruth/wiki/Creating-Templates) ready for you to start with. ### Health Score 34 — LowBetter than 75% of packages Maintenance10 Infrequent updates — may be unmaintained Popularity19 Limited adoption so far Community25 Small or concentrated contributor base Maturity74 Established project with proven stability Bus Factor2 2 contributors hold 50%+ of commits How is this calculated?**Maintenance (25%)** — Last commit recency, latest release date, and issue-to-star ratio. Uses a 2-year decay window. **Popularity (30%)** — Total and monthly downloads, GitHub stars, and forks. Logarithmic scaling prevents top-heavy scores. **Community (15%)** — Contributors, dependents, forks, watchers, and maintainers. Measures real ecosystem engagement. **Maturity (30%)** — Project age, version count, PHP version support, and release stability. ### Release Activity Cadence Every ~60 days Recently: every ~121 days Total 12 Last Release 3757d ago ### Community Maintainers ![](https://avatars.githubusercontent.com/u/7782231?v=4)[Carlos Martinez](/maintainers/c-martinez)[@c-martinez](https://github.com/c-martinez) --- Top Contributors [![arner](https://avatars.githubusercontent.com/u/6328508?v=4)](https://github.com/arner "arner (335 commits)")[![Bouncer](https://avatars.githubusercontent.com/u/103966425?v=4)](https://github.com/Bouncer "Bouncer (170 commits)")[![khamkham](https://avatars.githubusercontent.com/u/5898200?v=4)](https://github.com/khamkham "khamkham (120 commits)")[![c-martinez](https://avatars.githubusercontent.com/u/7782231?v=4)](https://github.com/c-martinez "c-martinez (106 commits)")[![oana-inel](https://avatars.githubusercontent.com/u/3357547?v=4)](https://github.com/oana-inel "oana-inel (91 commits)")[![lr292358](https://avatars.githubusercontent.com/u/4463818?v=4)](https://github.com/lr292358 "lr292358 (69 commits)")[![tacri](https://avatars.githubusercontent.com/u/6360418?v=4)](https://github.com/tacri "tacri (56 commits)")[![cwmeijer](https://avatars.githubusercontent.com/u/6087314?v=4)](https://github.com/cwmeijer "cwmeijer (24 commits)")[![Jorann](https://avatars.githubusercontent.com/u/336692?v=4)](https://github.com/Jorann "Jorann (8 commits)")[![MerelvanEmpel](https://avatars.githubusercontent.com/u/8793404?v=4)](https://github.com/MerelvanEmpel "MerelvanEmpel (4 commits)")[![schampilomatis](https://avatars.githubusercontent.com/u/6921086?v=4)](https://github.com/schampilomatis "schampilomatis (3 commits)")[![ancadumitrache](https://avatars.githubusercontent.com/u/1244371?v=4)](https://github.com/ancadumitrache "ancadumitrache (2 commits)")[![laroyo](https://avatars.githubusercontent.com/u/606086?v=4)](https://github.com/laroyo "laroyo (1 commits)") --- Tags crowdtruth-frameworkcrowdtruth-work-owevaluationjavascriptmicrotasksqualityframeworklaravel ### Embed Badge ![Health badge](/badges/crowdtruth-crowdtruth/health.svg) ``` [![Health](https://phpackages.com/badges/crowdtruth-crowdtruth/health.svg)](https://phpackages.com/packages/crowdtruth-crowdtruth) ``` ### Alternatives [laravel/laravel The skeleton application for the Laravel framework. 84.5k62.4M1.0k](/packages/laravel-laravel)[unopim/unopim UnoPim Laravel PIM 10.3k2.2k](/packages/unopim-unopim)[nasirkhan/laravel-starter A CMS like modular Laravel starter project. 1.4k2.7k](/packages/nasirkhan-laravel-starter)[codewithdennis/larament Larament is a time-saving starter kit to quickly launch Laravel 13.x projects. It includes FilamentPHP 5.x pre-installed and configured, along with additional tools and features to streamline your development workflow. 3861.7k](/packages/codewithdennis-larament)[kompo/kompo Laravel & Vue.js FullStack Components for Rapid Application Development 11914.4k42](/packages/kompo-kompo)[ecotone/laravel Ecotone for Laravel — CQRS, Event Sourcing, Sagas, Durable Workflows, and Outbox on top of Laravel Queue, via PHP attributes. 21313.7k3](/packages/ecotone-laravel) PHPackages © 2026 [Directory](/)[Categories](/categories)[Trending](/trending)[Changelog](/changelog)[Analyze](/analyze)