CUbRIK is a framework for the development of search-based applications.
Multimedia search processing done in CUbRIK (across query content analysis and relevance feedback processing) benefit from:
– High level metadata gathered from crowdsourcing and as a by-product of playing from games with a purpose
– Techniques for social web and community activity analysis and trust evaluation
– A library of new and existing content processing components and algorithms certified and integrated on the platform
– A reference data model for the logical structure of data to be processed, entities and relationships,configuration of the components in the system, information model for content providers, content owners, users and developers of search-based applications
– Entity-based time and space awareness, supported by a knowledge base of spatio-temporal entities (locations, events, trends) correlated with rich semantic associations.
CUbRIK works on enhancing the quality of search experience by augmenting the precision and the relevance of results when machine intelligence fails or is unable to remove uncertainty. The approach is not to emulate, but rather to incorporate human and social capabilities from feature extraction to search and validation of multimedia content: becausewe, the humans, still have a lot to teach machines about semantic understanding of multimedia content!
CUbRIK search-based applications are able to mix Crowd-based, GWAP-basedand pure-machine computation thanks to pipelines that describe programmable workflow of tasks allocated to executors (e.g. software components for data analysis, metadata indexing, search engines, presentation of results, or others) and able to combine different types of intelligence, according to the specific application requirements.
CUbRIK will prove the feasibility of its approach and the benefits of the integration of machine, human and social computation for multimedia search by showcasing Demonstrators and Vertical Application.
- Demonstrators prove increased precision of multimedia search done the CUbRIK way
- Vertical Applications exemplify selected use cases where machine-based processing is combined with human and social intelligence.
Prototype developments are exploited Fashion and Digital Humanities.
The inspiring principle of CUbRIK is about creating a “white-box” version of a multimedia content & query processing system.
The functionalities of query processing, content analysis and relevance feedback processing are unbundled into a set of search processing orchestrations (referred to as Pipelinesable to mix open source and third-party components, to instantiate algorithms and to aggregate Automatic Computation Jobs (automatic workflow or SMILA Workflow) and Human Computation (Crowd-enabled or GWAP-enabled)Tasks.
Metadata are extracted from media collections, using the software mix that best fits the need, and specific components can be included to process multimodal queries or to analyze user’s feedback in novel ways.
The architecture is an example of differential design: based on a SMILA underlying framework, modified to support programmable workflows and asynchronous task execution as it is required to mix automatic operations (Jobs) and human activities (Tasks) chained in a sequence. CUbRIK inherits and extends SMILA capabilities of easy integration of data source connectors, search engines, sophisticated analysis methods and other components by gaining scalability and reliability out-of-the-box.
