Multimedia Content Annotations for Rapid Exploitation in Multi-Screen Environments

Cordis link

Objective

The objective of MECANEX project is to devise innovative tools for automatic annotation and editorial support of multimedia content, extraction of personalized information and adaption of enriched multimedia content in multi-screen environments. As the prerequisites of production houses, broadcasters, advertising agencies and online publishing companies for enriched multimedia content increase rapidly, the need of such innovative tools is undeniable. Furthermore, the adaption of enriched multimedia content to multi-screen environments, which enable the automatic porting to different target platforms, such as regular web pages, mobile pages and mobile apps, as well as TV applications, is a significant asset for the global market of enriched multimedia content. The major novelty of MECANEX is the creation of SaaS (REST based) toolkit that provides the aforementioned functionalities by bringing together complementary expertise and technology of different European companies and research institutes, in order to provide a cost effective multi-screen video metadata enrichment solution for SMEs. Such tools will be adopted by production and post-production companies, as well as by content providers or relevant service oriented companies and will be adapted into the creation process. MECANEX will offer SMEs to semi automatically enrich video collections for multi-screen application, as well rapid prototype multi-screen concepts and test related interaction models that eventually will improve the chance for SMEs to develop more solid business models, marketing and advertising campaigns. Furthermore, a social and personalization tool is envisioned, which will collect consumers’ preferences and will provide feedback to the aforementioned tools. In a nutshell, automatic annotation, editorial, multi-screen and social and personalization tools will be integrated into a working MECANEX toolkit ready for the production of multimedia content, as well as the advertising agencies

Social Recommendation Tool

Code

We focus on the design, development and evaluation of a framework consisting of personalization, relevance feedback and recommendation mechanisms, as a principal method for the creation of enriched multimedia content targeted to each user’s needs, preferences and interests. As the multimedia content proliferates along with its consumption by the users, more effective ways of presenting it to the viewers are demanded in order to facilitate them with the multimedia content search and selection and improve their Quality of Experience (QoE). The main contribution is the introduction of a holistic framework that offers personalized enriched multimedia content, by extending the recommendation process to the set of enrichments that accompany the video except from the video itself and by collecting explicit and implicit relevance feedback from the interactions of the user with both the video and its enrichments. We evaluate the proposed framework following a two-step approach. We perform extended experiments by applying reasonably simulated user interactions, in order to calibrate its parameters that refer to multiple aspects of the enriched multimedia content, aiming at high performance in terms of QoE and by integrating our proposed recommender framework within the MECANEX streaming platform in order to perform user studies about its usability within a realistic environment of use.