Automatic semantic video annotation in wide domain videos based on similarity and commonsense knowledgebases

In this paper, we introduce a novel framework for automatic Semantic Video Annotation. As this framework detects possible events occurring in video clips, it forms the annotating base of video search engine. To achieve this purpose, the system has to able to operate on uncontrolled wide-domain videos. Thus, all layers have to be based on generic features. This framework aims to bridge the "semantic gap", which is the difference between the low-level visual features and the human's perception, by finding videos with similar visual events, then analyzing their free text annotation to find a common area then to decide the best description for this new video using commonsense knowledgebases. Experiments were performed on wide-domain video clips from the TRECVID 2005 BBC rush standard database. Results from these experiments show promising integrity between those two layers in order to find expressing annotations for the input video. These results were evaluated based on retrieval performance

Action Recognition in Videos: from Motion Capture Labs to the Web

This paper presents a survey of human action recognition approaches based on visual data recorded from a single video camera. We propose an organizing framework which puts in evidence the evolution of the area, with techniques moving from heavily constrained motion capture scenarios towards more challenging, realistic, "in the wild" videos. The proposed organization is based on the representation used as input for the recognition task, emphasizing the hypothesis assumed and thus, the constraints imposed on the type of video that each technique is able to address. Expliciting the hypothesis and constraints makes the framework particularly useful to select a method, given an application. Another advantage of the proposed organization is that it allows categorizing newest approaches seamlessly with traditional ones, while providing an insightful perspective of the evolution of the action recognition task up to now. That perspective is the basis for the discussion in the end of the paper, where we also present the main open issues in the area.Comment: Preprint submitted to CVIU, survey paper, 46 pages, 2 figures, 4 table

A Systematic Review of Existing Data Mining Approaches Envisioned for Knowledge Discovery from Multimedia

The extensive use of multimedia technologies extended the applicability of information technology to a large extent which results enormous generation of complex multimedia contents over the internet. Therefore the number of multimedia contents available to the user is also exponentially increasing. In this digital era of the cloud-enabled Internet of Things (IoT), analysis of complex video and image data plays a crucial role.It aims to extract meaningful information as the distributed storages and processing elements within a bandwidth constraint network seek optimal solutions to increase the throughput along with an optimal trade-off between computational complexity and power consumption. However, due to complex characteristics of visual patterns and variations in video frames, it is not a trivial task to discover meaningful information and correlation. Hence, data mining has emerged as a field which has diverse aspects presently in extracting meaningful hidden patterns from the complex image and video data considering different pattern classification approach. The study mostly investigates the existing data-mining tools and their performance metric for the purpose of reviewing this research track.It also highlights the relationship between frequent patterns and discriminativefeatures associated with a video object. Finally, the study addresses the existing research issues to strengthen up the future direction of research towards video analytics and pattern recognition

Using association rule mining to enrich semantic concepts for video retrieval

In order to achieve true content-based information retrieval on video we should analyse and index video with high-level semantic concepts in addition to using user-generated tags and structured metadata like title, date, etc. However the range of such high-level semantic concepts, detected either manually or automatically, usually limited compared to the richness of information content in video and the potential vocabulary of available concepts for indexing. Even though there is work to improve the performance of individual concept classifiers, we should strive to make the best use of whatever partial sets of semantic concept occurrences are available to us. We describe in this paper our method for using association rule mining to automatically enrich the representation of video content through a set of semantic concepts based on concept co-occurrence patterns. We describe our experiments on the TRECVid 2005 video corpus annotated with the 449 concepts of the LSCOM ontology. The evaluation of our results shows the usefulness of our approach

An empirical study of inter-concept similarities in multimedia ontologies

Generic concept detection has been a widely studied topic in recent research on multimedia analysis and retrieval, but the issue of how to exploit the structure of a multimedia ontology as well as different inter-concept relations, has not received similar attention. In this paper, we present results from our empirical analysis of different types of similarity among semantic concepts in two multimedia ontologies, LSCOM-Lite and CDVP-206. The results show promise that the proposed methods may be helpful in providing insight into the existing inter-concept relations within an ontology and selecting the most facilitating set of concepts and hierarchical relations. Such an analysis as this can be utilized in various tasks such as building more reliable concept detectors and designing large-scale ontologies