A framework for automatic semantic video annotation

The rapidly increasing quantity of publicly available videos has driven research into developing automatic tools for indexing, rating, searching and retrieval. Textual semantic representations, such as tagging, labelling and annotation, are often important factors in the process of indexing any video, because of their user-friendly way of representing the semantics appropriate for search and retrieval. Ideally, this annotation should be inspired by the human cognitive way of perceiving and of describing videos. The difference between the low-level visual contents and the corresponding human perception is referred to as the ‘semantic gap’. Tackling this gap is even harder in the case of unconstrained videos, mainly due to the lack of any previous information about the analyzed video on the one hand, and the huge amount of generic knowledge required on the other. This paper introduces a framework for the Automatic Semantic Annotation of unconstrained videos. The proposed framework utilizes two non-domain-specific layers: low-level visual similarity matching, and an annotation analysis that employs commonsense knowledgebases. Commonsense ontology is created by incorporating multiple-structured semantic relationships. Experiments and black-box tests are carried out on standard video databases for action recognition and video information retrieval. White-box tests examine the performance of the individual intermediate layers of the framework, and the evaluation of the results and the statistical analysis show that integrating visual similarity matching with commonsense semantic relationships provides an effective approach to automated video annotation

Predicting Student Failure in an Introductory Programming Course with Multiple Back-Propagation

One of the most challenging tasks in computer science and similar courses consists of both teaching and learning computer programming. Usually this requires a great deal of work, dedication, and motivation from both teachers and students. Accordingly, ever since the first programming languages emerged, the problems inherent to programming teaching and learning have been studied and investigated. The theme is very serious, not only for the important concepts underlying computer science courses but also for reducing the lack of motivation, failure, and abandonment that result from students frustration. Therefore, early identification of potential problems and immediate response is a fundamental aspect to avoid student’s failure and reduce dropout rates. In this paper, we propose a machine-learning (neural network) predictive model of student failure based on the student profile, which is built throughout programming classes by continuously monitoring and evaluating student activities. The resulting model allows teachers to early identify students that are more likely to fail, allowing them to devote more time to those students and try novel strategies to improve their programming skills

Video databases annotation enhancing using commonsense knowledgebases for indexing and retrieval

The rapidly increasing amount of video collections, especially on the web, motivated the need for intelligent automated annotation tools for searching, rating, indexing and retrieval purposes. These videos collections contain all types of manually annotated videos. As this annotation is usually incomplete and uncertain and contains misspelling words, search using some keywords almost do retrieve only a portion of videos which actually contains the desired meaning. Hence, this annotation needs filtering, expanding and validating for better indexing and retrieval. In this paper, we present a novel framework for video annotation enhancement, based on merging two widely known commonsense knowledgebases, namely WordNet and ConceptNet. In addition to that, a comparison between these knowledgebases in video annotation domain is presented. Experiments were performed on random widedomain video clips, from the vimeo.com website. Results show that searching for a video over enhanced tags, based on our proposed framework, outperforms searching using the original tags. In addition to that, the annotation enhanced by our framework outperforms both those enhanced by WordNet and ConceptNet individually, in terms of tags enrichment ability, concept diversity and most importantly retrieval performance.</p

Graph-based video sequence matching using dominant colour graph profile (DCGP)

This paper presents a fast and effective technique for videos’ visual similarity detection and measurement using compact fixed-length signatures. The proposed technique (dominant colour graph profile DCGP) extracts and encodes the spatio-temporal information of a given video shot into a graph-based structure (tree) that fully captures this vital information. The graph structured properties are utilized to construct a fixed-length video signature of 112 decimal values per video shot. The encoded spatio-temporal information is extracted following channelling each video frame into a block-based structure, where the positions of respective blocks are tracked across video frames and encoded into multiple DCGP trees. The proposed technique provides a high matching speed (&gt;2000 fps) and robust retrieval performance. The experiments on various standard and challenging datasets shows the framework’s robust performance, in terms of both, retrieval and computational performances