Automatic semantic video annotation in wide domain videos based on similarity and commonsense knowledgebases

In this paper, we introduce a novel framework for automatic Semantic Video Annotation. As this framework detects possible events occurring in video clips, it forms the annotating base of video search engine. To achieve this purpose, the system has to able to operate on uncontrolled wide-domain videos. Thus, all layers have to be based on generic features. This framework aims to bridge the "semantic gap", which is the difference between the low-level visual features and the human's perception, by finding videos with similar visual events, then analyzing their free text annotation to find a common area then to decide the best description for this new video using commonsense knowledgebases. Experiments were performed on wide-domain video clips from the TRECVID 2005 BBC rush standard database. Results from these experiments show promising integrity between those two layers in order to find expressing annotations for the input video. These results were evaluated based on retrieval performance

Video databases annotation enhancing using commonsense knowledgebases for indexing and retrieval

The rapidly increasing amount of video collections, especially on the web, motivated the need for intelligent automated annotation tools for searching, rating, indexing and retrieval purposes. These videos collections contain all types of manually annotated videos. As this annotation is usually incomplete and uncertain and contains misspelling words, search using some keywords almost do retrieve only a portion of videos which actually contains the desired meaning. Hence, this annotation needs filtering, expanding and validating for better indexing and retrieval. In this paper, we present a novel framework for video annotation enhancement, based on merging two widely known commonsense knowledgebases, namely WordNet and ConceptNet. In addition to that, a comparison between these knowledgebases in video annotation domain is presented. Experiments were performed on random wide-domain video clips, from the \emph{vimeo.com} website. Results show that searching for a video over enhanced tags, based on our proposed framework, outperforms searching using the original tags. In addition to that, the annotation enhanced by our framework outperforms both those enhanced by WordNet and ConceptNet individually, in terms of tags enrichment ability, concept diversity and most importantly retrieval performance

Content-based Video Retrieval by Integrating Spatio-Temporal and Stochastic Recognition of Events

As amounts of publicly available video data grow the need to query this data efficiently becomes significant. Consequently content-based retrieval of video data turns out to be a challenging and important problem. We address the specific aspect of inferring semantics automatically from raw video data. In particular, we introduce a new video data model that supports the integrated use of two different approaches for mapping low-level features to high-level concepts. Firstly, the model is extended with a rule-based approach that supports spatio-temporal formalization of high-level concepts, and then with a stochastic approach. Furthermore, results on real tennis video data are presented, demonstrating the validity of both approaches, as well us advantages of their integrated us

TagBook: A Semantic Video Representation without Supervision for Event Detection

We consider the problem of event detection in video for scenarios where only few, or even zero examples are available for training. For this challenging setting, the prevailing solutions in the literature rely on a semantic video representation obtained from thousands of pre-trained concept detectors. Different from existing work, we propose a new semantic video representation that is based on freely available social tagged videos only, without the need for training any intermediate concept detectors. We introduce a simple algorithm that propagates tags from a video's nearest neighbors, similar in spirit to the ones used for image retrieval, but redesign it for video event detection by including video source set refinement and varying the video tag assignment. We call our approach TagBook and study its construction, descriptiveness and detection performance on the TRECVID 2013 and 2014 multimedia event detection datasets and the Columbia Consumer Video dataset. Despite its simple nature, the proposed TagBook video representation is remarkably effective for few-example and zero-example event detection, even outperforming very recent state-of-the-art alternatives building on supervised representations.Comment: accepted for publication as a regular paper in the IEEE Transactions on Multimedi

Key Frame Generation to Generate Activity Strip Based on Similarity Calculation

Management of video data is done for several purposes, such as to make the information more meaningful. Research has been conducted to manage the video in terms of detecting activity in a video. There are three stages to generate activity strip: the data source stage (preparation of the frames), the processing stage (analysis of the activity), and the final stage (the collection of key frames). The generation of activity strip is done by calculating the difference of the pixel values of two frames to detect a similarity. In this research, we used SAD (Sum of Absolute Difference) method to calculate the value of the difference of the frame. Similar frames can be grouped in the same cluster. Each cluster is considered as one frame (or multiple frames) to serve as a key frame. The key frames are used for the representation of the activity strip. A collection of activity strip will be arranged sequentially and continuously for the activity generation

The Zoological Paradox

Through on-site field investigation, studies of animal behaviors, and Jon Coe’s zoo enrichment principles, this thesis proposes that zoo architec-ture should prioritize the needs of the animals by incorporating behavioral and environmental enrichments into its design. Enrichments refer to the pro- cess of providing the appropriate behav-ioral and environmental stimuli that foster the animals to exhibit their own natural behaviors. Adapting Jon Coe’s strategies and Heini Hediger’s concept of territory as precedents, “A Zoological Paradox” proposes the reimagining of the zoo typology by integrating existing enrich-ment methodologies, as well as allowing human visitors to interact and engage in the process of enriching the animals’ well-being. Through strategies of introducing choices and novelty to the “Animal Folly” enclosure, and allowing a visitor’s participation, the reimagined zoo offers the animals a sense of freedom within captivity; a temporary relief from stress within their confines. A continuous network of “corridors” replaces conventional enclosures; reflecting each animal’s territorial path-ways and further providing a sense of choice and freedom. Shifting the visitor’s role from a mere spectator to an active participant not only enhances his or her experience at the zoo, but also reshapes the percep-tion that these enrichments are what really matter. Rather than a “naturalistic facade,” enrichments become the most crucial component for animals born and raised in captivity

A framework for automatic semantic video annotation

The rapidly increasing quantity of publicly available videos has driven research into developing automatic tools for indexing, rating, searching and retrieval. Textual semantic representations, such as tagging, labelling and annotation, are often important factors in the process of indexing any video, because of their user-friendly way of representing the semantics appropriate for search and retrieval. Ideally, this annotation should be inspired by the human cognitive way of perceiving and of describing videos. The difference between the low-level visual contents and the corresponding human perception is referred to as the ‘semantic gap’. Tackling this gap is even harder in the case of unconstrained videos, mainly due to the lack of any previous information about the analyzed video on the one hand, and the huge amount of generic knowledge required on the other. This paper introduces a framework for the Automatic Semantic Annotation of unconstrained videos. The proposed framework utilizes two non-domain-specific layers: low-level visual similarity matching, and an annotation analysis that employs commonsense knowledgebases. Commonsense ontology is created by incorporating multiple-structured semantic relationships. Experiments and black-box tests are carried out on standard video databases for action recognition and video information retrieval. White-box tests examine the performance of the individual intermediate layers of the framework, and the evaluation of the results and the statistical analysis show that integrating visual similarity matching with commonsense semantic relationships provides an effective approach to automated video annotation

Developing Activities on Uncharismatic Animals Found at Omaha\u27s Henry Doorly Zoo and Aquarium

The Education Department at Omaha’s Henry Doorly Zoo and Aquarium has a number of educational activities based on charismatic species found there. I wished to develop activities the department may use for less charismatic or misunderstood species at the zoo. I began by surveying the exhibits at the zoo and taking note of what species or groups didn’t receive as much visitor engagement, or what comments were made on species that were incorrect. I then began to brainstorm and narrow down potential species that I could develop an educational activity around. As I began to develop and create activities, a trend I noticed was the higher presence of activities based around uncharismatic aquatic species as opposed to uncharismatic terrestrial species; this was only mostly since aquatic species tended to fill more exotic niches or possess more diverse adaptations compared to terrestrial species. The current final product has ten activities I developed over the course of my summer internship at the Henry Doorly Zoo

Towards a great ape dictionary : inexperienced humans understand common nonhuman ape gestures

Funding: This research received funding from the European Union’s 8th Framework 287 Programme, Horizon 2020, under grant agreement no 802719 to CH (https://ec.europa.eu/info/research-and-innovation/funding/funding-opportunities/funding-programmes-and-open-calls/horizon-2020_en). This work was supported by Gorilla Awards in Behavioural Science who provided the Gorilla.sc licensing fee and an unlimited participant award to KG (https://gorilla.sc/).In the comparative study of human and nonhuman communication, ape gesturing provided the first demonstrations of flexible, intentional communication outside human language. Rich repertoires of these gestures have been described in all ape species, bar one: us. Given that the majority of great ape gestural signals are shared, and their form appears biologically inherited, this creates a conundrum: Where did the ape gestures go in human communication? Here, we test human recognition and understanding of 10 of the most frequently used ape gestures. We crowdsourced data from 5,656 participants through an online game, which required them to select the meaning of chimpanzee and bonobo gestures in 20 videos. We show that humans may retain an understanding of ape gestural communication (either directly inherited or part of more general cognition), across gesture types and gesture meanings, with information on communicative context providing only a marginal improvement in success. By assessing comprehension, rather than production, we accessed part of the great ape gestural repertoire for the first time in adult humans. Cognitive access to an ancestral system of gesture appears to have been retained after our divergence from other apes, drawing deep evolutionary continuity between their communication and our own.Publisher PDFPeer reviewe

Smart carnivores think twice: Red fox delays scavenging on conspecific carcasses to reduce parasite risk

M.M. was supported by a research contract Ramon y Cajal from the MINECO (RYC-2015-19231) . This study was partly funded by the Spanish Ministry of Economy, Industry and Competitiveness and EU ERDF funds through the project CGL2017-89905-R.The recent SARS-CoV-2 epidemic has highlighted the need to prevent emerging and re-emerging diseases, which means that we must approach the study of diseases from a One Health perspective. The study of pathogen transmission in wildlife is challenging, but it is unquestionably key to understand how epidemiological interactions occur at the wildlife-domestic-human interface. In this context, studying parasite avoidance behaviours may provide essential insights on parasite transmission, host-parasite coevolution, and energy flow through food-webs. However, the strategies of avoiding trophically transmitted parasites in mammalian carnivores have received little scientific attention. Here, we explore the behaviour of red foxes (Vulpes vulpes) and other mammalian carnivores at conspecific and heterospecific carnivore carcasses using videos recorded by camera traps. We aim to determine 1) the factors influencing the probability of foxes to practice cannibalism, and 2) whether the scavenging behaviour of foxes differ when facing conspecific vs. heterospecific carcasses. We found that red foxes were generally reluctant to consume mesocarnivore carrion, especially of conspecifics. When recorded, consumption by foxes was delayed several days (heterospecific carcasses) or weeks (conspecific carcasses) after carcass detection. Other mammalian scavengers showed a similar pattern. Also, meat-borne parasite transmission from wild carnivore carcasses to domestic dogs and cats was highly unlikely. Our findings challenge the widespread assumption that cannibalistic or intra-specific scavenging is a major transmission route for Trichinella spp. and other meat-borne parasites, especially for the red fox. Overall, our results suggest that the feeding decisions of scavengers are probably shaped by two main contrasting forces, namely the nutritional reward provided by carrion of phylogenetically similar species and the risk of acquiring meat-borne parasites shared with these species. This study illustrates how the detailed monitoring of carnivore behaviour is essential to assess the epidemiological role of these hosts in the maintenance and dispersion of parasites of public and animal health relevance.MINECO RYC-2015-19231Spanish Ministry of Economy, Industry and CompetitivenessEuropean Commission CGL2017-89905-