The VIA Annotation Software for Images, Audio and Video

In this paper, we introduce a simple and standalone manual annotation tool for images, audio and video: the VGG Image Annotator (VIA). This is a light weight, standalone and offline software package that does not require any installation or setup and runs solely in a web browser. The VIA software allows human annotators to define and describe spatial regions in images or video frames, and temporal segments in audio or video. These manual annotations can be exported to plain text data formats such as JSON and CSV and therefore are amenable to further processing by other software tools. VIA also supports collaborative annotation of a large dataset by a group of human annotators. The BSD open source license of this software allows it to be used in any academic project or commercial application.Comment: to appear in Proceedings of the 27th ACM International Conference on Multimedia (MM '19), October 21-25, 2019, Nice, France. ACM, New York, NY, USA, 4 page

A Metadata Generation System with Semantic Understanding for Video Retrieval in Film Production

In film production, metadata plays an important role in original raw video indexing and classification within the industrial post-production software. Inspired by deep visual-semantic methods, we propose an automated image information extraction process to extend the diversity of metadata entities for massive large-scale raw video searching and retrieval. In this paper, we introduce the proposed system architecture and modules, integrating semantic annotation models and user-demand-oriented information fusion. We conducted experiments to validate the effectiveness of our system on Film Raw Video Semantic Annotation Dataset (Film-RVSAD) and Slate Board Template Dataset (SBTD), two benchmark datasets built for cinematography-related semantic annotation and slate detection. Experimental results show that the proposed system provides an effective strategy to improve the efficiency of metadata generation and transformation, which is necessary and convenient for collaborative work in the filmmaking process.Comment: Accepted by 2022 IEEE International Conference on Virtual Reality and Visualization (ICVRV), received Best Paper Awar

Interactive web-based track editing and management

Title from PDF of title page (University of Missouri--Columbia, viewed on September 13, 2012).The entire thesis text is included in the research.pdf file; the official abstract appears in the short.pdf file; a non-technical public abstract appears in the public.pdf file.Thesis advisor: Dr. Kannappan PalaniappanIncludes bibliographical references.M. S. University of Missouri--Columbia 2012."May 2012"Image and video analysis is the process of extracting useful information from an image. With advances in optical and computational technologies, image and video analysis is becoming an increasingly important tool in biological, medical and defense applications. Motion characteristics and behaviors of objects can be studied from image sequences, such as the spatio-temporal behavior of cells and organisms in microscopy videos or vehicles and people in surveillance videos. Manual supervised and automatic object tracking are used to study migration, lineage, cellular events and population scale dynamics in microscopy applications and multi-target behavior in defense applications. FireFly is a rich multimedia web-based tool based on Adobe Flash and Flex with server side PHP and MySQL, for managing image collections, collaborative ground truth generation, manual and supervised analysis of images and video including labeling, annotation, ground truthing, algorithm output inspection, comparison and correction, etc. The main objective of this project is to extend FireFly and add interactive editing and updating tracking information by automatically propagating the track information to upstream and downstream frames in the annotation/ video event database

Realization of Semantic Atom Blog

Web blog is used as a collaborative platform to publish and share information. The information accumulated in the blog intrinsically contains the knowledge. The knowledge shared by the community of people has intangible value proposition. The blog is viewed as a multimedia information resource available on the Internet. In a blog, information in the form of text, image, audio and video builds up exponentially. The multimedia information contained in an Atom blog does not have the capability, which is required by the software processes so that Atom blog content can be accessed, processed and reused over the Internet. This shortcoming is addressed by exploring OWL knowledge modeling, semantic annotation and semantic categorization techniques in an Atom blog sphere. By adopting these techniques, futuristic Atom blogs can be created and deployed over the Internet

The CAMOMILE collaborative annotation platform for multi-modal, multi-lingual and multi-media documents

In this paper, we describe the organization and the implementation of the CAMOMILE collaborative annotation framework for multimodal, multimedia, multilingual (3M) data. Given the versatile nature of the analysis which can be performed on 3M data, the structure of the server was kept intentionally simple in order to preserve its genericity, relying on standard Web technologies. Layers of annotations, defined as data associated to a media fragment from the corpus, are stored in a database and can be managed through standard interfaces with authentication. Interfaces tailored specifically to the needed task can then be developed in an agile way, relying on simple but reliable services for the management of the centralized annotations. We then present our implementation of an active learning scenario for person annotation in video, relying on the CAMOMILE server; during a dry run experiment, the manual annotation of 716 speech segments was thus propagated to 3504 labeled tracks. The code of the CAMOMILE framework is distributed in open source.Peer ReviewedPostprint (author's final draft

Query independent measures of annotation and annotator impact

The modern-day web-user plays a far more active role in the creation of content for the web as a whole. In this paper we present Annoby, a free-text annotation system built to give users a more interactive experience of the events of the Rugby World Cup 2007. Annotations can be used for query-independent ranking of both the annotations and the original recorded video footage (or documents) which has been annotated, based on the social interactions of a community of users. We present two algorithms, AuthorRank and MessageRank, designed to take advantage of these interactions so as to provide a means of ranking documents by their social impact

Adaptive Information Cluster at Dublin City University

The Adaptive Information Cluster (AIC) is a collaboration between Dublin City University and University College Dublin, and in the AIC at DCU, we investigate and develop as one stream of our research activities, various content analysis tools that can automatically index and structure video information. This includes movies or CCTV footage and the motivation is to support useful searching and browsing features for the envisaged end-users of such systems. We bring in the HCI perspective to this highly-technically-oriented research by brainstorming, generating scenarios, sketching and prototyping the user-interfaces to the resulting video retrieval systems we develop, and we conduct usability studies to better understand the usage and opinions of such systems so as to guide the future direction of our technological research

A Semantic Web Annotation Tool for a Web-Based Audio Sequencer

Music and sound have a rich semantic structure which is so clear to the composer and the listener, but that remains mostly hidden to computing machinery. Nevertheless, in recent years, the introduction of software tools for music production have enabled new opportunities for migrating this knowledge from humans to machines. A new generation of these tools may exploit sound samples and semantic information coupling for the creation not only of a musical, but also of a "semantic" composition. In this paper we describe an ontology driven content annotation framework for a web-based audio editing tool. In a supervised approach, during the editing process, the graphical web interface allows the user to annotate any part of the composition with concepts from publicly available ontologies. As a test case, we developed a collaborative web-based audio sequencer that provides users with the functionality to remix the audio samples from the Freesound website and subsequently annotate them. The annotation tool can load any ontology and thus gives users the opportunity to augment the work with annotations on the structure of the composition, the musical materials, and the creator's reasoning and intentions. We believe this approach will provide several novel ways to make not only the final audio product, but also the creative process, first class citizens of the Semantic We