AudioPairBank: Towards A Large-Scale Tag-Pair-Based Audio Content Analysis

Recently, sound recognition has been used to identify sounds, such as car and river. However, sounds have nuances that may be better described by adjective-noun pairs such as slow car, and verb-noun pairs such as flying insects, which are under explored. Therefore, in this work we investigate the relation between audio content and both adjective-noun pairs and verb-noun pairs. Due to the lack of datasets with these kinds of annotations, we collected and processed the AudioPairBank corpus consisting of a combined total of 1,123 pairs and over 33,000 audio files. One contribution is the previously unavailable documentation of the challenges and implications of collecting audio recordings with these type of labels. A second contribution is to show the degree of correlation between the audio content and the labels through sound recognition experiments, which yielded results of 70% accuracy, hence also providing a performance benchmark. The results and study in this paper encourage further exploration of the nuances in audio and are meant to complement similar research performed on images and text in multimedia analysis.Comment: This paper is a revised version of "AudioSentibank: Large-scale Semantic Ontology of Acoustic Concepts for Audio Content Analysis

Assessing interpersonal trust in an ambient intelligence negotiation system

This paper describes an approach to assess and measure trust based on a specific Ambient Intelligence environment. The primary aim of this work is to address and expand on this line of research by investigating the possibility of measuring trust based on quantifiable behavior. To do so, we present a brief review of the existing definitions of trust and define trust in the context of an Ambient Intelligence (AmI) scenario. Further, we propose a formal definition so that the analysis of trust in this kind of scenarios can be developed. Thus, it is suggested the use of Ambient Intelligence techniques that use a trust data model to collect and evaluate relevant information based on the assumption that observable trust between two entities (parties) results in certain typical behaviors. This will establish the foundation for the prediction of such aspects based on the analysis of people’s interaction with technological environments, providing new potentially interesting trust assessment tools.This work has been supported by COMPETE: POCI-01-0145-FEDER-007043 and FCT - Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) within the Project Scope UID/CEC/00319/2013.info:eu-repo/semantics/publishedVersio

Enhanced P2P Services Providing Multimedia Content

The retrieval facilities of most Peer-to-Peer (P2P) systems are limited to queries based on unique identifiers or small sets of keywords. Unfortunately, this approach is very inadequate and inefficient when a huge amount of multimedia resources is shared. To address this major limitation, we propose an original image and video sharing system, in which a user is able to interactively search interesting resources by means of content-based image and video retrieval techniques. In order to limit the network traffic load, maximizing the usefulness of each peer contacted in the query process, we also propose the adoption of an adaptive overlay routing algorithm, exploiting compact representations of the multimedia resources shared by each peer. Experimental results confirm the validity of the proposed approach, that is capable of dynamically adapting the network topology to peer interests, on the basis of query interactions among users

Similarity searching in sequence databases under time warping.

Wong, Siu Fung.Thesis submitted in: December 2003.Thesis (M.Phil.)--Chinese University of Hong Kong, 2004.Includes bibliographical references (leaves 77-84).Abstracts in English and Chinese.Abstract --- p.iiAcknowledgement --- p.viChapter 1 --- Introduction --- p.1Chapter 2 --- Preliminary --- p.6Chapter 2.1 --- Dynamic Time Warping (DTW) --- p.6Chapter 2.2 --- Spatial Indexing --- p.10Chapter 2.3 --- Relevance Feedback --- p.11Chapter 3 --- Literature Review --- p.13Chapter 3.1 --- Searching Sequences under Euclidean Metric --- p.13Chapter 3.2 --- Searching Sequences under Dynamic Time Warping Metric --- p.17Chapter 4 --- Subsequence Matching under Time Warping --- p.21Chapter 4.1 --- Subsequence Matching --- p.22Chapter 4.1.1 --- Sequential Search --- p.22Chapter 4.1.2 --- Indexing Scheme --- p.23Chapter 4.2 --- Lower Bound Technique --- p.25Chapter 4.2.1 --- Properties of Lower Bound Technique --- p.26Chapter 4.2.2 --- Existing Lower Bound Functions --- p.27Chapter 4.3 --- Point-Based indexing --- p.28Chapter 4.3.1 --- Lower Bound for subsequences matching --- p.28Chapter 4.3.2 --- Algorithm --- p.35Chapter 4.4 --- Rectangle-Based indexing --- p.37Chapter 4.4.1 --- Lower Bound for subsequences matching --- p.37Chapter 4.4.2 --- Algorithm --- p.41Chapter 4.5 --- Experimental Results --- p.43Chapter 4.5.1 --- Candidate ratio vs Width of warping window --- p.44Chapter 4.5.2 --- CPU time vs Number of subsequences --- p.45Chapter 4.5.3 --- CPU time vs Width of warping window --- p.46Chapter 4.5.4 --- CPU time vs Threshold --- p.46Chapter 4.6 --- Summary --- p.47Chapter 5 --- Relevance Feedback under Time Warping --- p.49Chapter 5.1 --- Integrating Relevance Feedback with DTW --- p.49Chapter 5.2 --- Query Reformulation --- p.53Chapter 5.2.1 --- Constraint Updating --- p.53Chapter 5.2.2 --- Weight Updating --- p.55Chapter 5.2.3 --- Overall Strategy --- p.58Chapter 5.3 --- Experiments and Evaluation --- p.59Chapter 5.3.1 --- Effectiveness of the strategy --- p.61Chapter 5.3.2 --- Efficiency of the strategy --- p.63Chapter 5.3.3 --- Usability --- p.64Chapter 5.4 --- Summary --- p.71Chapter 6 --- Conclusion --- p.72Chapter A --- Deduction of Data Bounding Hyper-rectangle --- p.74Chapter B --- Proof of Theorem2 --- p.76Bibliography --- p.77Publications --- p.8

Enhancing comprehension of complex data visualizations: Framework and techniques based on signature exploration

This thesis presents a framework and set of readily applicable techniques for enhancing comprehension of complex data visualizations. Central to the work has been the definition and exploration of a new concept, signature exploration. Visualization is being used increasingly to help make sense of large sets of data and information. Abstractions of complex data can be performed to reduce the dimensions to 2 or 3 for display. Novel or established representations can be used that allow direct mapping of greater numbers of attributes, and of a variety of data structures. There is an ever expanding set of visualization tools available. Two questions face the user: how to choose appropriate displays and how to understand the resultant graphic. This thesis examines how to support the user’s comprehension in this context. The work makes the following three main contributions to enhancing comprehension of complex data visualizations: the definition and application of signature exploration, a concept describing the exploration of visualization behaviour using specially constructed data; the proposal of a framework for the design of visualization systems for increased comprehension; the introduction of two new forms of interaction - which are here described as visual data tracking and feature fingerprinting. The central theme for the exploration presented in this work is the notion that a user wants to take data that is known in some way, put this into the visualization process and assess the resultant visual depiction. This intuitive desire has been captured in the definition of the concept, signature exploration. Signature exploration describes the exploration of the behaviour of visual representations using specially constructed datasets that contain features of interest. The datasets are used to explore the signatures of different visual representations and mathematical transformations. The thesis defines and illustrates signature exploration, with five proposed approaches: generic dataset provision; user-construction of data; querying; insertion of landmarks; elicitation and application of feedback data. These applications of signature exploration, together with analysis of the comprehension challenges presented by different aspects of visualization, and established work to support user comprehension, form the basis of the framework for increased user comprehension. Example software has been developed within the context of a visualization application that employs a number of visualization algorithms to generate graphics for multivariate or proximity data. Principal Components Analysis, Principal Coordinates Analysis and distance metrics of various kinds are the algorithms used. An additional interface is given to the user, to perform signature exploration. The work has resulted in the specification of a set of techniques that developers can readily apply. Two new interaction forms are described: visual data tracking - bi-directional brushing and linking between representations also allowing change of position or value; feature fingerprinting - synthetic additions to real-world datasets to provide the user with calibration of the visual depiction

Computer Learning of Subjectivity

tic control knobs." For example, the Wold model for retrieving perceptually similar visual patterns has knobs corresponding to periodicity, directionality, and randomness [2]. However, rarely can models with semantic control knobs be found. Even when they exist, it is an effort to know how to optimally set them. Moreover, usually a person does not make a single query, but a succession of queries, with slight variations each time. Therefore, she not only needs to know how to set the knobs when initiating a query session, but also how to adjust them with each new query. What I have described is the current trend in content-based retrieval and annotation systems, and it needs to change. The system must recognize that the user's goals evolve while they browse; subjectivity, mood-dependence, and fickleness are to be expected. Furthermore, a system that tracks the evolving goals of a subjective human will also be helpful for the difficult but common query sessions best described as "I'll kn