Improving Usability of Interactive Graphics Specification and Implementation with Picking Views and Inverse Transformations

Specifying and programming graphical interactions are difficult tasks, notably because designers have difficulties to express the dynamics of the interaction. This paper shows how the MDPC architecture improves the usability of the specification and the implementation of graphical interaction. The architecture is based on the use of picking views and inverse transforms from the graphics to the data. With three examples of graphical interaction, we show how to express them with the architecture, how to implement them, and how this improves programming usability. Moreover, we show that it enables implementing graphical interaction without a scene graph. This kind of code prevents from errors due to cache consistency management

Towards a Unified Knowledge-Based Approach to Modality Choice

This paper advances a unified knowledge-based approach to the process of choosing the most appropriate modality or combination of modalities in multimodal output generation. We propose a Modality Ontology (MO) that models the knowledge needed to support the two most fundamental processes determining modality choice – modality allocation (choosing the modality or set of modalities that can best support a particular type of information) and modality combination (selecting an optimal final combination of modalities). In the proposed ontology we model the main levels which collectively determine the characteristics of each modality and the specific relationships between different modalities that are important for multi-modal meaning making. This ontology aims to support the automatic selection of modalities and combinations of modalities that are suitable to convey the meaning of the intended message

Semantic Image Retrieval via Active Grounding of Visual Situations

We describe a novel architecture for semantic image retrieval---in particular, retrieval of instances of visual situations. Visual situations are concepts such as "a boxing match," "walking the dog," "a crowd waiting for a bus," or "a game of ping-pong," whose instantiations in images are linked more by their common spatial and semantic structure than by low-level visual similarity. Given a query situation description, our architecture---called Situate---learns models capturing the visual features of expected objects as well the expected spatial configuration of relationships among objects. Given a new image, Situate uses these models in an attempt to ground (i.e., to create a bounding box locating) each expected component of the situation in the image via an active search procedure. Situate uses the resulting grounding to compute a score indicating the degree to which the new image is judged to contain an instance of the situation. Such scores can be used to rank images in a collection as part of a retrieval system. In the preliminary study described here, we demonstrate the promise of this system by comparing Situate's performance with that of two baseline methods, as well as with a related semantic image-retrieval system based on "scene graphs.

Crowd-sourced Photographic Content for Urban Recreational Route Planning

Routing services are able to provide travel directions for users of all modes of transport. Most of them are focusing on functional journeys (i.e. journeys linking given origin and destination with minimum cost) while paying less attention to recreational trips, in particular leisure walks in an urban context. These walks are additionally predefined by time or distance and as their purpose is the process of walking itself, the attractiveness of areas that are passed by can be an important factor in route selection. This factor is hard to be formalised and requires a reliable source of information, covering the entire street network. Previous research shows that crowd-sourced data available from photo-sharing services has a potential for being a measure of space attractiveness, thus becoming a base for a routing system that suggests leisure walks, and ongoing PhD research aims to build such system. This paper demonstrates findings on four investigated data sources (Flickr, Panoramio, Picasa and Geograph) in Central London and discusses the requirements to the algorithm that is going to be implemented in the second half of this PhD research. Visual analytics was chosen as a method for understanding and comparing obtained datasets that contain hundreds of thousands records. Interactive software was developed to find a number of problems, as well as to estimate the suitability of the sources in general. It was concluded that Picasa and Geograph have problems making them less suitable for further research while Panoramio and Flickr require filtering to remove photographs that do not contribute to understanding of local attractiveness. Based on this analysis a number of filtering methods were proposed in order to improve the quality of datasets and thus provide a more reliable measure to support urban recreational routing

How Do Gestures Influence Thinking and Speaking? The Gesture-for-Conceptualization Hypothesis.

Peer reviewedPostprin