Phrasing Bimanual Interaction for Visual Design

Architects and other visual thinkers create external representations of their ideas to support early-stage design. They compose visual imagery with sketching to form abstract diagrams as representations. When working with digital media, they apply various visual operations to transform representations, often engaging in complex sequences. This research investigates how to build interactive capabilities to support designers in putting together, that is phrasing, sequences of operations using both hands. In particular, we examine how phrasing interactions with pen and multi-touch input can support modal switching among different visual operations that in many commercial design tools require using menus and tool palettes—techniques originally designed for the mouse, not pen and touch. We develop an interactive bimanual pen+touch diagramming environment and study its use in landscape architecture design studio education. We observe interesting forms of interaction that emerge, and how our bimanual interaction techniques support visual design processes. Based on the needs of architects, we develop LayerFish, a new bimanual technique for layering overlapping content. We conduct a controlled experiment to evaluate its efficacy. We explore the use of wearables to identify which user, and distinguish what hand, is touching to support phrasing together direct-touch interactions on large displays. From design and development of the environment and both field and controlled studies, we derive a set methods, based upon human bimanual specialization theory, for phrasing modal operations through bimanual interactions without menus or tool palettes

Barehand Mode Switching in Touch and Mid-Air Interfaces

Raskin defines a mode as a distinct setting within an interface where the same user input will produce results different to those it would produce in other settings. Most interfaces have multiple modes in which input is mapped to different actions, and, mode-switching is simply the transition from one mode to another. In touch interfaces, the current mode can change how a single touch is interpreted: for example, it could draw a line, pan the canvas, select a shape, or enter a command. In Virtual Reality (VR), a hand gesture-based 3D modelling application may have different modes for object creation, selection, and transformation. Depending on the mode, the movement of the hand is interpreted differently. However, one of the crucial factors determining the effectiveness of an interface is user productivity. Mode-switching time of different input techniques, either in a touch interface or in a mid-air interface, affects user productivity. Moreover, when touch and mid-air interfaces like VR are combined, making informed decisions pertaining to the mode assignment gets even more complicated. This thesis provides an empirical investigation to characterize the mode switching phenomenon in barehand touch-based and mid-air interfaces. It explores the potential of using these input spaces together for a productivity application in VR. And, it concludes with a step towards defining and evaluating the multi-faceted mode concept, its characteristics and its utility, when designing user interfaces more generally

Spatial Motion Doodles: Sketching Animation in VR Using Hand Gestures and Laban Motion Analysis

International audienceWe present a method for easily drafting expressive character animation by playing with instrumented rigid objects. We parse the input 6D trajectories (position and orientation over time)-called spatial motion doodles-into sequences of actions and convert them into detailed character animations using a dataset of parameterized motion clips which are automatically fitted to the doodles in terms of global trajectory and timing. Moreover, we capture the expres-siveness of user-manipulation by analyzing Laban effort qualities in the input spatial motion doodles and transferring them to the synthetic motions we generate. We validate the ease of use of our system and the expressiveness of the resulting animations through a series of user studies, showing the interest of our approach for interactive digital storytelling applications dedicated to children and non-expert users, as well as for providing fast drafting tools for animators

Different expressions of the same mode: a recent dialogue between archaeological and contemporary drawing practices

In this article we explore what we perceive as pertinent features of shared experience at the excavations of an Iron Age Hillfort at Bodfari, North Wales, referencing artist, archaeologist and examples of seminal art works and archaeological records resulting through inter-disciplinary collaboration. We explore ways along which archaeological and artistic practices of improvisation become entangled and productive through their different modes of mark-making. We contend that marks and memories of artist and archaeologist alike emerge interactively, through the mutually constituting effects of the object of study, the tools of exploration, and the practitioners themselves, when they are enmeshed in the cross-modally bound activities. These include, but are not limited to, remote sensing, surveying, mattocking, trowelling, drawing, photographing, videoing and sound recording. These marks represent the co-signatories: the gesture of the often anonymous practitioners, the voice of the deposits, as well as the imprint of the tools, and their interplay creates a multi-threaded narrative documenting their modes of intra-action, in short our practices. They occupy the conceptual space of paradata, and in the process of saturating the interstices of digital cognitive prosthetics they lend probity to their translations in both art form and archive

GESTURE AND THE AUDITORY IMPACT ON THE PERCEPTION OF SOUND: A PERFORMANCE GUIDE FOR UNDERSTANDING GESTURAL FUNCTION ON PERCUSSION PEDAGOGY AND PERFORMANCE

I will address the concept of physical gestures and discuss how they are applicable to the way percussionists visually and audibly perceive music. Through my research, I will focus on performance and pedagogy, seeking new possibilities for introducing percussionists at all ability levels to different articulations. Percussionists have traditionally been led to believe that they do not have the versatility over the initial musical attack and sustain that wind and string players do. This document has been developed to provide a more in-depth approach to how physical and musical gestures can function to create a more versatile and musical percussionist. This project will discuss both musical and physical gestures and the current research behind them, as well as a spectromorphology, a method developed by Dennis Smalley to analyze graphic notation from computer-generated scores. By applying spectromorphology to percussion scores, I will examine how musical gestures are present in solo and ensemble literature, and I will offer suggestions on how to effectively apply these findings to methods of study and performance. This project concludes with a thorough breakdown of standard articulations across marimba/mallet instruments, snare drum, and timpani and provides suggestions on how to functionally utilize gestures for these articulations while offering graphs for each category that highlight the focal points for continued use

Rebecoming analogue : groove, breakbeats and sampling

In this thesis I address two related questions: how does groove work in breakbeats, and how might it enable musical participation across time and space?In order to do this, I analyse breakbeats as they are heard in their original funk context and then in various subsequent genres for which they provide a percussive backbone via the process of recontextualization made possible by digital sampling. From this seemingly narrow focus, more broadly useful ideas about groove emerge and I discuss these in relation to current groovological thought. Of particular significance within my findings is the often‐overlooked role which timbre plays in groove. I propose that the groove in breakbeats operates as a result of timbral, as much as temporal, factors, and that breakbeats can therefore be seen to embody the complementary concepts of Wilson’s heterogeneous sound ideal and Small’s musicking. By exploring groove, breakbeats and sampling from a range of perspectives I show that the potent conceptual combination of musicking and the heterogeneous sound ideal accounts for the perennial appeal of breakbeats as a fundamental building block in contemporary popular music. In order to explore these ideas, following initial chapters that establish a theoretical framework, each successive chapter then deals with a particular manifestation of the breaks. Overall, this structure builds a kaleidoscopic conceptual picture that is appropriate to the multi--‐faceted nature of groove and the enduring versatility of breakbeats.[Note: audio samples available with hard copy of thesis only.

An evaluation of discrete and continuous mid-air loop and marking menu selection in optical see-through HMDs

© 2019 Copyright held by the owner/author(s). This paper investigates discrete and continuous hand-drawn loops and marks in mid-air as a selection input for gesture-based menu systemsonoptical see-through head-mounteddisplays (OST HMDs). We explore two fundamental methods of providing menu selection: the marking menu and the loop menu, and a hybrid method which combines the two. The loop menu design uses a selection mechanism with loops to approximate directional selections in a menu system. We evaluate the merits of loop and marking menu selection in an experiment with two phases and report that 1) the loop-based selection mechanism provides smooth and effective interaction; 2) users prioritize accuracy and comfort over speed for mid-air gestures; 3) users can exploit the flexibility of a final hybrid marking/loop menu design; and, finally, 4) users tend to chunk gestures depending on the selection task and their level of familiarity with the menu layout

A Multifaceted Performance Model for the Multiple Percussion Performance Practice: Performance Analysis of Select Works toward Developing a Graduate Curriculum

The purpose of this document is to articulate the various dimensions of the solo multiple percussion performance practice, the primary objective being to construct appropriate graduate curricula. It was found that the necessary skills and concepts could be categorized into four dimensions that make up the multi-faceted model: (1) equipment and setup, (2) notational style, (3) technical demands, and (4) conceptual performance. Requisite skills were posited from performance analyses of over fifty solo works that range from intermediate to advanced difficulty. Representative works are described that might effectively introduce and challenge these requisite skills and concepts, as are exemplary pieces that require their mastery. One piece that I performed for recital within my doctoral program was selected for each dimension to provide experiential details: The Anvil Chorus is used to describe equipment and setup, Psappha for notational style and devices, Thirteen Drums for technical facility, and Tunnels for the performance concept. I conclude the document with sample program curricula suggested for master’s and doctoral level, as well as curricula specific to each dimension of the performance model. It is my hope that the percussion instructor will use this model to analyze their own inventory and to include future additions to the literature for creating an effective and comprehensive graduate multiple percussion curriculum

Expressy : Using a Wrist-worn Inertial Measurement Unit to Add Expressiveness to Touch-based Interactions

Expressiveness, which we define as the extent to which rich and complex intent can be conveyed through action, is a vital aspect of many human interactions. For instance, paint on canvas is said to be an expressive medium, because it affords the artist the ability to convey multifaceted emotional intent through intricate manipulations of a brush. To date, touch devices have failed to offer users a level of expressiveness in their interactions that rivals that experienced by the painter and those completing other skilled physical tasks. We investigate how data about hand movement – provided by a motion sensor, similar to those found in many smart watches or fitness trackers – can be used to expand the expressiveness of touch interactions. We begin by introducing a conceptual model that formalizes a design space of possible expressive touch interactions. We then describe and evaluate Expressy, an approach that uses a wrist-worn inertial measurement unit to detect and classify qualities of touch interaction that extend beyond those offered by today’s typical sensing hardware. We conclude by describing a number of sample applications, which demonstrate the enhanced, expressive interaction capabilities made possible by Expressy

A Framework of Personality Cues for Conversational Agents

Conversational agents (CAs)—software systems emulating conversations with humans through natural language—reshape our communication environment. As CAs have been widely used for applications requiring human-like interactions, a key goal in information systems (IS) research and practice is to be able to create CAs that exhibit a particular personality. However, existing research on CA personality is scattered across different fields and researchers and practitioners face difficulty in understanding the current state of the art on the design of CA personality. To address this gap, we systematically analyze existing studies and develop a framework on how to imbue CAs with personality cues and how to organize the underlying range of expressive variation regarding the Big Five personality traits. Our framework contributes to IS research by providing an overview of CA personality cues in verbal and non-verbal language and supports practitioners in designing CAs with a particular personality