Perception and performance: an evaluation of multimodal feedback for the assessment of curve shape differences

The EU-funded SATIN project sought to provide a multimodal interface to aid product designers in judging the quality of curved shapes. This thesis outlines a research programme designed to assist in the exploration of fundamental issues related to this project, and provide a means to evaluate the success of such interfaces more generally. Therefore, three studies were undertaken with the aim of exploring the value of haptic and sound feedback in the perception of curve shape differences, and through the knowledge gained provide an evaluative framework for the assessment of such interfaces. The first study found that visual, haptic, and visual-haptic perception was insufficient to judge discontinuities in curvature without some further augmentation. This led to a second study which explored the use of sound for conveying curve shape information. It was found that sine waves or harmonic sounds were most suited to for this task. The third study combined visual-haptic and auditory information. It was found that sound improved the perception of curve shape differences, although this was dependent upon the type of sonification method used. Further to this, data from studies one and three were used to identify gradient as the active mechanism of curve shape differentiation and provided a model for the prediction of these differences. Similarly performance data (response time, accuracy, and confidence) were analysed to produce a model for the prediction of user performance at varying degrees of task difficulty. The research undertaken across these studies was used to develop a framework to evaluate multimodal interfaces for curve shape exploration. In particular a ‘discount’ psychophysical method was proposed, along with predictive tools for the creation of perceptual and performance metrics, plus guidelines to aid development. This research has added to fundamental knowledge and provided a useful framework through which future multimodal interfaces may be evaluated

Conceptual Simplification: an Empirical Investigation of a New Method for Analysis, Learning and Memorisation of Post-Tonal Piano Music

There is a gap in music performance, education and psychology in terms of memorisation training for post-tonal piano music. Despite the repertoire spanning over 100 years, pedagogues and professionals still lack effective tools for developing this skill. Existing research on this domain is mostly focused on observing practitioners’ behaviours during practice, to understand how these prepare for a memorised performance of a selected repertoire. However, the resulting Performance Cue Theory that emerges from these studies does not provide a systematic method to assist learning, but instead, explains performers’ behaviours to fulfil the given task. Furthermore, other important aspects of memorisation, such as the role of sleep for memory consolidation; influential parameters of performance practice, such as the abilities of perfect pitch and sight-reading; or the role of emotions have rarely been examined or simply omitted. This thesis focuses on testing, extending and formalising a new method for analysis, learning and memorisation of post-tonal piano music, named Conceptual Simplification. This presents a novel implementation to musical memorisation of group theory, number theory and geometry; and the paradigms of divide-and-conquer, decrease-and-conquer and transform-and-conquer. Therefore, it builds on mathematics and computer science to improve human memory and musical performance. However, as demonstrated with this thesis, Conceptual Simplification does not require any previous scientific training to be successfully implemented and works for different learning styles and types of complexity. From testing the parameters of perfect pitch, synaesthesia, sight-reading, emotions, sleep, mental practice, complexity and expertise; the most influential parameters for memorisation identified are perfect pitch, sight-reading, sleep and complexity. Additionally, a formal definition for complexity is formulated. Similarly, after testing different practice and performance strategies, the most effective strategies for memorisation identified are simplifying strategies and conceptual encoding strategies, included in Conceptual Simplification. Finally, it is also revealed the positive role of mental practice for coping with performance anxiety and self-sabotage. Throughout this thesis, Conceptual Simplification is tested through a series of studies with practitioners, who range from conservatoire piano students to international performers, including observation and analysis of the author’s own performing practice. The repertoire featured involves existing post-tonal and commissioned works. Although the scope of this thesis is limited to testing Conceptual Simplification for post-tonal piano music, this method could be adapted to other instrumentalists, singers and conductors; and musical genres. More ambitious applications might involve non-musical domains, since Conceptual Simplification essentially scaffolds complexity, proceeding in a non-linear manner and avoiding time-consuming procedures. The method also presents enough flexibility for other practitioners to incorporate additional strategies, adapting it to their needs accordingly. Finally, Conceptual Simplification also indicates promising additional benefits. Concretely, in preventing performance anxiety through greater confidence and reducing the potential for injuries that usually result from repeated practice. Conceptual Simplification’s systematic approach toward engaging conceptual memory and reasoning leads to more confident memorised performances, while needing less repetition during practice

Perception and performance: an evaluation of multimodal feedback for the assessment of curve shape differences

The EU-funded SATIN project sought to provide a multimodal interface to aid product designers in judging the quality of curved shapes. This thesis outlines a research programme designed to assist in the exploration of fundamental issues related to this project, and provide a means to evaluate the success of such interfaces more generally. Therefore, three studies were undertaken with the aim of exploring the value of haptic and sound feedback in the perception of curve shape differences, and through the knowledge gained provide an evaluative framework for the assessment of such interfaces. The first study found that visual, haptic, and visual-haptic perception was insufficient to judge discontinuities in curvature without some further augmentation. This led to a second study which explored the use of sound for conveying curve shape information. It was found that sine waves or harmonic sounds were most suited to for this task. The third study combined visual-haptic and auditory information. It was found that sound improved the perception of curve shape differences, although this was dependent upon the type of sonification method used. Further to this, data from studies one and three were used to identify gradient as the active mechanism of curve shape differentiation and provided a model for the prediction of these differences. Similarly performance data (response time, accuracy, and confidence) were analysed to produce a model for the prediction of user performance at varying degrees of task difficulty. The research undertaken across these studies was used to develop a framework to evaluate multimodal interfaces for curve shape exploration. In particular a ‘discount’ psychophysical method was proposed, along with predictive tools for the creation of perceptual and performance metrics, plus guidelines to aid development. This research has added to fundamental knowledge and provided a useful framework through which future multimodal interfaces may be evaluated