Computer supported interaction in distributed design teams

Thesis (Sc. D.)--Massachusetts Institute of Technology, Dept. of Civil and Environmental Engineering, 1998.Includes bibliographical references (leaves ).by Karim Mohie El Din Hussein.Sc.D

Augmentative communication device design, implementation and evaluation

The ultimate aim of this thesis was to design and implement an advanced software based Augmentative Communication Device (ACD) , or Voice Output Communication Aid NOCA), for non-vocal Learning Disabled individuals by applying current psychological models, theories, and experimental techniques. By taking account of potential user's cognitive and linguistic abilities a symbol based device (Easy Speaker) was produced which outputs naturalistic digitised human speech and sound and makes use of a photorealistic symbol set. In order to increase the size of the available symbol set a hypermedia style dynamic screen approach was employed. The relevance of the hypermedia metaphor in relation to models of knowledge representation and language processing was explored.Laboratory based studies suggested that potential user's could learn to productively operate the software, became faster and more efficient over time when performing set conversational tasks. Studies with unimpaired individuals supported the notion that digitised speech was less cognitively demanding to decode, or listen to.With highly portable, touch based, PC compatible systems beginning to appear it is hoped that the otherwise silent will be able to use the software as their primary means of communication with the speaking world. Extensive field trials over a six month period with a prototype device and in collaboration with user's caregivers strongly suggested this might be the case.Off-device improvements were also noted suggesting that Easy Speaker, or similar software has the potential to be used as a communication training tool. Such training would be likely 10 improve overall communicative effectiveness.To conclude, a model for successful ACD development was proposed

Acting Intuition into Sense: How Film Crews Make Sense with Embodied Ways of Knowing

This study contributes to a holistic understanding of sensemaking by going beyond the mind–body dualism. To do so, we focus analytically on a phenomenon that operates at the nexus of mind and body: intuition. By observing four film crews, we unpack how people act their intuition into sense – that is, how they transform, through action, an initial sense (intuition) that is tacit, intimate, and complex into one that is publicly displayed, simpler, and ordered (i.e., a developed sense). Our model identifies two sensemaking trajectories, each of which involves several bodily actions (e.g., displaying feelings, working hands-on, speaking assertively). These actions enable intuition to express a facet of itself and acquire new properties. This study makes three important contributions. First, it develops the holistic-relational character of sensemaking by locating it in the relations among multiple loci (cognition, language, body, and materiality) rather than in each one disjunctively. Second, it theorizes embodied sensemaking as a transformative process entailing a rich repertoire of bodily actions. Third, it extends sensemaking research by attending to the physicality and materiality of language in embodied sensemaking

Irony in a second language: exploring the comprehension of Japanese speakers of English

This thesis focuses on the extent to which non-native speakers of English understand potentially ironic utterances in a similar way to native speakers. Barbe (1995: 4) sees irony as one of ‘the final obstacles before achieving near native-speaker fluency.’ This assumption is supported by the findings of earlier studies (Bouton 1999, Lee 2002; Manowong 2011; Yamanaka 2003) which assumed a Gricean framework seeing irony as communicating the ‘opposite of what is said’ (Grice 1975, 1978). This thesis adopts instead the relevance-theoretic account of irony as echoic (Sperber and Wilson 1995; Wilson and Sperber 2012), arguing that previous work suffers from both problematic theoretical assumptions and flawed experimental methods. The thesis reports the findings of two experiments designed to examine similarities and differences between the responses of non-native speakers of English (here Japanese speakers) and native speakers and how similar or different the effects of prosody are for these groups. The first experiment, conducted by an online survey, provided surprising results, suggesting that Japanese speakers can respond to potentially ironical utterances similarly to native speakers. The second experiment, focusing on the effects of prosody, compared the groups with regard to response trends. Three prosodic contours were used in this study, labelled ‘basic’ (a kind of default, unmarked tone), ‘deadpan’ (with a narrower pitch range), and ‘exaggerated’ (with a wider pitch range). The results indicated that Japanese participants could perceive English prosodic structure in similar ways to native speakers and were affected by prosodic contours in similar ways. It also suggested that Japanese participants were affected less strongly by ‘exaggerated’ intonation and slightly more strongly by ‘deadpan’ tones. These findings suggest that a relevance-theoretic framework provides the means to carry out fuller investigations than carried out previously and to develop a more systematic explanation of the understanding of irony in a second language

Online avatar based interactions

The gridWorld project attempts to utilize 3D to develop an online multi-user visual chat system. GridWorld address ideas of how conversations in a virtual environment can be facilitated and enhanced by an abstract visual interface design. The visual interface was developed from research and examination of existing ideas, methodologies and application for development of user-embodiment, chat/virtual space, and interface useability towards the visualization of communication

Humour production in face-to-face interaction: a multimodal and cognitive study

El humor es una de las formas de comunicación más complejas que existen (Veale, Brône & Feyaerts, 2015). Entre las teorías lingüísticas sobre el humor, algunas tienen un enfoque semántico-pragmático, tales como la Semantic Script Theory of Humour (Raskin, 1984) o la General Theory of Verbal Humour (Attardo, 2001). Otras se inscriben en la Teoría de la Relevancia (Yus, 2016) y las hay también con una perspectiva más cognitiva (Giora, 1991, 2015; Coulson & Okley, 2005; Veale, Feyaerts & Brône, 2006). Por otra parte, se han realizado varios estudios sobre los marcadores multimodales de la ironía o el sarcasmo, cuyos resultados son dispares (Attardo, Eisterhold, Hay, and Poggi, 2003; Attardo, Pickering, and Baker, 2011; Attardo, Wagner, and Urios-Aparisi, 2011). Sin embargo, el humor no irónico ha sido objeto de menor estudio. Además, la mayor parte de los análisis se circunscriben al humor ensayado, con pocos estudios sobre el humor producido de forma espontánea (Bryant, 2010, Feyaerts, 2013; Tabacaru, 2014, etc.) y menos aún que conjuguen la perspectiva multimodal con la cognitiva. En esta tesis se analizan 14 entrevistas extraídas de The Late Show with Stephen Colbert con vistas a explicar la comunicación espontánea del humor desde el punto de vista multimodal y cognitivo. Los enunciados se han identificado como humorísticos cuando el público reaccionaba riendo. El análisis multimodal se ha realizado en ELAN, con cinco niveles de anotaciones: transcripción, tipo de humor (Feyaerts et al., 2010), mecanismo conceptual subyacente (Croft & Cruse, 2004), gestos y prosodia. El estudio prosódico se ha llevado a cabo con Praat, a fin de determinar si había un mayor contraste prosódico en enunciados humorísticos. Los resultados muestran que los mecanismos multimodales y cognitivos no difieren entre enunciados humorísticos y no humorísticos.Departamento de Filología InglesaDoctorado en Estudios Ingleses Avanzados: Lenguas y Culturas en Contact

Application of Machine Learning within Visual Content Production

We are living in an era where digital content is being produced at a dazzling pace. The heterogeneity of contents and contexts is so varied that a numerous amount of applications have been created to respond to people and market demands. The visual content production pipeline is the generalisation of the process that allows a content editor to create and evaluate their product, such as a video, an image, a 3D model, etc. Such data is then displayed on one or more devices such as TVs, PC monitors, virtual reality head-mounted displays, tablets, mobiles, or even smartwatches. Content creation can be simple as clicking a button to film a video and then share it into a social network, or complex as managing a dense user interface full of parameters by using keyboard and mouse to generate a realistic 3D model for a VR game. In this second example, such sophistication results in a steep learning curve for beginner-level users. In contrast, expert users regularly need to refine their skills via expensive lessons, time-consuming tutorials, or experience. Thus, user interaction plays an essential role in the diffusion of content creation software, primarily when it is targeted to untrained people. In particular, with the fast spread of virtual reality devices into the consumer market, new opportunities for designing reliable and intuitive interfaces have been created. Such new interactions need to take a step beyond the point and click interaction typical of the 2D desktop environment. The interactions need to be smart, intuitive and reliable, to interpret 3D gestures and therefore, more accurate algorithms are needed to recognise patterns. In recent years, machine learning and in particular deep learning have achieved outstanding results in many branches of computer science, such as computer graphics and human-computer interface, outperforming algorithms that were considered state of the art, however, there are only fleeting efforts to translate this into virtual reality. In this thesis, we seek to apply and take advantage of deep learning models to two different content production pipeline areas embracing the following subjects of interest: advanced methods for user interaction and visual quality assessment. First, we focus on 3D sketching to retrieve models from an extensive database of complex geometries and textures, while the user is immersed in a virtual environment. We explore both 2D and 3D strokes as tools for model retrieval in VR. Therefore, we implement a novel system for improving accuracy in searching for a 3D model. We contribute an efficient method to describe models through 3D sketch via an iterative descriptor generation, focusing both on accuracy and user experience. To evaluate it, we design a user study to compare different interactions for sketch generation. Second, we explore the combination of sketch input and vocal description to correct and fine-tune the search for 3D models in a database containing fine-grained variation. We analyse sketch and speech queries, identifying a way to incorporate both of them into our system's interaction loop. Third, in the context of the visual content production pipeline, we present a detailed study of visual metrics. We propose a novel method for detecting rendering-based artefacts in images. It exploits analogous deep learning algorithms used when extracting features from sketches

Effects of color and animation on visual short -term memory in computer-environment learning tasks

The goals of this research study were to gain a better understanding of the effects of color and animation on visual short-term memory (STM) in computer-environment learning tasks. The research methodology was based on an investigative design that attempted to address weaknesses of previous visual STM research (Klauer & Zhao, 2004) to provide information readily applicable to instructional design and technology. A 3 x 2 x 3 totally-within participants, repeated-measures Factorial Analysis of Variance (ANOVA) design was utilized to investigate limitations on visual short-term memory learning processes in computer-learning environments. Participant scores for accuracy of visual learning tasks and participant latency in responses to visual learning tasks were measured. Findings suggest achromatic color information interferes with visual STM learning tasks in specific contexts. Findings also suggest that the use of animations in visual STM learning tasks may interfere with complex visual STM learning. Numerous questions are raised for further research utilizing experimental designs that focus on visual STM learning tasks in contrast to designs that focus solely on identifying visual STM parameters.;Keywords. Computer aided instruction; Computer based instruction; Double dissociations; Dual coding; Multimedia learning theory; Information processing, Instructional design, Phonological loop, Short-term memory, Visual and spatial short-term memory, Visual stimuli, Visuospatial sketch pad, Working memory