A model for dynamic minimal mentalizing in dialogue

Buschmeier H, Kopp S. A model for dynamic minimal mentalizing in dialogue. Presented at the 12th Biannual Conference of the German Cognitive Science Society (KogWis 2014), Tübingen, Germany

A model for dynamic minimal mentalising in dialogue

<p>Buschmeier, H. & Kopp, S. (2014). A model for dynamic minimal mentalizing in dialogue. In <em>Proceedings of the 12th Biannual Conference of the German Cognitive Science Society,</em> pp. S32–S33, Tübingen, Germany. </p> <p>Poster presented at KogWis 2014, 1-4 October 2014, Tübingen Germany.</p

The shrink point: audiovisual integration of speech-gesture synchrony

Kirchhof C. The shrink point: audiovisual integration of speech-gesture synchrony. Bielefeld: Universität Bielefeld; 2017.Up to now, the focus in gesture research has long been on the production of speech-accompanying gestures and on how speech-gesture utterances contribute to communication. An issue that has mostly been neglected is in how far listeners even perceive the gesture-part of a multimodal utterance. For instance, there has been a major focus on the lexico-semiotic connection between spontaneously coproduced gestures and speech in gesture research (e.g., de Ruiter, 2007; Kita & Özyürek, 2003; Krauss, Chen & Gottesman, 2000). Due to the rather precise timing between the prosodic peak in speech with the most prominent stroke of the gesture phrase in production, Schegloff (1984) and Krauss, Morrel-Samuels and Colasante (1991; also Rauscher, Krauss & Chen, 1996), among others, coined the phenomenon of lexical affiliation. By following Krauss et al. (1991), the first empirical study of this dissertation investigates the nature of the semiotic relation between speech and gestures, focusing on its applicability to temporal perception and comprehension. When speech and lip movements diverge too far from the original production synchrony, this can be highly irritating to the viewer, even when audio and video stem from the same original recording (e.g., Vatakis, Navarra, Soto-Faraco & Spence, 2008; Feyereisen, 2007) – there is only a small temporal window of audiovisual integration (AVI) within which viewer-listeners can internally align discrepancies between lip movements and the speech supposedly produced by these (e.g. McGurk & MacDonald, 1976). Several studies in the area of psychophysics (e.g., Nishida, 2006; Fujisaki & Nishida, 2005) found that there is also a time window for the perceptual alignment of nonspeech visual and auditory signals. These and further studies on the AVI of speech-lip asynchronies have inspired research on the perception of speech-gesture utterances. McNeill, Cassell, and McCullough (1994; Cassell, McNeill & McCullough, 1999), for instance, discovered that listeners take up information even from artificially combined speech and gestures. More recent studies researching the AVI of speech and gestures have employed event-related potential (ERP) monitoring as a methodological means to investigate the perception of multimodal utterances (e.g., Gullberg & Holmqvist, 1999; 2006; Özyürek, Willems, Kita & Hagoort, 2007; Habets, Kita, Shao, Özyürek & Hagoort, 2011). While the aforementioned studies from the fields of psychophysics and speech-only and speech-gesture research have contributed greatly to theories of how listeners perceive multimodal signals, there has been a lack of explorations of natural data and of dyadic situations. This dissertation investigates the perception of naturally produced speech-gesture utterances by having participants rate the naturalness of synchronous and asynchronous versions of speech-gesture utterances using different qualitative and quantitative methodologies such as an online rating study and a preference task. Drawing, for example, from speech-gesture production models based on Levelt's (1989) model of speech production (e.g., de Ruiter, 1998; 2007; Krauss et al., 2000; Kita & Özyürek, 2003) and founding on the results and analyses of the studies conducted for this dissertation, I finally propose a model draft of a possible transmission cycle between Growth Point (e.g., McNeill, 1985; 1992) and Shrink Point, the perceptual counterpart to the Growth Point. This model includes the temporal and semantic alignment of speech and different gesture types as well as their audiovisual and conceptual integration during perception. The perceptual studies conducted within the scope of this dissertation have revealed varying temporal ranges in which an asynchrony in speechgesture utterances is integrable by the listener, especially iconic gestures

Neural and behavioral correlates of arithmetic development and learning in children

Arithmetic learning improves mathematical competence, which is necessary for successful daily life. However, little is known about the neural underpinnings of arithmetic learning during childhood, the age when individuals learn most of the mathematical skills and the vast majority of our knowledge comes from adult studies. In this dissertation project, four studies were conducted to investigate the neural and behavioral correlates of arithmetic development and learning in children. In Study 1 arithmetic development was evaluated longitudinally to see whether it is monotonous or there are intermediate phases in which certain domain-general processes become important but disappear later. In Study 2 arithmetic complexity was evaluated to see whether it relies on both magnitude and cognitive processes, such as in adults. In Study 3 it was asked whether the findings in adults are valid for children or are there intermediate stages. Furthermore, it was evaluated whether few training sessions are reflective of more long-term learning processes. In Study 4 the brain activation changes during the course of learning were measured to see whether they reveal similar changes as in after arithmetic learning. The findings revealed that different domain-general cognitive processes are involved in different steps of arithmetic development and learning. Furthermore, arithmetic achievement occurs in two steps in children, first from slow effortful procedural processes to fast compacted procedural processes, and then to retrieval processes. These changes are distinguishable after one and several training sessions, and also during the course of learning. The findings are integrated in a theoretical model of arithmetic achievement in children, which contains two phases: (i) the efficiency increase (from slow effortful procedural processes to fast compacted procedural processes) and (ii) the strategy change (from fast compacted procedural processes to retrieval processes) phases. The model was developed based on two principles of brain function, optimum performance and energy consumption, and supported by several empirical studies. Taken together, this dissertation project provides a comprehensive framework for arithmetic development and learning in children. The findings might be helpful to develop educational and therapeutic interventions and also a new measure of intervention outcomes, particularly in individuals with mathematical learning disabilities

Neurolinguistische Untersuchungen zur Antizipation des Turn-Endes bei gesprochener Sprache

Wesselmeier H. Neurolinguistische Untersuchungen zur Antizipation des Turn-Endes bei gesprochener Sprache. Bielefeld: Universität Bielefeld; 2017

Jahresbericht 2016 / Institut für Angewandte Informatik (KIT Scientific Reports ; 7736)

Im Jahresbericht 2016 des Instituts für Angewandte Informatik werden, nach einem kurzen Überblick über die Arbeiten, die Forschungsergebnisse dieses Jahres vorgestellt. Die Einordnung erfolgt entsprechend der Zuordnung der Projekte zu den Helmholtz-Programmen SCI, EMR, RE, TIS, BIFTM, STN und SBD. Es schließt sich eine Verzeichnis der im Berichtszeitraum erschienen Publikationen des Instituts an

Measuring pilot control behavior in control tasks with haptic feedback

The research goal of this thesis was to increase the understanding of effects of haptic feedback on human’s performance and control behavior. Firstly, we investigated the effectiveness of haptic aids on improving human’s performance in different control scenarios. Beneficial effects of haptic aids were shown in terms of human's performances and control effort. Comparisons with input-mixing systems showed that, although input-mixing systems yielded better performance than haptic aids in nominal conditions, participants recovered better from failures of haptic systems than from failures of input-mixing aids. Secondly, we investigated how humans adapt their dynamic responses to realize benefits of the haptic feedback. To achieve this goal, we developed novel identification methods to estimate human's neuromuscular dynamics in a multi-loop control task. The novel methods assumed a time-invariant behavior of humans responses. The novel methods were validated in simulation and applied to experimental data. Finally, novel methods were developed to account for time-varying behavior of human's responses. Different sets of numerical simulations were used to validate the novel methods. Then, the methods were applied to data obtained in human in-the-loop experiments