Embodied cognition: A field guide

The nature of cognition is being re-considered. Instead of emphasizing formal operations on abstract symbols, the new approach foregrounds the fact that cognition is, rather, a situated activity, and suggests that thinking beings ought therefore be considered first and foremost as acting beings. The essay reviews recent work in Embodied Cognition, provides a concise guide to its principles, attitudes and goals, and identifies the physical grounding project as its central research focus

Decision Making Under Threat: An Ecological Framework

Humans, like other animals, have evolved a set of neural circuits whose primary function is survival. In the case of predation, these circuits include "reactive fear" circuits involved in fast escape decisions, and "cognitive fear" circuits that are involved in more complex processing associated with slow strategic escape. In the context of flight initiation distance (FID), using neuroimaging combined with computational modeling, we support this differentiation of fear circuits by showing that fast escape decisions are elicited by the periaqueductal gray and midcingulate cortex, regions involved in reactive flight. Conversely, slower escape decisions rely on the hippocampus, posterior cingulate cortex, and prefrontal cortex, a circuit implicated in behavioral flexibility. We further tested whether individual differences in trait anxiety would impact escape behavior and neural responses to slow and fast attacking predators. Behaviorally, we found that trait anxiety was not related to escape decisions for fast threats, but individuals with higher trait anxiety escaped earlier during slow threats. Functional MRI showed that when subjects faced slow threats, trait anxiety positively correlated with activity in the vHPC, mPFC, amygdala and insula. Further, the strength of the functional coupling between the vHPC and mPFC was correlated with the degree of trait anxiety. A similar pattern of separation in survival circuits is also found in a follow up study utilizing the concept of margin of safety (MOS) with multivariate pattern analysis of fMRI data. In addition, we also discussed how decision making under threat was influenced by social factors such as reputation. Overall, these results provide new insights into decision making under threat and a separation of fear into reactive and cognitive circuits.</p

Perceptual Model-Driven Authoring of Plausible Vibrations from User Expectations for Virtual Environments

One of the central goals of design is the creation of experiences that are rated favorably in the intended application context. User expectations play an integral role in tactile product quality and tactile plausibility judgments alike. In the vibrotactile authoring process for virtual environments, vibra-tion is created to match the user’s expectations of the presented situational context. Currently, inefficient trial and error approaches attempt to match expectations implicitly. A more efficient, model-driven procedure based explicitly on tactile user expectations would thus be beneficial for author-ing vibrations. In everyday life, we are frequently exposed to various whole-body vibrations. Depending on their temporal and spectral proper-ties we intuitively associate specific perceptual properties such as “tin-gling”. This suggests a systematic relationship between physical parame-ters and perceptual properties. To communicate with potential users about such elicited or expected tactile properties, a standardized design language is proposed. It contains a set of sensory tactile perceptual attributes, which are sufficient to characterize the perceptual space of vibration encountered in everyday life. This design language enables the assessment of quantita-tive tactile perceptual specifications by laypersons that are elicited in situational contexts such as auditory-visual-tactile vehicle scenes. Howev-er, such specifications can also be assessed by providing only verbal de-scriptions of the content of these scenes. Quasi identical ratings observed for both presentation modes suggest that tactile user expectations can be quantified even before any vibration is presented. Such expected perceptu-al specifications are the prerequisite for a subsequent translation into phys-ical vibration parameters. Plausibility can be understood as a similarity judgment between elicited features and expected features. Thus, plausible vibration can be synthesized by maximizing the similarity of the elicited perceptual properties to the expected perceptual properties. Based on the observed relationships between vibration parameters and sensory tactile perceptual attributes, a 1-nearest-neighbor model and a regression model were built. The plausibility of the vibrations synthesized by these models in the context of virtual auditory-visual-tactile vehicle scenes was validat-ed in a perceptual study. The results demonstrated that the perceptual spec-ifications obtained with the design language are sufficient to synthesize vibrations, which are perceived as equally plausible as recorded vibrations in a given situational context. Overall, the demonstrated design method can be a new, more efficient tool for designers authoring vibrations for virtual environments or creating tactile feedback. The method enables further automation of the design process and thus potential time and cost reductions.:Preface III Abstract V Zusammenfassung VII List of Abbreviations XV 1 Introduction 1 1.1 General Introduction 1 1.1 Objectives of the Thesis 4 1.2 Structure of the Thesis 4 2. Tactile Perception in Real and Virtual Environments 7 2.1 Tactile Perception as a Multilayered Process 7 2.1.1 Physical Layer 8 2.1.2 Mechanoreceptor Layer 9 2.1.3 Sensory Layer 19 2.1.4 Affective Layer 26 2.2 Perception of Virtual Environments 29 2.2.1 The Place Illusion 29 2.2.2 The Plausibility Illusion 31 2.3 Approaches for the Authoring of Vibrations 38 2.3.1 Approaches on the Physical Layer 38 2.3.2 Approaches on the Mechanoreceptor Layer 40 2.3.3 Approaches on the Sensory Layer 40 2.3.4 Approaches on the Affective Layer 43 2.4 Summary 43 3. Research Concept 47 3.1 Research Questions 47 3.1.1 Foundations of the Research Concept 47 3.1.2 Research Concept 49 3.2 Limitations 50 4. Development of the Experimental Setup 53 4.1 Hardware 53 4.1.1 Optical Reproduction System 53 4.1.2 Acoustical Reproduction System 54 4.1.3 Whole-Body Vibration Reproduction System 56 4.2 Software 64 4.2.1 Combination of Reproduction Systems for Unimodal and Multimodal Presentation 64 4.2.2 Conducting Perceptual Studies 65 5. Assessment of a Sensory Tactile Design Language for Characterizing Vibration 67 5.1.1 Design Language Requirements 67 5.1.2 Method to Assess the Design Language 69 5.1.3 Goals of this Chapter 70 5.2 Tactile Stimuli 72 5.2.1 Generalization into Excitation Patterns 72 5.2.2 Definition of Parameter Values of the Excitation Patterns 75 5.2.3 Generation of the Stimuli 85 5.2.4 Summary 86 5.3 Assessment of the most relevant Sensory Tactile Perceptual Attributes 86 5.3.1 Experimental Design 87 5.3.2 Participants 88 5.3.3 Results 88 5.3.4 Aggregation and Prioritization 89 5.3.5 Summary 91 5.4 Identification of the Attributes forming the Design Language 92 5.4.1 Experimental Design 93 5.4.2 Participants 95 5.4.3 Results 95 5.4.4 Selecting the Elements of the Sensory Tactile Design Language 106 5.4.5 Summary 109 5.5 Summary and Discussion 109 5.5.1 Summary 109 5.5.2 Discussion 111 6. Quantification of Expected Properties with the Sensory Tactile Design Language 115 6.1 Multimodal Stimuli 116 6.1.1 Selection of the Scenes 116 6.1.2 Recording of the Scenes 117 6.1.3 Recorded Stimuli 119 6.2 Qualitative Communication in the Presence of Vibration 123 6.2.1 Experimental Design 123 6.2.2 Participants 124 6.2.3 Results 124 6.2.4 Summary 126 6.3 Quantitative Communication in the Presence of Vibration 126 6.3.1 Experimental Design 127 6.3.2 Participants 127 6.3.3 Results 127 6.3.4 Summary 129 6.4 Quantitative Communication in the Absence of Vibration 129 6.4.1 Experimental Design 130 6.4.2 Participants 132 6.4.3 Results 132 6.4.4 Summary 134 6.5 Summary and Discussion 135 7. Synthesis Models for the Translation of Sensory Tactile Properties into Vibration 137 7.1 Formalization of the Tactile Plausibility Illusion for Models 139 7.1.1 Formalization of Plausibility 139 7.1.2 Model Boundaries 143 7.2 Investigation of the Influence of Vibration Level on Attribute Ratings 144 7.2.1 Stimuli 145 7.2.2 Experimental Design 145 7.2.3 Participants 146 7.2.4 Results 146 7.2.5 Summary 148 7.3 Comparison of Modulated Vibration to Successive Impulse-like Vibration 148 7.3.1 Stimuli 149 7.3.2 Experimental Design 151 7.3.3 Participants 151 7.3.4 Results 151 7.3.5 Summary 153 7.4 Synthesis Based on the Discrete Estimates of a k-Nearest-Neighbor Classifier 153 7.4.1 Definition of the K-Nearest-Neighbor Classifier 154 7.4.2 Analysis Model 155 7.4.3 Synthesis Model 156 7.4.4 Interpolation of acceleration level for the vibration attribute profile pairs 158 7.4.5 Implementation of the Synthesis 159 7.4.6 Advantages and Disadvantages 164 7.5 Synthesis Based on the Quasi-Continuous Estimates of Regression Models 166 7.5.1 Overall Model Structure 168 7.5.2 Classification of the Excitation Pattern with a Support Vector Machine 171 7.5.3 General Approach to the Regression Models of each Excitation Pattern 178 7.5.4 Synthesis for the Impulse-like Excitation Pattern 181 7.5.5 Synthesis for the Bandlimited White Gaussian Noise Excitation Pattern 187 7.5.6 Synthesis for the Amplitude Modulated Sinusoidal Excitation Pattern 193 7.5.7 Synthesis for the Sinusoidal Excitation Pattern 199 7.5.8 Implementation of the Synthesis 205 7.5.9 Advantages and Disadvantages of the Approach 208 7.6 Validation of the Synthesis Models 210 7.6.1 Stimuli 212 7.6.2 Experimental Design 212 7.6.3 Participants 214 7.6.4 Results 214 7.6.5 Summary 219 7.7 Summary and Discussion 219 7.7.1 Summary 219 7.7.2 Discussion 222 8. General Discussion and Outlook 227 Acknowledgment 237 References 237Eines der zentralen Ziele des Designs von Produkten oder virtuellen Um-gebungen ist die Schaffung von Erfahrungen, die im beabsichtigten An-wendungskontext die Erwartungen der Benutzer erfüllen. Gegenwärtig versucht man im vibrotaktilen Authoring-Prozess mit ineffizienten Trial-and-Error-Verfahren, die Erwartungen an den dargestellten, virtuellen Situationskontext implizit zu erfüllen. Ein effizienteres, modellgetriebenes Verfahren, das explizit auf den taktilen Benutzererwartungen basiert, wäre daher von Vorteil. Im Alltag sind wir häufig verschiedenen Ganzkörper-schwingungen ausgesetzt. Abhängig von ihren zeitlichen und spektralen Eigenschaften assoziieren wir intuitiv bestimmte Wahrnehmungsmerkmale wie z.B. “kribbeln”. Dies legt eine systematische Beziehung zwischen physikalischen Parametern und Wahrnehmungsmerkmalen nahe. Um mit potentiellen Nutzern über hervorgerufene oder erwartete taktile Eigen-schaften zu kommunizieren, wird eine standardisierte Designsprache vor-geschlagen. Sie enthält eine Menge von sensorisch-taktilen Wahrneh-mungsmerkmalen, die hinreichend den Wahrnehmungsraum der im Alltag auftretenden Vibrationen charakterisieren. Diese Entwurfssprache ermög-licht die quantitative Beurteilung taktiler Wahrnehmungsmerkmale, die in Situationskontexten wie z.B. auditiv-visuell-taktilen Fahrzeugszenen her-vorgerufen werden. Solche Wahrnehmungsspezifikationen können jedoch auch bewertet werden, indem der Inhalt dieser Szenen verbal beschrieben wird. Quasi identische Bewertungen für beide Präsentationsmodi deuten darauf hin, dass die taktilen Benutzererwartungen quantifiziert werden können, noch bevor eine Vibration präsentiert wird. Die erwarteten Wahr-nehmungsspezifikationen sind die Voraussetzung für eine anschließende Übersetzung in physikalische Schwingungsparameter. Plausible Vibratio-nen können synthetisiert werden, indem die erwarteten Wahrnehmungs-merkmale hervorgerufen werden. Auf der Grundlage der beobachteten Beziehungen zwischen Schwingungs¬parametern und sensorisch-taktilen Wahrnehmungsmerkmalen wurden ein 1-Nearest-Neighbor-Modell und ein Regressionsmodell erstellt. Die Plausibilität der von diesen Modellen synthetisierten Schwingungen im Kontext virtueller, auditorisch-visuell-taktiler Fahrzeugszenen wurde in einer Wahrnehmungsstudie validiert. Die Ergebnisse zeigten, dass die mit der Designsprache gewonnenen Wahr-nehmungsspezifikationen ausreichen, um Schwingungen zu synthetisieren, die in einem gegebenen Situationskontext als ebenso plausibel empfunden werden wie aufgezeichnete Schwingungen. Die demonstrierte Entwurfsme-thode stellt ein neues, effizienteres Werkzeug für Designer dar, die Schwingungen für virtuelle Umgebungen erstellen oder taktiles Feedback für Produkte erzeugen.:Preface III Abstract V Zusammenfassung VII List of Abbreviations XV 1 Introduction 1 1.1 General Introduction 1 1.1 Objectives of the Thesis 4 1.2 Structure of the Thesis 4 2. Tactile Perception in Real and Virtual Environments 7 2.1 Tactile Perception as a Multilayered Process 7 2.1.1 Physical Layer 8 2.1.2 Mechanoreceptor Layer 9 2.1.3 Sensory Layer 19 2.1.4 Affective Layer 26 2.2 Perception of Virtual Environments 29 2.2.1 The Place Illusion 29 2.2.2 The Plausibility Illusion 31 2.3 Approaches for the Authoring of Vibrations 38 2.3.1 Approaches on the Physical Layer 38 2.3.2 Approaches on the Mechanoreceptor Layer 40 2.3.3 Approaches on the Sensory Layer 40 2.3.4 Approaches on the Affective Layer 43 2.4 Summary 43 3. Research Concept 47 3.1 Research Questions 47 3.1.1 Foundations of the Research Concept 47 3.1.2 Research Concept 49 3.2 Limitations 50 4. Development of the Experimental Setup 53 4.1 Hardware 53 4.1.1 Optical Reproduction System 53 4.1.2 Acoustical Reproduction System 54 4.1.3 Whole-Body Vibration Reproduction System 56 4.2 Software 64 4.2.1 Combination of Reproduction Systems for Unimodal and Multimodal Presentation 64 4.2.2 Conducting Perceptual Studies 65 5. Assessment of a Sensory Tactile Design Language for Characterizing Vibration 67 5.1.1 Design Language Requirements 67 5.1.2 Method to Assess the Design Language 69 5.1.3 Goals of this Chapter 70 5.2 Tactile Stimuli 72 5.2.1 Generalization into Excitation Patterns 72 5.2.2 Definition of Parameter Values of the Excitation Patterns 75 5.2.3 Generation of the Stimuli 85 5.2.4 Summary 86 5.3 Assessment of the most relevant Sensory Tactile Perceptual Attributes 86 5.3.1 Experimental Design 87 5.3.2 Participants 88 5.3.3 Results 88 5.3.4 Aggregation and Prioritization 89 5.3.5 Summary 91 5.4 Identification of the Attributes forming the Design Language 92 5.4.1 Experimental Design 93 5.4.2 Participants 95 5.4.3 Results 95 5.4.4 Selecting the Elements of the Sensory Tactile Design Language 106 5.4.5 Summary 109 5.5 Summary and Discussion 109 5.5.1 Summary 109 5.5.2 Discussion 111 6. Quantification of Expected Properties with the Sensory Tactile Design Language 115 6.1 Multimodal Stimuli 116 6.1.1 Selection of the Scenes 116 6.1.2 Recording of the Scenes 117 6.1.3 Recorded Stimuli 119 6.2 Qualitative Communication in the Presence of Vibration 123 6.2.1 Experimental Design 123 6.2.2 Participants 124 6.2.3 Results 124 6.2.4 Summary 126 6.3 Quantitative Communication in the Presence of Vibration 126 6.3.1 Experimental Design 127 6.3.2 Participants 127 6.3.3 Results 127 6.3.4 Summary 129 6.4 Quantitative Communication in the Absence of Vibration 129 6.4.1 Experimental Design 130 6.4.2 Participants 132 6.4.3 Results 132 6.4.4 Summary 134 6.5 Summary and Discussion 135 7. Synthesis Models for the Translation of Sensory Tactile Properties into Vibration 137 7.1 Formalization of the Tactile Plausibility Illusion for Models 139 7.1.1 Formalization of Plausibility 139 7.1.2 Model Boundaries 143 7.2 Investigation of the Influence of Vibration Level on Attribute Ratings 144 7.2.1 Stimuli 145 7.2.2 Experimental Design 145 7.2.3 Participants 146 7.2.4 Results 146 7.2.5 Summary 148 7.3 Comparison of Modulated Vibration to Successive Impulse-like Vibration 148 7.3.1 Stimuli 149 7.3.2 Experimental Design 151 7.3.3 Participants 151 7.3.4 Results 151 7.3.5 Summary 153 7.4 Synthesis Based on the Discrete Estimates of a k-Nearest-Neighbor Classifier 153 7.4.1 Definition of the K-Nearest-Neighbor Classifier 154 7.4.2 Analysis Model 155 7.4.3 Synthesis Model 156 7.4.4 Interpolation of acceleration level for the vibration attribute profile pairs 158 7.4.5 Implementation of the Synthesis 159 7.4.6 Advantages and Disadvantages 164 7.5 Synthesis Based on the Quasi-Continuous Estimates of Regression Models 166 7.5.1 Overall Model Structure 168 7.5.2 Classification of the Excitation Pattern with a Support Vector Machine 171 7.5.3 General Approach to the Regression Models of each Excitation Pattern 178 7.5.4 Synthesis for the Impulse-like Excitation Pattern 181 7.5.5 Synthesis for the Bandlimited White Gaussian Noise Excitation Pattern 187 7.5.6 Synthesis for the Amplitude Modulated Sinusoidal Excitation Pattern 193 7.5.7 Synthesis for the Sinusoidal Excitation Pattern 199 7.5.8 Implementation of the Synthesis 205 7.5.9 Advantages and Disadvantages of the Approach 208 7.6 Validation of the Synthesis Models 210 7.6.1 Stimuli 212 7.6.2 Experimental Design 212 7.6.3 Participants 214 7.6.4 Results 214 7.6.5 Summary 219 7.7 Summary and Discussion 219 7.7.1 Summary 219 7.7.2 Discussion 222 8. General Discussion and Outlook 227 Acknowledgment 237 References 23

Ongoing Tracking of Engagement in Motor Learning

Teaching motor skills such as playing music, handwriting, and driving, can greatly benefit from recently developed technologies such as wearable gloves for haptic feedback or robotic sensorimotor exoskeletons for the mediation of effective human-human and robot-human physical interactions. At the heart of such teacher-learner interactions still stands the critical role of the ongoing feedback a teacher can get about the student's engagement state during the learning and practice sessions. Particularly for motor learning, such feedback is an essential functionality in a system that is developed to guide a teacher on how to control the intensity of the physical interaction, and to best adapt it to the gradually evolving performance of the learner. In this paper, our focus is on the development of a near real-time machine-learning model that can acquire its input from a set of readily available, noninvasive, privacy-preserving, body-worn sensors, for the benefit of tracking the engagement of the learner in the motor task. We used the specific case of violin playing as a target domain in which data were empirically acquired, the latent construct of engagement in motor learning was carefully developed for data labeling, and a machine-learning model was rigorously trained and validated

Investigating the Correlation Between Presence and Reaction Time in Mixed Reality

Measuring presence is critical to improving user involvement and performance in Mixed Reality (MR). \emph{Presence}, a crucial aspect of MR, is traditionally gauged using subjective questionnaires, leading to a lack of time-varying responses and susceptibility to user bias. Inspired by the existing literature on the relationship between presence and human performance, the proposed methodology systematically measures a user's reaction time to a visual stimulus as they interact within a manipulated MR environment. We explore the user reaction time as a quantity that can be easily measured using the systemic tools available in modern MR devices. We conducted an exploratory study (N=40) with two experiments designed to alter the users' sense of presence by manipulating \emph{place illusion} and \emph{plausibility illusion}. We found a significant correlation between presence scores and reaction times with a correlation coefficient -0.65, suggesting that users with a higher sense of presence responded more swiftly to stimuli. We develop a model that estimates a user's presence level using the reaction time values with high accuracy of up to 80\%. While our study suggests that reaction time can be used as a measure of presence, further investigation is needed to improve the accuracy of the model

Somatic ABC's: A Theoretical Framework for Designing, Developing and Evaluating the Building Blocks of Touch-Based Information Delivery

abstract: Situations of sensory overload are steadily becoming more frequent as the ubiquity of technology approaches reality--particularly with the advent of socio-communicative smartphone applications, and pervasive, high speed wireless networks. Although the ease of accessing information has improved our communication effectiveness and efficiency, our visual and auditory modalities--those modalities that today's computerized devices and displays largely engage--have become overloaded, creating possibilities for distractions, delays and high cognitive load; which in turn can lead to a loss of situational awareness, increasing chances for life threatening situations such as texting while driving. Surprisingly, alternative modalities for information delivery have seen little exploration. Touch, in particular, is a promising candidate given that it is our largest sensory organ with impressive spatial and temporal acuity. Although some approaches have been proposed for touch-based information delivery, they are not without limitations including high learning curves, limited applicability and/or limited expression. This is largely due to the lack of a versatile, comprehensive design theory--specifically, a theory that addresses the design of touch-based building blocks for expandable, efficient, rich and robust touch languages that are easy to learn and use. Moreover, beyond design, there is a lack of implementation and evaluation theories for such languages. To overcome these limitations, a unified, theoretical framework, inspired by natural, spoken language, is proposed called Somatic ABC's for Articulating (designing), Building (developing) and Confirming (evaluating) touch-based languages. To evaluate the usefulness of Somatic ABC's, its design, implementation and evaluation theories were applied to create communication languages for two very unique application areas: audio described movies and motor learning. These applications were chosen as they presented opportunities for complementing communication by offloading information, typically conveyed visually and/or aurally, to the skin. For both studies, it was found that Somatic ABC's aided the design, development and evaluation of rich somatic languages with distinct and natural communication units.Dissertation/ThesisPh.D. Computer Science 201