Search Process as Transitions Between Neural States

Search is one of the most performed activities on the World Wide Web. Various conceptual models postulate that the search process can be broken down into distinct emotional and cognitive states of searchers while they engage in a search process. These models significantly contribute to our understanding of the search process. However, they are typically based on self-report measures, such as surveys, questionnaire, etc. and therefore, only indirectly monitor the brain activity that supports such a process. With this work, we take one step further and directly measure the brain activity involved in a search process. To do so, we break down a search process into five time periods: a realisation of Information Need, Query Formulation, Query Submission, Relevance Judgment and Satisfaction Judgment. We then investigate the brain activity between these time periods. Using functional Magnetic Resonance Imaging (fMRI), we monitored the brain activity of twenty-four participants during a search process that involved answering questions carefully selected from the TREC-8 and TREC 2001 Q/A Tracks. This novel analysis that focuses on transitions rather than states reveals the contrasting brain activity between time periods – which enables the identification of the distinct parts of the search process as the user moves through them. This work, therefore, provides an important first step in representing the search process based on the transitions between neural states. Discovering more precisely how brain activity relates to different parts of the search process will enable the development of brain-computer interactions that better support search and search interactions, which we believe our study and conclusions advance

“Some like it hot”:spectators who score high on the personality trait openness enjoy the excitement of hearing dancers breathing without music

Music is an integral part of dance. Over the last 10 years, however, dance stimuli (without music) have been repeatedly used to study action observation processes, increasing our understanding of the influence of observer’s physical abilities on action perception. Moreover, beyond trained skills and empathy traits, very little has been investigated on how other observer or spectators’ properties modulate action observation and action preference. Since strong correlations have been shown between music and personality traits, here we aim to investigate how personality traits shape the appreciation of dance when this is presented with three different music/sounds. Therefore, we investigated the relationship between personality traits and the subjective esthetic experience of 52 spectators watching a 24 min lasting contemporary dance performance projected on a big screen containing three movement phrases performed to three different sound scores: classical music (i.e., Bach), an electronic sound-score, and a section without music but where the breathing of the performers was audible. We found that first, spectators rated the experience of watching dance without music significantly different from with music. Second, we found that the higher spectators scored on the Big Five personality factor openness, the more they liked the no-music section. Third, spectators’ physical experience with dance was not linked to their appreciation but was significantly related to high average extravert scores. For the first time, we showed that spectators’ reported entrainment to watching dance movements without music is strongly related to their personality and thus may need to be considered when using dance as a means to investigate action observation processes and esthetic preferences

Differences in fMRI intersubject correlation while viewing unedited and edited videos of dance performance

Intersubject Correlation (ISC) analysis of fMRI data provides insight into how continuous streams of sensory stimulation are processed by groups of observers. Although edited movies are frequently used as stimuli in ISC studies, there has been little direct examination of the effect of edits on the resulting ISC maps. In this study we showed 16 observers two audiovisual movie versions of the same dance. In one experimental condition there was a continuous view from a single camera (Unedited condition) and in the other condition there were views from different cameras (Edited condition) that provided close up views of the feet or face and upper body. We computed ISC maps for each condition, as well as created a map that showed the difference between the conditions. The results from the Unedited and Edited maps largely overlapped in the occipital and temporal cortices, although more voxels were found for the Edited map. The difference map revealed greater ISC for the Edited condition in the Postcentral Gyrus, Lingual Gyrus, Precentral Gyrus and Medial Frontal Gyrus, while the Unedited condition showed greater ISC in only the Superior Temporal Gyrus. These findings suggest that the visual changes associated with editing provide a source of correlation in maps obtained from edited film, and highlight the utility of using maps to evaluate the difference in ISC between conditions

Improving Motor Imagination with Support of RealTime LORETA Neurofeedback

Recording cortical activity during imagined leg movement is a challenging task due to cortical representation of legs deeper within the central sulcus. Therefore Brain Computer Interface (BCI) studies typically rely on imagined movement of both legs [1]. Activity of deeper cortical structures can be estimated offline from multichannel Electroencephalography (EEG) by using LORETA numerical method [2]. LORETA can also be calculated in real time to provide an instantaneous estimate of brain activity, but currently available solution supports only up to 19 channels (BrainAvatar, BrainMaster, Inc). In this study we propose a custom designed real time LORETA neurofeedback based on multichannel EEG to increase cortical activity at the central sulcus during continuous imagining tapping with one leg only. This strategy could be useful in neurorehabilitation of hemiplegia (i.e. stroke)

Visual Perception of Humanoid Movement

We examined similarity judgements of arm movements generated by different control strategies with the goal of producing natural looking movements on humanoid robots and virtual humans. We examined a variety of movements generated by human motion capture data as well as fourteen differenct synthetic motion generation algorithms that were developed based on human motor production theories and computational considerations. In experiments we displayed motion clips generated by these 15 different methods on both a humanoid robot and a computer graphic character and obtained judgements of similarity between pairs of movements. Experimental results reveal that for movements with obviously different paths as occurred with two production techniques then, as expected, hand paths dominated in the perception of similarity. However, for roughly similar paths as occurred for the other techniques then judgements about fast movements appeared to be based on their velocity profile while judgements to slow movements were based on more detailed representation of the movement

The Visual Categorization of Humanoid Movement as Natural

We examined naturalness judgements of movements generated by different control strategies with the goal of producing natural looking movements on humanoid robots and virtual humans. We displayed motion clips on both a humanoid robot and a computer graphic character. Fourteen synthetic motion generation algorithms were developed based onhuman motor production theories. Human movement was also motion captured. Experimental results relate the actuator bandwidth, the level of computational complexity in motion generation, and the perceived naturalness of motion. In Experiment 1 it was found that for the humanoid robot, low ratings of naturalness were obtained for rapid movement. However Experiment 2 indicated that this effect appeared to not be due to specific features of the motor production techniques but instead due to a movement artifact generated by the humanoid robot

Neuropsychological model of the realization of information need

The main goal of information retrieval (IR) is to satisfy information need (IN). IN refers to a complex concept: at the very initial state of the phenomenon (that is, at a visceral level), even the searcher may not be aware of its existence. Thus, despite advances in the past few decades in both the IR and relevant scientific communities, we do not fully understand how an IN emerges and how it is physically manifested. In this article we aim to inform a holistic view of the realization of IN using functional magnetic resonance imaging. We collected new data of brain activity of 24 participants while they formulated and stated a realization of IN in a Question Answering task, focusing on a distributed set of brain regions associated with activities related to IN, found in our previous study. Results of a functional connectivity analysis led us to propose a neuropsychological model of the realization of IN. Our model consists of three components: (a) a successful memory retrieval component, (b) an information flow regulation component, and (c) a high-level perception component. We believe this study constitutes an important step in unraveling the nature of IN and how to better satisfy IN

Hyperalignment of motor cortical areas based on motor imagery during action observation

Multivariate Pattern Analysis (MVPA) has grown in importance due to its capacity to use both coarse and fine scale patterns of brain activity. However, a major limitation of multivariate analysis is the difficulty of aligning features across brains, which makes MVPA a subject specific analysis. Recent work by Haxby et al. (2011) introduced a method called Hyperalignment that explored neural activity in ventral temporal cortex during object recognition and demonstrated the ability to align individual patterns of brain activity into a common high dimensional space to facilitate Between Subject Classification (BSC). Here we examined BSC based on Hyperalignment of motor cortex during a task of motor imagery of three natural actions (lift, knock and throw). To achieve this we collected brain activity during the combined tasks of action observation and motor imagery to a parametric action space containing 25 stick-figure blends of the three natural actions. From these responses we derived Hyperalignment transformation parameters that were used to map subjects’ representational spaces of the motor imagery task in the motor cortex into a common model representational space. Results showed that BSC of the neural response patterns based on Hyperalignment exceeded both BSC based on anatomical alignment as well as a standard Within Subject Classification (WSC) approach. We also found that results were sensitive to the order in which participants entered the Hyperalignment algorithm. These results demonstrate the effectiveness of Hyperalignment to align neural responses across subject in motor cortex to enable BSC of motor imagery

Event segmentation and biological motion perception in watching dance

We used a combination of behavioral, computational vision and fMRI methods to examine human brain activity while viewing a 386 s video of a solo Bharatanatyam dance. A computational analysis provided us with a Motion Index (MI) quantifying the silhouette motion of the dancer throughout the dance. A behavioral analysis using 30 naïve observers provided us with the time points where observers were most likely to report event boundaries where one movement segment ended and another began. These behavioral and computational data were used to interpret the brain activity of a different set of 11 naïve observers who viewed the dance video while brain activity was measured using fMRI. Results showed that the Motion Index related to brain activity in a single cluster in the right Inferior Temporal Gyrus (ITG) in the vicinity of the Extrastriate Body Area (EBA). Perception of event boundaries in the video was related to the BA44 region of right Inferior Frontal Gyrus as well as extensive clusters of bilateral activity in the Inferior Occipital Gyrus which extended in the right hemisphere towards the posterior Superior Temporal Sulcus (pSTS)