Effects of conversation content on viewing dyadic conversations

People typically follow conversations closely with their gaze. We asked whether this viewing is influenced by what is actually said in the conversation and by the viewer’s psychological condition. We recorded the eye movements of healthy (N = 16) and depressed (N = 25) participants while they were viewing video clips. Each video showed two people, each speaking one line of dialogue about socio-emotionally important (i.e., personal) or unimportant topics (matter-of-fact). Between the spoken lines, the viewers made more saccadic shifts between the discussants, and looked more at the second speaker, in personal vs. matter-of-fact conversations. Higher depression scores were correlated with less looking at the currently speaking discussant. We conclude that subtle social attention dynamics can be detected from eye movements and that these dynamics are sensitive to the observer’s psychological condition, such as depression

Emotion modulates eye movement patterns and subsequent memory for the gist and details of movie scenes

n/

The impact of music and stretched time on pupillary responses and eye movements in slow-motion film scenes

This study investigated the effects of music and playback speed on arousal and visual perception in slow-motion scenes taken from commercial films. Slow-motion scenes are a ubiquitous film technique and highly popular. Yet the psychological effects of mediated time-stretching compared to real-time motion have not been empirically investigated. We hypothesised that music affects arousal and attentional processes. Furthermore, we assumed that playback speed influences viewers’ visual perception, resulting in a higher number of eye movements and larger gaze dispersion. Thirty-nine participants watched three film excerpts in a repeated-measures design in conditions with or without music and in slow motion vs. adapted real-time motion (both visual-only). Results show that music in slow-motion film scenes leads to higher arousal compared to no music as indicated by larger pupil diameters in the former. There was no systematic effect of music on visual perception in terms of eye movements. Playback speed influenced visual perception in eye movement parameters such that slow motion resulted in more and shorter fixations as well as more saccades compared to adapted real-time motion. Furthermore, in slow motion there was a higher gaze dispersion and a smaller centre bias, indicating that individuals attended to more detail in slow motion scenes

Visual Fixation Durations and Saccade Amplitudes: Shifting Relationship in a Variety of Conditions

Is there any relationship between visual fixation durations and saccade amplitudes in free exploration of pictures and scenes? In four experiments with naturalistic stimuli, we compared eye movements during early and late phases of scene perception. Influences of repeated presentation of similar stimuli (Experiment 1), object density (Experiment 2), emotional stimuli (Experiment 3) and mood induction (Experiment 4) were examined. The results demonstrate a systematic increase in the durations of fixations and a decrease for saccadic amplitudes over the time course of scene perception. This relationship was very stable across the variety of studied conditions. It can be interpreted in terms of a shifting balance of the two modes of visual information processing

Fearful faces have a sensory advantage in the competition for awareness

Only a subset of visual signals give rise to a conscious percept. Threat signals, such as fearful faces, are particularly salient to human vision. Research suggests that fearful faces are evaluated without awareness and preferentially promoted to conscious perception. This agrees with evolutionary theories that posit a dedicated pathway specialized in processing threat-relevant signals. We propose an alternative explanation for this "fear advantage." Using psychophysical data from continuous flash suppression (CFS) and masking experiments, we demonstrate that awareness of facial expressions is predicted by effective contrast: the relationship between their Fourier spectrum and the contrast sensitivity function. Fearful faces have higher effective contrast than neutral expressions and this, not threat content, predicts their enhanced access to awareness. Importantly, our findings do not support the existence of a specialized mechanism that promotes threatening stimuli to awareness. Rather, our data suggest that evolutionary or learned adaptations have molded the fearful expression to exploit our general-purpose sensory mechanisms

Effects of Conversation Content on Viewing Dyadic Conversations

People typically follow conversations closely with their gaze. We asked whether this viewing is influenced by what is actually said in the conversation and by the viewer's psychological condition. We recorded the eye movements of healthy (N = 16) and depressed (N = 25) participants while they were viewing video clips. Each video showed two people, each speaking one line of dialogue about socio-emotionally important (i.e., personal) or unimportant topics (matter-of-fact). Between the spoken lines, the viewers made more saccadic shifts between the discussants, and looked more at the second speaker, in personal vs. matter-of-fact conversations. Higher depression scores were correlated with less looking at the currently speaking discussant. We conclude that subtle social attention dynamics can be detected from eye movements and that these dynamics are sensitive to the observer's psychological condition, such as depression

Preferences, emotions, and visual attention in the first-person shooter game experience

First-person shooter (FPS) games are one of the most popular yet notorious genres of digital games. They contain visceral emotional content and require intense visual attention from players, leading some people to appreciate and others to resent these types of games. This thesis investigated individual differences in the game experience of FPS games by exploring how preferences for violent game dynamics (e.g., preferences for shooting, killing, and exploding) affect players’ emotions and perceptions of curiosity, vitality, and self-efficacy. In addition, the thesis explored how visual attention skills affect the viewing of FPS games as indexed by viewers’ eye movements. In Study I, the role of visual attention skills in viewing FPS gameplay videos was explored. The results showed that viewers’ eye movements tended to progress from a diffuse scanning mode towards a more focal and central viewing mode as time passed. Visual qualities and saliency of events also guided eye movements. Individual differences in visual attention skills (namely, the ability to track multiple objects, perform a visual search for targets, and to see rapidly appearing serial targets) were related to what was attended to in the screen. The role of visual attention skills on eye movements was more prominent during visually distinct events. In sum, the results showed that specific visual attention skills predicted eye movement patterns during FPS gameplay video viewing. Study II explored whether game dynamics preferences and emotion-related responses to an FPS game are connected. Participants’ heart rate, electrodermal activity, and electric activity of facial muscles were recorded as indexes of emotion-related responses both during playing (active participation) and gameplay video viewing (passive watching). The participants also rated their level of experienced arousal and valence. The results showed that there were individual differences in physiological emotion-related responses as a function of dynamics preferences, especially in measures of physiological arousal. Those who liked violent dynamics showed a rather stable level of physiological arousal state both when playing and when viewing the game. In contrast, participants who disliked violent dynamics showed an overall higher level of physiological arousal during playing than when viewing, and the level of arousal increased across time in both conditions. The results on facial muscle activity likewise showed that activity differed between people who liked versus disliked violent dynamics. However, the results were somewhat conflicting: those who liked violent game dynamics showed a steep increase in the activity of the corrugator supercilii muscle, an index of negative valence. Instead, those who disliked the dynamics showed less increase in corrugator supercilii activity. The dynamics preferences did not affect self-reported emotional valence or arousal. Thus, the results highlight that game dynamics preferences were associated with physiological signals, although they may not be a straightforward index of emotions in a gaming context. In Study III, associations between game dynamics preferences and self-reported experiences of vitality, self-efficacy, and curiosity were explored both in association with life in general and with playing an FPS video game. The results showed that players who were neutral or mildly positive towards violent content experienced stable levels of vitality, curiosity, and emotional valence both in life in general and when playing. They also experienced a slight decline in self-efficacy in the playing context. Conversely, those who disliked violent dynamics experienced a clear decline in all of these measures in the playing context. Thus, game dynamics preferences were connected with wider experiential reflections related to playing. Overall, the results of all three studies showed why there is individual variation in the playing experience: players and viewers have differing skills and preferences. These skills and preferences affect how players and viewers pay attention to the game, and what kind of emotional reactions and experiences they have. This is valuable for understanding the psychological outcomes of FPS games, as well as why people hold differing opinions about these types of games. Likewise, the results have importance for game design, as they show that players respond in different ways to game contents. Thus, it may be fruitful to personalize and tailor game contents based on players’ preferences and visual attention skills.Mieltymykset, tunteet ja näönvarainen tarkkaavuus ensimmäisen persoonan ammuntapelien pelikokemuksessa Ensimmäisen persoonan ammuntapelit (FPS-pelit) ovat sekä yksi suosituimmista että pahamaineisimmista digitaalisten pelien genreistä. Ne sisältävät voimakasta tunnepitoista sisältöä ja vaativat äärimmäistä näönvaraista tarkkaavuutta. Näiden seikkojen takia toiset arvostavat ja toiset paheksuvat kyseisiä pelejä. Tässä väitöskirjassa tutkittiin yksilöllisiä eroja FPS-pelien pelikokemuksessa selvittämällä, kuinka mieltymykset väkivaltaisiin pelidynamiikkoihin (esimerkiksi mieltymykset ampumiseen, tappamiseen ja räjäyttämiseen) vaikuttavat pelaajien tunnetiloihin ja kokemuksiin uteliaisuudesta, elinvoimaisuudesta ja minäpystyvyydestä. Tämän lisäksi väitöskirjassa tutkittiin silmänliikkeitä tarkastelemalla kuinka näönvaraisen tarkkaavuuden taidot ovat yhteydessä FPS-pelivideoiden katseluun. Tutkimuksessa I tarkasteltiin, miten näönvaraisen tarkkaavuuden taidot vaikuttavat FPS-pelivideoiden katseluun. Tutkimuksessa havaittiin, että silmänliikkeet etenivät laajemmasta ja hajaantuneemmasta silmäilystä kohti pinta-alaltaan pienempää ja lähempänä näytön keskustaa olevaa aluetta. Lisäksi erilaisten pelitapahtumien visuaaliset ominaisuudet ja huomiota herättävyys suuntasivat silmänliikkeitä. Yksilölliset erot näönvaraisen tarkkaavuuden taidoissa (tässä tutkimuksessa taidot seurata useita liikkuvia kohteita, etsiä kohteita ja nähdä nopeasti peräkkäin ilmestyviä kohteita) olivat yhteydessä siihen, miten katselija tarkasteli pelinäkymää. Näönvaraisen tarkkaavuuden taitojen yhteys silmänliikkeisiin tuli esiin etenkin visuaalisesti toisistaan erottuvien pelitapahtumien aikana. Yhteenvetona tutkimuksen tulokset osoittivat, että tietyt näönvaraisen tarkkaavuuden taidot ennustavat silmänliikkeitä FPS-pelivideon katselun aikana. Tutkimuksessa II selvitettiin, ovatko pelidynamiikkamieltymykset ja FPS-peliin liittyvät tunnereaktiot yhteydessä toisiinsa. Tutkimukseen osallistujien sykettä, ihon sähkönjohtavuutta ja kasvolihasten sähköistä aktiivisuutta mitattiin indikaatioina tunnereaktioista sekä pelaamisen (aktiivinen osallistuminen) että pelivideon katselun (passiivinen tarkkailu) aikana. Osallistujat myös arvioivat oman tunnetilansa koettua virittävyyttä ja valenssia. Tulokset osoittivat, että yksilöiden välillä oli eroja fysiologisissa tunteisiin liittyvissä reaktioissa riippuen siitä, millaiset pelidynamiikkamieltymykset heillä oli. Tämä näkyi erityisesti fysiologisissa autonomisen hermoston tilaa kuvaavissa mittareissa. Väkivaltaisista dynamiikoista pitävillä osallistujilla oli suhteellisen tasainen fysiologisen aktivaation taso sekä pelatessa että pelivideota katsellessa. Sen sijaan niillä osallistujilla jotka eivät pitäneet väkivaltaisista dynamiikoista oli kaiken kaikkiaan korkeampi fysiologisen aktivaation taso pelatessa kuin pelivideota katsellessa, ja aktivaation taso kasvoi ajan kuluessa molemmissa tilanteissa. Kasvojen lihasten sähköiseen toimintaan liittyvät tulokset niin ikään osoittivat, että väkivaltaisista dynamiikoista pitävien ja niitä vieroksuvien henkilöiden välillä oli eroja. Tulokset olivat kuitenkin jossain määrin ristiriitaisia: väkivaltaisista dynamiikoista pitävillä osallistujilla negatiivista valenssia indikoiva corrugator supercilii -lihaksen aktiivisuus lisääntyi ajan kuluessa huomattavasti. Sen sijaan osallistujilla jotka eivät pitäneet väkivaltaisista dynamiikoista corrugator supercilii -lihaksen aktiivisuuden lisääntyminen oli lievempää. Pelidynamiikkamieltymykset eivät olleet yhteydessä osallistujien omiin arvioihin tunnekokemuksen virittävyydestä ja valenssista. Täten tulokset osoittivat, että pelidynamiikkamieltymykset olivat yhteydessä fysiologisiin vasteisiin, mutta niitä ei voida käyttää täysin mutkattomina mittareina tunteista pelikontekstissa. Tutkimuksessa III tarkasteltiin pelidynamiikkamieltymysten yhteyttä uteliaisuuden, elinvoimaisuuden ja minäpystyvyyden kokemuksiin elämässä ylipäätään ja FPS-pelin pelaamiseen liittyen. Tutkimuksessa havaittiin, että uteliaisuus, elinvoimaisuus ja tunteen valenssi olivat samankaltaiset sekä elämässä ylipäätään että pelatessa mikäli pelaaja suhtautui väkivaltaisiin dynamiikkoihin neutraalisti tai jonkin verran positiivisesti. Tällaisilla pelaajilla minäpystyvyys oli kuitenkin jonkin verran alhaisempi pelitilanteessa verrattuna elämään ylipäätään. Sen sijaan pelaajat jotka eivät pitäneet väkivaltaisista dynamiikoista arvioivat kaikkien näiden kokemusten olevan selvästi huonompia pelatessa. Pelidynamiikkamieltymykset olivat siis yhteydessä laajempiin reflektiivisiin kokemuksiin pelaamisesta. Väitöstutkimuksen tulokset auttavat ymmärtämään, miksi pelikokemuksessa on yksilöllistä vaihtelua: pelaajat ja katsojat eroavat taidoiltaan ja mieltymyksiltään. Nämä taidot ja mieltymykset ovat yhteydessä siihen, millä tavoin pelaajat ja katselijat kiinnittävät huomiota peliin ja minkälaisia tunnereaktioita ja kokemuksia heillä on. Näiden seikkojen huomioiminen on tärkeää FPS-pelien psykologisten vaikutusten ja peleihin liittyvien eriävien mielipiteiden ymmärtämiseksi. Tuloksilla on lisäksi merkitystä pelisuunnittelulle, sillä ne osoittavat, että pelaajat reagoivat eri tavoin pelisisältöihin. Tämän vuoksi voisi olla hyödyllistä muovata pelisisältöjä yksilöllisesti pelaajien mieltymysten ja näönvaraisen tarkkaavuuden taitojen mukaan

Diagnostic information use to understand brain mechanisms of facial expression categorization

Proficient categorization of facial expressions is crucial for normal social interaction. Neurophysiological, behavioural, event-related potential, lesion and functional neuroimaging techniques can be used to investigate the underlying brain mechanisms supporting this seemingly effortless process, and the associated arrangement of bilateral networks. These brain areas exhibit consistent and replicable activation patterns, and can be broadly defined to include visual (occipital and temporal), limbic (amygdala) and prefrontal (orbitofrontal) regions. Together, these areas support early perceptual processing, the formation of detailed representations and subsequent recognition of expressive faces. Despite the critical role of facial expressions in social communication and extensive work in this area, it is still not known how the brain decodes nonverbal signals in terms of expression-specific features. For these reasons, this thesis investigates the role of these so-called diagnostic facial features at three significant stages in expression recognition; the spatiotemporal inputs to the visual system, the dynamic integration of features in higher visual (occipitotemporal) areas, and early sensitivity to features in V1. In Chapter 1, the basic emotion categories are presented, along with the brain regions that are activated by these expressions. In line with this, the current cognitive theory of face processing reviews functional and anatomical dissociations within the distributed neural “face network”. Chapter 1 also introduces the way in which we measure and use diagnostic information to derive brain sensitivity to specific facial features, and how this is a useful tool by which to understand spatial and temporal organisation of expression recognition in the brain. In relation to this, hierarchical, bottom-up neural processing is discussed along with high-level, top-down facilitatory mechanisms. Chapter 2 describes an eye-movement study that reveals inputs to the visual system via fixations reflect diagnostic information use. Inputs to the visual system dictate the information distributed to cognitive systems during the seamless and rapid categorization of expressive faces. How we perform eye-movements during this task informs how task-driven and stimulus-driven mechanisms interact to guide the extraction of information supporting recognition. We recorded eye movements of observers who categorized the six basic categories of facial expressions. We use a measure of task-relevant information (diagnosticity) to discuss oculomotor behaviour, with focus on two findings. Firstly, fixated regions reveal expression differences. Secondly, by examining fixation sequences, the intersection of fixations with diagnostic information increases in a sequence of fixations. This suggests a top-down drive to acquire task-relevant information, with different functional roles for first and final fixations. A combination of psychophysical studies of visual recognition together with the EEG (electroencephalogram) signal is used to infer the dynamics of feature extraction and use during the recognition of facial expressions in Chapter 3. The results reveal a process that integrates visual information over about 50 milliseconds prior to the face-sensitive N170 event-related potential, starting at the eye region, and proceeding gradually towards lower regions. The finding that informative features for recognition are not processed simultaneously but in an orderly progression over a short time period is instructive for understanding the processes involved in visual recognition, and in particular the integration of bottom-up and top-down processes. In Chapter 4 we use fMRI to investigate the task-dependent activation to diagnostic features in early visual areas, suggesting top-down mechanisms as V1 traditionally exhibits only simple response properties. Chapter 3 revealed that diagnostic features modulate the temporal dynamics of brain signals in higher visual areas. Within the hierarchical visual system however, it is not known if an early (V1/V2/V3) sensitivity to diagnostic information contributes to categorical facial judgements, conceivably driven by top-down signals triggered in visual processing. Using retinotopic mapping, we reveal task-dependent information extraction within the earliest cortical representation (V1) of two features known to be differentially necessary for face recognition tasks (eyes and mouth). This strategic encoding of face images is beyond typical V1 properties and suggests a top-down influence of task extending down to the earliest retinotopic stages of visual processing. The significance of these data is discussed in the context of the cortical face network and bidirectional processing in the visual system. The visual cognition of facial expression processing is concerned with the interactive processing of bottom-up sensory-driven information and top-down mechanisms to relate visual input to categorical judgements. The three experiments presented in this thesis are summarized in Chapter 5 in relation to how diagnostic features can be used to explore such processing in the human brain leading to proficient facial expression categorization