24 research outputs found
Eye-tracking evidence for multimodal language-graphics comprehension: The role of integrated conceptual representations
Proceedings of the NODALIDA 2009 workshop
Multimodal Communication — from Human Behaviour to Computational Models.
Editors: Costanza Navarretta, Patrizia Paggio, Jens Allwood,
Elisabeth Alsén and Yasuhiro Katagiri.
NEALT Proceedings Series, Vol. 6 (2009), 9-14.
© 2009 The editors and contributors.
Published by
Northern European Association for Language
Technology (NEALT)
http://omilia.uio.no/nealt .
Electronically published at
Tartu University Library (Estonia)
http://hdl.handle.net/10062/9208
MAGiC: A multimodal framework for analysing gaze in dyadic communication
The analysis of dynamic scenes has been a challenging domain in eye tracking research. This study presents a framework, named MAGiC, for analyzing gaze contact and gaze aversion in face-to-face communication. MAGiC provides an environment that is able to detect and track the conversation partner’s face automatically, overlay gaze data on top of the face video, and incorporate speech by means of speech-act annotation. Specifically, MAGiC integrates eye tracking data for gaze, audio data for speech segmentation, and video data for face tracking. MAGiC is an open source framework and its usage is demonstrated via publicly available video content and wiki pages. We explored the capabilities of MAGiC through a pilot study and showed that it facilitates the analysis of dynamic gaze data by reducing the annotation effort and the time spent for manual analysis of video data
The Role of Visual Features in Text-Based CAPTCHAs: An fNIRS Study for Usable Security
To mitigate dictionary attacks or similar undesirable automated attacks to
information systems, developers mostly prefer using CAPTCHA challenges as Human
Interactive Proofs (HIPs) to distinguish between human users and scripts.
Appropriate use of CAPTCHA requires a setup that balances between robustness
and usability during the design of a challenge. The previous research reveals
that most usability studies have used accuracy and response time as measurement
criteria for quantitative analysis. The present study aims at applying optical
neuroimaging techniques for the analysis of CAPTCHA design. The functional
Near-Infrared Spectroscopy technique was used to explore the hemodynamic
responses in the prefrontal cortex elicited by CAPTCHA stimulus of varying
types. )e findings suggest that regions in the left and right dorsolateral and
right dorsomedial prefrontal cortex respond to the degrees of line occlusion,
rotation, and wave distortions present in a CAPTCHA. The systematic addition of
the visual effects introduced nonlinear effects on the behavioral and
prefrontal oxygenation measures, indicative of the emergence of Gestalt effects
that might have influenced the perception of the overall CAPTCHA figure.Comment: 24 pages, 32 figures, Computational Intelligence and Neuroscienc
Multimodal Comprehension of Language and Graphics: Graphs with and without annotations
An experimental investigation into interaction between language and information graphics in multimodal documents served as the basis for this study. More specifically, our purpose was to investigate the role of linguistic annotations in graph-text documents. Participants were presented with three newspaper articles in the following conditions: one text-only, one text plus non-annotated graph, and one text plus annotated graph. Results of the experiment showed that, on one hand, annotations play a bridging role for integration of information contributed by different representational modalities. On the other hand, linguistic annotations have negative effects on recall, possibly due to attention divided by the different parts of a document
Gaze aversion in conversational settings: An investigation based on mock job interview
We report the results of an empirical study on gaze aversion during dyadic human-to-human conversation in an interview setting. To address various methodological challenges in as- sessing gaze-to-face contact, we followed an approach where the experiment was conducted twice, each time with a different set of interviewees. In one of them the interviewer’s gaze was tracked with an eye tracker, and in the other the interviewee’s gaze was tracked. The gaze sequences obtained in both experiments were analyzed and modeled as Discrete-Time Markov Chains. The results show that the interviewer made more frequent and longer gaze contacts compared to the interviewee. Also, the interviewer made mostly diagonal gaze aversions, whereas the interviewee made sideways aversions (left or right). We discuss the relevance of this research for Human-Robot Interaction, and discuss some future research problems
Proceedings of the Graduate Student Symposium of the 7th International Conference on the Theory and Application of Diagrams, July 5 2012
Proceedings of the Graduate Student Symposium held at the 7th International Conference on the Theory and Application of Diagrams, ( Diagrams 2012 ), held at the University of Kent on July 5, 2012. Dr. Nathaniel Miller, professor of in the School of Mathematical Sciences at UNC, served on the symposium organizing committee
IDEST: International Database of Emotional Short Texts
We introduce a database (IDEST) of 250 short stories rated for valence, arousal, and comprehensibility in two languages. The texts, with a narrative structure telling a story in the first person and controlled for length, were originally written in six different languages (Finnish, French, German, Portuguese, Spanish, and Turkish), and rated for arousal, valence, and comprehensibility in the original language. The stories were translated into English, and the same ratings for the English translations were collected via an internet survey tool (N = 573). In addition to the rating data, we also report readability indexes for the original and English texts. The texts have been categorized into different story types based on their emotional arc. The texts score high on comprehensibility and represent a wide range of emotional valence and arousal levels. The comparative analysis of the ratings of the original texts and English translations showed that valence ratings were very similar across languages, whereas correlations between the two pairs of language versions for arousal and comprehensibility were modest. Comprehensibility ratings correlated with only some of the readability indexes. The database is published in osf.io/9tga3, and it is freely available for academic research.This project was conducted as a part of E-READ COST action (IS1404). The first author received project funding from the Academy of Finland (decision number 334266). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.info:eu-repo/semantics/publishedVersio
Recommended from our members
Refixation Strategies in Sentential Word Reading: An Exploration by Linked Linear Mixed Models
The current study undertakes refixation patterns on words in sentential reading. Utilizing a Linked Linear Mixed Model approach, the analysis focused on words with a single fixation and the first fixation from words with a double fixation. The model findings revealed a relationship between refixation probability and fixation locations, with initial fixations tending to occur closer to the beginning of a word in instances of higher refixation likelihood. Incorporating predicted and residual values of the fixation location models into the fixation duration models resulted in congruence in the observed fixation locations, durations, and residual values. Finally, the models revealed differences between progressive and regressive second fixations
Recommended from our members