Visual world studies of conversational perspective taking: similar findings, diverging interpretations

Visual-world eyetracking greatly expanded the potential for insight into how listeners access and use common ground during situated language comprehension. Past reviews of visual world studies on perspective taking have largely taken the diverging findings of the various studies at face value, and attributed these apparently different findings to differences in the extent to which the paradigms used by different labs afford collaborative interaction. Researchers are asking questions about perspective taking of an increasingly nuanced and sophisticated nature, a clear indicator of progress. But this research has the potential not only to improve our understanding of conversational perspective taking. Grappling with problems of data interpretation in such a complex domain has the unique potential to drive visual world researchers to a deeper understanding of how to best map visual world data onto psycholinguistic theory. I will argue against this interactional affordances explanation, on two counts. First, it implies that interactivity affects the overall ability to form common ground, and thus provides no straightforward explanation of why, within a single noninteractive study, common ground can have very large effects on some aspects of processing (referential anticipation) while having negligible effects on others (lexical processing). Second, and more importantly, the explanation accepts the divergence in published findings at face value. However, a closer look at several key studies shows that the divergences are more likely to reflect inconsistent practices of analysis and interpretation that have been applied to an underlying body of data that is, in fact, surprisingly consistent. The diverging interpretations, I will argue, are the result of differences in the handling of anticipatory baseline effects (ABEs) in the analysis of visual world data. ABEs arise in perspective-taking studies because listeners have earlier access to constraining information about who knows what than they have to referential speech, and thus can already show biases in visual attention even before the processing of any referential speech has begun. To be sure, these ABEs clearly indicate early access to common ground; however, access does not imply integration, since it is possible that this information is not used later to modulate the processing of incoming speech. Failing to account for these biases using statistical or experimental controls leads to over-optimistic assessments of listeners’ ability to integrate this information with incoming speech. I will show that several key studies with varying degrees of interactional affordances all show similar temporal profiles of common ground use during the interpretive process: early anticipatory effects, followed by bottom-up effects of lexical processing that are not modulated by common ground, followed (optionally) by further late effects that are likely to be post-lexical. Furthermore, this temporal profile for common ground radically differs from the profile of contextual effects related to verb semantics. Together, these findings are consistent with the proposal that lexical processes are encapsulated from common ground, but cannot be straightforwardly accounted for by probabilistic constraint-based approaches

Maps as a visual language: A Chinese perspective

One primary goal of cartographic research is to improve cartographic communication. Psychophysical and cognitive research has assisted our understanding of the map use process. The present study is from a perspective of maps as a visual language. This study hypothesizes that (1) the map symbol system constitutes a visual ideographic language and (2) cartographic communication may be improved by applying the methods of teaching visual ideographic languages as a second language. Chinese script originated in primitive drawings of concrete things--pictographs--and ideographs. These became stylized and combined, and were expanded greatly in number. Although the characters came to include phonetic symbols, the script can be used as a completely visual language and is not structured as a parallel to the phonetic language as are alphabetic languages. Furthermore, written Chinese is processed mentally much more holistically and requires more reader-origin organization than alphabetic languages. maps have all the fundamental attributes of Chinese writing. maps with their many non-phonetic symbols are essentially visual. Both cartographic symbols and early Chinese characters are often mimetic. To understand maps, symbols must be put into relation with other symbols that are not arranged linearly. Similarly, to understand Chinese, each. character must be put into relation with other characters that can be sequenced vertically or horizontally and left to right or right to left. Studies of teaching Chinese as a Second Language stress that a variety of approaches are necessary in teaching such a complex, high-level cognitive process. The basics of lexicon and syntax need rote learning, substitution exercises and much experience. All these components and approaches could be applied to a map use teaching programme. (Abstract shortened by UMI.

Anatomy and physiology of word-selective visual cortex: from visual features to lexical processing

Over the past 2 decades, researchers have tried to uncover how the human brain can extract linguistic information from a sequence of visual symbols. The description of how the brain's visual system processes words and enables reading has improved with the progressive refinement of experimental methodologies and neuroimaging techniques. This review provides a brief overview of this research journey. We start by describing classical models of object recognition in non-human primates, which represent the foundation for many of the early models of visual word recognition in humans. We then review functional neuroimaging studies investigating the word-selective regions in visual cortex. This research led to the differentiation of highly specialized areas, which are involved in the analysis of different aspects of written language. We then consider the corresponding anatomical measurements and provide a description of the main white matter pathways carrying neural signals crucial to word recognition. Finally, in an attempt to integrate structural, functional, and electrophysiological findings, we propose a view of visual word recognition, accounting for spatial and temporal facets of word-selective neural processes. This multi-modal perspective on the neural circuitry of literacy highlights the relevance of a posterior-anterior differentiation in ventral occipitotemporal cortex for visual processing of written language and lexical features. It also highlights unanswered questions that can guide us towards future research directions. Bridging measures of brain structure and function will help us reach a more precise understanding of the transformation from vision to language

Using Metacognitive Reflection to Improve Student Learning

The study of art, especially perspective, involves the use of specialized vocabulary words. Vocabulary words can be difficult to comprehend, but when students learn to use the specialized vocabulary or academic language of a subject, the learner is better able to think about the content. While academic language is only a part of a visual art curriculum, students need support from the teacher to learn it. Metacognitive reflection (MR) offers a method to increase student learning of academic language in art specifically, and other subjects in general. Teacher feedback naturally occurs in response to students’ reflections and gives the learner direction and motivation to continue learning. This quasi-experimental study used a repeated measures design with a sample of intact middle school visual art classes to determine the influence of MR and teacher feedback on students’ ability to learn and retain academic language related to perspective drawing as measured by a multiple-choice test. This study was conducted three separate times, to improve validity. While the MR treatment groups attained and maintained greater mean gains overall, post-hoc tests revealed that differences between groups in two of three studies were not statistically significant. The groups who engaged in reflection with feedback added a weighted mean gain of d = .37 to their posttest score beyond that of the comparison groups. This finding provides moderate evidence for the efficacy of practicing reflection with feedback in favor of conventional teaching alone

Spatial and Visual Perspective-Taking via View Rotation and Relation Reasoning for Embodied Reference Understanding

Embodied Reference Understanding studies the reference understanding in an embodied fashion, where a receiver is required to locate a target object referred to by both language and gesture of the sender in a shared physical environment. Its main challenge lies in how to make the receiver with the egocentric view access spatial and visual information relative to the sender to judge how objects are oriented around and seen from the sender, i.e., spatial and visual perspective-taking. In this paper, we propose a REasoning from your Perspective (REP) method to tackle the challenge by modeling relations between the receiver and the sender and the sender and the objects via the proposed novel view rotation and relation reasoning. Specifically, view rotation first rotates the receiver to the position of the sender by constructing an embodied 3D coordinate system with the position of the sender as the origin. Then, it changes the orientation of the receiver to the orientation of the sender by encoding the body orientation and gesture of the sender. Relation reasoning models the nonverbal and verbal relations between the sender and the objects by multi-modal cooperative reasoning in gesture, language, visual content, and spatial position. Experiment results demonstrate the effectiveness of REP, which consistently surpasses all existing state-of-the-art algorithms by a large margin, i.e., +5.22% absolute accuracy in terms of Prec0.5 on YouRefIt.Comment: ECCV 2022. Code: http://github.com/ChengShiest/REP-ER

The Effects of Hypertext Glosses on L2 Vocabulary Acquisition: A Meta-Analysis

In the field of second language acquisition (SLA), "comprehensible input" (Krashen, 1985) has been considered a critical factor to help learners acquire foreign and second languages (L2). From this perspective, the notion of extensive or free voluntary reading (Day & Bamford, 1998; Krashen, 1993) has emerged that L2 learners should be given more pleasure reading by minimizing a burden look-up behavior. At the same time, technology innovation has made it possible for extensive reading to occur through technology over the past decades. In particular with hypertext glosses or multimedia annotations, a number of studies have indicated that hypertext glossed input is comprehensible input and has made it possible for L2 readers to benefit all from extensive reading. This study examines (1) effects of hypertext gloss use on L2 vocabulary acquisition in computerized reading contexts, and (2) which specific combination of either text-only (single) or text + visual (multiple) hypertext glosses is more effective on L2 vocabulary acquisition and 3) What potential moderators to systematically account for between study variation are. In addition, it aims to synthesize characteristics of studies, technology use and research methods from empirical research studies for a comprehensible and insightful review of the effect of hypertext glosses on L2 vocabulary acquisition. Meta-analysis as a quantitative method was conducted to synthesize overall findings of empirical studies by calculating a standardized mean difference effect size. From 300 papers considered, 10 met the Criteria for Inclusion through a final filtering process, and were finally meta-analyzed to extract effect sizes in the present study. On the basis of 35 weighted mean effect size, 0.46 (Cohen, 1988: medium), the magnitude of text + visual (multiple) hypertext gloss combination was moderately effective on L2 vocabulary acquisition when L2 learners were given two conditions: a text-only or a text + visual hypertext glosses. The results revealed that various L2 learners, including English as a second or foreign language (ESL/EFL), Spanish as a foreign language (SFL), Japanese as a foreign language (JFL), and German as a foreign language (GFL), benefit from multiple hypertext glosses while reading computerized texts. In terms of research design, hypertext gloss studies have been almost always conducted in settings of class session-based quasi-experiment design with a researcher-developed program at a university or college level. More implications are discussed for future research

The morphology of public open spaces as visual opportunity fields: a space syntax approach based on revelation and VGA maps customization with SalaScript

This paper explores the visual dynamics of the morphology of public open spaces. Resorting to space syntax concepts, visual graph analysis (VGA), and the innovative use of its standard tool Depthmap, a set of case studies is analysed under the perspective of visual opportunity fields and building upon the established concept and metrics of Revelation. A series of related novel measures and visualizations are developed, which are only possible, in Depthmap environment, by the advanced use of its scripting language: SalaScript. Despite Depthmap being the standard tool among the space syntax community, there is a lack of references to the explicit use of SalaScript. One major objective of this paper is to illustrate, and document, its possibilities to a broad audience by extending, customizing and introducing a more interactive approach in handling VGA maps. We present SalaScript functionalities and its use in the development of visual analysis scripts that allow the study of revelation, which we then apply to a set of real public open spaces case studies.info:eu-repo/semantics/publishedVersio

Museums as disseminators of niche knowledge: Universality in accessibility for all

Accessibility has faced several challenges within audiovisual translation Studies and gained great opportunities for its establishment as a methodologically and theoretically well-founded discipline. Initially conceived as a set of services and practices that provides access to audiovisual media content for persons with sensory impairment, today accessibility can be viewed as a concept involving more and more universality thanks to its contribution to the dissemination of audiovisual products on the topic of marginalisation. Against this theoretical backdrop, accessibility is scrutinised from the perspective of aesthetics of migration and minorities within the field of the visual arts in museum settings. These aesthetic narrative forms act as modalities that encourage the diffusion of ‘niche’ knowledge, where processes of translation and interpretation provide access to all knowledge as counter discourse. Within this framework, the ways in which language is used can be considered the beginning of a type of local grammar in English as lingua franca for interlingual translation and subtitling, both of which ensure access to knowledge for all citizens as a human rights principle and regardless of cultural and social differences. Accessibility is thus gaining momentum as an agent for the democratisation and transparency of information against media discourse distortions and oversimplifications

Towards Real-World Writing Assistance: A Chinese Character Checking Benchmark with Faked and Misspelled Characters

Writing assistance is an application closely related to human life and is also a fundamental Natural Language Processing (NLP) research field. Its aim is to improve the correctness and quality of input texts, with character checking being crucial in detecting and correcting wrong characters. From the perspective of the real world where handwriting occupies the vast majority, characters that humans get wrong include faked characters (i.e., untrue characters created due to writing errors) and misspelled characters (i.e., true characters used incorrectly due to spelling errors). However, existing datasets and related studies only focus on misspelled characters mainly caused by phonological or visual confusion, thereby ignoring faked characters which are more common and difficult. To break through this dilemma, we present Visual-C$^3$, a human-annotated Visual Chinese Character Checking dataset with faked and misspelled Chinese characters. To the best of our knowledge, Visual-C$^3$ is the first real-world visual and the largest human-crafted dataset for the Chinese character checking scenario. Additionally, we also propose and evaluate novel baseline methods on Visual-C$^3$. Extensive empirical results and analyses show that Visual-C$^3$ is high-quality yet challenging. The Visual-C$^3$ dataset and the baseline methods will be publicly available to facilitate further research in the community.Comment: Work in progres