A view on the iconic turn from a semiotic perspective

Media are not only a means of communication. From a cognitive perspective, they may be viewed as components of an external, auxiliary memory system (Schönpflug 1997), and contemporary cognitive science “construes cognition as a complex system in which cognitive processes are ‘embodied, situated’ in environments, and ‘distributed’ across people and artifacts” (Nersessian 2007: 2). In man-machine communication, man-man-communication via digital machinery and especially in the World Wide Web (Heintz 2006, Steels 2006) the “external” components of this system have taken on more and more of the characteristics of our individual, “internal”, living and active memory with its richness of sensual and symbolic formats. The intellectual challenge in the drafts of the “masterminds” of hypertext (Eisenstein) and multimedia (Lintsbakh) was the detection of temporal/spatial, mathematical and linguistic correspondences between such different sensual and symbolic representations (Bulgakova 2007, Tsivian 2007). The so called “iconic” or “pictorial turn” was pulled along by the digital turn, and it may in turn have stimulated and accelerated the digital turn

Speech and gesture in spatial language and cognition among the Yucatec Mayas

In previous analyses of the influence of language on cognition, speech has been the main channel examined. In studies conducted among Yucatec Mayas, efforts to determine the preferred frame of reference in use in this community have failed to reach an agreement (Bohnemeyer & Stolz, 2006; Levinson, 2003 vs. Le Guen, 2006, 2009). This paper argues for a multimodal analysis of language that encompasses gesture as well as speech, and shows that the preferred frame of reference in Yucatec Maya is only detectable through the analysis of co-speech gesture and not through speech alone. A series of experiments compares knowledge of the semantics of spatial terms, performance on nonlinguistic tasks and gestures produced by men and women. The results show a striking gender difference in the knowledge of the semantics of spatial terms, but an equal preference for a geocentric frame of reference in nonverbal tasks. In a localization task, participants used a variety of strategies in their speech, but they all exhibited a systematic preference for a geocentric frame of reference in their gestures

TEST: A Tropic, Embodied, and Situated Theory of Cognition

TEST is a novel taxonomy of knowledge representations based on three distinct hierarchically organized representational features: Tropism, Embodiment, and Situatedness. Tropic representational features reflect constraints of the physical world on the agent’s ability to form, reactivate, and enrich embodied (i.e., resulting from the agent’s bodily constraints) conceptual representations embedded in situated contexts. The proposed hierarchy entails that representations can, in principle, have tropic features without necessarily having situated and/or embodied features. On the other hand, representations that are situated and/or embodied are likely to be simultaneously tropic. Hence while we propose tropism as the most general term, the hierarchical relationship between embodiment and situatedness is more on a par, such that the dominance of one component over the other relies on the distinction between offline storage vs. online generation as well as on representation-specific properties

Language and memory for object location

In three experiments, we investigated the influence of two types of language on memory for object location: demonstratives (this, that) and possessives (my, your). Participants first read instructions containing demonstratives/possessives to place objects at different locations, and then had to recall those object locations (following object removal). Experiments 1 and 2 tested contrasting predictions of two possible accounts of language on object location memory: the Expectation Model (Coventry, Griffiths, & Hamilton, 2014) and the congruence account (Bonfiglioli, Finocchiaro, Gesierich, Rositani, & Vescovi, 2009). In Experiment 3, the role of attention allocation as a possible mechanism was investigated. Results across all three experiments show striking effects of language on object location memory, with the pattern of data supporting the Expectation Model. In this model, the expected location cued by language and the actual location are concatenated leading to (mis)memory for object location, consistent with models of predictive coding (Bar, 2009; Friston, 2003)

Identifying the task characteristics that predict children's construction task performance

Construction tasks form a major part of children’s play and can be linked to achievement in maths and science. However there is a lack of understanding of construction task ability and development. Therefore, there is little foundation for the applied use of construction tasks, such as in teaching or research, as there are no apparent methods for assessing difficulty. This empirical research identifies four construction task characteristics that impact on cognition and predict construction task difficulty in children aged 7-8 and 10-11 years and adults. The results also reveal a developmental trajectory in construction ability. The research provides a method to quantify, predict and control the complexity of construction tasks for future research and to inform applied use

Perceptual Pluralism

Perceptual systems respond to proximal stimuli by forming mental representations of distal stimuli. A central goal for the philosophy of perception is to characterize the representations delivered by perceptual systems. It may be that all perceptual representations are in some way proprietarily perceptual and differ from the representational format of thought (Dretske 1981; Carey 2009; Burge 2010; Block ms.). Or it may instead be that perception and cognition always trade in the same code (Prinz 2002; Pylyshyn 2003). This paper rejects both approaches in favor of perceptual pluralism, the thesis that perception delivers a multiplicity of representational formats, some proprietary and some shared with cognition. The argument for perceptual pluralism marshals a wide array of empirical evidence in favor of iconic (i.e., image-like, analog) representations in perception as well as discursive (i.e., language-like, digital) perceptual object representations

Spatial demonstratives, perceptual space, and linguistic diversity

versus extrapersonal space; object location memor

Spatial representations of numbers and letters in children

Different lines of evidence suggest that children's mental representations of numbers are spatially organized in form of a mental number line. It is, however, still unclear whether a spatial organization is specific for the numerical domain or also applies to other ordinal sequences in children. In the present study, children (n = 129) aged 8–9 years were asked to indicate the midpoint of lines flanked by task-irrelevant digits or letters. We found that the localization of the midpoint was systematically biased toward the larger digit. A similar, but less pronounced, effect was detected for letters with spatial biases toward the letter succeeding in the alphabet. Instead of assuming domain-specific forms of spatial representations, we suggest that ordinal information expressing relations between different items of a sequence might be spatially coded in children, whereby numbers seem to convey this kind of information in the most salient way

The temporary nature of number-space interactions

It is commonly accepted that the mental representation and processing of numbers and of space are tightly linked. This is evident from studies that have shown relations between math ability and visuospatial skill. Also, math instruction and education rely strongly on visuospatial tools and strategies. The dominant explanation for these number—space interactions is that the mental representation of numbers takes the form of a mental number line with numbers positioned in ascending order according to our reading habits. A long-standing debate is whether the link between numbers and space can be considered as evidence for a spatial number representation in long-term semantic memory, or whether this spatial frame is a temporary representation that emerges in working memory (WM) during task execution. We summarise our recent work that suggests basic number processing tasks do not operate on a long-term spatial memory representation, but on a representation constructed in serial order WM, where the elements are spatially coded as a function of their ordinal position in the memorised sequence. Implications for a new theoretical framework linking serial order WM and basic number processing are discussed