Preschool children's use of perceptual-motor knowledge and hierarchical representational skills for tool making

Although other animals can make simple tools, the expanded and complex material culture of humans is unprecedented in the animal kingdom. Tool making is a slow and late-developing ability in humans, and preschool children find making tools to solve problems very challenging. This difficulty in tool making might be related to the lack of familiarity with the tools and may be overcome by children's long term perceptual-motor knowledge. Thus, in this study, the effect of tool familiarity on tool making was investigated with a task in which 5-to-6-yearold children (n = 75) were asked to remove a small bucket from a vertical tube. The results show that children are better at tool making if the tool and its relation to the task are familiar to them (e.g., soda straw). Moreover, we also replicated the finding that hierarchical complexity and tool making were significantly related. Results are discussed in light of the ideomotor approach

Team DMG at CMCL 2022 Shared Task: Transformer Adapters for the Multi- and Cross-Lingual Prediction of Human Reading Behavior

In this paper, we present the details of our approaches that attained the second place in the shared task of the ACL 2022 Cognitive Modeling and Computational Linguistics Workshop. The shared task is focused on multi- and cross-lingual prediction of eye movement features in human reading behavior, which could provide valuable information regarding language processing. To this end, we train `adapters' inserted into the layers of frozen transformer-based pretrained language models. We find that multilingual models equipped with adapters perform well in predicting eye-tracking features. Our results suggest that utilizing language- and task-specific adapters is beneficial and translating test sets into similar languages that exist in the training set could help with zero-shot transferability in the prediction of human reading behavior

Less Descriptive yet Discriminative: Quantifying the Properties of Multimodal Referring Utterances via CLIP

In this work, we use a transformer-based pre-trained multimodal model, CLIP, to shed light on the mechanisms employed by human speakers when referring to visual entities. In particular, we use CLIP to quantify the degree of descriptiveness (how well an utterance describes an image in isolation) and discriminativeness (to what extent an utterance is effective in picking out a single image among similar images) of human referring utterances within multimodal dialogues. Overall, our results show that utterances become less descriptive over time while their discriminativeness remains unchanged. Through analysis, we propose that this trend could be due to participants relying on the previous mentions in the dialogue history, as well as being able to distill the most discriminative information from the visual context. In general, our study opens up the possibility of using this and similar models to quantify patterns in human data and shed light on the underlying cognitive mechanisms

The PhotoBook Dataset: Building Common Ground through Visually Grounded Dialogue

This paper introduces the PhotoBook dataset, a large-scale collection of visually-grounded, task-oriented dialogues in English designed to investigate shared dialogue history accumulating during conversation. Taking inspiration from seminal work on dialogue analysis, we propose a data-collection task formulated as a collaborative game prompting two online participants to refer to images utilising both their visual context as well as previously established referring expressions. We provide a detailed description of the task setup and a thorough analysis of the 2,500 dialogues collected. To further illustrate the novel features of the dataset, we propose a baseline model for reference resolution which uses a simple method to take into account shared information accumulated in a reference chain. Our results show that this information is particularly important to resolve later descriptions and underline the need to develop more sophisticated models of common ground in dialogue interaction