SALSA: A Novel Dataset for Multimodal Group Behavior Analysis

Studying free-standing conversational groups (FCGs) in unstructured social settings (e.g., cocktail party ) is gratifying due to the wealth of information available at the group (mining social networks) and individual (recognizing native behavioral and personality traits) levels. However, analyzing social scenes involving FCGs is also highly challenging due to the difficulty in extracting behavioral cues such as target locations, their speaking activity and head/body pose due to crowdedness and presence of extreme occlusions. To this end, we propose SALSA, a novel dataset facilitating multimodal and Synergetic sociAL Scene Analysis, and make two main contributions to research on automated social interaction analysis: (1) SALSA records social interactions among 18 participants in a natural, indoor environment for over 60 minutes, under the poster presentation and cocktail party contexts presenting difficulties in the form of low-resolution images, lighting variations, numerous occlusions, reverberations and interfering sound sources; (2) To alleviate these problems we facilitate multimodal analysis by recording the social interplay using four static surveillance cameras and sociometric badges worn by each participant, comprising the microphone, accelerometer, bluetooth and infrared sensors. In addition to raw data, we also provide annotations concerning individuals' personality as well as their position, head, body orientation and F-formation information over the entire event duration. Through extensive experiments with state-of-the-art approaches, we show (a) the limitations of current methods and (b) how the recorded multiple cues synergetically aid automatic analysis of social interactions. SALSA is available at http://tev.fbk.eu/salsa.Comment: 14 pages, 11 figure

Higher physical fitness levels are associated with less language decline in healthy ageing

Healthy ageing is associated with decline in cognitive abilities such as language. Aerobic fitness has been shown to ameliorate decline in some cognitive domains, but the potential benefits for language have not been examined. In a cross-sectional sample, we investigated the relationship between aerobic fitness and tip-of-the-tongue states. These are among the most frequent cognitive failures in healthy older adults and occur when a speaker knows a word but is unable to produce it. We found that healthy older adults indeed experience more tip-of-the-tongue states than young adults. Importantly, higher aerobic fitness levels decrease the probability of experiencing tip-of-the-tongue states in healthy older adults. Fitness-related differences in word finding abilities are observed over and above effects of age. This is the first demonstration of a link between aerobic fitness and language functioning in healthy older adults

The phonological status of English oral stops after tautosyllabic /s/ : evidence from speakers' classificatory behaviour

The classification of oral stops after tautosyllabic /s/ in English is an old phonological problem to which different solutions have been proposed. In an attempt to provide experimental evidence on the classification of oral bilabial stops after tautosyllabic /s/ by native speakers of English, a concept formation experiment was conducted. The results showed that out of the four main phonological theoretical views on the classification of oral stops after tautosyllabic /s/, the solution which treats those speech segments as allophones of the phonemes /p, t, k/ is the most plausible from the point of view of language users' classificatory behaviour

Neural bases of learning and recognition of statistical regularities

First published: 09 January 2020Statistical learning is a set of cognitive mechanisms allowing for extracting regularities from the environment and segmenting continuous sensory input into discrete units. The current study used functional magnetic resonance imaging (fMRI) (N = 25) in conjunction with an artificial language learning paradigm to provide new insight into the neural mechanisms of statistical learning, considering both the online process of extracting statistical regularities and the subsequent offline recognition of learned patterns. Notably, prior fMRI studies on statistical learning have not contrasted neural activation during the learning and recognition experimental phases. Here, we found that learning is supported by the superior temporal gyrus and the anterior cingulate gyrus, while subsequent recognition relied on the left inferior frontal gyrus. Besides, prior studies only assessed the brain response during the recognition of trained words relative to novel nonwords. Hence, a further key goal of this study was to understand how the brain supports recognition of discrete constituents from the continuous input versus recognition of mere statistical structure that is used to build new constituents that are statistically congruent with the ones from the input. Behaviorally, recognition performance indicated that statistically congruent novel tokens were less likely to be endorsed as parts of the familiar environment than discrete constituents. fMRI data showed that the left intraparietal sulcus and angular gyrus support the recognition of old discrete constituents relative to novel statistically congruent items, likely reflecting an additional contribution from memory representations for trained items.The research was supported by the Spanish Ministry of Economy and Competitiveness (MINECO) through the “Severo Ochoa” Programme for Centres/Units of Excellence in R&D (SEV-2015- 490), and project Grant RTI2018-098317-B-I00 awarded to M.O., by the Basque Government through project Grant PI-2017-25 awarded to D.S., and by the European Commission as Marie Skłodowska-Curie Fellowship DLV-792331 to L.P

Evaluating devices for the measurement of auditory-evoked fetal movement

Determining normal and abnormal fetal function in utero in order to better predict which fetuses are at risk for adverse outcome is critical. However, the medical imaging tools that could assist with diagnosis are very expensive and rarely available in the developing world. In this study, we developed a prototype audio-motio-tachograph (AMTG), which measures fetal movements through the recording of abdominal wall deformations and tested it in Rwanda. First, we showed that AMTG detected fetal signals and that fetuses respond to complex acoustic stimuli. In order to improve the sensitivity of the device, we then measured whole abdominal wall deformations in an automated way using a lab-based 3D optical measurement system, in which fringes are projected and the deflections recorded with a camera. We found that abdominal wall deformations can be measured accurately with a non-invasive measurement apparatus. Overall, we conclude that wearable modalities provide a promising alternative assessment capacity in fetal research, especially in low income countries

Directional adposition use in English, Swedish and Finnish

Directional adpositions such as to the left of describe where a Figure is in relation to a Ground. English and Swedish directional adpositions refer to the location of a Figure in relation to a Ground, whether both are static or in motion. In contrast, the Finnish directional adpositions edellä (in front of) and jäljessä (behind) solely describe the location of a moving Figure in relation to a moving Ground (Nikanne, 2003). When using directional adpositions, a frame of reference must be assumed for interpreting the meaning of directional adpositions. For example, the meaning of to the left of in English can be based on a relative (speaker or listener based) reference frame or an intrinsic (object based) reference frame (Levinson, 1996). When a Figure and a Ground are both in motion, it is possible for a Figure to be described as being behind or in front of the Ground, even if neither have intrinsic features. As shown by Walker (in preparation), there are good reasons to assume that in the latter case a motion based reference frame is involved. This means that if Finnish speakers would use edellä (in front of) and jäljessä (behind) more frequently in situations where both the Figure and Ground are in motion, a difference in reference frame use between Finnish on one hand and English and Swedish on the other could be expected. We asked native English, Swedish and Finnish speakers’ to select adpositions from a language specific list to describe the location of a Figure relative to a Ground when both were shown to be moving on a computer screen. We were interested in any differences between Finnish, English and Swedish speakers. All languages showed a predominant use of directional spatial adpositions referring to the lexical concepts TO THE LEFT OF, TO THE RIGHT OF, ABOVE and BELOW. There were no differences between the languages in directional adpositions use or reference frame use, including reference frame use based on motion. We conclude that despite differences in the grammars of the languages involved, and potential differences in reference frame system use, the three languages investigated encode Figure location in relation to Ground location in a similar way when both are in motion. Levinson, S. C. (1996). Frames of reference and Molyneux’s question: Crosslingiuistic evidence. In P. Bloom, M.A. Peterson, L. Nadel & M.F. Garrett (Eds.) Language and Space (pp.109-170). Massachusetts: MIT Press. Nikanne, U. (2003). How Finnish postpositions see the axis system. In E. van der Zee & J. Slack (Eds.), Representing direction in language and space. Oxford, UK: Oxford University Press. Walker, C. (in preparation). Motion encoding in language, the use of spatial locatives in a motion context. Unpublished doctoral dissertation, University of Lincoln, Lincoln. United Kingdo

Influence of Early Bilingual Exposure in the Developing Human Brain.

190 p.La adquisición del lenguaje es un proceso que ese encuentra determinado tanto por mecanismos de desarrollo cognitivo, como por la experiencia lingüística durante los primeros años de vida. Aunque se trata de un proceso relativamente complejo, los bebés muestran una gran habilidad para el aprendizaje del lenguaje. Un entorno de aprendizaje lingüístico bilingüe podría considerarse aun más complejo, ya que los bebés están expuestos a las características lingüísticas de dos lenguas simultáneamente. En primer lugar, los bebés que crecen en un entorno bilingüe tienen que ser capaces de darse cuenta de que están expuestos a dos lenguas diferentes, y posteriormente deben separar y aprender las características especificas de cada una de ellas; por ejemplo, los distintos fonemas, palabras o estructuras gramaticales. Aunque la exposición lingüística total de los bebés bilingües debería ser comparable a la de los bebés monolingües, es probable que la exposición a cada una de las lenguas de su entorno sea menor, ya que tienen que dividir su tiempo de exposición entre ambas. Si bien los bebés bilingües parecen no tener problemas para enfrentarse a un contexto de aprendizaje potencialmente más complejo, ya que alcanzan las distintas etapas de adquisición del lenguaje a un ritmo similar a los bebés monolingües, sí se han observado adaptaciones a nivel conductual y a nivel de funcionamiento cerebral que podrían producirse como consecuencia de este contexto.Basque Center on cognition, brain and languag