Towards responsive Sensitive Artificial Listeners

This paper describes work in the recently started project SEMAINE, which aims to build a set of Sensitive Artificial Listeners – conversational agents designed to sustain an interaction with a human user despite limited verbal skills, through robust recognition and generation of non-verbal behaviour in real-time, both when the agent is speaking and listening. We report on data collection and on the design of a system architecture in view of real-time responsiveness

Voice processing abilities in children with autism, children with specific language impairments and young typically developing children

It is well established that people with autism have impaired face processing, but much less is known about voice processing in autism. Four experiments were therefore carried out to assess (1) familiar voice-face and sound-object matching; (2) familiar voice recognition; (3) unfamiliar voice discrimination; and (4) vocal affect naming and vocal-facial affect matching. In Experiments 1 and 2 language-matched children with specific language impairment (SLI) were the controls. In Experiments 3 and 4 language-matched children with SLI and young mainstream children were the controls. The results were unexpected: the children with autism were not impaired relative to controls on Experiments 1, 2 and 3, and were superior to the children with SLI on both parts of Experiment 4, although impaired on affect matching relative to the mainstream children. These results are interpreted in terms of an unexpected impairment of voice processing in the children with SLI associated partly, but not wholly, with an impairment of cross-modal processing. Performance on the experimental tasks was not associated with verbal or nonverbal ability in either of the clinical groups. The implications of these findings for understanding autism and SLI are discussed

Communication difficulties following right hemisphere stroke : applying evidence to clinical management

Following reports in the 1960s that language may be affected by right hemisphere (RH) lesions, many limitations to effective communication in the right hemisphere damaged (RHD) population have been described and evidenced. However, stereotypical portrayals and descriptions of carefully selected cases may be misleading as to the extent of communication deficits. In many of the parameters in which RHD patients are presented as typically impaired, e.g. discourse skills, a less severe picture may emerge where data from the non-brain damaged (NBD) population are considered, with age and education variables controlled. Subsequent to RHD, some people show deficit on some communication measures, but many of these communication behaviours are also present in some NBD adults. Thus diagnosis of deficit must be made with reference both to the healthy peer population and the individual's pre-lesion behaviour. The authors' right RH stroke research programme includes studies of incidence of communication deficit, comparisons of RHD and NBD groups in various spoken discourse and comprehension tasks, comparison of RHD groups of different ages, detailed analysis of topic within discourse in RHD and NBD groups, family members' views of communication behaviour following RHD, and the natural course of communication change during the first year after RH stroke. The findings from several studies are summarised and used as the basis for management recommendations, which may guide future outcome research. There is an urgent need for the evaluation of communication management programmes, to determine whether therapists may with confidence offer an effective intervention service to those people whose communication skills are affected by RHD

Resisting the Far-Right: Indigenous Perspectives, Community Arts and Story-Based Strategy

This article explores how we might resist and confront anti-immigration and anti-refugee politics by addressing the social and historical well-spring from which these discriminatory and damaging politics emerge and take sustenance. In doing this, I draw upon the concept of story-based strategy and the idea that our potential to address this issue relies on our capacity to fundamentally shift the dominant ways in which people understand and engage with it. This discussion occurs with reference to one practical application of story-based strategy – a community-arts project titled Stories of Hope and Migration – which attempted to re-frame the migration and refugee debate in Australia by funnelling it through a localised Indigenous perspective. In so doing, this article challenges the way in which early British migrants and their descendants have continually excised themselves from the rhetoric of migration, and furthermore, suggests that through a more nuanced conversation regarding the migration stories of all non-Aboriginal people, we might better promote a more historically aware, compassionate and inclusive society

A transparent framework towards the context-sensitive recognition of conversational engagement

Modelling and recognising affective and mental user states is an urging topic in multiple research fields. This work suggests an approach towards adequate recognition of such states by combining state-of-the-art behaviour recognition classifiers in a transparent and explainable modelling framework that also allows to consider contextual aspects in the inference process. More precisely, in this paper we exemplify the idea of our framework with the recognition of conversational engagement in bi-directional conversations. We introduce a multi-modal annotation scheme for conversational engagement. We further introduce our hybrid approach that combines the accuracy of state-of-the art machine learning techniques, such as deep learning, with the capabilities of Bayesian Networks that are inherently interpretable and feature an important aspect that modern approaches are lacking - causal inference. In an evaluation on a large multi-modal corpus of bi-directional conversations, we show that this hybrid approach can even outperform state-of-the-art black-box approaches by considering context information and causal relations

Diagnostic colours of emotions

This thesis investigates the role of colour in the cognitive processesing of emotional information. The research is guided by the effect of colour diagnosticity which has been shown previously to influence recognition performance of several types of objects as well as natural scenes. The research presented in Experiment 1 examined whether colour information is considered a diagnostic perceptual feature of seven emotional categories: happiness, sadness, anger, fear, disgust, surprise and neutral. Participants (N = 119), who were naïve to the specific purpose and expectations of the experiment, chose colour more than any other perceptual quality (e.g. shape and tactile information) as a feature that describes the seven emotional categories. The specific colour features given for the six basic emotions were consistently different from those given to the non-emotional neutral category. While emotional categories were often described by chromatic colour features (e.g. red, blue, orange) the neutral category was often ascribed achromatic colour features (e.g. white, grey, transparent) as the most symptomatic perceptual qualities for its description. The emotion 'anger' was unique in being the only emotion showing an agreement higher that 50% of the total given colour features for one particular colour - red. Confirming that colour is a diagnostic feature of emotions led to the examination of the effect of diagnostic colours of emotion on recognition memory for emotional words and faces: the effect, if any, of appropriate and inappropriate colours (matched with emotion) on the strength of memory for later recognition of faces and words (Experiments 2 & 3). The two experiments used retention intervals of 15 minutes and one week respectively and the colour-emotion associations were determined for each individual participant. Results showed that regardless of the subject’s consistency level in associating colours with emotions, and compared with the individual inappropriate or random colours, individual appropriate colours of emotions significantly enhance recognition memory for six basic emotional faces and words. This difference between the individual inappropriate colours or random colours and the individual appropriate colours of emotions was not found to be significant for non-emotional neutral stimuli. Post hoc findings from both experiments further show that appropriate colours of emotion are associated more consistently than inappropriate colours of emotions. This suggests that appropriate colour-emotion associations are unique both in their strength of association and in the form of their representation. Experiment 4 therefore aimed to investigate whether appropriate colour-emotion associations also trigger an implicit automatic cognitive system that allows faster naming times for appropriate versus inappropriate colours of emotional word carriers. Results from the combined Emotional-Semantic Stroop task confirm the above hypothesis and therefore imply that colour plays a substantial role not only in our conceptual representations of objects but also in our conceptual representations of basic emotions. The resemblance of the present findings collectively to those found previously for objects and natural scenes suggests a common cognitive mechanism for the processing of emotional diagnostic colours and the processing of diagnostic colours of objects or natural scenes. Overall, this thesis provides the foundation for many future directions of research in the area of colour and emotion as well as a few possible immediate practical implications

Multimodal Affect Recognition: Current Approaches and Challenges

Many factors render multimodal affect recognition approaches appealing. First, humans employ a multimodal approach in emotion recognition. It is only fitting that machines, which attempt to reproduce elements of the human emotional intelligence, employ the same approach. Second, the combination of multiple-affective signals not only provides a richer collection of data but also helps alleviate the effects of uncertainty in the raw signals. Lastly, they potentially afford us the flexibility to classify emotions even when one or more source signals are not possible to retrieve. However, the multimodal approach presents challenges pertaining to the fusion of individual signals, dimensionality of the feature space, and incompatibility of collected signals in terms of time resolution and format. In this chapter, we explore the aforementioned challenges while presenting the latest scholarship on the topic. Hence, we first discuss the various modalities used in affect classification. Second, we explore the fusion of modalities. Third, we present publicly accessible multimodal datasets designed to expedite work on the topic by eliminating the laborious task of dataset collection. Fourth, we analyze representative works on the topic. Finally, we summarize the current challenges in the field and provide ideas for future research directions

Music Therapy by Proxy: Using Humanised Images in Song

Developing awareness, exploration and expression of emotionally sensitive issues can be difficult for some clients in music therapy. They may find it hard to express emotion through improvised music and may turn instead to the perceived security of the repetition of known songs.This paper presents the results from a completed research PhD, a qualitative case study based on naturalistic clinical practice, which examined the song choices of one woman in a medium-secure forensic unit over the three-year course of her music therapy. A descriptive narrative account was subjected to analysis according to a modified form of therapeutic narrative analysis (Aldridge and Aldridge 2002), resulting in the abstraction of a series of generative metaphoric images, framed within a chronological series of events. Crucially, these images were found to be humanised figures, yet they were also emotionally decentred or depersonalised. When approached from the philosophical and methodological perspective of behaviourism, which views these as conditioned responses associating music with life experiences as part of a process of developing self-identity, such images can be seen to provide an unspoken voice for the client’s feelings to be expressed in a manner that is personally revealing, socially acceptable, culturally accessible and therapeutically constructive.I assert that using these third-person characters as a form of proxy facilitates mutual reference and experimentation, and places music firmly at the heart of a socially constructed process of music therapy

System upgrade: realising the vision for UK education

A report summarising the findings of the TEL programme in the wider context of technology-enhanced learning and offering recommendations for future strategy in the area was launched on 13th June at the House of Lords to a group of policymakers, technologists and practitioners chaired by Lord Knight. The report – a major outcome of the programme – is written by TEL director Professor Richard Noss and a team of experts in various fields of technology-enhanced learning. The report features the programme’s 12 recommendations for using technology-enhanced learning to upgrade UK education