Crossmodal content binding in information-processing architectures

Operating in a physical context, an intelligent robot faces two fundamental problems. First, it needs to combine information from its different sensors to form a representation of the environment that is more complete than any of its sensors on its own could provide. Second, it needs to combine high-level representations (such as those for planning and dialogue) with its sensory information, to ensure that the interpretations of these symbolic representations are grounded in the situated context. Previous approaches to this problem have used techniques such as (low-level) information fusion, ontological reasoning, and (high-level) concept learning. This paper presents a framework in which these, and other approaches, can be combined to form a shared representation of the current state of the robot in relation to its environment and other agents. Preliminary results from an implemented system are presented to illustrate how the framework supports behaviours commonly required of an intelligent robot

Spectators’ aesthetic experiences of sound and movement in dance performance

In this paper we present a study of spectators’ aesthetic experiences of sound and movement in live dance performance. A multidisciplinary team comprising a choreographer, neuroscientists and qualitative researchers investigated the effects of different sound scores on dance spectators. What would be the impact of auditory stimulation on kinesthetic experience and/or aesthetic appreciation of the dance? What would be the effect of removing music altogether, so that spectators watched dance while hearing only the performers’ breathing and footfalls? We investigated audience experience through qualitative research, using post-performance focus groups, while a separately conducted functional brain imaging (fMRI) study measured the synchrony in brain activity across spectators when they watched dance with sound or breathing only. When audiences watched dance accompanied by music the fMRI data revealed evidence of greater intersubject synchronisation in a brain region consistent with complex auditory processing. The audience research found that some spectators derived pleasure from finding convergences between two complex stimuli (dance and music). The removal of music and the resulting audibility of the performers’ breathing had a significant impact on spectators’ aesthetic experience. The fMRI analysis showed increased synchronisation among observers, suggesting greater influence of the body when interpreting the dance stimuli. The audience research found evidence of similar corporeally focused experience. The paper discusses possible connections between the findings of our different approaches, and considers the implications of this study for interdisciplinary research collaborations between arts and sciences

A case study of professional musicians and their perceptions of the impact of aural learning experiences in developing musicianship

This narrative case study examines the perceptions of three professional musicians on the impact of focused aural experiences (FAEs) in developing their musicianship. The participants include a professional instrumental performer, the director of a non-profit jazz school, and a retired high school band teacher. Each participant has engaged in performing, teaching, composing, arranging, and/or conducting in some capacity at various stages of their careers. The research explores the roles that focused aural experiences may have had in shaping each individual’s musical development by examining their perceptions of the impacts of such experiences on their overall musicianship. Focused aural experiences (FAEs) are defined here as any formal or informal activities in which the individual learned music by ear, learned to play an instrument by ear, or listened to music with focused intent. Relevant formal activities might include taking formal aural training classes in a school setting, transcribing, and playing music by ear, and studying orchestration or composition through aural transcription. For the purposes of this study, focused aural experiences will be specifically differentiated from musical engagements where notation is more emphasized such as performing music through score reading, sight reading, score analysis, or notation-based music theory. This study aims to provide a perspective on the complex experiences that shape musical development specifically through the lens of FAEs, describe the nature of those experiences, and offer another pathway towards understanding the nature of music learning, the development of musicianship, and notions about the multidimensional nature of talent and ability

On the edge: ICT and the transformation of professional legal education

Information and communications technology in professional legal education courses is perceived as problematic for teachers and course designers. It is so not because technology is inherently difficult or strange, but because at a deep level it can threaten the practice and identity of teachers. However the contextual challenges of their position, caught between academy and practice, may actually enable professional legal educators to take account of new technologies. The article discusses this proposal, using the example of the incremental development of a discussion forum. It suggests that the tools of pragmatist and transformative meta-theory may point the way forward for professional legal educators to create their own community of practice in the use of ICT in professional legal learning

Dialogue, New Media and Children's Intellectual Development: Re-Thinking Malaysian Teaching and Learning Approaches

This thesis examines the extent to which ‘Talk or Dialogue’ contributes to children’s cognitive and psychological development when it is experienced through technological devices. The work analyzes some of the sociocultural theories of children’s speech, cognitive learning, psychological functions, sociocultural learning context, dialogic teaching and learning approaches in the classroom, social interaction and the use of social tools. The theory of speech is built on the Vygotskian notion of language as the prime cultural and psychological tool for children’s learning development in a sociocultural environment. Lev S. Vygotsky emphasised that the development of cognitive processes in children includes thinking, reasoning and understanding of a conceptualised social interaction. These processes are core to children’s intellectual learning. Vygotsky and the neo-Vygotskians emphasised the use of Speech, Talk or Dialogue and the Zone of Proximal Development (ZPD) concept in children’s learning development. In the UK, it was evident that the Talk teaching and learning approach (Mercer & Littleton, 2007) contributed effectively to children’s learning achievements. This approach reinforces Talk or Dialogue collaboratively in the classroom with the ZPD concept. The significance of the Talk teaching approach has encouraged this study to examine further children’s speech and the use of technological devices. Hence, a theoretical discourse methodology on children’s Talk or Dialogue was examined for the research outcomes. The aim is to devise a new teaching and learning approach that contributes to the Malaysian children’s intellectual development inside and outside the classroom through the use of Talk or Dialogue. As a result, a Dialogic framework is articulated based on four existing educational theories of children’s speech and learning. This framework is vital to contribute directly to the Malaysia Education Department Blueprint 2013-2025 in promoting children’s intellectual development. For that reason, two approaches are proposed which emphasise children’s psychological functions of perception, attention, sensory motor-operations and memory through the use of Talk and technological devices. These approaches accentuate the ZPD concept between the teachers and children for learning and activity games. This is the study’s contribution to new knowledge

An integrated theory of language production and comprehension

Currently, production and comprehension are regarded as quite distinct in accounts of language processing. In rejecting this dichotomy, we instead assert that producing and understanding are interwoven, and that this interweaving is what enables people to predict themselves and each other. We start by noting that production and comprehension are forms of action and action perception. We then consider the evidence for interweaving in action, action perception, and joint action, and explain such evidence in terms of prediction. Specifically, we assume that actors construct forward models of their actions before they execute those actions, and that perceivers of others' actions covertly imitate those actions, then construct forward models of those actions. We use these accounts of action, action perception, and joint action to develop accounts of production, comprehension, and interactive language. Importantly, they incorporate well-defined levels of linguistic representation (such as semantics, syntax, and phonology). We show (a) how speakers and comprehenders use covert imitation and forward modeling to make predictions at these levels of representation, (b) how they interweave production and comprehension processes, and (c) how they use these predictions to monitor the upcoming utterances. We show how these accounts explain a range of behavioral and neuroscientific data on language processing and discuss some of the implications of our proposal

Maurice Merleau-Ponty, Marshall McLuhan and Communication Ethics: The Taming of Americanitis

In 1880 neurologist George Miller Beard identified the diagnosis of neurasthenia. Popularly referred to as Americanitis, Beard treated an increasing number of people for symptoms of anxiety, malaise and gastric discomfort. Per Beard, the illness resulted from the rapidly changing mechanical landscape of modernity. Similarly, contemporary Media Ecology literature suggests that Americanitis continues amidst our current digital moment. Manifest as narcissistically anxious nervous exhaustion, digital Americanitis results due to technological encouragement of existential and communicative closure, thereby negatively implicating the human condition, human communication and communication ethics practices. As such, this project considers Marshall McLuhan’s Media Ecology to examine the communicative phenomena of Americanitis. Based on affable grounding assumptions as well as calls in recent literature, McLuhan’s work is read through the presently underrepresented Media Ecology scholarly approach of existential phenomenology. In particular, this Merleau-Pontean existential phenomenological reading enhances understanding of the implicit theory of human communication and communication ethics informing McLuhan’s Media Ecology criticism. Once elucidated, McLuhan’s theoretical assumptions, aims and ends comport with theoretical dimensions of Merleau-Ponty’s existential phenomenology to reveal how and why technology encourages our digital Americanitis. When placed into conversation, the two thinkers also offer possible responses to our ills – approaches to “taming” Americanitis

Sonic Interactions in Virtual Environments: the Egocentric Audio Perspective of the Digital Twin

The relationships between the listener, physical world and virtual environment (VE) should not only inspire the design of natural multimodal interfaces but should be discovered to make sense of the mediating action of VR technologies. This chapter aims to transform an archipelago of studies related to sonic interactions in virtual environments (SIVE) into a research field equipped with a first theoretical framework with an inclusive vision of the challenges to come: the egocentric perspective of the auditory digital twin. In a VE with immersive audio technologies implemented, the role of VR simulations must be enacted by a participatory exploration of sense-making in a network of human and non-human agents, called actors. The guardian of such locus of agency is the auditory digital twin that fosters intra-actions between humans and technology, dynamically and fluidly redefining all those configurations that are crucial for an immersive and coherent experience. The idea of entanglement theory is here mainly declined in an egocentric-spatial perspective related to emerging knowledge of the listener's perceptual capabilities. This is an actively transformative relation with the digital twin potentials to create movement, transparency, and provocative activities in VEs. The chapter contains an original theoretical perspective complemented by several bibliographical references and links to the other book chapters that have contributed significantly to the proposal presented here.Comment: 46 pages, 5 figures. Pre-print version of the introduction to the book "Sonic Interactions in Virtual Environments" in press for Springer's Human-Computer Interaction Series, Open Access license. The pre-print editors' copy of the book can be found at https://vbn.aau.dk/en/publications/sonic-interactions-in-virtual-environments - full book info: https://sive.create.aau.dk/index.php/sivebook

Annotation of negotiation processes in joint-action dialogues

Situated dialogic corpora are invaluable resources for understanding the complex relationship between language, perception, and action as they are based on naturalistic dialogue situations in which the interactants are given shared goals to be accomplished in the real world. In such situations, verbal interactions are intertwined with actions, and shared goals can only be achieved via dynamic negotiation processes based on common ground constructed from discourse history as well as the interactants' knowledge about the status of actions. In this paper, we propose four major dimensions of collaborative tasks that affect the negotiation processes among interactants, and, hence, the structure of the dialogue. Based on a review of available dialogue corpora and annotation manuals, we show that existing annotation schemes so far do not adequately account for the complex dialogue processes in situated task-based scenarios. We illustrate the effects of specific features of a scenario using annotated samples of dialogue taken from the literature as well as our own corpora, and end with a brief discussion of the challenges ahead