Gesture based word (re)acquisition with a virtual agent in augmented reality: A preliminary study

From an evolutionary perspective, language and gesture belong together as a system, serving communication on both an abstract and a physical level. In aphasia, when language is impaired, patients make use of gestures. Laboratory research has provided evidence that gesture can support aphasia rehabilitation, or more specifically, anomia rehabilitation. Here, we test an anomia gesture-based rehabilitation scenario with a virtual trainer (VT) in augmented reality (AR) as a therapy simulation. Thirty German-speaking participants were trained on 27 biand three-syllabic words of Vimmi, an artificial language. Each Vimmi word was paired to a function word in German. The participants were divided into two Groups of 15 and 15 persons. Group A learned word pairs by observing the gestures performed by the VT and additionally imitating them. Group B learned 27 word-pairs by observing the VT standing still and listening to them. Participants were trained singularly for 3 days, alternating one day of training with one day of rest for memory consolidation. Word retention was assessed immediately after each training session by means of free and cued recall tests administered electronically. Group A and Group B did not differ in word retention. When subdividing participants in high and low performers, interactions showed that high performers benefitted more from gesture-based training than low performers. The data in this preliminary study do not speak in favour of VTs as possible tools in gesture-based AR language rehabilitation. Technology might have, in this case, detrimental effects on word learning

A Virtual Agent as Vocabulary Trainer: Iconic Gestures Help to Improve Learners’ Memory Performance

An important and often laborious task in foreign language acquisition is vocabulary learning. Research has repeatedly demonstrated that performing iconic gestures together with novel words has a beneficial effect on learning performance. Can these findings be transferred onto virtual agents applied in gesture-supported vocabulary training? We present a study investigating whether iconic gestures performed by a virtual agent and imitated by learners have an impact on verbal memory for words in a foreign language. In a within-subject design we compared participants’ memory performance achieved with the help of a virtual agent and those achieved with the help of a human trainer regarding both short-term learning effects and long-term decay effects. The overall results demonstrate improved memory scores when participants learned with a virtual agent. Especially high performers could profit from gesture-supported training with a virtual agent

Symmetric and asymmetric action integration during cooperative object manipulation in virtual environments

Cooperation between multiple users in a virtual environment (VE) can take place at one of three levels. These are defined as where users can perceive each other (Level 1), individually change the scene (Level 2), or simultaneously act on and manipulate the same object (Level 3). Despite representing the highest level of cooperation, multi-user object manipulation has rarely been studied. This paper describes a behavioral experiment in which the piano movers' problem (maneuvering a large object through a restricted space) was used to investigate object manipulation by pairs of participants in a VE. Participants' interactions with the object were integrated together either symmetrically or asymmetrically. The former only allowed the common component of participants' actions to take place, but the latter used the mean. Symmetric action integration was superior for sections of the task when both participants had to perform similar actions, but if participants had to move in different ways (e.g., one maneuvering themselves through a narrow opening while the other traveled down a wide corridor) then asymmetric integration was superior. With both forms of integration, the extent to which participants coordinated their actions was poor and this led to a substantial cooperation overhead (the reduction in performance caused by having to cooperate with another person)

Developing serious games for cultural heritage: a state-of-the-art review

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been less well considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in entertainment games technology. As a result, the field of serious heritage games concerns itself with recent advances in computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On the other hand, the main strengths of serious gaming applications may be generalised as being in the areas of communication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In this report, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in serious heritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several case studies illustrating the application of methods and technologies used in cultural heritage are presented

Computational Physics on Graphics Processing Units

The use of graphics processing units for scientific computations is an emerging strategy that can significantly speed up various different algorithms. In this review, we discuss advances made in the field of computational physics, focusing on classical molecular dynamics, and on quantum simulations for electronic structure calculations using the density functional theory, wave function techniques, and quantum field theory.Comment: Proceedings of the 11th International Conference, PARA 2012, Helsinki, Finland, June 10-13, 201

The Art of Research: A Divergent/Convergent Framework and Opportunities for Science-Based Approaches

Applying science to the current art of producing engineering and research knowledge has proven difficult, in large part because of its seeming complexity. We posit that the microscopic processes underlying research are not so complex, but instead are iterative and interacting cycles of divergent (generation of ideas) and convergent (testing and selecting of ideas) thinking processes. This reductionist framework coherently organizes a wide range of previously disparate microscopic mechanisms which inhibit these processes. We give examples of such inhibitory mechanisms and discuss how deeper scientific understanding of these mechanisms might lead to dis-inhibitory interventions for individuals, networks and institutional levels

An empirical investigation of performance overhead in cross-platform mobile development frameworks

The heterogeneity of the leading mobile platforms in terms of user interfaces, user experience, programming language, and ecosystem have made cross-platform development frameworks popular. These aid the creation of mobile applications – apps – that can be executed across the target platforms (typically Android and iOS) with minimal to no platform-specific code. Due to the cost- and time-saving possibilities introduced through adopting such a framework, researchers and practitioners alike have taken an interest in the underlying technologies. Examining the body of knowledge, we, nonetheless, frequently encounter discussions on the drawbacks of these frameworks, especially with regard to the performance of the apps they generate. Motivated by the ongoing discourse and a lack of empirical evidence, we scrutinised the essential piece of the cross-platform frameworks: the bridge enabling cross-platform code to communicate with the underlying operating system and device hardware APIs. The study we present in the article benchmarks and measures the performance of this bridge to reveal its associated overhead in Android apps. The development of the artifacts for this experiment was conducted using five cross-platform development frameworks to generate Android apps, in addition to a baseline native Android app implementation. Our results indicate that – for Android apps – the use of cross-platform frameworks for the development of mobile apps may lead to decreased performance compared to the native development approach. Nevertheless, certain cross-platform frameworks can perform equally well or even better than native on certain metrics which highlights the importance of well-defined technical requirements and specifications for deliberate selection of a cross-platform framework or overall development approach