A prototype for a conversational companion for reminiscing about images

This work was funded by the COMPANIONS project sponsored by the European Commission as part of the Information Society Technologies (IST) programme under EC grant number IST-FP6-034434. Companions demonstrators can be seen at: http://www.dcs.shef.ac.uk/∼roberta/companions/Web/.This paper describes an initial prototype of the Companions project (www.companions-project.org): the Senior Companion (SC), designed to be a platform to display novel approaches to: (1) The use of Information Extraction (IE) techniques to extract the content of incoming dialogue utterances after an ASR phase. (2) The conversion of the input to RDF form to allow the generation of new facts from existing ones, under the control of a Dialogue Manager (DM), that also has access to stored knowledge and knowledge accessed in real time from the web, all in RDF form. (3) A DM expressed as a stack and network virtual machine that models mixed initiative in dialogue control. (4) A tuned dialogue act detector based on corpus evidence. The prototype platform was evaluated, and we describe this; it is also designed to support more extensive forms of emotion detection carried by both speech and lexical content, as well as extended forms of machine learning. We describe preliminary studies and results for these, in particular a novel approach to enabling reinforcement learning for open dialogue systems through the detection of emotion in the speech signal and its deployment as a form of a learned DM, at a higher level than the DM virtual machine and able to direct the SC’s responses to a more emotionally appropriate part of its repertoire. © 2010 Elsevier Ltd. All rights reserved.peer-reviewe

Demonstration of a prototype for a conversational companion for reminiscing about images

This work was funded by the Companions project (2006-2009) sponsored by the European Commission as part of the Information Society Technologies (IST) programme under EC grant number IST-FP6-034434.This paper describes an initial prototype demonstrator of a Companion, designed as a platform for novel approaches to the following: 1) The use of Information Extraction (IE) techniques to extract the content of incoming dialogue utterances after an Automatic Speech Recognition (ASR) phase, 2) The conversion of the input to Resource Descriptor Format (RDF) to allow the generation of new facts from existing ones, under the control of a Dialogue Manger (DM), that also has access to stored knowledge and to open knowledge accessed in real time from the web, all in RDF form, 3) A DM implemented as a stack and network virtual machine that models mixed initiative in dialogue control, and 4) A tuned dialogue act detector based on corpus evidence. The prototype platform was evaluated, and we describe this briefly; it is also designed to support more extensive forms of emotion detection carried by both speech and lexical content, as well as extended forms of machine learning.peer-reviewe

Co-present photo sharing on mobile devices

This dissertation researches current approaches to photo sharing. We have found that most current methods of photo sharing are not as compelling as traditional photo sharing - with the increasing in popularity of digital photography, consumers do not print photos as often as before and thus typically require a group display (such as a PC) to view their photographs collectively. This dissertation describes a mobile application that attempts to support traditional photo sharing activities by allowing users to share photos with other co-present users by synchronizing the display on multiple mobile devices. Various floor control policies (software locks that determine when someone can control the displays) were implemented. The behaviour of groups of users was studied to determine how people would use this application for sharing photos and how various floor control policies affect this behaviour

A framework for developing a conversational agent to improve normal age- associated memory loss and increase subjective wellbeing

Research has developed a baseline conversational agent (CA) framework that experiments suggest may improve normal ageing memory problems and increase Subjective Wellbeing (SWB) in participants aged 60+ with normal age-associated memory loss. In 2008, 1.3 million people in the United Kingdom were aged 85+, this figure is projected to reach 3.3 million by 2033 (Morse, 2010). Thus, as the population profile changes, ageing memory impairment problems will become acuter (Morse, 2010). The number of people worldwide with diagnosed clinical memory problems is expected to double every 20 years to 66 million by 2030 and 115 million by 2050 (Casey et al., 2016, Prince et al., 2013). Improving memory impairment reduces distress for individuals and enhances wellbeing and independence (Dorin, 2007); (Wagner et al., 2010). The quality of life in old age can be improved by increasing SWB (George, 2010) that is concerned with how people experience the quality of their lives and includes both emotional reactions and cognitive judgments (George, 2010). Experiments performed as part of the pilot study suggested evidence of increased SWB and improved memory after use of the CA. To support these early findings, modification to the agent and further experimentation was undertaken. Further work enhanced the preliminary work that was carried out and provided the opportunity to run further, more in-depth evaluations of the CA as both a reminiscence aid and as an improver of SWB. This PhD study applied for and gained ethical approval (SE111219) from the Faculty of Science & Engineering Ethics Committee, Manchester Metropolitan University on 25 October 2012

Co-Present Photo Sharing on Mobile Devices

The paper reports a mobile application that allows users to share photos with other co-present users by synchronizing the display on multiple mobile devices. Various floor control policies (software locks that determine when someone can control the displays) were implemented. The behaviour of groups of users was studied to determine how people would use this application for sharing photos and how various floor control policies affect this behaviour. Explicit policies was shown to be the best strategy for structured presentations, but when all locks were removed, the users created a new form of social interaction which seemed to be a more compelling use of the technology than the original, intended, application

Question-Answering Virtual Humans based on Pre-recorded Testimonies for Holocaust Education.

With the continued application of gaming for training and education, which has seen exponential growth over the past two decades, this book offers an insightful introduction to the current developments and applications of game technologies within educational settings, with cutting-edge academic research and industry insights, providing a greater understanding into current and future developments and advances within this field. Following on from the success of the first volume in 2011, researchers from around the world presents up-to-date research on a broad range of new and emerging topics such as serious games and emotion, games for music education and games for medical training, to gamification, bespoke serious games, and adaptation of commercial off-the shelf games for education and narrative design, giving readers a thorough understanding of the advances and current issues facing developers and designers regarding games for training and education. This second volume of Serious Games and Edutainment Applications offers further insights for researchers, designers and educators who are interested in using serious games for training and educational purposes, and gives game developers with detailed information on current topics and developments within this growing area

Mixed Reality Storytelling for Social Engagement with Older Adults

Aging is a natural process that brings social and physical challenges among adults, due to which they have to make shifts in habits and routines. Adults belonging to an older age group lose touch with people which provides an opportunity for researchers to think and implement novel ways to engage the population with their loved ones. The advent of technology within the Mixed Reality (MR) space aims to facilitate diverse groups of people to engage in immersive and interactive ways, opening possibilities to address the predicament of aging in an isolated environment. Utilizing participatory design in a virtual setting, inclusive design frameworks and design thinking practices, the contributions of this research are to present the broad concepts of storytelling, social engagement and Mixed Reality existing in the literature, and then take inspiration to co-design a Mixed Reality storytelling system with older adults and their friends & family for the purpose of cultivating meaningful social connections through sharing stories