This paper proposes formative plans for the concept of a framework called VIOLA (Virbots for Independent Online Living Applications) with the potential to integrate an artificial cognitive system (ACS) with real and virtual worlds. The aim is to endow a robot with advanced perception, action and social interaction capabilities, achieved through translating learning and social interaction techniques currently utilised only in virtual worlds, to real world systems. The framework would aspire to enhance human-robotic interaction through the application of accelerated virtual learning outcomes to real-world social interactions and scenarios. A home-based implementation of this framework may demonstrate the capabilities of the concept to illustrate a socially aware cognitive system simultaneously driving and learning from multiple virtual robots (Virbots), and robots, on both functional and social levels

Bonnell, B.

Burden, D.

de Freitas, Sara

Dunwell, Ian

Duthen, Y.

Graf, B.

Panzoli, D.

Peters, Christopher

Tscheligi, M.

Peters, C.

Dunwell, I.

de Freitas, S.

Research Repository

  MURDOCH RESEARCH REPOSITORY       Peters, C., Dunwell, I., de Freitas, S., Panzoli, D., Graf, B., Tscheligi, M., Burden, D., Duthen, Y. and Bonnell, B. (2009) VIOLA: Concept of a new cognitive framework to enhance the capabilities of interactive service robots using virtual worlds. In: Beyond Gray Droids workshop: Domestic Robot Design for the 21st Century, 1 September, Cambridge, UK.     http://researchrepository.murdoch.edu.au/27766/           It is posted here for your personal use. No further distribution is permitted.    VIOLA: Concept of a New Cognitive Framework to Enhance the Capabilities of Interactive Service Robots using Virtual Worlds  Christopher Peters Faculty of Engineering and Computing Coventry University United Kingdom +44 (0)2476887688 Christopher.Peters@coventry.ac.uk  David Panzoli Research Institute on Informatics University of Toulouse  France +33 561128833 panzoli@irit.fr  David Burden Daden Limited Birmingham, B13 9SG United Kingdom +44 (0)121 698 8520 david.burden@daden.co.uk  Ian Dunwell Serious Games Institute Coventry University United Kingdom +44 (0)2476158250 idunwell@cad.coventry.ac.uk  Birgit Graf Fraunhofer IPA Nobelstr. 12 D-70569 Stuttgart +49 711 970 1910 birgit.graf@ipa.fraunhofer.de  Yves Duthen Research Institute on Informatics  University of Toulouse France +33 5674949400 duthen@irit.frSara de Freitas Serious Games Institute Coventry University United Kingdom +44 (0)24 7615 8208 s.defreitas@coventry.ac.uk  Manfred Tscheligi HCI & Usability Unit, ICT&S Center University of Salzburg 5020 Salzburg, Austria +43 662 8044 4811 manfred.tscheligi@sbg.ac.at  Bruno Bonnell Robopolis 107 Boulevard Beaumarchais 75003 Paris France bb@robopolis.com   ABSTRACT This paper proposes formative plans for the concept of a framework called VIOLA (Virbots for Independent Online Living Applications) with the potential to integrate an artificial cognitive system (ACS) with real and virtual worlds. The aim is to endow a robot with advanced perception, action and social interaction capabilities, achieved through translating learning and social interaction techniques currently utilised only in virtual worlds, to real world systems. The framework would aspire to enhance human-robotic interaction through the application of accelerated virtual learning outcomes to real-world social interactions and scenarios. A home-based implementation of this framework may demonstrate the capabilities of the concept to illustrate a socially aware cognitive system simultaneously driving and learning from multiple virtual robots (Virbots), and robots, on both functional and social levels.  Keywords Adaptive real-world systems, artificial cognitive systems, autonomous robots, virtual environments, human-robot interaction, home assistance  1. INTRODUCTION Over the last two decades there has been an incremental development in service robotics engineering. One of the major challenges, still unsolved, regards cognitive learning capabilities enabling robots to solve complex tasks in everyday environments. This has limited the practical application of such systems in real time dynamic environments. Cognitive systems operating in real world environments need to be flexible and robust. A key way in which this can be achieved is by endowing the systems with the ability to learn, so as to be able to adapt to dynamic situations.  The VIOLA concept envisions a shift beyond the use of virtual environments as merely low autonomy, direct control systems e.g. tele-operation interfaces, moving instead towards their use as collaborative learning environments for supporting autonomous and social robotic interactions. It represents a shift in how learning processes can be captured by cognitive systems and robots, by integrating real and virtual environment technologies. The virtual environment enables the adoption of simulation techniques that are not practical solely with real world systems. While virtual environments are now being used as control systems or simulators for robotic technology, our framework could advance scientific knowledge by using virtual environments as collaborative learning environments for the artificial cognitive system (ACS). The ability for the ACS to drive multiple virtual robots, or Virbots, simultaneously allows for user interaction to be harnessed for accelerated learning. This represents a fundamental benefit: the system does not require a real world presence in order to assimilate learning, and can thus be trained by multiple users in parallel – in addition to developing itself using the simulation capabilities afforded by the virtual environment. As a potential proof of concept, a cognitive aware home-based implementation is envisioned for a mobile robot supporting people within the home. Such a robot could be used to assist people with mobility issues, such as the elderly who are living independently in the home, enriching the quality of their lives, in addition to a host of other applications. 2. BACKGROUND This work builds on crossovers from research strands in the domains of virtual environments, artificial life (or A-Life) and robotics [7]. 2.1 Virtual Environments and Robotics One of the most common applications for virtual environments in a robotics context is as a means to simulate robot behaviour without requiring costly real-world development of hardware. Baselt et al. [1] used the term VirBot to refer to a virtual reality robot driven by commands from a real-world operator. Examples of the use of virtual environments as a tool for direct robotic manipulation include [11], who use a virtual environment as a control system for a robot in tele-rehabilitation. By interacting with objects in the virtual world using a haptic, force-feedback interface, the user controls two real-world robots manipulating real-world objects. The applications of virtual environments of closest relevance to this work are those seeking to utilise it as an extension of the robot’s ‘brain’, in order to impart the ability to artificially simulate and visualise real or hypothetical situations. For example, [12] used a virtual environment to describe objects from the real-world, which were subsequently analysed by obstacle-avoidance algorithms. The robot was thus controlled according to the content of the virtual environment. A 2006 US patent exists in this area (7099745), outlining the potential for the use of a virtual world as a control system.   2.2 A-Life and Robotics In early work, [3] explored the potential of A-Life for driving a robot. Two points were emphasised in his study. First, calibration, or relating the parameters of the simulation to real world environments greatly benefits from the adaptiveness of A-Life systems. Secondly, these approaches typically have a lower memory footprint since instructions are emergent rather than pre-programmed, resulting in better reactivity of the robot. [9] evolve both the shape and the controller of autonomous robots. [2] self-robotics’ project aims to embed, inside a robot, an inner model of its body and environment, enabling it to be “consciously” aware of its position and actions. Self-organising robots are today widely recognised as a promising next step in robotics research (see [9] for an exhaustive review).  2.3 Home-based Systems Some key advances in home-based systems can be related to the ubiquitous computing paradigm, focusing on the provision of a smart home: that is, a home environment that is “a small world where all kinds of smart devices are continuously working to make inhabitants’ lives more comfortable” [4]. This is accomplished by providing remote control of devices and automated management and enhanced services through intelligent devices (see for example, the Aware Home Research Initiative at GeorgiaTech and MavHome, Managing an Intelligent Versatile Home [14]). A key challenge remains to augment such environments with reliable, capable, smart mobile platforms that can enhance existing capabilities. The VIOLA concept seeks to explore this potential, and leverage the advantages of sophisticated artificial and virtual life models alongside the modelling of robot behaviour within virtual environments. Dealing with dynamic environments and interacting with users is still a difficult task to solve in simulation alone. However, it is important to stress the far broader implications of VIOLA – the robot could not only be capable of learning about its physical environment based on virtual interaction, it could also be able to learn and relate virtual experiences on a much more general level, enabling complex social as well as physical interactions.  Figure 1. Illustration of the VIOLA framework concept and possible interactions with virtual environment and A-Life techniques. 3. FRAMEWORK The VIOLA concept may be viewed as a high-level architecture that integrates real and virtual systems under the control of a single cognitive system. This framework provides a generic guideline for implementation on a wide range of platforms to address many different application scenarios. The key components (see Figure 1) are (i) Virbots, or virtual robots and (ii) the artificial cognitive system, ACS. We describe them in more detail next. 3.1 Virbots A key notion behind the framework is the concept of the Virbot as the embodiment of the ACS in the virtual world, whilst the physical robot is a simultaneous embodiment of the ACS in the real world. This permits the transfer of knowledge and skills between the virtual and the real embodiments, thus leveraging the demonstrated efficacy of virtual learning environments to real world systems driven by an ACS.   Figure. 2 Overview of potential interactions within a VIOLA framework. The ACS (right) consists of a layered model capable of controlling a physical and/or virtual embodiment, for example a physical robot operating within the real environment or an agent within a virtual environment. The ACS may learn through interactions with multiple remote users and/or a single user situated locally to the physical embodiment.  3.2 Artificial Cognitive System (ACS) The development ACS, which handles the cognitive processing of the system independently of its embodiment, whether virtual or real, is of key importance. The ACS interfaces between (i) the physical robot operating in the real world, (ii) the Virbot, operating in a virtual world and (iii) the user(s). The ACS provides cognitive capabilities, such as perceiving the environment using several different sensors, navigating and manipulating objects, autonomously managing goals, planning and learning to execute complex cognitive and social tasks, and communicating with the user. Since the learning and control strategies used in the Virbot are linked to perception and action both in the real and virtual world, the ACS allows the system to learn from multiple users interacting with the Virbot in the virtual environment. An important advantage of this method is that it does not necessitate the use of the real robot: users may train the system when the physical robot is not operational.  3.3 The User Interface The main novelty of the interface lies in the introduction of an additional level of interaction with the learning environment. While in the past, the user interacted directly with the robot, it will now be possible, if desired by the user, for them to interact with the Virbot. This flexibility is advantageous, since in some circumstances either interface type may be more appropriate depending on the task or scenario at hand. For example, low-level interaction with the robot allows simpler direct communication, while interacting with the Virbot frees the user from low-level tasks to permit more natural communication with the system, enabling the user to gain a better understanding of the context of the interaction and perhaps a better awareness and control of the situation. 4. HOME-BASED SCENARIO The VIOLA framework involves the concept of a hybrid system, bringing together the practical benefits of static systems and the capabilities of mobile platforms. In order to capture all relevant requirements for the framework coming from real application scenarios, a cognitive-aware home-based scenario has been devised. This home assistance scenario involves a mobility restricted local user, for example an elderly person, who is being supported by a number of remote users who are caregivers (see Figure 2). In this scenario, two important physical components are a smart, home environment and the robot (for instance, in our scenario, a Robotic Home Assistant Care-O-bot® [8]). The home environment incorporates embedded sensor technologies, for detecting details regarding the state of the local user and their location. The robot acts as a mobile facilitator capable of conducting a number of simple, helpful tasks, of great aid to the mobility restricted user, for example fetching various objects. It may be controlled at different levels by the caregivers and the ACS. The purpose of the virtual environment is to provide a control interface for the caregivers when the robot and smart environment is online, and also to allow the ACS to conduct hypothetical scenarios and to be trained (and learn independently) when it is not. The ability for the ACS to drive multiple Virbots simultaneously allows for user interaction to be harnessed for accelerated learning. There are two ways of accelerating learning; the first is by allowing multiple users to train Virbots in parallel in order to inform the ACS. In this scenario, multiple caregivers may engage in the virtual environment to train the ACS (through a Virbot) in the way it should act given specific circumstances regarding the local user. The second way is to use an evolutionary learning approach to allow the ACS to learn without the necessity of direct user input.  5. OUTLOOK Cognitive systems operating in real world environments need to be flexible and robust. A key way in which this can be achieved is by endowing the systems with the ability to learn, so as to be able to adapt to dynamic situations. Here, we present our formative work on concepts utilising virtual environment and A-Life technologies to enhance learning.  There are two main issues with modern service robotics systems that we hope could be addressed with this framework. The first is a limitation with artificial cognitive systems (ACS) as a whole - that they require significant computational processing power, particularly for higher order cognitive functioning. This is exacerbated in mobile robotics systems, where all the processing often occurs on board the robot. In the era of distributed computing, this problematic is rescinding as there is a general trend towards distributing the processing. Significantly, however, it remains time consuming to train the robot. The second issue is the challenge in providing naturalistic communications between humans and robots [5][6]. In existing systems, many restrictions exist regarding the depth, ease and naturalness of social interactions with robots [13]. Leveraging the research advances in the virtual world (in particular the use of intelligent virtual agents as ‘chatbots’) has opened new avenues for supporting increasingly natural and deeper interactions. These areas have the potential to produce significant advances to support accelerated learning and advanced AI/A-Life techniques, and the potential of this field for learning transfer to hard systems, such as robotics systems, is only beginning to be explored due to the maturity of the technologies. It is hoped that such a framework may provide dual benefits for virtual environments and robotics. For virtual environments, the meshing between A-Life techniques and virtual world agents could produce benefits in terms of more complex behaviours of agents and the development of new applications integrating real world and virtual systems. For robotics, the development of the home-based implementation may support more complex emergent behaviour achieved through the virtualisation of simulation for enhanced learning and adaptation. 6. ACKNOWLEDGMENTS We thankfully acknowledge contributions in relation to this work from: Astrid Weiss and Daniela Wurhofer, HCI Unit, University of Salzburg; Catherine Simon, Robopolis; Nicolas Lassabe, University of Toulouse; and Martin Hägele, Fraunhofer IPA. 7. REFERENCES  [1] Baselt, D.R., Lee, G.U., Natesan, M., Metzger, S.W., Sheehan, P.E., Colton, R.J., Savage-Carmona, J., Billinghurst, M. and Holden, A. 1998. The Virbot: a virtual reality robot driven with multimodal commands. Journal of Expert Systems with Applications, Volume 15, Number 3, pp. 413-419. [2] Bongard, J., Zykov, V. and Lipson, H. 2006. Resilient Machines Through Continuous Self-Modeling, Science Vol. 314, No. 5802, pp. 1118 – 1121. [3] Brooks, R. 1992. Artificial Life and Real Robots. Proceedings of the First European Conference on Artificial Life.  3-10. [4] Cook, D. and Das, S. 2004. Smart Environments: Technology, Protocols and Applications, Wiley Series on Parallel and Distributed Computing, Wiley-Interscience. [5] Dautenhahn, K. 2007. Socially intelligent robots: dimensions of human-robot interaction, Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1480), pp. 679-704. [6] Dautenhahn, K., Woods, S., Kaouri, C., Walters, M., Koay, K. L., and Werry I. 2005. What is a Robot Companion - Friend, Assistant or Butler?, Proc. IROS 2005, IEEE IRS/RSJ International Conference on Intelligent Robots and Systems, August 2-6, 2005, Edmonton, Alberta Canada, pp. 1488-1493. [7] Gamez, D., Newcombe, R., Holland, O. and Knight, R. 2006. Two Simulation Tools for Biologically Inspired Virtual Robotics, Proceedings of the IEEE 5th Chapter Conference on Advances in Cybernetic Systems, Sheffield, 2006, pp. 85-90. [8] Graf, B., Parlitz, C. and Hägele, M. 2009. Robotic Home Assistant Care-O-bot® 3 - Product Vision and Innovation Platform. In: Proceedings of HCI International 2009, San Diego, USA. [9] Lipson, H. and Pollack, J.B. 2000. Automatic design and Manufacture of Robotic Lifeforms. Nature, Vol. 406, pp. 974-978. [10] Nolfi, S. and Floreano, D. 2000. Evolutionary Robotics: The Biology, Intelligence, and Technology of Self-Organizing Machines. Cambridge, MA: MIT Press/Bradford Books. [11] Tang, J., Carignan, C. and Gattewar, S. 2005. Virtual Environment for Robotic Tele-Rehabilitation. In Proceedings of the 9th International Conference on Rehabilitation Robotics (ICORR), pp. 365-370. [12] Voisan, E., Volosencu, C. Leu, A., and Dragan, F. 2008. Fault detection using virtual environment and wireless robot. In Proceedings of the 9th WSEAS international Conference on international Conference on Automation and information. Recent Advances in Electrical Engineering. World Scientific and Engineering Academy and Society (WSEAS), pp. 190-193 [13] Walters, M.L., Syrdal, D.S., Dautenhahn, K., te Boekhorst, R. and Koay, K.L. 2008. Avoiding the uncanny valley: robot appearance, personality and consistency of behavior in an attention-seeking home scenario for a robot companion. Autonomous Robots, Volume 24, Number 2 / February, 2008, pp. 159-178. [14] Youngblood, M., Cook, D. and Holder, L. 2005. Seamlessly engineering a smart environment, 2005 IEEE Conference on  Systems, Man and Cybernetics, Vol. 1, pp. 548-553. 

VIOLA: Concept of a new cognitive framework to enhance the capabilities of interactive service robots using virtual worlds

Coventry University Pure Portal

VIOLA: concept of a new cognitive framework to enhance the capabilities of interactive service robots using virtual worlds

https://researchrepository.murdoch.edu.au/id/eprint/27766/1/interactive_service_robots_using_virtual_worlds.pdf

VIOLA: concept of a new cognitive framework to enhance the capabilities of interactive service robots using virtual worlds

Abstract

Similar works

Full text

Available Versions

Research Repository

Coventry University Pure Portal