Virtual humans: thirty years of research, what next?

In this paper, we present research results and future challenges in creating realistic and believable Virtual Humans. To realize these modeling goals, real-time realistic representation is essential, but we also need interactive and perceptive Virtual Humans to populate the Virtual Worlds. Three levels of modeling should be considered to create these believable Virtual Humans: 1) realistic appearance modeling, 2) realistic, smooth and flexible motion modeling, and 3) realistic high-level behaviors modeling. At first, the issues of creating virtual humans with better skeleton and realistic deformable bodies are illustrated. To give a level of believable behavior, challenges are laid on generating on the fly flexible motion and complex behaviours of Virtual Humans inside their environments using a realistic perception of the environment. Interactivity and group behaviours are also important parameters to create believable Virtual Humans which have challenges in creating believable relationship between real and virtual humans based on emotion and personality, and simulating realistic and believable behaviors of groups and crowds. Finally, issues in generating realistic virtual clothed and haired people are presente

Recreating Daily life in Pompeii

[EN] We propose an integrated Mixed Reality methodology for recreating ancient daily life that features realistic simulations of animated virtual human actors (clothes, body, skin, face) who augment real environments and re-enact staged storytelling dramas. We aim to go further from traditional concepts of static cultural artifacts or rigid geometrical and 2D textual augmentations and allow for 3D, interactive, augmented historical character-based event representations in a mobile and wearable setup. This is the main contribution of the described work as well as the proposed extensions to AR Enabling technologies: a VR/AR character simulation kernel framework with real-time, clothed virtual humans that are dynamically superimposed on live camera input, animated and acting based on a predefined, historically correct scenario. We demonstrate such a real-time case study on the actual site of ancient Pompeii.The work presented has been supported by the Swiss Federal Office for Education and Science and the EU IST programme, in frame of the EU IST LIFEPLUS 34545 and EU ICT INTERMEDIA 38417 projects.Magnenat-Thalmann, N.; Papagiannakis, G. (2010). Recreating Daily life in Pompeii. Virtual Archaeology Review. 1(2):19-23. https://doi.org/10.4995/var.2010.4679OJS192312P. MILGRAM, F. KISHINO, (1994) "A Taxonomy of Mixed Reality Visual Displays", IEICE Trans. Information Systems, vol. E77-D, no. 12, pp. 1321-1329R. AZUMA, Y. BAILLOT, R. BEHRINGER, S. FEINER, S. JULIER, B. MACINTYRE, (2001) "Recent Advances in Augmented Reality", IEEE Computer Graphics and Applications, November/December http://dx.doi.org/10.1109/38.963459D. STRICKER, P. DÄHNE, F. SEIBERT, I. CHRISTOU, L. ALMEIDA, N. IOANNIDIS, (2001) "Design and Development Issues for ARCHEOGUIDE: An Augmented Reality-based Cultural Heritage On-site Guide", EuroImage ICAV 3D Conference in Augmented Virtual Environments and Three-dimensional Imaging, Mykonos, Greece, 30 May-01 JuneW. WOHLGEMUTH, G. TRIEBFÜRST, (2000)"ARVIKA: augmented reality for development, production and service", DARE 2000 on Designing augmented reality environments, Elsinore, Denmark http://dx.doi.org/10.1145/354666.354688H. TAMURA, H. YAMAMOTO, A. KATAYAMA, (2001) "Mixed reality: Future dreams seen at the border between real and virtual worlds", Computer Graphics and Applications, vol.21, no.6, pp.64-70 http://dx.doi.org/10.1109/38.963462M. PONDER, G. PAPAGIANNAKIS, T. MOLET, N. MAGNENAT-THALMANN, D. THALMANN, (2003) "VHD++ Development Framework: Towards Extendible, Component Based VR/AR Simulation Engine Featuring Advanced Virtual Character Technologies", IEEE Computer Society Press, CGI Proceedings, pp. 96-104 http://dx.doi.org/10.1109/cgi.2003.1214453Archaeological Superintendence of Pompeii (2009), http://www.pompeiisites.orgG. PAPAGIANNAKIS, S. SCHERTENLEIB, B. O'KENNEDY , M. POIZAT, N.MAGNENAT-THALMANN, A. STODDART, D.THALMANN, (2005) "Mixing Virtual and Real scenes in the site of ancient Pompeii",Journal of CAVW, p 11-24, Volume 16, Issue 1, John Wiley and Sons Ltd, FebruaryEGGES, A., PAPAGIANNAKIS, G., MAGNENAT-THALMANN, N., (2007) "Presence and Interaction in Mixed Reality", The Visual Computer, Springer-Verlag Volume 23, Number 5, MaySEO H., MAGNENAT-THALMANN N. (2003), An Automatic Modeling of Human Bodies from Sizing Parameters. In ACM SIGGRAPH, Symposium on Interactive 3D Graphics, pp19-26, pp234. http://dx.doi.org/10.1145/641480.641487VOLINO P., MAGNENAT-THALMANN N. (2006), Resolving Surface Collisions through Intersection Contour Minimization. In ACM Transactions on Graphics (Siggraph 2006 proceedings), 25(3), pp 1154-1159. http://dx.doi.org/10.1145/1179352.1142007http://dx.doi.org/10.1145/1141911.1142007PAPAGIANNAKIS, G., SINGH, G., MAGNENAT-THALMANN, N., (2008) "A survey of mobile and wireless technologies for augmented reality systems", Journal of Computer Animation and Virtual Worlds, John Wiley and Sons Ltd, 19, 1, pp. 3-22, February http://dx.doi.org/10.1002/cav.22

On the simulation of interactive non-verbal behaviour in virtual humans

Development of virtual humans has focused mainly in two broad areas - conversational agents and computer game characters. Computer game characters have traditionally been action-oriented - focused on the game-play - and conversational agents have been focused on sensible/intelligent conversation. While virtual humans have incorporated some form of non-verbal behaviour, this has been quite limited and more importantly not connected or connected very loosely with the behaviour of a real human interacting with the virtual human - due to a lack of sensor data and no system to respond to that data. The interactional aspect of non-verbal behaviour is highly important in human-human interactions and previous research has demonstrated that people treat media (and therefore virtual humans) as real people, and so interactive non-verbal behaviour is also important in the development of virtual humans. This paper presents the challenges in creating virtual humans that are non-verbally interactive and drawing corollaries with the development history of control systems in robotics presents some approaches to solving these challenges - specifically using behaviour based systems - and shows how an order of magnitude increase in response time of virtual humans in conversation can be obtained and that the development of rapidly responding non-verbal behaviours can start with just a few behaviours with more behaviours added without difficulty later in development

Developing serious games for cultural heritage: a state-of-the-art review

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been less well considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in entertainment games technology. As a result, the field of serious heritage games concerns itself with recent advances in computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On the other hand, the main strengths of serious gaming applications may be generalised as being in the areas of communication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In this report, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in serious heritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several case studies illustrating the application of methods and technologies used in cultural heritage are presented

Serious Games in Cultural Heritage

Although the widespread use of gaming for leisure purposes has been well documented, the use of games to support cultural heritage purposes, such as historical teaching and learning, or for enhancing museum visits, has been less well considered. The state-of-the-art in serious game technology is identical to that of the state-of-the-art in entertainment games technology. As a result the field of serious heritage games concerns itself with recent advances in computer games, real-time computer graphics, virtual and augmented reality and artificial intelligence. On the other hand, the main strengths of serious gaming applications may be generalised as being in the areas of communication, visual expression of information, collaboration mechanisms, interactivity and entertainment. In this report, we will focus on the state-of-the-art with respect to the theories, methods and technologies used in serious heritage games. We provide an overview of existing literature of relevance to the domain, discuss the strengths and weaknesses of the described methods and point out unsolved problems and challenges. In addition, several case studies illustrating the application of methods and technologies used in cultural heritage are presented

A proposed framework of an interactive semi-virtual environment for enhanced education of children with autism spectrum disorders

Education of people with special needs has recently been considered as a key element in the field of medical education. Recent development in the area of information and communication technologies may enable development of collaborative interactive environments which facilitate early stage education and provide specialists with robust tools indicating the person's autism spectrum disorder level. Towards the goal of establishing an enhanced learning environment for children with autism this paper attempts to provide a framework of a semi-controlled real-world environment used for the daily education of an autistic person according to the scenarios selected by the specialists. The proposed framework employs both real-world objects and virtual environments equipped with humanoids able to provide emotional feedback and to demonstrate empathy. Potential examples and usage scenarios for such environments are also described