The virtual human technique already can provide vivid and believable human behaviour in more and more scenarios. Virtual humans are expected to replace real humans in hazardous situations to undertake tests and feed back valuable information. This paper will introduce a virtual human with a novel collision-based synthetic vision, short-term memory model and a capability to implement the emotion calculation and decision making. The virtual character based on this model can ‘see’ what is in his field of view (FOV) and remember those objects. After that, a group of affective computing equations have been introduced. These equations have been implemented into a proposed emotion calculation process to enlighten emotion for virtual intelligent huma

Kang, J

Wright, D K

Zhao, Y

English

Brunel University Research Archive

SYNTHETIC VISION AND EMOTION CALCULATION IN INTELLIGENT VIRTUAL HUMAN MODELING Y. Zhao, J. Kang and D. K. Wright School of Engineering and Design, Brunel University, Uxbridge, Middlesex, UB8 3PH, UK ABSTRACT The virtual human technique already can provide vivid and believable human behaviour in more and more scenarios. Virtual humans are expected to replace real humans in hazardous situations to undertake tests and feed back valuable information. This paper will introduce a virtual human with a novel collision-based synthetic vision, short-term memory model and a capability to implement the emotion calculation and decision making. The virtual character based on this model can ‘see’ what is in his field of view (FOV) and remember those objects. After that, a group of affective computing equations have been introduced. These equations have been implemented into a proposed emotion calculation process to enlighten emotion for virtual intelligent human. Keywords: Virtual Human, Field of Vision, Synthetic Vision, Emotion Calculation INTRODUCTION Based on fast development of last decade, nowadays, the research into virtual human modeling has been wildly accepted. The virtual intelligent human technique already can provide vivid and believable human behaviour in more and more scenarios. Virtual humans are expected to replace real humans in hazardous situations to undertake tests and feed back valuable information. Furthermore, Research into intelligent virtual humans also has been applied in computer game design, ergonomic simulation, security training and film industry.   The main purpose of this paper is to build a multi-applied virtual human with realistic synthetic vision and short-term memory model. This paper also proposes a possible way to implement the emotion calculation and decision making for the virtual human model [1]. A novel collision-based synthetic vision combined with a vision-based short-term memory system has firstly been developed. The virtual character based on this model can ‘see’ what is in his field of view (FOV) and remember those objects. After that, a group of affective computing equations have been introduced. These equations have been implemented into a proposed emotion calculation process [2] to enlighten emotion for virtual intelligent human. METHODS Synthetic Vision and Memory Based on an understanding of human modeling [2], Synthetic perception is the important bridge between a virtual character and its virtual environment. As the majority of information collected by humans is through vision, we consider synthetic vision as the first choice when simulating more realistic virtual human behaviour in a virtual environment.    An important notion in the synthetic vision is the field of vision (FOV). We notice that the FOV is actually an irregular conical shape. The FOV of human is normally a volume projects forward from the face of the virtual character with 60 degrees above the horizon, 75 degrees under the horizon, 100 degrees to both the left and right hand sides when the virtual character is looking horizontally forward [3]. A tapered volume is modeled to simulate human’s field of vision in our research to indicate this irregular conical shape. For a graphical illustration, see Figure 1. It is necessary to point out that the FOV volume is a transparent during our simulation.  Fig.1 Field of Vision for Virtual Human Modeling The tapered FOV volume extends to an effective radius fovR . The radius fovR  should be influenced by the visibility in the environment. We set a visibility variable xV  for the FOV volume.   fov x fovR V R′ = i     xV ( 1)≤   (eq.1) When virtual objects collide into the FOV volume in the virtual environment, those objects in the FOV volume will be stored into a stack. This is also why it is called collision-based synthetic vision.   Initially, we assume there is no object in FOV volume. The visibility of an object to the virtual character could be simply presented as the time of collision between FOV and the object: if the time of collision is an odd number, then the object must be inside the FOV and visible for the virtual character; if the time of collision for a certain object and FOV is an even number, then the object must be outside the FOV and invisible for the virtual character.   Secondly, it is necessary to get the coordinates of a virtual character’s head and objects. For the synthetic 3D space been built, it is easy to get the coordinates of all objects and characters. The corresponding direction vector from a virtual character to any object can be calculated based on those coordinates. If the direction vector from a virtual character to one object is similar to the direction vector to another object, then the distance from a virtual character to each of the object will be calculated. The object with the shorter distance will be treated as visible. The object with a longer distance will shoot rays from its vertexes to the peak of the FOV volume. If one of the rays does not intersect any other object in the 3D space, the object is visible for the virtual character. For example, there is a desk in a virtual room and a chair behind it. Both the desk and chair are in the FOV volume. However, because the chair is behind the desk and small enough, the chair is invisible for the virtual character. Therefore, the chair should be stored in stack. The method is an improvement of Tu’s occlusion test for visibility of seaweed [4].                               Compared with the methods which analyses pixels of a rendered image from a virtual character’s point of view [5, 6, 7], the collision-based synthetic vision model proposed here is biologically more faithful as compared to the view of a real human. Compared with the computing visibility method [4], the collision-based synthetic vision model proposed here is simpler and requires less time to calculate. It provides more reasonable and accurate synthetic vision for virtual human in a virtual environment.   A time tolerance constraint is also added to the stack. It helps to build virtual character’s short-term memory. Especially, the constraint gives our virtual character a capability to remember those objects which have left its FOV for a short period. This function will make virtual character’s behavior more realistic when it reactive relies on its own memory and experience. The process of virtual human’s short-term memory modeling is shown in Figure 2.  Fig.2 The flow chart for virtual human’s short-term memory Emotion calculation Emotion is a certain form of cognitive reflection to an environment. It is a subjective and attitudinal experience of whether environmental factors are meeting the human’s desires. It can be interpreted that human consciousness is a reflection of entities in an environment that is inhabitanted by the subject. The item entity here means the object of existence or an environment factor. As a kind of subjective consciousness, emotions also should have their own corresponding entities.   “Whether environmental factors are meeting the human’s desires” is a typical evaluation problem. The standard for this evaluation is whether the human’s desires are met. So, if we could find the way to measure “meeting human’s desires”, we also could find the corresponding entities of emotions.  Ortony had proposed an emotion theory and listed a group of input factors which may affect the intensity of emotion [8]. We assume that those effective factors are measurable and appropriate to evaluate whether the human’s desires are met. Therefore, those input factors can also be used to represent the corresponding entities of emotions.  We noticed that the intensity of one-dimensional emotion ( ) does not changing linearly with the value of input factor (V ). It actually logarithmic increases when the value difference enlarging ( ).  1EVΔ 1 log(1 )E A V= + Δ  (eq.2) VΔ means the value difference between the input factor and its reference. Also know as, the value of human’s desire: . is the reference value of input factor. A is the intensity index of emotion ( ) /r rV V V VΔ = − rV This equation can explain why the change of human emotion seems sensitive and accurate when the intensity of one’s emotion is relatively week. However, human could hardly feel the changes of emotion when the intensity of emotion reaches to a certain level. A case in point, we normally feel more annoyed Emotion Calculation FOV Volume Stack Visibility Test123Time Toleranceabout lost a car than a magazine. But we might not be able to experience the difference of more angry when we are already in fury.  The intensity of emotion is declined by time. E is the intensity of an emotion. is the initial intensity of the emotion. is the decline index. T  is duration of the emotion. 0Etd 0 exp( )tE E d T= −       (eq.3) 0T >For multi-dimensional emotions, the details of implement have been introduced in previous paper [2]. RESULTS A 7 cameras optical motion capture system is conducted for recording whole body movement. 40 markers are attached on human body for getting enough details of motion, as shown in Fig 1. Some influential parameters to motion capture have been chosen to classify individual motion. Such as, age, gender, stature, emotion, personality, etc. Furthermore, motion examples are carefully designed in order to get more expressive emotional motions through each individual’s body language.   3D body surface model has been built s of each previous motion-captured individual. A 3D body scanner from , USA is used to create realistic human body surface. Geomagic Studio 8 is used for further refinement of the 3D body surface. 2[ ]TC The body surface model then combined with motion capture data to generate vivid, realistic virtual human. Figure 3 show some examples of virtual human generated though the human modeling method [9].     Fig.3 virtual human with postures A 3D virtual indoor environment has been built in Virtools™ to test the proposed virtual human modelling approach. For a simplified example scenario, see Figure 4.  Fig.4 A virtual indoor environment Example Scenario Virtual character Eva is standing in a lounge. Initially, she is wandering in the lounge and detects the environment through her collision-based synthetic vision. While wandering, Eva will store objects in her short-term memory stack according to the principles mentioned earlier. What is stored in those three stacks will be used to generate inputs for processes which affect different aspects of virtual character’s reactions. For example, the attention process, emotion calculation process and decision making process. Those processes will be looped as the virtual character changes its position to provide real-time synthetic vision, short-term memory, emotion states and decision.  One criterion of validation of our method was to store objects in corresponding stacks correctly in real-time. Figure 5 demonstrates four possible situations in the virtual environment and the corresponding states of the short-term memory stack. In Case 1, a virtual character, Peter, can see both Eva and an object; in Case 2, an object is blocked by Eva, however, the time tolerance allows the object to exist in Peter’s short-term memory stack for a short period, even if the object has left his FOV volume as in Case 4; Case 3 shows the object will be deleted from his short-term memory stack if the time tolerance expired.          Case1                                        Case 2                                        Case 3                                        Case4 Fig.5 Four possible situations and corresponding states in short-term memory stack After synthetic vision, virtual character is required to generate emotional reaction based on his/her memory stack. An emotion calculation process is designed to evaluate virtual character’s emotion state in VIrtools™ , as shown in Figure 6. Those emotion states will be used to trigger emotional classified motion database.  Fig.6 A part of emotion calculation processing CONCLUSIONS This paper proposed a possible method to supply inputs for emotion calculation and decision making processes to our virtual human modeling research. A short-term memory stack structure based on a novel collision-based synthetic vision method has been introduced and simulated.  The novel synthetic vision method detects whether an object was in a virtual observer’s field of vision only by the collision times between the object and the FOV volume. It does not decide the visibility of all objects through calculating the distance from the observer to each vertex of object. This is quite suitable for a real-time virtual reality system.  On the other hand, the approach still has aspects which need to be improved. For example, long-term memory is not considered in this approach. Also, the synthetic vision in our approach is still improvable, etc. All these aspects will be the objects for future research.  The approaches presented here are integrated into a more complex intelligent virtual human model based on Virtools™. The target of modelling such an intelligent virtual character is to build a flexible intelligent virtual human model, including behaviour choice and other more complex emotion-affected decision making processes. ACKNOWLEDGMENTS Acknowledgments REFERENCES [1] J. Kang, D.K. Wright, S. F. Qin, Y.Zhao, “Modeling Human Behaviours and Reactions Under Dangerous Environment,” Proceedings of the 42nd Annual Rocky Mountain Bioengineering Symposium, 8-10 April 2005. [2] Y.Zhao, J. Kang, D.K. Wright, “Emotion-Affected Decision Making in Human Simulation,” Proceedings of the 43rd Annual Rocky Mountain Bioengineering Symposium, 7-9 April 2006. [3] A. P. Spense, E. B. Mason, “Human Anatony and Physiology,” Fourth Edition, 1992. West Pubbishing Company. [4] X. Tu, “Perception Modeling for Behavioral Animation of Fishes,” Proc. of the 2nd Pacific Conference on Computer Graphics, Beijing, China, 1994. [5] H. Noser, D. Thalmann, “Synthetic vision and audition for digital actors,” Computer Graphics Forum, 1995. 14(3): p. 325-336. [6] James J. Kuffner, Jean-Claude Latombe, “Fast Synthetic Vision, Memory, and Learning Models for Virtual Humans,” Computer Animation Proceedings, 1999: p. 118-127. [7] N. Courty, “A New Application for Saliency Maps: Synthetic Vision of Autonomous Actors,” IEEE Int. Conf. on Image Processing, ICIP'03 Spain, September 2003. [8] A. Ortony, G. Clore, A. Collins, “The Cognitive Structure of Emotions,” Cambridge University Press, 1988 [9] J. Kang, B. Badi, Y. Zhao, D. K. Wright, “Human Motion Modelling and Simulation by Anatomical Approach”, WSEAS Transacions on Computers, Issue 6, Vol 5, June 2006. 

A New Application for Saliency Maps: Synthetic Vision of Autonomous Actors,”

Emotion-Affected Decision Making in Human Simulation,”

Human Anatony and Physiology,” Fourth Edition,

Human Motion Modelling and Simulation by Anatomical Approach”,

Jean-Claude Latombe, “Fast Synthetic Vision, Memory, and Learning Models for Virtual Humans,” Computer Animation Proceedings,

Modeling Human Behaviours and Reactions Under Dangerous Environment,”

Perception Modeling for Behavioral Animation of Fishes,”

Synthetic vision and audition for digital actors,” Computer Graphics Forum,

The Cognitive Structure of Emotions,”

Synthetic vision and emotion calculation in intelligent virtual human modeling

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.132.9044

Synthetic vision and emotion calculation in intelligent virtual human modeling

Abstract

Similar works

Full text

Available Versions

Brunel University Research Archive