Head-coupled perspective in computer games

This work presents the new method of interaction in computer application, especially firstand third-person player games. The new control technique called “Head-Aim” is based on Head-Coupled Perspective imagery method and can be used as an independent method of aiming in games as alternative to conventional targeting (with use of popular control devices like computer mouse or gamepad) or combined with them, extending their functionality and interaction realism. Proposed pointing method can also be used in non-game application to allow people control them even if they are unable to use regular control devices. Effectiveness of Head-Aim technique was compared with convetional methods of aiming in specially designed game named “Facepace”. Promising results of these tests are presented. Also a robust, fast and computationally cheap method of head tracking is proposed. Presented algorithm combines background subtrakction and new color-based tracking method enabling tracking user’s head under various lightning conditions. Method allows Head-Coupled Perspective technique to be performed without latencies even with use of a low quality webcam and enables viewing scene in 3D (and use of presented Head Aim method) on every machine, without use of glasses or special devices

Acoustic chase : designing an interactive audio environment to stimulate human body movement

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2004.Includes bibliographical references (p. 58-60).An immersive audio environment was created that explores how humans react to commands imposed by a machine generating its acoustic stimuli on the basis of tracked body movement. In this environment, different states of human and machine action are understood as a balance of power that moves back and forth between the apparatus and the human being. This system is based on spatial sounds that are designed to stimulate body movements. The physical set-up consists of headphones with attached sensors to pick up the movements of the head. Mathematic models calculate the behavior of the sound, its virtual motion path relative to the person, and how it changes over time.by Simon Karl Josef Schiessl.S.M

A Virtual Testbed for Fish-Tank Virtual Reality: Improving Calibration with a Virtual-in-Virtual Display

With the development of novel calibration techniques for multimedia projectors and curved projection surfaces, volumetric 3D displays are becoming easier and more affordable to build. The basic requirements include a display shape that defines the volume (e.g. a sphere, cylinder, or cuboid) and a tracking system to provide each user's location for the perspective corrected rendering. When coupled with modern graphics cards, these displays are capable of high resolution, low latency, high frame rate, and even stereoscopic rendering; however, like many previous studies have shown, every component must be precisely calibrated for a compelling 3D effect. While human perceptual requirements have been extensively studied for head-tracked displays, most studies featured seated users in front of a flat display. It remains unclear if results from these flat display studies are applicable to newer, walk-around displays with enclosed or curved shapes. To investigate these issues, we developed a virtual testbed for volumetric head-tracked displays that can measure calibration accuracy of the entire system in real-time. We used this testbed to investigate visual distortions of prototype curved displays, improve existing calibration techniques, study the importance of stereo to performance and perception, and validate perceptual calibration with novice users. Our experiments show that stereo is important for task performance, but requires more accurate calibration, and that novice users can make effective use of perceptual calibration tools. We also propose a novel, real-time calibration method that can be used to fine-tune an existing calibration using perceptual feedback. The findings from this work can be used to build better head-tracked volumetric displays with an unprecedented amount of 3D realism and intuitive calibration tools for novice users

Convergência de games e realidade virtual para treinamento de manutenção em redes de energia em linha viva

Orientador : Prof. Dr. Klaus de GeusCoorienntador: Prof. Dr.Sérgio ScheerTese (doutorado) - Universidade Federal do Paraná, Setor de Ciencias Exatas e Setor de Tecnologia, Programa de Pós-Graduação em Métodos Numéricos em Engenharia. Defesa: Curitiba, 2011Bibliografia: fls. 116-135Área de concentração: Mecanica computacionalResumo: T_ecnicas de manuten_c~ao em linha viva s~ao aquelas desempenhadas em redes de distribui _c~ao de energia sem que haja interrup_c~ao no servi_co. Essa pr_atica evita custos e transtornos, por_em _e considerada de alto risco. Assim, a efetividade de treinamento pr_evio _e de grande import^ancia e, portanto, constantes esfor_cos v^em sendo feitos no sentido de buscar recursos que possam melhorar a reten_c~ao de conhecimento e a qualidade da informa_c~ao a ser adquirida. Neste sentido, novas tecnologias tais como aquelas utilizadas nos modernos videogames e em sistemas de Realidade Virtual podem proporcionar experi^encias pr_aticas que outros mecanismos de aprendizado di_cilmente contemplariam, adicionando vantagens relacionadas a seguran_ca e custo. Este trabalho apresenta o desenvolvimento de um sistema de treinamento de atividades em linha viva, baseado nas tecnologias de Games e Realidade Virtual. Aspectos relativos ao desenvolvimento de uma solu_c~ao abrangente, contemplando todos os dispositivos de hardware para visualiza_c~ao e intera_c~ao, al_em do conjunto de sistemas de software utilizados, s~ao discutidos com foco em quest~oes tais como custo, facilidade de acesso e manuten_c~ao, tend^encias de mercado, facilidade de uso, aprendizado e durabilidade do equipamento. A pesquisa que fundamenta a proposta da plataforma _nal fornece um levantamento de caracter__sticas e limita_c~oes que at_e o presente momento t^em posto empecilhos ao uso mais amplo e abrangente de sistemas baseados em Games e Realidade Virtual para treinamento de atividades cr__ticas. Aspectos chave relacionados a usabilidade, ergonomia, design instrucional e interface de sistema, al_em do comportamento humano e do sistema cognitivo, entre outros, s~ao identi_cados e tratados. O desenvolvimento proporcionou como resultado pr_atico o prot_otipo de uma plataforma para treinamento de atividades em linha viva, a qual utiliza dispositivos como o Nintendo Wii RemoteR e televisores estereosc_opicos para prover um novo modelo de intera _c~ao e visualiza_c~ao focado na aplica_c~ao contemplada. Um cen_ario completo, incluindo modelos geom_etricos precisos, foram criados e integrados ao prot_otipo, o qual permite a simula_c~ao completa de uma atividade recorrente de manuten_c~ao em linha viva. Funcionalidades foram implementadas incluindo o comportamento f__sico dos objetos da cena, um modelo de navega_c~ao e visualiza_c~ao, e um modelo de sele_c~ao e manipula_c~ao de objetos virtuais. Esses dois modelos complementam-se em um mecanismo de intera_c~ao com o ambiente virtual inspirado em conceitos da computa_c~ao ub__qua e interfaces tang__veis. Uma proposta de avalia_c~ao preliminar do sistema desenvolvido _e apresentada juntamente com os resultados obtidos em sua aplica_c~ao a pro_ssionais da _area. Por _m, as contribui_c~oes cient___cas focadas no aprendizado mediante tecnologias de Games e Realidade Virtual s~ao discutidas.Abstract: Live line maintenance techniques live line are those performed in power distribution networks without the need for service interruption. This practice avoids costs and inconveniences, but is considered a high risk activity. Thus, the e_ectiveness of prior training is of great importance and, therefore, constant e_orts have been made to _nd mechanisms that can improve knowledge retention and the quality of information to be acquired. In this respect, new technologies such as those used in modern video games and virtual reality systems can provide practical experiences that other learning mechanisms hardly account for, adding advantages related to safety and cost. This work presents the development of a training system for live line activities, based on the paradigms of Games and Virtual Reality. Aspects related to the development of a comprehensive, which accounts for all hardware devices used for visualization and interaction, as well as all software systems used, are discussed. focusing on issues such as cost, ease of access and maintenance, market trends, ease of use, learning and durability. The research that supports the proposal of the _nal platform provides an overview of characteristics and limitations that until now have put obstacles to a wider and more comprehensive use of systems based on Games and Virtual Reality for training on critical activities. Key aspects related to usability, ergonomics, instructional design and system interface, as well as human behaviour and cognitive system, among others, are identi_ed and treated. The development of this work has provided, as practical result, a prototype of a training platform for live line activities, which uses devices such as Nintendo Wii Remote R and stereoscopic television sets to provide a new model of interaction and visualization focused on the application. Geometric models were created and integrated into the prototype, which allows full simulation of a recurring activity in live line maintenance. Features have been implemented including the physical behaviour of objects in the scene, a model for navigation and viewing, and a model for selection and manipulation of virtual objects. These two models complement each other, making up an interaction mechanism inspired on concepts of ubiquitous computing and tangible interfaces. A proposal for a preliminary evaluation procedure for the developed system is presented together with results obtained after its application to end users. Finally, the scienti_c contributions focused on technology-based learning through Games and Virtual Reality are discussed

3D Face Tracking Using Stereo Cameras with Whole Body View

All visual tracking tasks associated with people tracking are in a great demand for modern applications dedicated to make human life easier and safer. In this thesis, a special case of people tracking - 3D face tracking in whole body view video is explored. Whole body view video means that the tracked face typically occupies not more than 5-10% of the frame area. Currently there is no reliable tracker that can track a face in long-term whole body view videos with luminance cameras in the 3D space. I followed a non-classical approach to designing a 3D tracker: first a 2D face tracking algorithm was developed in one view and then extended into stereo tracking. I recorded and annotated my own extensive dataset specifically for 2D face tracking in whole body view video and evaluated 17 state of the art 2D tracking algorithms. Based on the TLD tracker, I developed a face adapted median flow tracker that shows superior results compared to state of the art generic trackers. I explored different ways of extending 2D tracking into 3D and developed a method of using the epipolar constraint to check consistency of 3D tracking results. This method allows to detect tracking failures early and improves overall 3D tracking accuracy. I demonstrated how a Kinect based method can be compared to visual tracking methods and compared four different visual tracking methods running on low resolution fisheye stereo video and the Kinect face tracking application. My main contributions are: - I developed a face adaptation of generic trackers that improves tracking performance in long-term whole body view videos. - I designed a method of using the epipolar constraint to check consistency of 3D tracking results

Exploring 3D User Interface Technologies for Improving the Gaming Experience

3D user interface technologies have the potential to make games more immersive & engaging and thus potentially provide a better user experience to gamers. Although 3D user interface technologies are available for games, it is still unclear how their usage affects game play and if there are any user performance benefits. A systematic study of these technologies in game environments is required to understand how game play is affected and how we can optimize the usage in order to achieve better game play experience. This dissertation seeks to improve the gaming experience by exploring several 3DUI technologies. In this work, we focused on stereoscopic 3D viewing (to improve viewing experience) coupled with motion based control, head tracking (to make games more engaging), and faster gesture based menu selection (to reduce cognitive burden associated with menu interaction while playing). We first studied each of these technologies in isolation to understand their benefits for games. We present the results of our experiments to evaluate benefits of stereoscopic 3D (when coupled with motion based control) and head tracking in games. We discuss the reasons behind these findings and provide recommendations for game designers who want to make use of these technologies to enhance gaming experiences. We also present the results of our experiments with finger-based menu selection techniques with an aim to find out the fastest technique. Based on these findings, we custom designed an air-combat game prototype which simultaneously uses stereoscopic 3D, head tracking, and finger-count shortcuts to prove that these technologies could be useful for games if the game is designed with these technologies in mind. Additionally, to enhance depth discrimination and minimize visual discomfort, the game dynamically optimizes stereoscopic 3D parameters (convergence and separation) based on the user\u27s look direction. We conducted a within subjects experiment where we examined performance data and self-reported data on users perception of the game. Our results indicate that participants performed significantly better when all the 3DUI technologies (stereoscopic 3D, head-tracking and finger-count gestures) were available simultaneously with head tracking as a dominant factor. We explore the individual contribution of each of these technologies to the overall gaming experience and discuss the reasons behind our findings. Our experiments indicate that 3D user interface technologies could make gaming experience better if used effectively. The games must be designed to make use of the 3D user interface technologies available in order to provide a better gaming experience to the user. We explored a few technologies as part of this work and obtained some design guidelines for future game designers. We hope that our work will serve as the framework for the future explorations of making games better using 3D user interface technologies

Development of the components of a low cost, distributed facial virtual conferencing system

This thesis investigates the development of a low cost, component based facial virtual conferencing system. The design is decomposed into an encoding phase and a decoding phase, which communicate with each other via a network connection. The encoding phase is composed of three components: model acquisition (which handles avatar generation), pose estimation and expression analysis. Audio is not considered part of the encoding and decoding process, and as such is not evaluated. The model acquisition component is implemented using a visual hull reconstruction algorithm that is able to reconstruct real-world objects using only sets of images of the object as input. The object to be reconstructed is assumed to lie in a bounding volume of voxels. The reconstruction process involves the following stages: - Space carving for basic shape extraction; - Isosurface extraction to remove voxels not part of the surface of the reconstruction; - Mesh connection to generate a closed, connected polyhedral mesh; - Texture generation. Texturing is achieved by Gouraud shading the reconstruction with a vertex colour map; - Mesh decimation to simplify the object. The original algorithm has complexity O(n), but suffers from an inability to reconstruct concave surfaces that do not form part of the visual hull of the object. A novel extension to this algorithm based on Normalised Cross Correlation (NCC) is proposed to overcome this problem. An extension to speed up traditional NCC evaluations is proposed which reduces the NCC search space from a 2D search problem down to a single evaluation. Pose estimation and expression analysis are performed by tracking six fiducial points on the face of a subject. A tracking algorithm is developed that uses Normalised Cross Correlation to facilitate robust tracking that is invariant to changing lighting conditions, rotations and scaling. Pose estimation involves the recovery of the head position and orientation through the tracking of the triangle formed by the subject's eyebrows and nose tip. A rule-based evaluation of points that are tracked around the subject's mouth forms the basis of the expression analysis. A user assisted feedback loop and caching mechanism is used to overcome tracking errors due to fast motion or occlusions. The NCC tracker is shown to achieve a tracking performance of 10 fps when tracking the six fiducial points. The decoding phase is divided into 3 tasks, namely: avatar movement, expression generation and expression management. Avatar movement is implemented using the base VR system. Expression generation is facilitated using a Vertex Interpolation Deformation method. A weighting system is proposed for expression management. Its function is to gradually transform from one expression to the next. The use of the vertex interpolation method allows real-time deformations of the avatar representation, achieving 16 fps when applied to a model consisting of 7500 vertices. An Expression Parameter Lookup Table (EPLT) facilitates an independent mapping between the two phases. It defines a list of generic expressions that are known to the system and associates an Expression ID with each one. For each generic expression, it relates the expression analysis rules for any subject with the expression generation parameters for any avatar model. The result is that facial expression replication between any subject and avatar combination can be performed by transferring only the Expression ID from the encoder application to the decoder application. The ideas developed in the thesis are demonstrated in an implementation using the CoRgi Virtual Reality system. It is shown that the virtual-conferencing application based on this design requires only a bandwidth of 2 Kbps.Adobe Acrobat Pro 9.4.6Adobe Acrobat 9.46 Paper Capture Plug-i

Development of the components of a low cost, distributed facial virtual conferencing system

This thesis investigates the development of a low cost, component based facial virtual conferencing system. The design is decomposed into an encoding phase and a decoding phase, which communicate with each other via a network connection. The encoding phase is composed of three components: model acquisition (which handles avatar generation), pose estimation and expression analysis. Audio is not considered part of the encoding and decoding process, and as such is not evaluated. The model acquisition component is implemented using a visual hull reconstruction algorithm that is able to reconstruct real-world objects using only sets of images of the object as input. The object to be reconstructed is assumed to lie in a bounding volume of voxels. The reconstruction process involves the following stages: - Space carving for basic shape extraction; - Isosurface extraction to remove voxels not part of the surface of the reconstruction; - Mesh connection to generate a closed, connected polyhedral mesh; - Texture generation. Texturing is achieved by Gouraud shading the reconstruction with a vertex colour map; - Mesh decimation to simplify the object. The original algorithm has complexity O(n), but suffers from an inability to reconstruct concave surfaces that do not form part of the visual hull of the object. A novel extension to this algorithm based on Normalised Cross Correlation (NCC) is proposed to overcome this problem. An extension to speed up traditional NCC evaluations is proposed which reduces the NCC search space from a 2D search problem down to a single evaluation. Pose estimation and expression analysis are performed by tracking six fiducial points on the face of a subject. A tracking algorithm is developed that uses Normalised Cross Correlation to facilitate robust tracking that is invariant to changing lighting conditions, rotations and scaling. Pose estimation involves the recovery of the head position and orientation through the tracking of the triangle formed by the subject's eyebrows and nose tip. A rule-based evaluation of points that are tracked around the subject's mouth forms the basis of the expression analysis. A user assisted feedback loop and caching mechanism is used to overcome tracking errors due to fast motion or occlusions. The NCC tracker is shown to achieve a tracking performance of 10 fps when tracking the six fiducial points. The decoding phase is divided into 3 tasks, namely: avatar movement, expression generation and expression management. Avatar movement is implemented using the base VR system. Expression generation is facilitated using a Vertex Interpolation Deformation method. A weighting system is proposed for expression management. Its function is to gradually transform from one expression to the next. The use of the vertex interpolation method allows real-time deformations of the avatar representation, achieving 16 fps when applied to a model consisting of 7500 vertices. An Expression Parameter Lookup Table (EPLT) facilitates an independent mapping between the two phases. It defines a list of generic expressions that are known to the system and associates an Expression ID with each one. For each generic expression, it relates the expression analysis rules for any subject with the expression generation parameters for any avatar model. The result is that facial expression replication between any subject and avatar combination can be performed by transferring only the Expression ID from the encoder application to the decoder application. The ideas developed in the thesis are demonstrated in an implementation using the CoRgi Virtual Reality system. It is shown that the virtual-conferencing application based on this design requires only a bandwidth of 2 Kbps.Adobe Acrobat Pro 9.4.6Adobe Acrobat 9.46 Paper Capture Plug-i