Using CNNs For Users Segmentation In Video See-Through Augmented Virtuality

In this paper, we present preliminary results on the use of deep learning techniques to integrate the users self-body and other participants into a head-mounted video see-through augmented virtuality scenario. It has been previously shown that seeing users bodies in such simulations may improve the feeling of both self and social presence in the virtual environment, as well as user performance. We propose to use a convolutional neural network for real time semantic segmentation of users bodies in the stereoscopic RGB video streams acquired from the perspective of the user. We describe design issues as well as implementation details of the system and demonstrate the feasibility of using such neural networks for merging users bodies in an augmented virtuality simulation.Comment: 6 pages, 6 figures. Published in the 2nd International Conference on Artificial Intelligence & Virtual Reality (IEEE AIVR 2019

Enhancing presence in head-mounted display environments by visual body feedback using head-mounted cameras

Abstract—A fully-articulated visual representation of a user in an immersive virtual environment (IVE) can enhance the user’s subjective sense of feeling present in the virtual world. Usually this requires the user to wear a full-body motion capture suit to track real-world body movements and to map them to a virtual body model. In this paper we present an augmented virtuality approach that allows to incorporate a realistic view of oneself in virtual environments using cameras attached to head mounted displays. The described system can easily be integrated into typical virtual reality setups. Egocentric camera images captured by a video-see-through system are segmented in real-time into foreground, showing parts of the user’s body, e. g., her hands or feet, and background. The segmented foreground is then displayed as inset in the user’s current view of the virtual world. Thus the user is able to see her physical body in an arbitrary virtual world, including individual characteristics such as skin pigmentation and hairiness. Keywords-Head-mounted displays; augmented virtuality; presence I

Aesthetiography : The next Milestone in the Confluence of Media

Art and technology always evolved simultaneously, often inspiring and complimenting each other. This can be observed at every point in the history of media technologies. We suggest that the evolution of media has always been looking forward to an ultimate form of representation of reality that one could experience - a perfect Simulated Reality. Over the years the forms of media have been evolving and improving in order to create stronger perceptual as well as psychological illusions. Today virtual reality and associated technologies help evoke illusion of reality strong enough to make one believe to be entirely immersed and present in an artificial world. Observing these developments, we believe that the distinct goal of tomorrow’s media would be to create a perfect experience of perceptual illusion with the help of multisensory mediation. In this pursuit of an ultimate representational media, different media technologies will converge. This meeting point of cinema, virtual reality and associated new-media technologies in the near future, is what we would like to refer to as ‘Aesthetiography’ - the art and science of capturing (or creating) and reproducing an absolute perceptual experience. We propose that it would be the next milestone in the confluence of media

Zefektivnění provádění výrobních procesů v rámci organizační přípravy výroby

Disertační práce se zabývá tématem moderními vizualizačními nástroji a jejich využitím v průmyslové praxi, především v rámci technické přípravy výroby. Práce je koncipována tak, že z počátku obsahuje rešerši současného stavu poznání ve zkoumané oblasti. Na základě rešeršní části je definována užší oblast výzkumu a jsou stanoveny cíle a hypotézy práce. V další části je definován postup výzkumu, který vede k návrhu metodiky pro zefektivnění provádění výrobních procesů v rámci organizační přípravy výroby. V rámci návrhu metodiky budou nalezeny charakteristiky výrobních procesů, které mají vliv na výběr vhodného vizualizačního nástroje pro provádění budoucího výrobního procesu. Dále bude zkoumán vztah Proces -Nástroj tak, aby bylo možné sestavit multikriteriální funkci a logiku použití moderních vizualizačních nástrojů. Práce končí ověřením navržené metodiky a definováním přínosů, které tato práce přináší jak v teoretické rovině, tak i v oblasti praktického uplatnění metodiky v průmyslových podnicích. Výsledná metodika by měla sloužit výrobním podnikům jako pomoc při rozhodování o nasazení moderních vizualizačních nástrojů v rámci organizační přípravy výroby.ObhájenoThe dissertation deals with the topic of modern visualization tools and their use in industrial practice, especially in the technical preparation of production. The work is designed in such a way that from the beginning, it contains a survey of the current state of knowledge in the researched area. On the basis of the research part, a narrower area of research is defined and the goals and hypotheses of the work are determined. In the next part, the research procedure is defined, which leads to the design of a methodology for making production processes more efficient as part of the organizational preparation of production. As part of the design of the methodology, the characteristics of the production processes will be found, which influence the selection of a suitable visualization tool for the implementation of the future production process. Furthermore, the Process-Tool relationship will be investigated so that it is possible to construct a multi-criteria function and the logic of using modern visualization tools. The work ends by verifying the proposed methodology and defining the benefits that this work brings both on a theoretical level and in the field of practical application of the methodology in industrial enterprises. The resulting methodology should serve manufacturing companies as an aid in deciding on the deployment of modern visualization tools as part of the organizational preparation of production

Personalized Digital Body: Enhancing Body Ownership and Spatial Presence in Virtual Reality

person\u27s sense of acceptance of a virtual body as his or her own is generally called virtual body ownership (VBOI). Having such a mental model of one\u27s own body transferred to a virtual human surrogate is known to play a critical role in one\u27s sense of presence in a virtual environment. Our focus in this dissertation is on top-down processing based on visual perception in both the visuomotor and the visuotactile domains, using visually personalized body cues. The visual cues we study here range from ones that we refer to as direct and others that we classify as indirect. Direct cues are associated with body parts that play a central role in the task we are performing. Such parts typically dominate a person\u27s foveal view and will include one or both of their hands. Indirect body cues come from body parts that are normally seen in our peripheral view, e.g., legs and torso, and that are often observed through some mediation and are not directly associated with the current task. This dissertation studies how and to what degree direct and indirect cues affect a person\u27s sense of VBOI for which they are receiving direct and, sometimes, inaccurate cues, and to investigate the relationship between enhanced virtual body ownership and task performance. Our experiments support the importance of a personalized representation, even for indirect cues. Additionally, we studied gradual versus instantaneous transition between one\u27s own body and a virtual surrogate body, and between one\u27s real-world environment and a virtual environment. We demonstrate that gradual transition has a significant influence on virtual body ownership and presence. In a follow-on study, we increase fidelity by using a personalized hand. Here, we demonstrate that a personalized hand significantly improves dominant visual illusions, resulting in more accurate perception of virtual object sizes

3D Multi-user interactive visualization with a shared large-scale display

When the multiple users interact with a virtual environment on a largescale display there are several issues that need to be addressed to facilitate the interaction. In the thesis, three main topics for collaborative visualization are discussed; display setup, interactive visualization, and visual fatigue. The problems that the author is trying to address in this thesis are how multiple users can interact with a shared large-scale display depending on the display setups and how they can interact with the shared visualization in a way that doesn’t lead to visual fatigue. The first user study (Chapter 3) explores the display setups for multi-user interaction with a shared large-display. The author describes the design of the three main display setups (a shared view, a split screen, and a split screen with navigation information) and a demonstration using these setups. The user study found that the split screen and the split screen with navigation information can improve users’ confidence and reduce frustration level and are more preferred than a shared view. However, a shared view can still provide effective interaction and collaboration and the display setups cannot have a large impact on usability and workload. From the first study, the author employed a shared view for multi-user interactive visualization with a shared large-scale display due to the advantages of the shared view. To improve interactive visualization with a shared view for multiple users, the author designed and conducted the second user study (Chapter 4). A conventional interaction technique, the mean tracking method, was not effective for more than three users. In order to overcome the limitation of the current multi-user interactive visualization techniques, two interactive visualization techniques (the Object Shift Technique and Activity-based Weighted Mean Tracking method) were developed and were evaluated in the second user study. The Object Shift Technique translates the virtual objects in the opposite direction of movement of the Point of View (PoV) and the Activity-based Weighted Mean Tracking method assigns the higher weight to active users in comparison with stationary users to determine the location of the PoV. The results of the user study showed that these techniques can support collaboration, improve interactivity, and provide similar visual discomfort compared to the conventional method. The third study (Chapter 5) describes how to reduce visual fatigue for 3D stereoscopic visualization with a single point of view (PoV). When multiple users interact with 3D stereoscopic VR using multi-user interactive visualization techniques and they are close to the virtual objects, they can perceive 3D visual fatigue from the large disparity. To reduce the 3D visual fatigue, an Adaptive Interpupillary Distance (Adaptive IPD) adjustment technique was developed. To evaluate the Adaptive IPD method, the author compared to traditional 3D stereoscopic and the monoscopic visualization techniques. Through the user experiments, the author was able to confirm that the proposed method can reduce visual discomfort, yet maintain compelling depth perception as the result provided the most preferable 3D stereoscopic visualization experience. For these studies, the author developed a software framework and designed a set of experiments (Chapter 6). The framework architecture that contains the three main ideas are described. A demonstration application for multidimensional decision making was developed using the framework. The primary contributions of this thesis include a literature review of multiuser interaction with a shared large-scale display, deeper insights into three display setups for multi-user interaction, development of the Object Shift Techniques, the Activity-based Weighted Mean Tracking method, and the Adaptive Interpupillary Distance Adjustment technique, the evaluation of the three novel interaction techniques, development of a framework for supporting a multi-user interaction with a shared large-scale display and its application to multi-dimensional decision making VR system

Creación de experiencias de realidad aumentada realistas por usuarios finales

Programa de Doctorado en Ciencia y Tecnología Informática por la Universidad Carlos III de MadridPresidente: Juan Manuel Dodero Beardo.- Secretario: Andrea Bellucci.- Vocal: Camino Fernández Llama

Réalité évoquée, des rêves aux simulations (un cadre conceptuel de la réalité au regard de la présence)

Dans cette recherche, nous présentons le concept de Réalité Évoquée ( Evoked Reality ) afin d'essayer de relier différentes notions entourant la présence et la réalité au sein d'un cadre commun. Nous introduisons et illustrons le concept en tant que illusion de la réalité (Réalité Évoquée) qui évoque un sentiment de présence (Présence Évoquée) dans nos esprits. Nous distinguons les concepts de Réalité Média-Évoquée et Réalité Auto-Évoquée et nous les définissons clairement. Le concept de Réalité Évoquée nous permet d'introduire un modèle tripolaire de la réalité, qui remet en cause le modèle classique des deux pôles. Nous présentons également un modèle graphique appelé Reality-Presence Map (Carte Réalité-Présence) qui nous permet de localiser et d'analyser toutes les expériences cognitives concernant la présence et la réalité. Nous explorons également les qualia et la subjectivité de nos expériences de Réalité Évoquée. Deux expériences ont été réalisées : l'une dans le domaine de la Réalité Média-Évoquée et l'autre dans celui de l'Auto-Évoquée. Les expériences nous ont permis de valider nos hypothèses et de réaliser que nos recherches empiriques pouvaient encore être poussées plus loin encore. Enfin, nous illustrons les différentes implications et nous examinons les applications et les utilisations possibles de notre concept, en particulier dans le domaine de la recherche sur la présence. En outre, nous proposons d'étendre la recherche sur la présence au-delà du domaine de la réalité virtuelle et des moyens de communication et de l'étudier dans une perspective plus large que celle des sciences cognitives. Nous sommes convaincus que ce concept de Réalité Évoquée et le modèle proposé peuvent avoir des applications significatives dans l'étude de la présence et dans l'exploration des possibilités qui dépassent la réalité virtuelle.In this research, we introduce the concept of "Evoked Reality" in an attempt to bring together various ideas on presence and reality onto a common platform. The concept we propose and illustrate is in fact an 'illusion of reality' (Evoked Realty) that simply evokes a 'sense of presence' (Evoked Presence) in our minds. We clearly define and differentiate between a Media-Evoked and a Self-Evoked Reality. That helped us introduce the Three Pole Reality Model that redefines the classical Two Pole Reality Model. We also present a graphical model called Reality-Presence Map, which would help us locate and analyse every possible cognitive experience relating to presence and reality. We also explore the qualia and subjectivity of our experiences of Evoked Reality. Two experiments were conducted, one in the area of Media-Evoked Reality and one in Self-Evoked Reality. The experiments in fact lead to fruitful conclusions regarding our hypotheses and help us understand what could be further empirically studied. Ultimately, we illustrate different implications and shed light on prospective applications and uses of our concept, especially in the area of research on presence. In addition, we strongly suggest that we must open up presence research beyond the domain of virtual reality and communication media, and examine it from a broader perspective of cognitive science. We strongly believe that this concept of Evoked Reality and the proposed model may have significant applications in the study of presence, and in exploring the possibilities beyond virtual reality.PARIS-Arts et Métiers (751132303) / SudocSudocFranceF