Interactivity:the missing link between virtual reality technology and drug discovery pipelines

The potential of virtual reality (VR) to contribute to drug design and development has been recognised for many years. Hardware and software developments now mean that this potential is beginning to be realised, and VR methods are being actively used in this sphere. A recent advance is to use VR not only to visualise and interact with molecular structures, but also to interact with molecular dynamics simulations of 'on the fly' (interactive molecular dynamics in VR, IMD-VR), which is useful not only for flexible docking but also to examine binding processes and conformational changes. iMD-VR has been shown to be useful for creating complexes of ligands bound to target proteins, e.g., recently applied to peptide inhibitors of the SARS-CoV-2 main protease. In this review, we use the term 'interactive VR' to refer to software where interactivity is an inherent part of the user VR experience e.g., in making structural modifications or interacting with a physically rigorous molecular dynamics (MD) simulation, as opposed to simply using VR controllers to rotate and translate the molecule for enhanced visualisation. Here, we describe these methods and their application to problems relevant to drug discovery, highlighting the possibilities that they offer in this arena. We suggest that the ease of viewing and manipulating molecular structures and dynamics, and the ability to modify structures on the fly (e.g., adding or deleting atoms) makes modern interactive VR a valuable tool to add to the armoury of drug development methods.Comment: 19 pages, 3 figure

Developing a mixed reality assistance system based on projection mapping technology for manual operations at assembly workstations.

ABSTRACT Manual tasks play an important role in social sustainable manufacturing enterprises. Commonly, manual operations are used for low volume productions, but are not limited to. Operational models in manufacturing sisters cased of x-to-order paradigms (e.g. assembly-to-order) may require manual operations to speed-up the ramp-up time of new product configuration assemblies. The implications of manual operations in any production line may imply that any manufacturing or assembly process become more susceptible to human errors and therefore translate into delays, defects and/or poor product quality. In this scenario, virtual and augmented realities can offer significant advantages to support the human operator in manual operations. This research work presents the development of a mixed (virtual and augmented) reality assistance system that permits real-time support in manual operations. A review of mixed reality techniques and technologies was conducted, where it was determined to use a projection mapping solution for the proposed assistance system. According to the specific requirements of the demonstration environment, hardware and software components were chosen. The developed mixed reality assistance system was able to guide any user without any prior knowledge through the successful completion of the specific assembly task

FACING EXPERIENCE: A PAINTER’S CANVAS IN VIRTUAL REALITY

Full version unavailable due to 3rd party copyright restrictions.This research investigates how shifts in perception might be brought about through the development of visual imagery created by the use of virtual environment technology. Through a discussion of historical uses of immersion in art, this thesis will explore how immersion functions and why immersion has been a goal for artists throughout history. It begins with a discussion of ancient cave drawings and the relevance of Plato’s Allegory of the Cave. Next it examines the biological origins of “making special.” The research will discuss how this concept, combined with the ideas of “action” and “reaction,” has reinforced the view that art is fundamentally experiential rather than static. The research emphasizes how present-day virtual environment art, in providing a space that engages visitors in computer graphics, expands on previous immersive artistic practices. The thesis examines the technical context in which the research occurs by briefly describing the use of computer science technologies, the fundamentals of visual arts practices, and the importance of aesthetics in new media and provides a description of my artistic practice. The aim is to investigate how combining these approaches can enhance virtual environments as artworks. The computer science of virtual environments includes both hardware and software programming. The resultant virtual environment experiences are technologically dependent on the types of visual displays being used, including screens and monitors, and their subsequent viewing affordances. Virtual environments fill the field of view and can be experienced with a head mounted display (HMD) or a large screen display. The sense of immersion gained through the experience depends on how tracking devices and related peripheral devices are used to facilitate interaction. The thesis discusses visual arts practices with a focus on how illusions shift our cognition and perception in the visual modalities. This discussion includes how perceptual thinking is the foundation of art experiences, how analogies are the foundation of cognitive experiences and how the two intertwine in art experiences for virtual environments. An examination of the aesthetic strategies used by artists and new media critics are presented to discuss new media art. This thesis investigates the visual elements used in virtual environments and prescribes strategies for creating art for virtual environments. Methods constituting a unique virtual environment practice that focuses on visual analogies are discussed. The artistic practice that is discussed as the basis for this research also concentrates on experiential moments and shifts in perception and cognition and references Douglas Hofstadter, Rudolf Arnheim and John Dewey. iv Virtual environments provide for experiences in which the imagery generated updates in real time. Following an analysis of existing artwork and critical writing relative to the field, the process of inquiry has required the creation of artworks that involve tracking systems, projection displays, sound work, and an understanding of the importance of the visitor. In practice, the research has shown that the visitor should be seen as an interlocutor, interacting from a first-person perspective with virtual environment events, where avatars or other instrumental intermediaries, such as guns, vehicles, or menu systems, do not to occlude the view. The aesthetic outcomes of this research are the result of combining visual analogies, real time interactive animation, and operatic performance in immersive space. The environments designed in this research were informed initially by paintings created with imagery generated in a hypnopompic state or during the moments of transitioning from sleeping to waking. The drawings often emphasize emotional moments as caricatures and/or elements of the face as seen from a number of perspectives simultaneously, in the way of some cartoons, primitive artwork or Cubist imagery. In the imagery, the faces indicate situations, emotions and confrontations which can offer moments of humour and reflective exploration. At times, the faces usurp the space and stand in representation as both face and figure. The power of the placement of the caricatures in the paintings become apparent as the imagery stages the expressive moment. The placement of faces sets the scene, establishes relationships and promotes the honesty and emotions that develop over time as the paintings are scrutinized. The development process of creating virtual environment imagery starts with hand drawn sketches of characters, develops further as paintings on “digital canvas”, are built as animated, three-dimensional models and finally incorporated into a virtual environment. The imagery is generated while drawing, typically with paper and pencil, in a stream of consciousness during the hypnopompic state. This method became an aesthetic strategy for producing a snappy straightforward sketch. The sketches are explored further as they are worked up as paintings. During the painting process, the figures become fleshed out and their placement on the page, in essence brings them to life. These characters inhabit a world that I explore even further by building them into three dimensional models and placing them in computer generated virtual environments. The methodology of developing and placing the faces/figures became an operational strategy for building virtual environments. In order to open up the range of art virtual environments, and develop operational strategies for visitors’ experience, the characters and their facial features are used as navigational strategies, signposts and methods of wayfinding in order to sustain a stream of consciousness type of navigation. Faces and characters were designed to represent those intimate moments of self-reflection and confrontation that occur daily within ourselves and with others. They sought to reflect moments of wonderment, hurt, curiosity and humour that could subsequently be relinquished for more practical or purposeful endeavours. They were intended to create conditions in which visitors might reflect upon their emotional state, v enabling their understanding and trust of their personal space, in which decisions are made and the nature of world is determined. In order to extend the split-second, frozen moment of recognition that a painting affords, the caricatures and their scenes are given new dimensions as they become characters in a performative virtual reality. Emotables, distinct from avatars, are characters confronting visitors in the virtual environment to engage them in an interactive, stream of consciousness, non-linear dialogue. Visitors are also situated with a role in a virtual world, where they were required to adapt to the language of the environment in order to progress through the dynamics of a drama. The research showed that imagery created in a context of whimsy and fantasy could bring ontological meaning and aesthetic experience into the interactive environment, such that emotables or facially expressive computer graphic characters could be seen as another brushstroke in painting a world of virtual reality

Developing a user-centered accessible virtual reality video environment for severe visual disabilities

We address a timely issue of accessibility for visual information through the means of videos. Using emerging technologies (Head Mounted Virtual Reality Displays) and a user-centred design approach, we provide people with severe visual disabilities with a bespoke platform for accessing and viewing videos. We report on newly created test methods for measuring acuity within virtual spaces and reactions of impaired individuals, which informed our platform's design, to inform similar designs and allow testing and refinement for ecological and external validity. A prototype software for accessible virtual reality video viewing is presented, with a subsequent user evaluation to test the software, and a newer virtual reality head mounted display to determine usability while measuring how visually impaired users utilize elements in a virtual environment. We give guidance, based on empirical evidence, and advocate that although VR technologies are currently primarily targeted at a generic audience (gaming and entertainment), they can and should be further developed as assistive tools that enable independent living and increase the quality of life for those with disabilities, and specifically severe visual impairments

Modelling and use of SysML behaviour models for achieving dynamic use cases of technical products in different VR-systems

Digital methods and models help the product designers in performing early evaluations on a product that eventually help to gain understanding about a product’s behaviour and its interactions with neighbouring systems in its later life-phases. Virtual Reality (VR) is a technology that can facilitate the early evaluation process by showing later life situations of a product as early as at the design stage. However, the application of VR in the industry is currently limited due to high model preparation effort and poor reusability of already prepared models. Therefore, this thesis pursues towards the development of a method that can facilitate the early evaluations of the product in VR and thus, facilitate the use of VR in the product development process. This method aims at achieving generic behavioural descriptions for use in VR that can be reused as well to form dynamic use cases of a product in different VR-systems. The focus lies on reducing the overall preparation effort of VR-models and on achieving high reusability of already created models. The core components of the thesis consist of the use of Model Based Systems Engineering (MBSE) to develop generic behavioural model descriptions, their use in building different use cases of a product in one VR-system and their reuse in different VR-systems as well. The Systems Modeling Language (SysML) is used to describe the behavioural models, the modelling process is described systematically and is also summarized in the form of general-purpose guidelines for later use. Furthermore, a dedicated physics engine is used to perform the physical calculations on virtual objects in VR and is integrated with the SysML. These SysML behaviour models together with the physics engine are used to achieve a real-time product use case simulation inside VR. The same SysML behaviour models are used across different VR-systems to achieve real-time simulations and to validate their reuse. Two VR prototypes are developed to demonstrate the effectivity and use of the presented method. Finally, one of the prototypes is put to the empirical evaluation performed with the help of experts from academia as well as the industry.Digitale Methode und Modellen ermöglichen den Produktdesignern eine frühzeitige Evaluierung des Produkts, damit sie das Verhalten des Produkts und seine Interaktionen mit benachbarten Systemen in seinen späteren Lebensphasen besser verstehen können. Virtual Reality (VR) ist eine Technologie, die zum frühen Evaluierungsprozess beitragen kann, indem spätere Lebenssituationen eines Produkts schon in der Entwurfsphase angezeigt werden können. Die Anwendung von VR in der Industrie ist jedoch derzeit aufgrund des hohen Modellaufbereitungsaufwands und der limitierten Wiederverwendbarkeit vorhandener Modelle begrenzt. Daher befasst sich diese Arbeit mit der Entwicklung einer Methode, die die frühzeitige Evaluierung des Produkts innerhalb von VR und die Verwendung von VR im Produktentwicklungsprozess erleichtern kann. Diese Methode befasst sich mit dem Prozess der Entwicklung allgemeiner Verhaltensbeschreibungen zur Verwendung in VR, die auch wiederverwendet werden können, um dynamische Anwendungsfälle eines Produkts in den verschiedenen VR-Systemen abzubilden. Der Fokus liegt auf der Reduzierung des gesamten Aufbereitungsaufwands von VR-Modellen und auf das Verwirklichen einer hohen Wiederverwendbarkeit bereits vorhandener Modelle. Die Kernkomponenten der Arbeit bestehen in der Verwendung von Model Based Systems Engineering (MBSE) zur Entwicklung allgemeingültiger Verhaltensmodellbeschreibungen, ihrer Verwendung beim Erstellen verschiedener Anwendungsfälle eines Produkts in einem VR-System und ihrer Wiederverwendung in den verschiedenen VR-Systemen. Die Systems Modeling Language (SysML) wird zur Beschreibung der Verhaltensmodelle verwendet, der Modellierungsprozess wird systematisch beschrieben und auch in Form allgemeiner Anwendungsrichtlinien für die spätere Verwendung zusammengefasst. Darüber hinaus wird eine dedizierte Physik-Engine verwendet, um die physikalischen Berechnungen für virtuelle Objekte in VR durchzuführen, welche auch mit SysML integriert ist. Diese SysML-Verhaltensmodelle zusammen mit der Physik-Engine bilden eine echtzeitfähige Produktanwendungssimulation in VR. Dieselben SysML-Verhaltensmodelle werden für verschiedene VR-Systeme verwendet, um Echtzeitsimulationen abzubilden und ihre Wiederverwendung zu validieren. Zwei VR-Prototypen wurden entwickelt, um die Wirksamkeit und Verwendung der vorgestellten Methoden zu demonstrieren. Schließlich wurde einer der Prototypen einer empirischen Untersuchung unterzogen, die mithilfe von Experten aus Wissenschaft und Industrie durchgeführt wurde

Measuring The Impact of Narrative on Player\u27s Presence and Immersion in A First Person Game Environment

In the virtual environments (VE) literature, presence has been described as the feeling whereby an individual feels as if he or she is actually in the VE. In the videogame literature, the related concept of immersion is viewed as an effect facilitating player engagement. This thesis examines how narrative and graphics quality influence presence and immersion in a first person game. Three levels of narrative and graphics quality are used in an empirical study: text narrative with high quality graphics, no text narrative with high quality graphics, and no text narrative with low quality graphics. Results showed that there is a significant difference in players’ presence and immersion with rich narrative provided through text narrative together with high quality graphics versus no narrative and low quality graphics, and the use of text narrative results in greater presence and immersion than high quality graphics alone

Interfaces for human-centered production and use of computer graphics assets

L'abstract è presente nell'allegato / the abstract is in the attachmen