How to Build a Patient-Specific Hybrid Simulator for Orthopaedic Open Surgery: Benefits and Limits of Mixed-Reality Using the Microsoft HoloLens

Orthopaedic simulators are popular in innovative surgical training programs, where trainees gain procedural experience in a safe and controlled environment. Recent studies suggest that an ideal simulator should combine haptic, visual, and audio technology to create an immersive training environment. This article explores the potentialities of mixed-reality using the HoloLens to develop a hybrid training system for orthopaedic open surgery. Hip arthroplasty, one of the most common orthopaedic procedures, was chosen as a benchmark to evaluate the proposed system. Patient-specific anatomical 3D models were extracted from a patient computed tomography to implement the virtual content and to fabricate the physical components of the simulator. Rapid prototyping was used to create synthetic bones. The Vuforia SDK was utilized to register virtual and physical contents. The Unity3D game engine was employed to develop the software allowing interactions with the virtual content using head movements, gestures, and voice commands. Quantitative tests were performed to estimate the accuracy of the system by evaluating the perceived position of augmented reality targets. Mean and maximum errors matched the requirements of the target application. Qualitative tests were carried out to evaluate workload and usability of the HoloLens for our orthopaedic simulator, considering visual and audio perception and interaction and ergonomics issues. The perceived overall workload was low, and the self-assessed performance was considered satisfactory. Visual and audio perception and gesture and voice interactions obtained a positive feedback. Postural discomfort and visual fatigue obtained a nonnegative evaluation for a simulation session of 40 minutes. These results encourage using mixed-reality to implement a hybrid simulator for orthopaedic open surgery. An optimal design of the simulation tasks and equipment setup is required to minimize the user discomfort. Future works will include Face Validity, Content Validity, and Construct Validity to complete the assessment of the hip arthroplasty simulator

How to Build a Patient-Specific Hybrid Simulator for Orthopaedic Open Surgery: Benefits and Limits of Mixed-Reality Using the Microsoft HoloLens

Orthopaedic simulators are popular in innovative surgical training programs, where trainees gain procedural experience in a safe and controlled environment. Recent studies suggest that an ideal simulator should combine haptic, visual, and audio technology to create an immersive training environment. This article explores the potentialities of mixed-reality using the HoloLens to develop a hybrid training system for orthopaedic open surgery. Hip arthroplasty, one of the most common orthopaedic procedures, was chosen as a benchmark to evaluate the proposed system. Patient-specific anatomical 3D models were extracted from a patient computed tomography to implement the virtual content and to fabricate the physical components of the simulator. Rapid prototyping was used to create synthetic bones. The Vuforia SDK was utilized to register virtual and physical contents. The Unity3D game engine was employed to develop the software allowing interactions with the virtual content using head movements, gestures, and voice commands. Quantitative tests were performed to estimate the accuracy of the system by evaluating the perceived position of augmented reality targets. Mean and maximum errors matched the requirements of the target application. Qualitative tests were carried out to evaluate workload and usability of the HoloLens for our orthopaedic simulator, considering visual and audio perception and interaction and ergonomics issues. The perceived overall workload was low, and the self-assessed performance was considered satisfactory. Visual and audio perception and gesture and voice interactions obtained a positive feedback. Postural discomfort and visual fatigue obtained a nonnegative evaluation for a simulation session of 40 minutes. These results encourage using mixed-reality to implement a hybrid simulator for orthopaedic open surgery. An optimal design of the simulation tasks and equipment setup is required to minimize the user discomfort. Future works will include Face Validity, Content Validity, and Construct Validity to complete the assessment of the hip arthroplasty simulator

Mobility Aware Optimization in the Metaverse

Metaverse applications that incorporate Mobile Augmented Reality (MAR) provide mixed and immersive experiences by amalgamating the virtual with the physical world. Notably, due to their multi-modality such applications are demanding in terms of energy consumption, computing and caching resources to efficiently support foreground interactions of participating users and rich background content. In this paper, the metaverse service is decomposed and anchored at suitable edge caching/computing nodes in 5G and beyond networks to enable efficient processing of background metaverse region models embedded with target AROs. To achieve that, a joint optimization problem is proposed, which explicitly considers the user physical mobility, service decomposition, and the balance between service delay, user perception quality and power consumption. A wide set of numerical investigations reveal that, the proposed scheme could provide optimal decision making and outperform other nominal baseline schemes which are oblivious of user mobility as well as do not consider service decomposition

"You feel real to me, Samantha": the matter of technology in Spike Jonze's Her

This essay will argue that Spike Jonze’s Her demonstrates a key idea in posthumanist new materialist theory: that matter is essential for posthuman interaction and communication. It also examines the requirement for embodiment on the part of the digital entity as well as the human, in this case the operating system Samantha. As the film presents an artificially intelligent operating system that ultimately moves beyond matter, it provides a case study for the importance of matter and the consequences of de-materialization. In this article, posthumanism names this era in which relationships between humans and technologies have become increasingly digitised, and the cluster of theoretical concepts which have arisen to interrogate this state of affairs. It is not seen as departure, rather as part of the continuing relationship between humans and technologies. Posthumanist new materialism is drawn on for its emphasis on and insights into embodiment and materiality. Theodore experiences Samantha in an embodied way, although their interactions are mediated. It is not a virtual experience, because, as Hansen notes, there is no “pure” virtual reality, only “mixed reality”. Samantha must use Theodore’s body to navigate the material world, but her experience of this world is “perception without affection” in Bergson’s terms: a radically different perception because of her lack of a physical body of her own, a “diffractive” perception that is not presented in the film. When Samantha moves to a processing platform beyond matter, even meaning itself becomes elusive as it is no longer, as Barad describes it “material discursive”, and Samantha’s implied exit from human society at the end of the film underscores the pre-eminent place of matter in human and posthuman life

Software techniques for improving head mounted displays to create comfortable user experiences in virtual reality

Head Mounted Displays (HMDs) allow users to experience Virtual Reality (VR) with a great level of immersion. Advancements in hardware technologies have led to a reduction in cost of producing good quality VR HMDs bringing them out from research labs to consumer markets. However, the current generation of HMDs suffer from a few fundamental problems that can deter their widespread adoption. For this thesis, we explored two techniques to overcome some of the challenges of experiencing VR when using HMDs. When experiencing VR with HMDs strapped to your head, even simple physical tasks like drinking a beverage can be difficult and awkward. We explored mixed reality renderings that selectively incorporate the physical world into the virtual world for interactions with physical objects. We conducted a user study comparing four rendering techniques that balance immersion in the virtual world with ease of interaction with the physical world. Users of VR systems often experience vection, the perception of self-motion in the absence of any physical movement. While vection helps to improve presence in VR, it often leads to a form of motion sickness called cybersickness. Prior work has discovered that changing vection (changing the perceived speed or moving direction) causes more severe cybersickness than steady vection (walking at a constant speed or in a constant direction). Based on this idea, we tried to reduce cybersickness caused by character movements in a First Person Shooter (FPS) game in VR. We propose Rotation Blurring (RB), uniformly blurring the screen during rotational movements to reduce cybersickness. We performed a user study to evaluate the impact of RB in reducing cybersickness and found that RB led to an overall reduction in sickness levels of the participants and delayed its onset. Participants who experienced acute levels of cybersickness benefited significantly from this technique

What is quantum mechanics trying to tell us?

I explore whether it is possible to make sense of the quantum mechanical description of physical reality by taking the proper subject of physics to be correlation and only correlation, and by separating the problem of understanding the nature of quantum mechanics from the hard problem of understanding the nature of objective probability in individual systems, and the even harder problem of understanding the nature of conscious awareness. The resulting perspective on quantum mechanics is supported by some elementary but insufficiently emphasized theorems. Whether or not it is adequate as a new Weltanschauung, this point of view toward quantum mechanics provides a different perspective from which to teach the subject or explain its peculiar character to people in other fields.Comment: 37 pages, no figures. This is the published version of the lecture notes that expand on my earlier ``Ithaca interpretation of quantum mechanics'', quant-ph/9609013. ``Wootters' theorem'' has become the SSC theorem, an earlier citation has been added, and a joke about Talmudic scholarship has been dropped at the request of a refere

Emotional Qualities of VR Space

The emotional response a person has to a living space is predominantly affected by light, color and texture as space-making elements. In order to verify whether this phenomenon could be replicated in a simulated environment, we conducted a user study in a six-sided projected immersive display that utilized equivalent design attributes of brightness, color and texture in order to assess to which extent the emotional response in a simulated environment is affected by the same parameters affecting real environments. Since emotional response depends upon the context, we evaluated the emotional responses of two groups of users: inactive (passive) and active (performing a typical daily activity). The results from the perceptual study generated data from which design principles for a virtual living space are articulated. Such a space, as an alternative to expensive built dwellings, could potentially support new, minimalist lifestyles of occupants, defined as the neo-nomads, aligned with their work experience in the digital domain through the generation of emotional experiences of spaces. Data from the experiments confirmed the hypothesis that perceivable emotional aspects of real-world spaces could be successfully generated through simulation of design attributes in the virtual space. The subjective response to the virtual space was consistent with corresponding responses from real-world color and brightness emotional perception. Our data could serve the virtual reality (VR) community in its attempt to conceive of further applications of virtual spaces for well-defined activities.Comment: 12 figure