Stereoscopic bimanual interaction for 3D visualization

Virtual Environments (VE) are being widely used in various research fields for several decades such as 3D visualization, education, training and games. VEs have the potential to enhance the visualization and act as a general medium for human-computer interaction (HCI). However, limited research has evaluated virtual reality (VR) display technologies, monocular and binocular depth cues, for human depth perception of volumetric (non-polygonal) datasets. In addition, a lack of standardization of three-dimensional (3D) user interfaces (UI) makes it challenging to interact with many VE systems. To address these issues, this dissertation focuses on evaluation of effects of stereoscopic and head-coupled displays on depth judgment of volumetric dataset. It also focuses on evaluation of a two-handed view manipulation techniques which support simultaneous 7 degree-of-freedom (DOF) navigation (x,y,z + yaw,pitch,roll + scale) in a multi-scale virtual environment (MSVE). Furthermore, this dissertation evaluates auto-adjustment of stereo view parameters techniques for stereoscopic fusion problems in a MSVE. Next, this dissertation presents a bimanual, hybrid user interface which combines traditional tracking devices with computer-vision based "natural" 3D inputs for multi-dimensional visualization in a semi-immersive desktop VR system. In conclusion, this dissertation provides a guideline for research design for evaluating UI and interaction techniques

Touching 3D data:interactive visualization of cosmological simulations

Visualisatie richt zich op het aanvullen van de zintuigen en het voorstellingsvermogen van wetenschappers zodat ze hun gegevens beter kunnen begrijpen. Dit is een interactief en iteratief proces waar informatierepresentatie, interactieve verkenning en het nemen van beslissingen een grote rol spelen. Het doel is om door middel van dit iteratieve proces inzicht te verkrijgen in het probleem en de onderliggende gegevens totdat er voldoende begrip is. Bij deze visuele verkenning is een hoge mate van interactiviteit essentieel voor het op eem efficiënte manier behalen van dit doel omdat het de gebruiker dan de mogelijkheid geeft om nieuwe ideëen uit te proberen, terugkoppeling te verkrijgen en de verkenning op basis daarvan bij te sturen. In dit proefschrift rapporteren we over ons onderzoek naar de uitdagingen voor natuurlijke interactie en de verkenning van gegevens die gerepresenteerd worden in drie dimensies. In dit proefschrift hebben we de visualisatie van astronomische gegevens als centraal voorbeeld genomen van een toepassingsdomein waarin de nadruk ligt op op drie-dimensionale puntwolkgegevens van numerieke simulaties, zoals simulaties van galactische dynamica of hoog-dimensionale informatie uit deeltjessystemen. We introduceren twee intuïtieve en efficiënte interactie-technieken voor de verkenning van gegevens in drie dimensies. De eerste techniek helpt gebruikers te navigeren in drie dimensies en de tweede techniek geeft gebruikers de mogelijkheid om eenvoudig een subset van deeltjes te selecteren. Daarnaast integreren we deze twee technieken in een visueel-analytische computerapplicatie om zo wetenschappers te helpen nuttige informatie te extraheren en inzicht te verkrijgen

Autonomous Sweet Pepper Harvesting for Protected Cropping Systems

In this letter, we present a new robotic harvester (Harvey) that can autonomously harvest sweet pepper in protected cropping environments. Our approach combines effective vision algorithms with a novel end-effector design to enable successful harvesting of sweet peppers. Initial field trials in protected cropping environments, with two cultivar, demonstrate the efficacy of this approach achieving a 46% success rate for unmodified crop, and 58% for modified crop. Furthermore, for the more favourable cultivar we were also able to detach 90% of sweet peppers, indicating that improvements in the grasping success rate would result in greatly improved harvesting performance

Designing a robotic port system for laparo-endoscopic single-site surgery

Current research and development in the field of surgical interventions aim to reduce the invasiveness by using few incisions or natural orifices in the body to access the surgical site. Considering surgeries in the abdominal cavity, the Laparo-Endoscopic Single-site Surgery (LESS) can be performed through a single incision in the navel, reducing blood loss, post-operative trauma, and improving the cosmetic outcome. However, LESS results in less intuitive instrument control, impaired ergonomic, loss of depth and haptic perception, and restriction of instrument positioning by a single incision. Robot-assisted surgery addresses these shortcomings, by introducing highly articulated, flexible robotic instruments, ergonomic control consoles with 3D visualization, and intuitive instrument control algorithms. The flexible robotic instruments are usually introduced into the abdomen via a rigid straight port, such that the positioning of the tools and therefore the accessibility of anatomical structures is still constrained by the incision location. To address this limitation, articulated ports for LESS are proposed by recent research works. However, they focus on only a few aspects, which are relevant to the surgery, such that a design considering all requirements for LESS has not been proposed yet. This partially originates in the lack of anatomical data of specific applications. Further, no general design guidelines exist and only a few evaluation metrics are proposed. To target these challenges, this thesis focuses on the design of an articulated robotic port for LESS partial nephrectomy. A novel approach is introduced, acquiring the available abdominal workspace, integrated into the surgical workflow. Based on several generated patient datasets and developed metrics, design parameter optimization is conducted. Analyzing the surgical procedure, a comprehensive requirement list is established and applied to design a robotic system, proposing a tendon-driven continuum robot as the articulated port structure. Especially, the aspects of stiffening and sterile design are addressed. In various experimental evaluations, the reachability, the stiffness, and the overall design are evaluated. The findings identify layer jamming as the superior stiffening method. Further, the articulated port is proven to enhance the accessibility of anatomical structures and offer a patient and incision location independent design

Virtual Valcamonica: collaborative exploration of prehistoric petroglyphs and their surrounding environment in multi-user virtual reality

In this paper, we present a novel, multi-user, virtual reality environment for the interactive, collaborative 3D analysis of large 3D scans and the technical advancements that were necessary to build it: a multi-view rendering system for large 3D point clouds, a suitable display infrastructure and a suite of collaborative 3D interaction techniques. The cultural heritage site of Valcamonica in Italy with its large collection of prehistoric rock-art served as an exemplary use case for evaluation. The results show that our output-sensitive level-of-detail rendering system is capable of visualizing a 3D dataset with an aggregate size of more than 14 billion points at interactive frame rates. The system design in this exemplar application results from close exchange with a small group of potential users: archaeologists with expertise in rock-art and allows them to explore the prehistoric art and its spatial context with highly realistic appearance. A set of dedicated interaction techniques was developed to facilitate collaborative visual analysis. A multi-display workspace supports the immediate comparison of geographically distributed artifacts. An expert review of the final demonstrator confirmed the potential for added value in rock-art research and the usability of our collaborative interaction techniques