Sensing and mapping for interactive performance

This paper describes a trans-domain mapping (TDM) framework for translating meaningful activities from one creative domain onto another. The multi-disciplinary framework is designed to facilitate an intuitive and non-intrusive interactive multimedia performance interface that offers the users or performers real-time control of multimedia events using their physical movements. It is intended to be a highly dynamic real-time performance tool, sensing and tracking activities and changes, in order to provide interactive multimedia performances. From a straightforward definition of the TDM framework, this paper reports several implementations and multi-disciplinary collaborative projects using the proposed framework, including a motion and colour-sensitive system, a sensor-based system for triggering musical events, and a distributed multimedia server for audio mapping of a real-time face tracker, and discusses different aspects of mapping strategies in their context. Plausible future directions, developments and exploration with the proposed framework, including stage augmenta tion, virtual and augmented reality, which involve sensing and mapping of physical and non-physical changes onto multimedia control events, are discussed

Interactive 3D Digital Models for Anatomy and Medical Education

This chapter explores the creation and use of interactive, three-dimensional (3D), digital models for anatomy and medical education. Firstly, it looks back over the history and development of virtual 3D anatomy resources before outlining some of the current means of their creation; including photogrammetry, CT and surface scanning, and digital modelling, outlining advantages and disadvantages for each. Various means of distribution are explored, including; virtual learning environments, websites, interactive PDF’s, virtual and augmented reality, bespoke applications, and 3D printing, with a particular focus on the level of interactivity each method offers. Finally, and perhaps most importantly, the use of such models for education is discussed. Questions addressed include; How can such models best be used to enhance student learning? How can they be used in the classroom? How can they be used for selfdirected study? As well as exploring if they could one day replace human specimens, and how they complement the rise of online and e-learning

Evaluation of an augmented reality application for learning neuroanatomy in psychology

Neuroanatomy is difficult for psychology students because of spatial visualization and the relationship among brain structures. Some technologies have been implemented to facilitate the learning of anatomy using three-dimensional (3D) visualization of anatomy contents. Augmented reality (AR) is a promising technology in this field. A mobile AR application to provide the visualization of morphological and functional information of the brain was developed. A sample of 67 students of neuropsychology completed tests for visuospatial ability, anatomical knowledge, learning goals, and experience with technologies. Subsequently, they performed a learning activity using one of the visualization methods considered: a 3D method using the AR application and a two-dimensional (2D) method using a textbook to color, followed by questions concerning their satisfaction and knowledge. After using the alternative method, the students expressed their preference. The two methods improved knowledge equally, but the 3D method obtained higher satisfaction scores and was more preferred by students. The 3D method was also more preferred by the students who used this method during the activity. After controlling for the method used in the activity, associations were found between the preference of the 3D method because of its usability and experience with technologies. These results found that the AR application was highly valued by students to learn and was as effective as the textbook for this purpose

Imaging Informatics and the Human Brain Project: the Role of Structure, Review

(no abstract

The Connectome Viewer Toolkit: An Open Source Framework to Manage, Analyze, and Visualize Connectomes

Advanced neuroinformatics tools are required for methods of connectome mapping, analysis, and visualization. The inherent multi-modality of connectome datasets poses new challenges for data organization, integration, and sharing. We have designed and implemented the Connectome Viewer Toolkit – a set of free and extensible open source neuroimaging tools written in Python. The key components of the toolkit are as follows: (1) The Connectome File Format is an XML-based container format to standardize multi-modal data integration and structured metadata annotation. (2) The Connectome File Format Library enables management and sharing of connectome files. (3) The Connectome Viewer is an integrated research and development environment for visualization and analysis of multi-modal connectome data. The Connectome Viewer's plugin architecture supports extensions with network analysis packages and an interactive scripting shell, to enable easy development and community contributions. Integration with tools from the scientific Python community allows the leveraging of numerous existing libraries for powerful connectome data mining, exploration, and comparison. We demonstrate the applicability of the Connectome Viewer Toolkit using Diffusion MRI datasets processed by the Connectome Mapper. The Connectome Viewer Toolkit is available from http://www.cmtk.org

Object-based representation and analysis of light and electron microscopic volume data using Blender

This is the final version of the article. Available from the publisher via the DOI in this record.BACKGROUND: Rapid improvements in light and electron microscopy imaging techniques and the development of 3D anatomical atlases necessitate new approaches for the visualization and analysis of image data. Pixel-based representations of raw light microscopy data suffer from limitations in the number of channels that can be visualized simultaneously. Complex electron microscopic reconstructions from large tissue volumes are also challenging to visualize and analyze. RESULTS: Here we exploit the advanced visualization capabilities and flexibility of the open-source platform Blender to visualize and analyze anatomical atlases. We use light-microscopy-based gene expression atlases and electron microscopy connectome volume data from larval stages of the marine annelid Platynereis dumerilii. We build object-based larval gene expression atlases in Blender and develop tools for annotation and coexpression analysis. We also represent and analyze connectome data including neuronal reconstructions and underlying synaptic connectivity. CONCLUSIONS: We demonstrate the power and flexibility of Blender for visualizing and exploring complex anatomical atlases. The resources we have developed for Platynereis will facilitate data sharing and the standardization of anatomical atlases for this species. The flexibility of Blender, particularly its embedded Python application programming interface, means that our methods can be easily extended to other organisms.The research leading to these results received funding from the European Research Council under the European Union’s Seventh Framework Programme (FP7/2007-2013)/European Research Council Grant Agreement 260821

Performance Factors in Neurosurgical Simulation and Augmented Reality Image Guidance

Virtual reality surgical simulators have seen widespread adoption in an effort to provide safe, cost-effective and realistic practice of surgical skills. However, the majority of these simulators focus on training low-level technical skills, providing only prototypical surgical cases. For many complex procedures, this approach is deficient in representing anatomical variations that present clinically, failing to challenge users’ higher-level cognitive skills important for navigation and targeting. Surgical simulators offer the means to not only simulate any case conceivable, but to test novel approaches and examine factors that influence performance. Unfortunately, there is a void in the literature surrounding these questions. This thesis was motivated by the need to expand the role of surgical simulators to provide users with clinically relevant scenarios and evaluate human performance in relation to image guidance technologies, patient-specific anatomy, and cognitive abilities. To this end, various tools and methodologies were developed to examine cognitive abilities and knowledge, simulate procedures, and guide complex interventions all within a neurosurgical context. The first chapter provides an introduction to the material. The second chapter describes the development and evaluation of a virtual anatomical training and examination tool. The results suggest that learning occurs and that spatial reasoning ability is an important performance predictor, but subordinate to anatomical knowledge. The third chapter outlines development of automation tools to enable efficient simulation studies and data management. In the fourth chapter, subjects perform abstract targeting tasks on ellipsoid targets with and without augmented reality guidance. While the guidance tool improved accuracy, performance with the tool was strongly tied to target depth estimation – an important consideration for implementation and training with similar guidance tools. In the fifth chapter, neurosurgically experienced subjects were recruited to perform simulated ventriculostomies. Results showed anatomical variations influence performance and could impact outcome. Augmented reality guidance showed no marked improvement in performance, but exhibited a mild learning curve, indicating that additional training may be warranted. The final chapter summarizes the work presented. Our results and novel evaluative methodologies lay the groundwork for further investigation into simulators as versatile research tools to explore performance factors in simulated surgical procedures

Haptically assisted connection procedure for the reconstruction of dendritic spines

Dendritic spines are thin protrusions that cover the dendritic surface of numerous neurons in the brain and whose function seems to play a key role in neural circuits. The correct segmentation of those structures is difficult due to their small size and the resulting spines can appear incomplete. This paper presents a four-step procedure for the complete reconstruction of dendritic spines. The haptically driven procedure is intended to work as an image processing stage before the automatic segmentation step giving the final representation of the dendritic spines. The procedure is designed to allow both the navigation and the volume image editing to be carried out using a haptic device. A use case employing our procedure together with a commercial software package for the segmentation stage is illustrated. Finally, the haptic editing is evaluated in two experiments; the first experiment concerns the benefits of the force feedback and the second checks the suitability of the use of a haptic device as input. In both cases, the results shows that the procedure improves the editing accuracy