124 research outputs found
Augmented reality (AR) for surgical robotic and autonomous systems: State of the art, challenges, and solutions
Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human-robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future
Augmented Reality (AR) for Surgical Robotic and Autonomous Systems: State of the Art, Challenges, and Solutions
Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human–robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future
Probing the Unseen Depths of the Hepatic Microarchitecture via Multimodal Microscopy
Multimodal microscopy combines the advantages and strengths of different imaging modalities in order to holistically characterise the organisation of biological organisms and their comprising constituents under healthy and diseased conditions, down to the spatial resolution required to understand the morphology and function of such structures. Given the profound advantages conferred by such an approach, this work broadly aimed to develop and exploit various multimodal and multi-dimensional imaging modalities in a complimentary, combined and/or correlative manner – namely, three-dimensional scanning electron microscopy, transmission electron tomography, bright-field light microscopy, confocal laser scanning microscopy and X-ray micro-computed tomography – in order to characterise and collect new information on the normal and pathological microarchitecture of rodent and human liver tissue in 3-D under various experimental conditions. The data reported in this work includes a comparative analysis of a variety of sample preparation protocols applied to rat liver tissue to determine the suitability of such protocols for the application of serial block-face scanning electron microscopy (SBF-SEM). Next, 3-D modelling and morphometric analysis (utilising the premier SBF-SEM protocol) was performed in order to visualise and quantify key features of the hepatic microarchitecture. We further outline a large-volume correlative light and electron microscopy approach utilising selective molecular probes for confocal laser scanning microscopy (actin, lipids and nuclei), combined with the 3-D ultrastructure of the same structures of interest, as revealed by SBF-SEM (Chapter 2). Development of a straightforward combinatorial sample preparation approach, followed by a swift multimodal imaging approach – combining X-ray micro-computed tomography, bright-field light microscopy and serial section scanning electron microscopy – facilitated the cross correlation of structure-function information on the same sample across diverse length scales (Chapter 3). Next, we outline a novel “silver filler pre-embedding approach” in order to reduce artefactual charging, minimise dataset acquisition time and improve resolution and contrast in rat liver tissue prepared for SBF-SEM (Chapter 4). Next, we employ a complementary imaging approach involving serial section scanning electron microscopy and transmission electron tomography in order to comparatively analyse the structure and morphometric parameters of thousands of normal- and giant mitochondria in human patients diagnosed with non-alcoholic fatty liver disease. In so doing, we reveal functional alterations associated with mitochondrial gigantism and propose a mechanism for their formation (Chapter 5). Finally, the significance of the results obtained, and major scientific advances reported in this work are discussed in-depth against the relevant literature. This is proceeded by the future outlooks and research that remains to be done, followed by the main conclusions of this Ph.D thesis (Chapter 6). In summary, our findings firmly establish the immense importance and value of contemporary multimodal microscopy modalities in modern life science research, for holistically revealing cellular structures along the vast length scales amongst which they exist, under healthy and clinically relevant pathological conditions
Current research opportunities of image processing and computer vision
Image processing and computer vision is an important and essential area in today’s scenario. Several problems can be solved through computer vision techniques. There are a large number of challenges and opportunities which require skills in the field of computer vision to address them. Computer vision applications cover each band of the electromagnetic spectrum and there are numerous applications in every band. This article is targeted to the research students, scholars and researchers who are interested to solve the problems in the field of image processing and computer vision. It addresses the opportunities and current trends of computer vision applications in all emerging domains. The research needs are identified through available literature survey and classified in the corresponding domains. The possible exemplary images are collected from the different repositories available for research and shown in this paper. The opportunities mentioned in this paper are explained through the images so that a naive researcher can understand it well before proceeding to solve the corresponding problems. The databases mentioned in this article could be useful for researchers who are interested in further solving the problem. The motivation of the article is to expose the current opportunities in the field of image processing and computer vision along with corresponding repositories. Interested researchers who are working in the field can choose a problem through this article and can get the experimental images through the cited references for working further.
TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning
Background and objective:
Processing of medical images such as MRI or CT presents different challenges compared to RGB images typically used in computer vision. These include a lack of labels for large datasets, high computational costs, and the need of metadata to describe the physical properties of voxels. Data augmentation is used to artificially increase the size of the training datasets. Training with image subvolumes or patches decreases the need for computational power. Spatial metadata needs to be carefully taken into account in order to ensure a correct alignment and orientation of volumes.
Methods:
We present TorchIO, an open-source Python library to enable efficient loading, preprocessing, augmentation and patch-based sampling of medical images for deep learning. TorchIO follows the style of PyTorch and integrates standard medical image processing libraries to efficiently process images during training of neural networks. TorchIO transforms can be easily composed, reproduced, traced and extended. Most transforms can be inverted, making the library suitable for test-time augmentation and estimation of aleatoric uncertainty in the context of segmentation. We provide multiple generic preprocessing and augmentation operations as well as simulation of MRI-specific artifacts.
Results:
Source code, comprehensive tutorials and extensive documentation for TorchIO can be found at http://torchio.rtfd.io/. The package can be installed from the Python Package Index (PyPI) running pip install torchio. It includes a command-line interface which allows users to apply transforms to image files without using Python. Additionally, we provide a graphical user interface within a TorchIO extension in 3D Slicer to visualize the effects of transforms.
Conclusion:
TorchIO was developed to help researchers standardize medical image processing pipelines and allow them to focus on the deep learning experiments. It encourages good open-science practices, as it supports experiment reproducibility and is version-controlled so that the software can be cited precisely. Due to its modularity, the library is compatible with other frameworks for deep learning with medical images
- …