Towards registration of multimodal images of vocal folds based on mutual information.

International audienceThis paper deals with mutual information-based registration of multimodal images for laser phonomicrosurgery of the vocal folds. The images to be registered are white light images (white light camera) versus fluorescence images. This work is carried out within the framework of the European project RALP which involves the use of microrobotic system for endoluminal laser phonosurgery. The designed system includes two fiber bundles connected to a high speed camera and one fiber bundle used for fluorescence image. Using the mutual information based registration method, it will be possible to represent the visible information in the fluorescence image and use it in the other image

Application of metaheuristic optimization algorithms for image registration in mobile robot visual control

Visual Servoing (VS) of a mobile robot requires advanced digital image processing, and one of the techniques especially fitting for this complex task is Image Registration (IR). In general, IR involves the geometrical alignment of images, and it can be viewed as an optimization problem. Therefore, we propose Metaheuristic Optimization Algorithms (MOA) for IR in VS of a mobile robot. The comprehensive comparison study of three state-of-the-art MOA, namely the Slime Mould Algorithm (SMA), Harris Hawks Optimizer (HHO), and Whale Optimization Algorithm (WOA) is presented. The previously mentioned MOA used for IR are evaluated on 12 pairs of stereo images obtained by a mobile robot stereo vision system in a laboratory model of a manufacturing environment. The MATLAB software package is used for the implementation of the considered optimization algorithms. Acquired experimental results show that SMA outperforms HHO and WOA, while all three algorithms perform satisfactory alignment of images captured from various mobile robot poses

Application of metaheuristic optimization algorithms for image registration in mobile robot visual control

Visual Servoing (VS) of a mobile robot requires advanced digital image processing, and one of the techniques especially fitting for this complex task is Image Registration (IR). In general, IR involves the geometrical alignment of images, and it can be viewed as an optimization problem. Therefore, we propose Metaheuristic Optimization Algorithms (MOA) for IR in VS of a mobile robot. The comprehensive comparison study of three state-of-the-art MOA, namely the Slime Mould Algorithm (SMA), Harris Hawks Optimizer (HHO), and Whale Optimization Algorithm (WOA) is presented. The previously mentioned MOA used for IR are evaluated on 12 pairs of stereo images obtained by a mobile robot stereo vision system in a laboratory model of a manufacturing environment. The MATLAB software package is used for the implementation of the considered optimization algorithms. Acquired experimental results show that SMA outperforms HHO and WOA, while all three algorithms perform satisfactory alignment of images captured from various mobile robot poses

Robot Autonomy for Surgery

Autonomous surgery involves having surgical tasks performed by a robot operating under its own will, with partial or no human involvement. There are several important advantages of automation in surgery, which include increasing precision of care due to sub-millimeter robot control, real-time utilization of biosignals for interventional care, improvements to surgical efficiency and execution, and computer-aided guidance under various medical imaging and sensing modalities. While these methods may displace some tasks of surgical teams and individual surgeons, they also present new capabilities in interventions that are too difficult or go beyond the skills of a human. In this chapter, we provide an overview of robot autonomy in commercial use and in research, and present some of the challenges faced in developing autonomous surgical robots

Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectives

Ultrasound (US) is one of the most widely used modalities for clinical intervention and diagnosis due to the merits of providing non-invasive, radiation-free, and real-time images. However, free-hand US examinations are highly operator-dependent. Robotic US System (RUSS) aims at overcoming this shortcoming by offering reproducibility, while also aiming at improving dexterity, and intelligent anatomy and disease-aware imaging. In addition to enhancing diagnostic outcomes, RUSS also holds the potential to provide medical interventions for populations suffering from the shortage of experienced sonographers. In this paper, we categorize RUSS as teleoperated or autonomous. Regarding teleoperated RUSS, we summarize their technical developments, and clinical evaluations, respectively. This survey then focuses on the review of recent work on autonomous robotic US imaging. We demonstrate that machine learning and artificial intelligence present the key techniques, which enable intelligent patient and process-specific, motion and deformation-aware robotic image acquisition. We also show that the research on artificial intelligence for autonomous RUSS has directed the research community toward understanding and modeling expert sonographers' semantic reasoning and action. Here, we call this process, the recovery of the "language of sonography". This side result of research on autonomous robotic US acquisitions could be considered as valuable and essential as the progress made in the robotic US examination itself. This article will provide both engineers and clinicians with a comprehensive understanding of RUSS by surveying underlying techniques.Comment: Accepted by Medical Image Analysi

A system for synthetic vision and augmented reality in future flight decks

Rockwell Science Center is investigating novel human-computer interaction techniques for enhancing the situational awareness in future flight decks. One aspect is to provide intuitive displays that provide the vital information and the spatial awareness by augmenting the real world with an overlay of relevant information registered to the real world. Such Augmented Reality (AR) techniques can be employed during bad weather scenarios to permit flying in Visual Flight Rules (VFR) in conditions which would normally require Instrumental Flight Rules (IFR). These systems could easily be implemented on heads-up displays (HUD). The advantage of AR systems vs. purely synthetic vision (SV) systems is that the pilot can relate the information overlay to real objects in the world, whereas SV systems provide a constant virtual view, where inconsistencies can hardly be detected. The development of components for such a system led to a demonstrator implemented on a PC. A camera grabs video images which are overlaid with registered information. Orientation of the camera is obtained from an inclinometer and a magnetometer; position is acquired from GPS. In a possible implementation in an airplane, the on-board attitude information can be used for obtaining correct registration. If visibility is sufficient, computer vision modules can be used to fine-tune the registration by matching visual cues with database features. This technology would be especially useful for landing approaches. The current demonstrator provides a frame-rate of 15 fps, using a live video feed as background with an overlay of avionics symbology in the foreground. In addition, terrain rendering from a 1 arc sec. digital elevation model database can be overlaid to provide synthetic vision in case of limited visibility. For true outdoor testing (on ground level), the system has been implemented on a wearable computer

Biologically Inspired Optimization Methods for Image Registration in Visual Servoing of a Mobile Robot

Image registration (IR) represents image processing technique that is suitable for use in Visual Servoing (VS). This paper proposes the use of Biologically Inspired Optimization (BIO) methods for IR in VS of nonholonomic mobile robot. The comparison study of three different BIO methods is conducted, namely Genetic Algorithm (GA), Particle Swarm Optimization (PSO), and Grey Wolf Optimizer (GWO). The aforementioned optimization algorithms utilized for IR are tested on 24 images of manufacturing entities acquired by mobile robot stereo vision system. The considered algorithms are implemented in the MATLAB environment. The experimental results suggest satisfactory geometrical alignment after IR, whilst GA and PSO outperform GWO

CFVS: Coarse-to-Fine Visual Servoing for 6-DoF Object-Agnostic Peg-In-Hole Assembly

Robotic peg-in-hole assembly remains a challenging task due to its high accuracy demand. Previous work tends to simplify the problem by restricting the degree of freedom of the end-effector, or limiting the distance between the target and the initial pose position, which prevents them from being deployed in real-world manufacturing. Thus, we present a Coarse-to-Fine Visual Servoing (CFVS) peg-in-hole method, achieving 6-DoF end-effector motion control based on 3D visual feedback. CFVS can handle arbitrary tilt angles and large initial alignment errors through a fast pose estimation before refinement. Furthermore, by introducing a confidence map to ignore the irrelevant contour of objects, CFVS is robust against noise and can deal with various targets beyond training data. Extensive experiments show CFVS outperforms state-of-the-art methods and obtains 100%, 91%, and 82% average success rates in 3-DoF, 4-DoF, and 6-DoF peg-in-hole, respectively