Retrieval and Registration of Long-Range Overlapping Frames for Scalable Mosaicking of In Vivo Fetoscopy

Purpose: The standard clinical treatment of Twin-to-Twin Transfusion Syndrome consists in the photo-coagulation of undesired anastomoses located on the placenta which are responsible to a blood transfer between the two twins. While being the standard of care procedure, fetoscopy suffers from a limited field-of-view of the placenta resulting in missed anastomoses. To facilitate the task of the clinician, building a global map of the placenta providing a larger overview of the vascular network is highly desired. Methods: To overcome the challenging visual conditions inherent to in vivo sequences (low contrast, obstructions or presence of artifacts, among others), we propose the following contributions: (i) robust pairwise registration is achieved by aligning the orientation of the image gradients, and (ii) difficulties regarding long-range consistency (e.g. due to the presence of outliers) is tackled via a bag-of-word strategy, which identifies overlapping frames of the sequence to be registered regardless of their respective location in time. Results: In addition to visual difficulties, in vivo sequences are characterised by the intrinsic absence of gold standard. We present mosaics motivating qualitatively our methodological choices and demonstrating their promising aspect. We also demonstrate semi-quantitatively, via visual inspection of registration results, the efficacy of our registration approach in comparison to two standard baselines. Conclusion: This paper proposes the first approach for the construction of mosaics of placenta in in vivo fetoscopy sequences. Robustness to visual challenges during registration and long-range temporal consistency are proposed, offering first positive results on in vivo data for which standard mosaicking techniques are not applicable.Comment: Accepted for publication in International Journal of Computer Assisted Radiology and Surgery (IJCARS

Free Viewpoint Video Based on Stitching Technique

Image stitching is a technique used for creating one panoramic scene from multiple images. It is used in panoramic photography and video where the viewer can only scroll horizontally and vertically across the scene. However, stitching has not been used for creating free-viewpoint videos (FVV) where viewers can change their viewing points freely and smoothly while playing the video. current research, implemented FVV playing system using image stitching, this system allows users to enjoy the capability of moving their viewpoint freely and smoothly. To develop this system, user should capture MVV from different viewpoints and with appropriate region area for each pair of cameras then the system stitch the overlapped video to create stitched video/videos to display it in FVV playing system with applying freely and smoothly switching and interpolation of viewpoints over video playback. Current research evaluated the performance of video playing system based on system idea, system accuracy, smoothness, and user satisfaction. The results of evaluation have been very positive in most aspects

???????????? ????????? ????????? ??????????????? ?????? ????????? ????????? ??????????????? ?????????

Department of Biomedical EngineeringImage stitching is a well-known method to make panoramic image which has a wide field-of-view and high resolution. It has been used in various fields such as digital map, gigapixel imaging, and 360-degree camera. However, commercial stitching tools often fail, require a lot of processing time, and only work on certain images. The problems of existing tools are mainly caused by trying to stitch the wrong image pair. To overcome these problems, it is important to select suitable image pair for stitching in advance. Nevertheless, there are no universal standards to judge the good image pairs. Moreover, the derived stitching algorithms cannot be compatible with each other because they conform to their own available criteria. Here, we present universal stitching parameters and their conditions for selecting good image pairs. The proposed stitching parameters can be easily calculated through analysis of corresponding features and homography, which are basic elements in feature-based image stitching algorithm. In order to specify the conditions of the stitching parameters, we devised a new method to calculate stitching accuracy for qualifying stitching results into 3 classesgood, bad, and fail. With the classed stitching results, the values of the stitching parameters could be checked how they differ in each class. Through experiments with large datasets, the most valid parameter for each class is identified as filtering level which is calculated in corresponding feature analysis. In addition, supplemental experiments were conducted with various datasets to demonstrate the validity of the filtering level. As a result of our study, universal stitching parameters can judge the success of stitching, so that it is possible to prevent stitching errors through parameter verification test in advance. This paper can greatly contribute to guide for creating high performance and high efficiency stitching software by applying the proposed stitching conditions.ope

Smart environment monitoring through micro unmanned aerial vehicles

In recent years, the improvements of small-scale Unmanned Aerial Vehicles (UAVs) in terms of flight time, automatic control, and remote transmission are promoting the development of a wide range of practical applications. In aerial video surveillance, the monitoring of broad areas still has many challenges due to the achievement of different tasks in real-time, including mosaicking, change detection, and object detection. In this thesis work, a small-scale UAV based vision system to maintain regular surveillance over target areas is proposed. The system works in two modes. The first mode allows to monitor an area of interest by performing several flights. During the first flight, it creates an incremental geo-referenced mosaic of an area of interest and classifies all the known elements (e.g., persons) found on the ground by an improved Faster R-CNN architecture previously trained. In subsequent reconnaissance flights, the system searches for any changes (e.g., disappearance of persons) that may occur in the mosaic by a histogram equalization and RGB-Local Binary Pattern (RGB-LBP) based algorithm. If present, the mosaic is updated. The second mode, allows to perform a real-time classification by using, again, our improved Faster R-CNN model, useful for time-critical operations. Thanks to different design features, the system works in real-time and performs mosaicking and change detection tasks at low-altitude, thus allowing the classification even of small objects. The proposed system was tested by using the whole set of challenging video sequences contained in the UAV Mosaicking and Change Detection (UMCD) dataset and other public datasets. The evaluation of the system by well-known performance metrics has shown remarkable results in terms of mosaic creation and updating, as well as in terms of change detection and object detection

Real Time Panoramic Image Processing

Image stitching algorithms are able to join sets of images together and provide a wider field of a vision when compared with an image from a single standard camera. Traditional techniques for accomplishing this are able to adequately produce a stitch for a static set of images, but suffer when differing lighting conditions exist between the two images. Additionally, traditional techniques suffer from processing times that are too slow for real time use cases. We propose a solution which resolves the issues encountered by traditional image stitching techniques. To resolve the issues with lighting difference, two blending schemes have been implemented, a standard approach and a superpixel approach. To verify the integrity of the cached solution, a validation scheme has been implemented. Using this scheme, invalid solutions can be detected, and the cache regenerated. Finally, these components are packaged together in a parallel processing architecture to ensure that frame processing is never interrupted

An Intelligent Portable Aerial Surveillance System: Modeling and Image Stitching

Unmanned Aerial Vehicles (UAVs) have been widely used in modern warfare for surveillance, reconnaissance and even attack missions. They can provide valuable battlefield information and accomplish dangerous tasks with minimal risk of loss of lives and personal injuries. However, existing UAV systems are far from perfect to meet all possible situations. One of the most notable situations is the support for individual troops. Besides the incapability to always provide images in desired resolution, currently available systems are either too expensive for large-scale deployment or too heavy and complex for a single solder. Intelligent Portable Aerial Surveillance System (IPASS), sponsored by the Air Force Research Laboratory (AFRL), is aimed at developing a low-cost, light-weight unmanned aerial vehicle that can provide sufficient battlefield intelligence for individual troops. The main contributions of this thesis are two-fold (1) the development and verification of a model-based flight simulation for the aircraft, (2) comparison of image stitching techniques to provide a comprehensive aerial surveillance information from multiple vision. To assist with the design and control of the aircraft, dynamical models are established at different complexity levels. Simulations with these models are implemented in Matlab to study the dynamical characteristics of the aircraft. Aerial images acquired from the three onboard cameras are processed after getting the flying platform built. How a particular image is formed from a camera and the general pipeline of the feature-based image stitching method are first introduced in the thesis. To better satisfy the needs of this application, a homography-based stitching method is studied. This method can greatly reduce computation time with very little compromise in the quality of the panorama, which makes real-time video display of the surroundings on the ground station possible. By implementing both of the methods for image stitching using OpenCV, a quantitative comparison in the performance is accomplished

Optical quality assurance of GEM foils

An analysis software was developed for the high aspect ratio optical scanning system in the Detec- tor Laboratory of the University of Helsinki and the Helsinki Institute of Physics. The system is used e.g. in the quality assurance of the GEM-TPC detectors being developed for the beam diagnostics system of the SuperFRS at future FAIR facility. The software was tested by analyzing five CERN standard GEM foils scanned with the optical scanning system. The measurement uncertainty of the diameter of the GEM holes and the pitch of the hole pattern was found to be 0.5 {\mu}m and 0.3 {\mu}m, respectively. The software design and the performance are discussed. The correlation between the GEM hole size distribution and the corresponding gain variation was studied by comparing them against a detailed gain mapping of a foil and a set of six lower precision control measurements. It can be seen that a qualitative estimation of the behavior of the local variation in gain across the GEM foil can be made based on the measured sizes of the outer and inner holes.Comment: 12 pages, 29 figure

A study of smart device-based mobile imaging and implementation for engineering applications

Title from PDF of title page, viewed on June 12, 2013Thesis advisor: ZhiQiang ChenVitaIncludes bibliographic references (pages 76-82)Thesis (M.S.)--School of Computing and Engineering. University of Missouri--Kansas City, 2013Mobile imaging has become a very active research topic in recent years thanks to the rapid development of computing and sensing capabilities of mobile devices. This area features multi-disciplinary studies of mobile hardware, imaging sensors, imaging and vision algorithms, wireless network and human-machine interface problems. Due to the limitation of computing capacity that early mobile devices have, researchers proposed client-server module, which push the data to more powerful computing platforms through wireless network, and let the cloud or standalone servers carry out all the computing and processing work. This thesis reviewed the development of mobile hardware and software platform, and the related research done on mobile imaging for the past 20 years. There are several researches on mobile imaging, but few people aim at building a framework which helps engineers solving problems by using mobile imaging. With higher-resolution imaging and high-performance computing power built into smart mobile devices, more and more imaging processing tasks can be achieved on the device rather than the client-server module. Based on this fact, a framework of collaborative mobile imaging is introduced for civil infrastructure condition assessment to help engineers solving technical challenges. Another contribution in this thesis is applying mobile imaging application into home automation. E-SAVE is a research project focusing on extensive use of automation in conserving and using energy wisely in home automation. Mobile users can view critical information such as energy data of the appliances with the help of mobile imaging. OpenCV is an image processing and computer vision library. The applications in this thesis use functions in OpenCV including camera calibration, template matching, image stitching and Canny edge detection. The application aims to help field engineers is interactive crack detection. The other one uses template matching to recognize appliances in the home automation system.Introduction -- Background and related work -- Basic imaging processing methods for mobile applications -- Collaborative and interactive mobile imaging -- Mobile imaging for smart energy -- Conclusion and recommendation