高速ビジョンを用いたリアルタイムビデオモザイキングと安定化に関する研究

広島大学(Hiroshima University)博士(工学)Doctor of Engineeringdoctora

Smart environment monitoring through micro unmanned aerial vehicles

In recent years, the improvements of small-scale Unmanned Aerial Vehicles (UAVs) in terms of flight time, automatic control, and remote transmission are promoting the development of a wide range of practical applications. In aerial video surveillance, the monitoring of broad areas still has many challenges due to the achievement of different tasks in real-time, including mosaicking, change detection, and object detection. In this thesis work, a small-scale UAV based vision system to maintain regular surveillance over target areas is proposed. The system works in two modes. The first mode allows to monitor an area of interest by performing several flights. During the first flight, it creates an incremental geo-referenced mosaic of an area of interest and classifies all the known elements (e.g., persons) found on the ground by an improved Faster R-CNN architecture previously trained. In subsequent reconnaissance flights, the system searches for any changes (e.g., disappearance of persons) that may occur in the mosaic by a histogram equalization and RGB-Local Binary Pattern (RGB-LBP) based algorithm. If present, the mosaic is updated. The second mode, allows to perform a real-time classification by using, again, our improved Faster R-CNN model, useful for time-critical operations. Thanks to different design features, the system works in real-time and performs mosaicking and change detection tasks at low-altitude, thus allowing the classification even of small objects. The proposed system was tested by using the whole set of challenging video sequences contained in the UAV Mosaicking and Change Detection (UMCD) dataset and other public datasets. The evaluation of the system by well-known performance metrics has shown remarkable results in terms of mosaic creation and updating, as well as in terms of change detection and object detection

GIFT-Grab: Real-time C++ and Python multi-channel video capture, processing and encoding API

GIFT-Grab is an open-source API for acquiring, processing and encoding video streams in real time. GIFT-Grab supports video acquisition using various frame-grabber hardware as well as from standard-compliant network streams and video files. The current GIFT-Grab release allows for multi-channel video acquisition and encoding at the maximum frame rate of supported hardware – 60 frames per second (fps). GIFT-Grab builds on well-established highly configurable multimedia libraries including FFmpeg and OpenCV. GIFT-Grab exposes a simplified high-level API, aimed at facilitating integration into client applications with minimal coding effort. The core implementation of GIFT-Grab is in C++11. GIFT-Grab also features a Python API compatible with the widely used scientific computing packages NumPy and SciPy. GIFT-Grab was developed for capturing multiple simultaneous intra-operative video streams from medical imaging devices. Yet due to the ubiquity of video processing in research, GIFT-Grab can be used in many other areas. GIFT-Grab is hosted and managed on the software repository of the Centre for Medical Image Computing (CMIC) at University College London, and is also mirrored on GitHub. In addition it is available for installation from the Python Package Index (PyPI) via the pip installation tool

An in Depth Review Paper on Numerous Image Mosaicing Approaches and Techniques

Image mosaicing is one of the most important subjects of research in computer vision at current. Image mocaicing requires the integration of direct techniques and feature based techniques. Direct techniques are found to be very useful for mosaicing large overlapping regions, small translations and rotations while feature based techniques are useful for small overlapping regions. Feature based image mosaicing is a combination of corner detection, corner matching, motion parameters estimation and image stitching.Furthermore, image mosaicing is considered the process of obtaining a wider field-of-view of a scene from a sequence of partial views, which has been an attractive research area because of its wide range of applications, including motion detection, resolution enhancement, monitoring global land usage, and medical imaging. Numerous algorithms for image mosaicing have been proposed over the last two decades.In this paper the authors present a review on different approaches for image mosaicing and the literature over the past few years in the field of image masaicing methodologies. The authors take an overview on the various methods for image mosaicing.This review paper also provides an in depth survey of the existing image mosaicing algorithms by classifying them into several groups. For each group, the fundamental concepts are first clearly explained. Finally this paper also reviews and discusses the strength and weaknesses of all the mosaicing groups

Multi-Modal Imaging Techniques for Early Cancer Diagnostics

Cancer kills more Americans under the age of 75 than any other disease. Although most cancers occur in epithelial surfaces that can be directly visualized, the majority of cases are detected at an advanced stage. Optical imaging and spectroscopy may provide a solution to the need for non-invasive and effective early detection tools. These technologies are capable of examining tissue over a wide range of spatial scales, with widefield macroscopic imaging typically spanning several square-centimeters, and high resolution in vivo microscopy techniques enabling cellular and subcellular features to be visualized. This work presents novel technologies in two important areas of optical imaging: high resolution imaging and widefield imaging. For subcellular imaging applications, new high resolution endomicroscope techniques are presented with improved lateral resolution, larger field-of-view, increased contrast, decreased background signal, and reduced cost compared to existing devices. A new widefield optical technology called multi-modal spectral imaging is also developed. This technique provides real-time in vivo spectral data over a large field-of-view, which is useful for detecting biochemical alterations associated with neoplasia. The described devices are compared to existing technologies, tested using ex vivo tissue specimens, and evaluated for diagnostic potential in a multi-patient oral cancer clinical trial

Development Of A High Performance Mosaicing And Super-Resolution Algorithm

In this dissertation, a high-performance mosaicing and super-resolution algorithm is described. The scale invariant feature transform (SIFT)-based mosaicing algorithm builds an initial mosaic which is iteratively updated by the robust super resolution algorithm to achieve the final high-resolution mosaic. Two different types of datasets are used for testing: high altitude balloon data and unmanned aerial vehicle data. To evaluate our algorithm, five performance metrics are employed: mean square error, peak signal to noise ratio, singular value decomposition, slope of reciprocal singular value curve, and cumulative probability of blur detection. Extensive testing shows that the proposed algorithm is effective in improving the captured aerial data and the performance metrics are accurate in quantifying the evaluation of the algorithm

Deep-sea image processing

High-resolution seafloor mapping often requires optical methods of sensing, to confirm interpretations made from sonar data. Optical digital imagery of seafloor sites can now provide very high resolution and also provides additional cues, such as color information for sediments, biota and divers rock types. During the cruise AT11-7 of the Woods Hole Oceanographic Institution (WHOI) vessel R/V Atlantis (February 2004, East Pacific Rise) visual imagery was acquired from three sources: (1) a digital still down-looking camera mounted on the submersible Alvin, (2) observer-operated 1-and 3-chip video cameras with tilt and pan capabilities mounted on the front of Alvin, and (3) a digital still camera on the WHOI TowCam (Fornari, 2003). Imagery from the first source collected on a previous cruise (AT7-13) to the Galapagos Rift at 86°W was successfully processed and mosaicked post-cruise, resulting in a single image covering area of about 2000 sq.m, with the resolution of 3 mm per pixel (Rzhanov et al., 2003). This paper addresses the issues of the optimal acquisition of visual imagery in deep-seaconditions, and requirements for on-board processing. Shipboard processing of digital imagery allows for reviewing collected imagery immediately after the dive, evaluating its importance and optimizing acquisition parameters, and augmenting acquisition of data over specific sites on subsequent dives.Images from the deepsea power and light (DSPL) digital camera offer the best resolution (3.3 Mega pixels) and are taken at an interval of 10 seconds (determined by the strobe\u27s recharge rate). This makes images suitable for mosaicking only when Alvin moves slowly (≪1/4 kt), which is not always possible for time-critical missions. Video cameras provided a source of imagery more suitable for mosaicking, despite its inferiority in resolution. We discuss required pre-processing and imageenhancement techniques and their influence on the interpretation of mosaic content. An algorithm for determination of camera tilt parameters from acquired imagery is proposed and robustness conditions are discussed

Application of landscape mosaics for the biological quality assessment of subtidal 2 macroalgae communities using the CFR index

The assessment of anthropogenic impacts on coastal waters is an important task to accomplish under the European Water Framework Directive (WFD 2000/60/EEC). Macroalgae are one of the biological quality elements that must be considered, but their assessment has been generally limited to intertidal areas due to the difficulties and costs associated with working in subtidal areas. In this work, the suitability of using landscape mosaicing techniques is analyzed for the application of the “Quality of Rocky Bottoms” index (CFR by its Spanish acronym) in subtidal areas. For this purpose, the sensitivity and accuracy of both the indicators that compose the CFR index (characteristic macroalgae coverage, fraction of opportunistics and characteristic macroalgae richness) and the index itself were tested against different sampling surfaces and validated through direct applications of the CFR carried out in situ by scuba divers. The study was carried out at three sites, located on the coast of Cantabria (N. Spain), covering a variety of environmental conditions (depth ranges and anthropogenic pressures). Underwater video transects of 5-20 m length were recorded by scuba divers and processed with specialized software to build continuous image mosaics of the assessment sites. Each mosaic was inserted into a Geographical Information System where all distinguishable macroalgae species were identified and their coverages were estimated. Replicated subsamples of different areas (0.25 m2, 0.5 m2, 1 m2 and 2.5 m2) were tested from each mosaic for the estimation of both the single indicators and the CFR index itself. Main results showed that larger subsample areas produced higher and more accurate CFR values, mainly related to higher richness values and to smaller variability within the replicates. Accordingly, the minimum sample size required to carry out this type of studies was estimated to be of 2.5 m2, showing no significant differences with the total mosaics. At this spatial scale, the assessments of the CFR index using mosaics showed a significant correlation and an excellent agreement with the results obtained in situ. In summary, underwater video mosaicing techniques proved to be a useful tool for the application of the CFR index and could also be of great interest for the study of subtidal environments by allowing visualization of extensive seafloor areas