2,716 research outputs found
Online Mutual Foreground Segmentation for Multispectral Stereo Videos
The segmentation of video sequences into foreground and background regions is
a low-level process commonly used in video content analysis and smart
surveillance applications. Using a multispectral camera setup can improve this
process by providing more diverse data to help identify objects despite adverse
imaging conditions. The registration of several data sources is however not
trivial if the appearance of objects produced by each sensor differs
substantially. This problem is further complicated when parallax effects cannot
be ignored when using close-range stereo pairs. In this work, we present a new
method to simultaneously tackle multispectral segmentation and stereo
registration. Using an iterative procedure, we estimate the labeling result for
one problem using the provisional result of the other. Our approach is based on
the alternating minimization of two energy functions that are linked through
the use of dynamic priors. We rely on the integration of shape and appearance
cues to find proper multispectral correspondences, and to properly segment
objects in low contrast regions. We also formulate our model as a frame
processing pipeline using higher order terms to improve the temporal coherence
of our results. Our method is evaluated under different configurations on
multiple multispectral datasets, and our implementation is available online.Comment: Preprint accepted for publication in IJCV (December 2018
Evaluation of the damages caused by seismic events: First tests on supporting traditional multispectral classification with DSM
Seismic damages, as a roof entirely collapsed on the ground, are very difficult to be found using only multispectral classification
algorithms. The availability of high resolution stereopairs from satellite disclose new possible fields of application to estimate
changes and transformations of areas following catastrophic events. Combining both techniques it is obviously possible only when
stereoscopic and multispectral images are available. In this case, as for all monitoring studies, it is necessary to compare the present
situation to the pre-seismic one.
The pre-seismic situation can be advantageously studied by classic photogrammetric techniques based on aerial frames, that are
available in archives managed by photogrammetric companies and local government agencies. But it is also possible to extract the
pre-seismic morphology from digital maps, containing the three-dimensional characteristics of the buildings. The present research
tries to: a) improve the digital surface model extracted from Ikonos satellite images covering an area of central Italy (Foligno,
Umbria), through a pre-treatment of images and a manual editing b) study the best DSM models to improve the detection of height
difference, mainly in urban areas, and evaluate the results of the classification of land cover as further data to detect changes in
building shape. DSM obtained by three-dimensional maps have been compared with DSM extracted directly from aerial stereo-pairs
using different approaches. In the area under study a seismic event happened in September of the '97 causing relevant damages to
different urbanized centres of the area
Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery
One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions
Object-Based Greenhouse Classification from GeoEye-1 and WorldView-2 Stereo Imagery
Remote sensing technologies have been commonly used to perform greenhouse detection and mapping. In this research, stereo pairs acquired by very high-resolution optical satellites GeoEye-1 (GE1) and WorldView-2 (WV2) have been utilized to carry out the land cover classification of an agricultural area through an object-based image analysis approach, paying special attention to greenhouses extraction. The main novelty of this work lies in the joint use of single-source stereo-photogrammetrically derived heights and multispectral information from both panchromatic and pan-sharpened orthoimages. The main features tested in this research can be grouped into different categories, such as basic spectral information, elevation data (normalized digital surface model; nDSM), band indexes and ratios, texture and shape geometry. Furthermore, spectral information was based on both single orthoimages and multiangle orthoimages. The overall accuracy attained by applying nearest neighbor and support vector machine classifiers to the four multispectral bands of GE1 were very similar to those computed from WV2, for either four or eight multispectral bands. Height data, in the form of nDSM, were the most important feature for greenhouse classification. The best overall accuracy values were close to 90%, and they were not improved by using multiangle orthoimages
Enhancment of dense urban digital surface models from VHR optical satellite stereo data by pre-segmentation and object detection
The generation of digital surface models (DSM) of urban areas from very high resolution (VHR) stereo satellite imagery requires advanced methods. In the classical approach of DSM generation from stereo satellite imagery, interest points are extracted and correlated between the stereo mates using an area based matching followed by a least-squares sub-pixel refinement step. After a region growing the 3D point list is triangulated to the resulting DSM. In urban areas this approach fails due to the size of the correlation window, which smoothes out the usual steep edges of buildings. Also missing correlations as for partly – in one or both of the images – occluded areas will simply be interpolated in the triangulation step. So an urban DSM generated with the classical approach results in a very smooth DSM with missing steep walls, narrow streets and courtyards. To overcome these problems algorithms from computer vision are introduced and adopted to satellite imagery. These algorithms do not work using local optimisation like the area-based matching but try to optimize a (semi-)global cost function. Analysis shows that dynamic programming approaches based on epipolar images like dynamic line warping or semiglobal matching yield the best results according to accuracy and processing time. These algorithms can also detect occlusions – areas not visible in one or both of the stereo images. Beside these also the time and memory consuming step of handling and triangulating large point lists can be omitted due to the direct operation on epipolar images and direct generation of a so called disparity image fitting exactly on the first of the stereo images. This disparity image – representing already a sort of a dense DSM – contains the distances measured in pixels in the epipolar direction (or a no-data value for a detected occlusion) for each pixel in the image. Despite the global optimization of the cost function many outliers, mismatches and erroneously detected occlusions remain, especially if only one stereo pair is available. To enhance these dense DSM – the disparity image – a pre-segmentation approach is presented in this paper. Since the disparity image is fitting exactly on the first of the two stereo partners (beforehand transformed to epipolar geometry) a direct
correlation between image pixels and derived heights (the disparities) exist. This feature of the disparity image is exploited to integrate additional knowledge from the image into the DSM. This is done by segmenting the stereo image, transferring the segmentation information to the DSM and performing a statistical analysis on each of the created DSM segments. Based on this analysis and spectral information a coarse object detection and classification can be performed and in turn the DSM can be enhanced. After the description of the proposed method some results are shown and discussed
Deep learning in remote sensing: a review
Standing at the paradigm shift towards data-intensive science, machine
learning techniques are becoming increasingly important. In particular, as a
major breakthrough in the field, deep learning has proven as an extremely
powerful tool in many fields. Shall we embrace deep learning as the key to all?
Or, should we resist a 'black-box' solution? There are controversial opinions
in the remote sensing community. In this article, we analyze the challenges of
using deep learning for remote sensing data analysis, review the recent
advances, and provide resources to make deep learning in remote sensing
ridiculously simple to start with. More importantly, we advocate remote sensing
scientists to bring their expertise into deep learning, and use it as an
implicit general model to tackle unprecedented large-scale influential
challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin
Radiomeric and Geometric Analysis of Worldview-2 Stereo Scenes
WorldView-2 (WV-2) is DigitalGlobe's latest very high resolution optical sensor. Launched on October 8, 2009 and full operational since January 5, 2010, it flies along a sun-synchronous orbit at an altitude of 770km. The sensor is able to acquire panchromatic imagery at 0.46m ground resolution (0.52m at 20° off-nadir) and multispectral images in eight spectral bands at 1.8m resolution. In addition to the four typical multispectral bands (blue, green, red, near infrared), the sensors scans in the coastal (400-450nm), yellow (585-625nm), red edge (705-745nm) and near infrared-2 (860-1040nm) bands. Stereo images can be planned thanks to the ability of the sensor to rotate off-nadir up to +/-45°.
This paper describes the investigations on WV-2 that are carried out at the EU Joint Research Center (JRC) in Ispra (Italy), by the ISFEREA Team of the Istitute for the Protection and Security of the Citizen (IPSC). The purpose of the study is to evaluate the radiometric and geometric properties of the sensor and its potentials for 3D information extraction.
The images used for the analysis are a stereopair over North-West Italy, where ISFEREA has established a testfield.
The radiometry of the images is evaluated through different methods, including the estimation of the noise level (standard deviation of the digital number) in homogeneous and non-homogeneous areas. The images are oriented with an RPC-based approach: using GCP (ground control points), the given Rational Polynomial Coefficients (RPC) are improved with an affine transformation to compensate residual systematic errors. The achieved accuracy is investigated by varying the number and distribution of GCPs. For automatic 3D information assessment, Digital Surface Models (DSM) are generated with different commercial software available at JRC and compared with reference data. The processing steps are described in the paper and the results discussed in detail.JRC.DG.G.2-Global security and crisis managemen
- …