2,716 research outputs found

    Online Mutual Foreground Segmentation for Multispectral Stereo Videos

    Full text link
    The segmentation of video sequences into foreground and background regions is a low-level process commonly used in video content analysis and smart surveillance applications. Using a multispectral camera setup can improve this process by providing more diverse data to help identify objects despite adverse imaging conditions. The registration of several data sources is however not trivial if the appearance of objects produced by each sensor differs substantially. This problem is further complicated when parallax effects cannot be ignored when using close-range stereo pairs. In this work, we present a new method to simultaneously tackle multispectral segmentation and stereo registration. Using an iterative procedure, we estimate the labeling result for one problem using the provisional result of the other. Our approach is based on the alternating minimization of two energy functions that are linked through the use of dynamic priors. We rely on the integration of shape and appearance cues to find proper multispectral correspondences, and to properly segment objects in low contrast regions. We also formulate our model as a frame processing pipeline using higher order terms to improve the temporal coherence of our results. Our method is evaluated under different configurations on multiple multispectral datasets, and our implementation is available online.Comment: Preprint accepted for publication in IJCV (December 2018

    Evaluation of the damages caused by seismic events: First tests on supporting traditional multispectral classification with DSM

    Get PDF
    Seismic damages, as a roof entirely collapsed on the ground, are very difficult to be found using only multispectral classification algorithms. The availability of high resolution stereopairs from satellite disclose new possible fields of application to estimate changes and transformations of areas following catastrophic events. Combining both techniques it is obviously possible only when stereoscopic and multispectral images are available. In this case, as for all monitoring studies, it is necessary to compare the present situation to the pre-seismic one. The pre-seismic situation can be advantageously studied by classic photogrammetric techniques based on aerial frames, that are available in archives managed by photogrammetric companies and local government agencies. But it is also possible to extract the pre-seismic morphology from digital maps, containing the three-dimensional characteristics of the buildings. The present research tries to: a) improve the digital surface model extracted from Ikonos satellite images covering an area of central Italy (Foligno, Umbria), through a pre-treatment of images and a manual editing b) study the best DSM models to improve the detection of height difference, mainly in urban areas, and evaluate the results of the classification of land cover as further data to detect changes in building shape. DSM obtained by three-dimensional maps have been compared with DSM extracted directly from aerial stereo-pairs using different approaches. In the area under study a seismic event happened in September of the '97 causing relevant damages to different urbanized centres of the area

    Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

    Get PDF
    One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

    Object-Based Greenhouse Classification from GeoEye-1 and WorldView-2 Stereo Imagery

    Get PDF
    Remote sensing technologies have been commonly used to perform greenhouse detection and mapping. In this research, stereo pairs acquired by very high-resolution optical satellites GeoEye-1 (GE1) and WorldView-2 (WV2) have been utilized to carry out the land cover classification of an agricultural area through an object-based image analysis approach, paying special attention to greenhouses extraction. The main novelty of this work lies in the joint use of single-source stereo-photogrammetrically derived heights and multispectral information from both panchromatic and pan-sharpened orthoimages. The main features tested in this research can be grouped into different categories, such as basic spectral information, elevation data (normalized digital surface model; nDSM), band indexes and ratios, texture and shape geometry. Furthermore, spectral information was based on both single orthoimages and multiangle orthoimages. The overall accuracy attained by applying nearest neighbor and support vector machine classifiers to the four multispectral bands of GE1 were very similar to those computed from WV2, for either four or eight multispectral bands. Height data, in the form of nDSM, were the most important feature for greenhouse classification. The best overall accuracy values were close to 90%, and they were not improved by using multiangle orthoimages

    Enhancment of dense urban digital surface models from VHR optical satellite stereo data by pre-segmentation and object detection

    Get PDF
    The generation of digital surface models (DSM) of urban areas from very high resolution (VHR) stereo satellite imagery requires advanced methods. In the classical approach of DSM generation from stereo satellite imagery, interest points are extracted and correlated between the stereo mates using an area based matching followed by a least-squares sub-pixel refinement step. After a region growing the 3D point list is triangulated to the resulting DSM. In urban areas this approach fails due to the size of the correlation window, which smoothes out the usual steep edges of buildings. Also missing correlations as for partly – in one or both of the images – occluded areas will simply be interpolated in the triangulation step. So an urban DSM generated with the classical approach results in a very smooth DSM with missing steep walls, narrow streets and courtyards. To overcome these problems algorithms from computer vision are introduced and adopted to satellite imagery. These algorithms do not work using local optimisation like the area-based matching but try to optimize a (semi-)global cost function. Analysis shows that dynamic programming approaches based on epipolar images like dynamic line warping or semiglobal matching yield the best results according to accuracy and processing time. These algorithms can also detect occlusions – areas not visible in one or both of the stereo images. Beside these also the time and memory consuming step of handling and triangulating large point lists can be omitted due to the direct operation on epipolar images and direct generation of a so called disparity image fitting exactly on the first of the stereo images. This disparity image – representing already a sort of a dense DSM – contains the distances measured in pixels in the epipolar direction (or a no-data value for a detected occlusion) for each pixel in the image. Despite the global optimization of the cost function many outliers, mismatches and erroneously detected occlusions remain, especially if only one stereo pair is available. To enhance these dense DSM – the disparity image – a pre-segmentation approach is presented in this paper. Since the disparity image is fitting exactly on the first of the two stereo partners (beforehand transformed to epipolar geometry) a direct correlation between image pixels and derived heights (the disparities) exist. This feature of the disparity image is exploited to integrate additional knowledge from the image into the DSM. This is done by segmenting the stereo image, transferring the segmentation information to the DSM and performing a statistical analysis on each of the created DSM segments. Based on this analysis and spectral information a coarse object detection and classification can be performed and in turn the DSM can be enhanced. After the description of the proposed method some results are shown and discussed

    Deep learning in remote sensing: a review

    Get PDF
    Standing at the paradigm shift towards data-intensive science, machine learning techniques are becoming increasingly important. In particular, as a major breakthrough in the field, deep learning has proven as an extremely powerful tool in many fields. Shall we embrace deep learning as the key to all? Or, should we resist a 'black-box' solution? There are controversial opinions in the remote sensing community. In this article, we analyze the challenges of using deep learning for remote sensing data analysis, review the recent advances, and provide resources to make deep learning in remote sensing ridiculously simple to start with. More importantly, we advocate remote sensing scientists to bring their expertise into deep learning, and use it as an implicit general model to tackle unprecedented large-scale influential challenges, such as climate change and urbanization.Comment: Accepted for publication IEEE Geoscience and Remote Sensing Magazin

    Radiomeric and Geometric Analysis of Worldview-2 Stereo Scenes

    Get PDF
    WorldView-2 (WV-2) is DigitalGlobe's latest very high resolution optical sensor. Launched on October 8, 2009 and full operational since January 5, 2010, it flies along a sun-synchronous orbit at an altitude of 770km. The sensor is able to acquire panchromatic imagery at 0.46m ground resolution (0.52m at 20° off-nadir) and multispectral images in eight spectral bands at 1.8m resolution. In addition to the four typical multispectral bands (blue, green, red, near infrared), the sensors scans in the coastal (400-450nm), yellow (585-625nm), red edge (705-745nm) and near infrared-2 (860-1040nm) bands. Stereo images can be planned thanks to the ability of the sensor to rotate off-nadir up to +/-45°. This paper describes the investigations on WV-2 that are carried out at the EU Joint Research Center (JRC) in Ispra (Italy), by the ISFEREA Team of the Istitute for the Protection and Security of the Citizen (IPSC). The purpose of the study is to evaluate the radiometric and geometric properties of the sensor and its potentials for 3D information extraction. The images used for the analysis are a stereopair over North-West Italy, where ISFEREA has established a testfield. The radiometry of the images is evaluated through different methods, including the estimation of the noise level (standard deviation of the digital number) in homogeneous and non-homogeneous areas. The images are oriented with an RPC-based approach: using GCP (ground control points), the given Rational Polynomial Coefficients (RPC) are improved with an affine transformation to compensate residual systematic errors. The achieved accuracy is investigated by varying the number and distribution of GCPs. For automatic 3D information assessment, Digital Surface Models (DSM) are generated with different commercial software available at JRC and compared with reference data. The processing steps are described in the paper and the results discussed in detail.JRC.DG.G.2-Global security and crisis managemen
    • …
    corecore