37 research outputs found
Multi-label learning based semi-global matching forest
Semi-Global Matching (SGM) approximates a 2D Markov Random Field (MRF) via multiple 1D scanline optimizations, which serves as a good trade-off between accuracy and efficiency in dense matching. Nevertheless, the performance is limited due to the simple summation of the aggregated costs from all 1D scanline optimizations for the final disparity estimation. SGM-Forest improves the performance of SGM by training a random forest to predict the best scanline according to each scanline’s disparity proposal. The disparity estimated by the best scanline acts as reference to adaptively adopt close proposals for further post-processing. However, in many cases more than one scanline is capable of providing a good prediction. Training the random forest with only one scanline labeled may limit or even confuse the learning procedure when other scanlines can offer similar contributions. In this paper, we propose a multi-label classification strategy to further improve SGM-Forest. Each training sample is allowed to be described by multiple labels (or zero label) if more than one (or none) scanline gives a proper prediction. We test the proposed method on stereo matching datasets, from Middlebury, ETH3D, EuroSDR image matching benchmark, and the 2019 IEEE GRSS data fusion contest. The result indicates that under the framework of SGM-Forest, the multi-label strategy outperforms the single-label scheme consistently
Dense matching comparison between classical and deep learning based algorithms for remote sensing data
GA-Net-Pyramid: An Efficient End-to-End Network for Dense Matching
Dense matching plays a crucial role in computer vision and remote sensing, to rapidly provide stereo products using inexpensive hardware. Along with the development of deep learning, the Guided Aggregation Network (GA-Net) achieves state-of-the-art performance via the proposed Semi-Global Guided Aggregation layers and reduces the use of costly 3D convolutional layers. To solve the problem of GA-Net requiring large GPU memory consumption, we design a pyramid architecture to modify the model. Starting from a downsampled stereo input, the disparity is estimated and continuously refined through the pyramid levels. Thus, the disparity search is only applied for a small size of stereo pair and then confined within a short residual range for minor correction, leading to highly reduced memory usage and runtime. Tests on close-range, aerial, and satellite data demonstrate that the proposed algorithm achieves significantly higher efficiency (around eight times faster consuming only 20–40% GPU memory) and comparable results with GA-Net on remote sensing data. Thanks to this coarse-to-fine estimation, we successfully process remote sensing datasets with very large disparity ranges, which could not be processed with GA-Net due to GPU memory limitations
Early Detection of Forest Drought Stress with Very High Resolution Stereo and Hyperspectral Imagery
The project ‘Application of remote sensing for the early detection of drought stress at vulnerable forest sites (ForDroughtDet)’ is funded by the German Federal Agency of Agriculture and Food and aims to detect drought stress in an early phase using remote sensing techniques. In this project, three test sites in the south and middle part of Germany are selected. Three levels of observation and analyses are performed. In the first level, close-range stereo images and spectral information are captured with a research crane in Kranzberg forest. In the second level, three study sites are imaged twice in three years by airborne hyperspectral and stereo cameras. In the third level, the drought stress detection approach will be transferred to regional scale by satellite image. In this paper, we will briefly report our results from the first and second levels. In the first level, 3D models of the forest canopies are generated with the MC-CNN based dense matching approaches, with which the 3D shapes of the stressed and healthy trees are analysed. In addition, for the spectral analyses, different chlorophyll-sensitive indices are calculated and compared for the stressed and healthy trees. In order to further analyse the tree drought stress in the second level, a novel individual tree crown (ITC) segmentation approach is proposed and tested on the airborne stereo dataset
Middle and Late Pleistocene Denisovan subsistence at Baishiya Karst Cave
Genetic and fragmented palaeoanthropological data suggest that Denisovans were once widely distributed across eastern Eurasia . Despite limited archaeological evidence, this indicates that Denisovans were capable of adapting to a highly diverse range of environments. Here we integrate zooarchaeological and proteomic analyses of the late Middle to Late Pleistocene faunal assemblage from Baishiya Karst Cave on the Tibetan Plateau, where a Denisovan mandible and Denisovan sedimentary mitochondrial DNA were found . Using zooarchaeology by mass spectrometry, we identify a new hominin rib specimen that dates to approximately 48-32 thousand years ago (layer 3). Shotgun proteomic analysis taxonomically assigns this specimen to the Denisovan lineage, extending their presence at Baishiya Karst Cave well into the Late Pleistocene. Throughout the stratigraphic sequence, the faunal assemblage is dominated by Caprinae, together with megaherbivores, carnivores, small mammals and birds. The high proportion of anthropogenic modifications on the bone surfaces suggests that Denisovans were the primary agent of faunal accumulation. The chaîne opératoire of carcass processing indicates that animal taxa were exploited for their meat, marrow and hides, while bone was also used as raw material for the production of tools. Our results shed light on the behaviour of Denisovans and their adaptations to the diverse and fluctuating environments of the late Middle and Late Pleistocene of eastern Eurasia
Homogeneous pixel selection for distributed scatterers using multitemporal SAR data stacks
Land deformation and topography estimation using Synthetic Aperture Radar (SAR) interferometry has gained more and more attention over the last years due to its high quality. Up to now, there have been many high resolution SAR satellites launched, such as TerraSAR-X, TanDEM-X, and COSMO-SkyMed. Multitemporal interferometric SAR (InSAR) techniques have become important quantitative geodetic tools to monitor deformation time series. The Persistent Scatterer Interferometry (PSI) technique utilizes coherent radar targets exhibiting high phase stability. The technique is reliable for monitoring deformation with millimeter accuracy. Persistent Scatterers (PSs) e.g. man-made structures, boulders, and outcrops are widely available over a city, however, they have less density in non-urban areas. As a result, Distributed Scatterers (DSs) corresponding to image pixels belonging to areas of moderate coherence, where many neighboring pixels share similar reflectivity values (as they belong to the same object), are used in order to increase the density of measurement points, such as in the SqueeSAR, Small BAseline Subset (SBAS) techniques etc.
In order to improve the estimation accuracy of interferometric phase and coherence of DSs, multilooking is needed, where neighboring pixels are averaged together with the target pixel to reduce phase noise. Conventional boxcar kernel multilooking technique is based on the hypothesis of statistical homogeneity of the averaged pixels in a rectangular window surrounding the DS pixel. Nevertheless, as the size of the kernel increases, this hypothesis loses its validity. There can be loss of details as pixels arising from different statistical distributions are averaged together. Therefore, an adaptive multilooking technique is required to preserve the high resolution provided by modern satellites. This thesis aims to find reliable and robust methods to select homogeneous pixels, i.e. the pixels belonging to the same category of scatterers, to perform adaptive multilooking of DSs. The focus is on techniques that make use of the amplitude information of a multitemporal SAR data stack. Accordingly, two new methods, based on Confidence Interval (CI) of amplitude mean and CI of amplitude median have been developed as part of the thesis and principles behind these are clarified. Results of the selected methods are shown and compared using TerraSAR-X data from two different test regions (i.e. Lueneburg and Cologne). Performance, quality and robustness are analyzed. At last, the conclusions are summarized and some suggestions are proposed for future research. The algorithms can aid in accurate covariance matrix estimation and can be widely applied in topographic mapping and deformation monitoring
GA-Net-Pyramid: An Efficient End-to-End Network for Dense Matching
Dense matching plays a crucial role in computer vision and remote sensing, to rapidly provide stereo products using inexpensive hardware. Along with the development of deep learning, the Guided Aggregation Network (GA-Net) achieves state-of-the-art performance via the proposed Semi-Global Guided Aggregation layers and reduces the use of costly 3D convolutional layers. To solve the problem of GA-Net requiring large GPU memory consumption, we design a pyramid architecture to modify the model. Starting from a downsampled stereo input, the disparity is estimated and continuously refined through the pyramid levels. Thus, the disparity search is only applied for a small size of stereo pair and then confined within a short residual range for minor correction, leading to highly reduced memory usage and runtime. Tests on close-range, aerial, and satellite data demonstrate that the proposed algorithm achieves significantly higher efficiency (around eight times faster consuming only 20–40% GPU memory) and comparable results with GA-Net on remote sensing data. Thanks to this coarse-to-fine estimation, we successfully process remote sensing datasets with very large disparity ranges, which could not be processed with GA-Net due to GPU memory limitations
Comparison of the room’s dimensions and absorption distribution estimation performance using wave-based and geometrical acoustics dataset
Deep neural networks (DNNs) are trained to extract the room dimensions and absorption configurations from room transfer function (TF) measurements. This study investigates the performance of DNNs in room acoustic analyses, which are trained with wave-based (WB) and geometrical acoustics (GA) simulation data. WB simulation data provide a physically accurate representation of room acoustics including diffraction and interference, albeit with substantial computation demands. In contrast, GA data can be obtained more rapidly, but with reduced accuracy. We found that the DNN trained with WB training data exhibits enhanced estimation performance and generalization capabilities when applied to real-world measurements. This study underscores the trade-offs between training dataset generation speed and their performance of machine learning algorithms in the inverse proble