DSM Building Shape Refinement from Combined Remote Sensing Images based on Wnet-cGANs

We describe the workflow of a digital surface models (DSMs) refinement algorithm using a hybrid conditional generative adversarial network (cGAN) where the generative part consists of two parallel networks merged at the last stage forming a WNet architecture. The inputs to the so-called WNet-cGAN are stereo DSMs and panchromatic (PAN) half-meter resolution satellite images. Fusing these helps to propagate fine detailed information from a spectral image and complete the missing 3D knowledge from a stereo DSM about building shapes. Besides, it refines the building outlines and edges making them more rectangular and sharp

Long-Short Skip Connections in Deep Neural Networks for DSM Refinement

Detailed digital surface models (DSMs) from space-borne sensors are the key to successful solutions for many remote sensing problems, like environmental disaster simulations, change detection in rural and urban areas, 3D urban modeling for city planning and management, etc. Traditional methodologies, e.g., stereo matching, used to generate photogrammetric DSMs from stereo imagery, usually deliver low-quality results due to the matching errors in homogeneous areas or the lack of information when observing the scene under different viewing angles. This makes the tasks related to building reconstruction very challenging since in most cases it is difficult to recognize the type of roofs, especially if overlaid with trees. This work represents a continuation of research regarding the automatic optimization of building geometries in photogrammetric DSMs with half-meter resolution and introduces an improved generative adversarial network (GAN) architecture which allows to reconstruct complete and detailed building structures without neglecting even low-rise urban constructions. The generative part of the network is constructed in a way that it simultaneously processes height and intensity information, and combines short and long skip connections within one architecture. To improve different aspects of the surface, several loss terms are used, the contributions of which are automatically balanced during training. The obtained results demonstrate that the proposed methodology can achieve two goals without any manual intervention: improve the roof surfaces by making them more planar and also recognize and optimize even small residential buildings which are hard to detect

SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis

Synthesizing realistic images from human drawn sketches is a challenging problem in computer graphics and vision. Existing approaches either need exact edge maps, or rely on retrieval of existing photographs. In this work, we propose a novel Generative Adversarial Network (GAN) approach that synthesizes plausible images from 50 categories including motorcycles, horses and couches. We demonstrate a data augmentation technique for sketches which is fully automatic, and we show that the augmented data is helpful to our task. We introduce a new network building block suitable for both the generator and discriminator which improves the information flow by injecting the input image at multiple scales. Compared to state-of-the-art image translation methods, our approach generates more realistic images and achieves significantly higher Inception Scores.Comment: Accepted to CVPR 201

Deep learning for cardiac image segmentation: A review

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound (US) and major anatomical structures of interest (ventricles, atria and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research

Deep learning methods applied to digital elevation models: state of the art

Deep Learning (DL) has a wide variety of applications in various thematic domains, including spatial information. Although with limitations, it is also starting to be considered in operations related to Digital Elevation Models (DEMs). This study aims to review the methods of DL applied in the field of altimetric spatial information in general, and DEMs in particular. Void Filling (VF), Super-Resolution (SR), landform classification and hydrography extraction are just some of the operations where traditional methods are being replaced by DL methods. Our review concludes that although these methods have great potential, there are aspects that need to be improved. More appropriate terrain information or algorithm parameterisation are some of the challenges that this methodology still needs to face.Functional Quality of Digital Elevation Models in Engineering’ of the State Agency Research of SpainPID2019-106195RB- I00/AEI/10.13039/50110001103