Immersive Visual Information Mining for Exploring the Content of TerraSAR-X Archives

The amount of collected Earth Observation (EO) data is increasing intensively in order of several Terabytes a day. Consequently, new trends to explore and retrieve information from the available data is highly demanded. Recent proposed methods for exploring EO data are mainly based on Image Information Mining (IIM). Because the main steps of this approach are image features extraction, data reduction, and labeling, developing a new process chain, mainly based on human interaction, might be a promising solution. More precisely, this chain provides an active learning system by interacting human users with features descriptors in a virtual environment. The focus of this article is an Immersive Visual Information Mining in which feature descriptors/images are visualized in a 3-D virtual environment, so called CAVE. This environment also allows the users to manipulate the feature space to increase the performance of learning process in an interactive manner. In our experiments, we use a dataset of Synthetic Aperture Radar (SAR) images. To process the data, the contents of images are represented by three different feature descriptors comprising Gabor, adapted WLD, and Bag-of-Words

Saving Lives from Above: Person Detection in Disaster Response Using Deep Neural Networks

This paper focuses on person detection in aerial and drone imagery, which is crucial for various operations such as situational awareness, search and rescue, and safe delivery of supplies. We aim to improve disaster response efforts by enhancing the speed, safety, and effectiveness of the process. Therefore, we introduce a new person detection dataset comprising 311 annotated aerial and drone images, acquired from helicopters and drones in different scenes, including urban and rural areas, and for different scenarios, such as estimation of damage in disaster-affected zones, and search and rescue operations in different countries. The amount of data considered and level of detail of the annotations resulted in a total of 10,050 annotated persons. To detect people in aerial and drone images, we propose a multi-stage training procedure to improve YOLOv3's ability. The proposed procedure aims at addressing challenges such as variations in scenes, scenarios, people poses, as well as image scales and viewing angles. To evaluate the effectiveness of our proposed training procedure, we split our dataset into a training and a test set. The latter includes images acquired during real search and rescue exercises and operations, and is therefore representative for the challenges encountered during operational missions and suitable for an accurate assessment of the proposed models. Experimental results demonstrate the effectiveness of our proposed training procedure, as the model's average precision exhibits a relevant increase with respect to the baseline value

Airborne-Shadow: Towards Fine-Grained Shadow Detection in Aerial Imagery

Shadow detection is the first step in the process of shadow removal, which improves the understanding of complex urban scenes in aerial imagery for applications such as autonomous driving, infrastructure monitoring, and mapping. However, the limited annotation in existing datasets hinders the effectiveness of semantic segmentation and the ability of shadow removal algorithms to meet the fine-grained requirements of real-world applications. To address this problem, we present Airborne-Shadow (ASD), a meticulously annotated dataset for shadow detection in aerial imagery. Unlike existing datasets, ASD includes annotations for both heavy and light shadows, covering various structures ranging from buildings and bridges to smaller details such as poles and fences. Therefore, we define shadow detection tasks for multi-class, single class, and merging two classes. Extensive experiments show the challenges that state-of-the-art semantic segmentation and shadow detection algorithms face in handling different shadow sizes, scales, and fine details, while still achieving comparable results to conventional methods. We make the ASD dataset publicly available to encourage progress in shadow detection

EAGLE: Large-scale Vehicle Detection Dataset in Real-World Scenarios using Aerial Imagery

Multi-class vehicle detection from airborne imagery with orientation estimation is an important task in the near and remote vision domains with applications in traffic monitoring and disaster management. In the last decade, we have witnessed significant progress in object detection in ground imagery, but it is still in its infancy in airborne imagery, mostly due to the scarcity of diverse and large-scale datasets. Despite being a useful tool for different applications, current airborne datasets only partially reflect the challenges of real-world scenarios. To address this issue, we introduce EAGLE (oriEnted vehicle detection using Aerial imaGery in real-worLd scEnarios), a large-scale dataset for multi-class vehicle detection with object orientation information in aerial imagery. It features high-resolution aerial images composed of different real-world situations with a wide variety of camera sensor, resolution, flight altitude, weather, illumination, haze, shadow, time, city, country, occlusion, and camera angle. The annotation was done by airborne imagery experts with small- and large-vehicle classes. EAGLE contains 215,986 instances annotated with oriented bounding boxes defined by four points and orientation, making it by far the largest dataset to date in this task. It also supports researches on the haze and shadow removal as well as super-resolution and in-painting applications. We define three tasks: detection by (1) horizontal bounding boxes, (2) rotated bounding boxes, and (3) oriented bounding boxes. We carried out several experiments to evaluate several state-of-the-art methods in object detection on our dataset to form a baseline. Experiments show that the EAGLE dataset accurately reflects real-world situations and correspondingly challenging applications.Comment: Accepted in ICPR 202

AerialMPTNet: Multi-Pedestrian Tracking in Aerial Imagery Using Temporal and Graphical Features

Multi-pedestrian tracking in aerial imagery has several applications such as large-scale event monitoring, disaster management, search-and-rescue missions, and as input into predictive crowd dynamic models. Due to the challenges such as the large number and the tiny size of the pedestrians (e.g., 4 x 4 pixels) with their similar appearances as well as different scales and atmospheric conditions of the images with their extremely low frame rates (e.g., 2 fps), current state-of-the-art algorithms including the deep learning-based ones are unable to perform well. In this paper, we propose AerialMPTNet, a novel approach for multi-pedestrian tracking in geo-referenced aerial imagery by fusing appearance features from a Siamese Neural Network, movement predictions from a Long Short-Term Memory, and pedestrian interconnections from a GraphCNN. In addition, to address the lack of diverse aerial pedestrian tracking datasets, we introduce the Aerial Multi-Pedestrian Tracking (AerialMPT) dataset consisting of 307 frames and 44,740 pedestrians annotated. We believe that AerialMPT is the largest and most diverse dataset to this date and will be released publicly. We evaluate AerialMPTNet on AerialMPT and KIT AIS, and benchmark with several state-of-the-art tracking methods. Results indicate that AerialMPTNet significantly outperforms other methods on accuracy and time-efficiency.Comment: ICPR 202

Multiple Pedestrians and Vehicles Tracking in Aerial Imagery Using a Convolutional Neural Network

In this paper, we address various challenges in multi-pedestrian and vehicle tracking in high-resolution aerial imagery by intensive evaluation of a number of traditional and Deep Learning based Single- and Multi-Object Tracking methods. We also describe our proposed Deep Learning based Multi-Object Tracking method Aerial MPTNet that fuses appearance, temporal, and graphical information using a Siamese Neural Network, a Long Short-Term Memory, and a Graph Convolutional Neural Network module for more accurate and stable tracking. Moreover, we investigate the influence of the Squeeze-and-Excitation layers and Online Hard Example Mining on the performance of Aerial MPTNet. To the best of our knowledge, we are the first to use these two for regression-based Multi-Object Tracking. Additionally, we studied and compared the L1 and Huber loss functions. In our experiments, we extensively evaluate AerialMPTNet on three aerial Multi-Object Trackingdatasets, namely AerialMPT and KIT AIS pedestrian and vehicle datasets. Qualitative and quantitative results show that AerialMPTNet outperforms all previous methods for the pedestrian datasets and achieves competitive results for the vehicle dataset. In addition, Long Short-Term Memory and Graph Convolutional Neural Network modules enhance the tracking performance. Moreover, using Squeeze-and-Excitation and Online Hard Example Mining significantly helps for some cases while degrades the results for other cases. In addition, according to the results, L1 yields better results with respect to Huber loss for most of the scenarios. The presented results provide a deep insight into challenges and opportunities of the aerial Multi-Object Tracking domain, paving the way for future research

Automatic 3-D Building Model Reconstruction from Very High Resolution Stereo Satellite Imagery

Recent advances in the availability of very high-resolution (VHR) satellite data together with efficient data acquisition and large area coverage have led to an upward trend in their applications for automatic 3-D building model reconstruction which require large-scale and frequent updates, such as disaster monitoring and urban management. Digital Surface Models (DSMs) generated from stereo satellite imagery suffer from mismatches, missing values, or blunders, resulting in rough building shape representations. To handle 3-D building model reconstruction using such low-quality DSMs, we propose a novel automatic multistage hybrid method using DSMs together with orthorectified panchromatic (PAN) and pansharpened data (PS) of multispectral (MS) satellite imagery. The algorithm consists of multiple steps including building boundary extraction and decomposition, image-based roof type classification, and initial roof parameter computation which are prior knowledge for the 3-D model fitting step. To fit 3-D models to the normalized DSM (nDSM) and to select the best one, a parameter optimization method based on exhaustive search is used sequentially in 2-D and 3-D. Finally, the neighboring building models in a building block are intersected to reconstruct the 3-D model of connecting roofs. All corresponding experiments are conducted on a dataset including four different areas of Munich city containing 208 buildings with different degrees of complexity. The results are evaluated both qualitatively and quantitatively. According to the results, the proposed approach can reliably reconstruct 3-D building models, even the complex ones with several inner yards and multiple orientations. Furthermore, the proposed approach provides a high level of automation by limiting the number of primitive roof types and by performing automatic parameter initialization

Vehicle Occlusion Removal from Single Aerial Images Using Generative Adversarial Networks

Removing occluding objects such as vehicles from drivable areas allows precise extraction of road boundaries and related semantic objects such as lane-markings, which is crucial for several applications such as generating high-definition maps for autonomous driving. Conventionally, multiple images of the same area taken at different times or from various perspectives are used to remove occlusions and to reconstruct the occluded areas. Nevertheless, these approaches require large amounts of data, which are not always available. Furthermore, they do not work for static occlusions caused by, among others, parked vehicles. In this paper, we address occlusion removal based on single aerial images using generative adversarial networks (GANs), which are able to deal with the mentioned challenges. To this end, we adapt several state-of-the-art GAN-based image inpainting algorithms to reconstruct the missing information. Results indicate that the StructureFlow algorithm outperforms the competitors and the restorations obtained are robust, with high visual fidelity in real-world applications. Furthermore, due to the lack of annotated aerial vehicle removal datasets, we generate a new dataset for training and validating the algorithms, the Aerial Vehicle Occlusion Removal (AVOR) dataset. To the best of our knowledge, our work is the first to address vehicle removal using deep learning algorithms to enhance maps

The Association Between Serum Vitamin D Level and Nonalcoholic Fatty Liver Disease

Background: Nonalcoholic fatty liver disease (NAFLD) is a condition, in which triglyceride accumulates in hepatic cells without a history of alcohol consumption and is strongly associated with insulin resistance, obesity, diabetes mellitus, hypertension and dyslipidemia. The potential role of vitamin D in the pathogenesis of NAFLD has been reported. Objectives: The aim of this study was to determine the optimal vitamin D levels for prevention of NAFLD. Methods: In a cross-sectional study, 2,160 cases who referred to a university-affiliated health center were randomly selected and their demographic information, anthropometric and metabolic indices and also vitamin D levels were collected. Fatty liver was assessed by fatty liver index (FLI) and confirmed by FibroScan using controlled attenuation parameter (CAP). Based on the NAFLD score, the subjects were divided into two groups and the vitamin D cutoff point was calculated by ROC curve. Results: Based on the results, 745 patients (34.5%) had different degrees of fatty liver. Significant differences in the stiffness of liver tissue were observed between vitamin D categories (285.10 +/- 30.56 for severe, 251.82 +/- 42.63 for moderate and 201.02 +/- 36.08 for mild deficiency). According to the multivariate analysis, age, fasting insulin and vitamin D levels were found as the most significant factors in NAFLD pathogenesis. Vitamin D cutoff point was obtained 18 nmol/L in women and 21 nmol/L in men. Conclusions: The results indicated a significant association between vitamin D level and NAFLD score. Accordingly, increasing the public awareness to maintain a proper level of vitamin D may be a preventative strategy against NAFLD. Keywords Author Keywords:25-Hydroxyvitamin D; Nonalcoholic Fatty Liver Disease; Nonalcoholic Steatohepatitis; Obesity; Vitamin D Deficiency KeyWords Plus:25-HYDROXYVITAMIN D-3; INSULIN-RESISTANCE; STEATOHEPATITIS; EXPRESSION; SEVERITY; ALPHA; NAFLD; HISTOLOGY; CHILDREN; IMPAC