Description generation for remote sensing images using attributes

Image captioning generates a semantic description of an image. It deals with image understanding and text mining, which has made great progress in recent years. However, it is still a great challenge to bridge the “semantic gap” between low-level features and high-level semantics in remote sensing images, in spite of the improvement of image resolutions. In this paper, we present a new model with an attribute attention mechanism for the description generation of remote sensing images. Therefore, we have explored the impact of the attributes extracted from remote sensing images on the attention mechanism. The results of our experiments demonstrate the validity of our proposed model. The proposed method obtains six higher scores and one slightly lower, compared against several state of the art techniques, on the Sydney Dataset and Remote Sensing Image Caption Dataset (RSICD), and receives all seven higher scores on the UCM Dataset for remote sensing image captioning, indicating that the proposed framework achieves robust performance for semantic description in high-resolution remote sensing images

Hyperspectral Anomaly Detection Based on Low- Rank and Sparse Representation with Data-Driven Projection and Dictionary Construction

Hyperspectral image anomaly detection is an increasingly important research topic in remote sensing images understanding and interpretation. Recently, low-rank representation-based methods have attracted extensive attentions and achieved promising performances in hyperspectral anomaly detection. These methods assume that the hyperspectral data can be decomposed into two parts: the low-rank component representing the background and the residual part indicating the anomaly. More recently, sparse representation has been introduced to support the development of low-rank representation (LRR) models, which considers the sparsity of the representation. In order to improve the separability of the background and anomaly, we propose a novel hyperspectral anomaly detection based on low-rank and sparse representation with dictionary construction and data-driven projection. To construct a robust dictionary that contains all categories of the background objects whilst excluding the anomaly’s influence, we adopt a superpixel-based tensor low-rank decomposition method to generate a comprehensive and pure background dictionary. Considering the spectral redundancy in the hyperspectral data, data-driven projection is introduced to the low-rank representation to project the original data to a low-dimensional feature space to better separate the anomaly and the background. Experimental results on four real hyperspectral datasets show that the proposed anomaly detection method outperforms the other anomaly detectors.</p

Aerial road extraction based on an improved Generative Adversarial Network

Aerial photographs and satellite images are one of the resources used for earth observation. In practice, automated detection of roads on aerial images is of significant values for the application such as car navigation, law enforcement, and fire services. In this paper, we present a novel road extraction method from aerial images based on an improved generative adversarial network, which is an end-to-end framework only requiring a few samples for training. Experimental results on the Massachusetts Roads Dataset show that the proposed method provides better performance than several state of the art techniques in terms of detection accuracy, recall, precision and F1-score

Semantics and Contour Based Interactive Learning Network For Building Footprint Extraction

Building footprint extraction plays an important role in the analysis of remote sensing images and has an extensive range of applications. Obtaining precise boundaries of buildings remains a challenge in existing building extraction methods. Some previous works have made notable efforts to address this concern. However, most of these methods require cumbersome and expensive post-processing steps. Moreover, they ignored the correlation between building semantics and contours, which we believe is crucial for building footprint extraction. To mitigate this issue, our paper presents an intuitive and effective framework that explores semantic and contour cues of buildings and fully excavates their correlation. Specifically, we construct an interactive dual-stream decoder. The Intermediate connections within this decoder interactively transmit features between branches, contributing to learning correlations between semantics and contours. We propose the Semantic Collaboration Module (SCM) to strengthen the connection between the two branches. To further boost performance, we build the Multi-Scale Semantic Context Fusion Module (MSCF) to fuse semantic information from the higher and lower layers of the network, allowing the network to obtain superior feature representations. The experimental results on the WHU, INRIA, and Massachusetts building datasets demonstrate the superior performance of our method. </p

Negative Deterministic Information based Multiple Instance Learning for Weakly Supervised Object Detection and Segmentation

Weakly  supervised  object  detection  and  semanticsegmentation  with  image-level  annotations  have  attracted  ex-tensive   attention   due   to   their  high   label   efficiency.   Multipleinstance  learning  (MIL)  offers  a  feasible  solution  forthe  twotasks by treating each image as a bag with a series of instances(object  regions  or  pixels)  and  identifying  foreground  instancesthat contribute to bag classification. However, conventional MILparadigms  often  suffer  from  issues,  e.g.,  discriminative  instancedomination  and  missing  instances.In  this  paper,  weobservethat  negative  instances  usually  contain  valuable  deterministicinformation, which is the key to solving the two issues. Motivatedby  this,  we  proposea  novel  MIL  paradigm  based  on  negativedeterministic   information   (NDI),   termed   NDI-MIL,   whichisbased  on  two  core  designs  with  a  progressive  relation:  NDIcollection  and  negative  contrastive  learning.  In  NDI  collection,we  identify  and  distill  NDI  from  negative  instances  online  bya  dynamic  feature  bank.  The  collected  NDI  is  then  utilized  ina  negative  contrastive  learning  mechanism  to  locate  and  punishthose discriminative regions, by which the discriminative instancedomination and missing instances issues are effectively addressed,leading to improved object- and pixel-level localization accuracyand completeness. In addition, we design an NDI-guided instanceselection strategy to further enhance the systematic performance.Experimental  results  on  several  public  benchmarks,  includingPASCAL VOC 2007, PASCAL VOC 2012, and MS COCO, showthat  our  method  achieves  satisfactory  performance.  The  code  isavailable at: https://github.com/GC-WSL/NDI.</p

Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images

In this paper, we focus on the challenging multi-category instance segmentation problem in remote sensing images(RSIs), which aims at predicting the categories of all instances and localizing them with pixel-level masks. Although many land-mark frameworks have demonstrated promising performance in instance segmentation, the complexity in the background and scale variability instances still remain challenging for in-stance segmentation of RSIs. To address the above problems, we propose an end-to-end multi-category instance segmentation model, namely Semantic Attention and Scale Complementary Network, which mainly consists of a Semantic Attention (SEA)module and a Scale Complementary Mask Branch (SCMB).The SEA module contains a simple fully convolutional semantic segmentation branch with extra supervision to strengthen the activation of interest instances on the feature map and reduce the background noise’s interference. To handle the under-segmentation of geospatial instances with large varying scales, we design the SCMB that extends the original single-scale mask branch to trident mask branches and introduces complementary mask supervision at different scales to sufficiently leverage the multi-scale information. We conduct comprehensive experiments to evaluate the effectiveness of our proposed method on the iSAID dataset and the NWPU Instance Segmentation dataset and achieve promising performance

Negative Deterministic Information based Multiple Instance Learning for Weakly Supervised Object Detection and Segmentation

Weakly  supervised  object  detection  and  semanticsegmentation  with  image-level  annotations  have  attracted  ex-tensive   attention   due   to   their  high   label   efficiency.   Multipleinstance  learning  (MIL)  offers  a  feasible  solution  forthe  twotasks by treating each image as a bag with a series of instances(object  regions  or  pixels)  and  identifying  foreground  instancesthat contribute to bag classification. However, conventional MILparadigms  often  suffer  from  issues,  e.g.,  discriminative  instancedomination  and  missing  instances.In  this  paper,  weobservethat  negative  instances  usually  contain  valuable  deterministicinformation, which is the key to solving the two issues. Motivatedby  this,  we  proposea  novel  MIL  paradigm  based  on  negativedeterministic   information   (NDI),   termed   NDI-MIL,   whichisbased  on  two  core  designs  with  a  progressive  relation:  NDIcollection  and  negative  contrastive  learning.  In  NDI  collection,we  identify  and  distill  NDI  from  negative  instances  online  bya  dynamic  feature  bank.  The  collected  NDI  is  then  utilized  ina  negative  contrastive  learning  mechanism  to  locate  and  punishthose discriminative regions, by which the discriminative instancedomination and missing instances issues are effectively addressed,leading to improved object- and pixel-level localization accuracyand completeness. In addition, we design an NDI-guided instanceselection strategy to further enhance the systematic performance.Experimental  results  on  several  public  benchmarks,  includingPASCAL VOC 2007, PASCAL VOC 2012, and MS COCO, showthat  our  method  achieves  satisfactory  performance.  The  code  isavailable at: https://github.com/GC-WSL/NDI.</p

High-Quality Angle Prediction for Oriented Object Detection in Remote Sensing Images

Oriented object detection is a challenging task in remote sensing, where the detected objects can be represented by oriented bounding boxes (OBBs). Angle prediction in oriented object detection has been widely studied, due to its crucial role in object detection. However, the precision of angle prediction is severely limited by misalignments in most of the existing methods, including representation-, evaluation-, and optimization-based misalignments. To alleviate these misalignments, this paper presents a novel angle prediction method, called Angle Quality Estimation (AQE). Specifically, our proposed AQE transforms the angle prediction task into a distribution estimation task to address the representation misalignment problem and implicitly measure the quality of the predicted angles. Based on the estimated angle quality, we then propose a new metric to comprehensively evaluate the quality of OBBs. Then we propose an object aspect ratio based loss function to optimize angle prediction for addressing the optimization misalignment. Our proposed AQE is a plug-and-play method, which can be embedded on any existing oriented object detector. Experimental results on three public benchmarks, including DOTA, HRSC2016, and ICDAR2015 datasets, show that our method achieves better performance than the other state-of-the-art.</p

Connective tissue growth factor contributes to joint homeostasis and osteoarthritis severity by controlling the matrix sequestration and activation of latent TGFβ.

OBJECTIVES: One mechanism by which cartilage responds to mechanical load is by releasing heparin-bound growth factors from the pericellular matrix (PCM). By proteomic analysis of the PCM, we identified connective tissue growth factor (CTGF) and here investigate its function and mechanism of action. METHODS: Recombinant CTGF (rCTGF) was used to stimulate human chondrocytes for microarray analysis. Endogenous CTGF was investigated by in vitro binding assays and confocal microscopy. Its release from cut cartilage (injury CM) was analysed by Western blot under reducing and non-reducing conditions. A postnatal, conditional CtgfcKO mouse was generated for cartilage injury experiments and to explore the course of osteoarthritis (OA) by destabilisation of the medial meniscus. siRNA knockdown was performed on isolated human chondrocytes. RESULTS: The biological responses of rCTGF were TGFβ dependent. CTGF displaced latent TGFβ from cartilage and both were released on cartilage injury. CTGF and latent TGFβ migrated as a single high molecular weight band under non-reducing conditions, suggesting that they were in a covalent (disulfide) complex. This was confirmed by immunoprecipitation. Using CtgfcKO mice, CTGF was required for sequestration of latent TGFβ in the matrix and activation of the latent complex at the cell surface through TGFβR3. In vivo deletion of CTGF increased the thickness of the articular cartilage and protected mice from OA. CONCLUSIONS: CTGF is a latent TGFβ binding protein that controls the matrix sequestration and activation of TGFβ in cartilage. Deletion of CTGF in vivo caused a paradoxical increase in Smad2 phosphorylation resulting in thicker cartilage that was protected from OA