Dsfer-Net: A Deep Supervision and Feature Retrieval Network for Bitemporal Change Detection Using Modern Hopfield Networks

Change detection, as an important application for high-resolution remote sensing images, aims to monitor and analyze changes in the land surface over time. With the rapid growth in the quantity of high-resolution remote sensing data and the complexity of texture features, a number of quantitative deep learning-based methods have been proposed. Although these methods outperform traditional change detection methods by extracting deep features and combining spatial-temporal information, reasonable explanations about how deep features work on improving the detection performance are still lacking. In our investigations, we find that modern Hopfield network layers achieve considerable performance in semantic understandings. In this paper, we propose a Deep Supervision and FEature Retrieval network (Dsfer-Net) for bitemporal change detection. Specifically, the highly representative deep features of bitemporal images are jointly extracted through a fully convolutional Siamese network. Based on the sequential geo-information of the bitemporal images, we then design a feature retrieval module to retrieve the difference feature and leverage discriminative information in a deeply supervised manner. We also note that the deeply supervised feature retrieval module gives explainable proofs about the semantic understandings of the proposed network in its deep layers. Finally, this end-to-end network achieves a novel framework by aggregating the retrieved features and feature pairs from different layers. Experiments conducted on three public datasets (LEVIR-CD, WHU-CD, and CDD) confirm the superiority of the proposed Dsfer-Net over other state-of-the-art methods. Code will be available online (https://github.com/ShizhenChang/Dsfer-Net)

The effects of the allocation of attention on rapid scene categorization

It is well documented that observers are able to accurately extract the semantic information from natural scenes in 120 msec (Thorpe, Fize, & Merlot, 1996). This rapid categorization ability is often cited as evidence that the information that is required to categorize a scene originates from low-level visual information. Information related to an image’s spatial scales (Oliva & Schyns, 1997; Schyns & Oliva, 1994), phase (Joubert, Rousselet, Fabre-Thorpe, & Fize, 2009; Loschky et al., 2007, 2010; Loschky & Larson, 2008), overall summary statistics (Evans & Treisman, 2005), and colour (Castelhano & Henderson, 2008; 2005; Loschky & Simons, 2004; Oliva & Schyns, 2000) have all been shown to provide information that can be used to categorize a briefly presented image. The experiments reported in this dissertation were designed to address the overarching question of how the visual system selects diagnostic scene information? It addressed this question by examining the hypothesis that visual attention facilitates the selection of information that underpins rapid scene categorization. In order to investigate this hypothesis, the present work was divided into two main manuscripts. Manuscript 1 is presented in Chapter 2 and includes four experiments that were designed to investigate if attending to global and local levels of a scene facilitate categorization based on a scene’s coarse and fine information, respectively. This hypothesis was explored by asking observers to classify hybrid images. A hybrid image combines the coarse information (conveyed by an image’s low spatial frequencies) of one image (e.g., a city) and the fine information (conveyed by an image’s high spatial frequencies) of a second image (e.g., a highway). Experiments 1 and 2 showed that although observers could classify hybrid images based on both fine and coarse information (i.e., as either a city or a highway scene; Experiment 1), observers preferred to base categorization on coarse content (Experiment 2). Experiment 3 demonstrated that categorization based on coarse content was facilitated when observers were prompted to attend globally to scenes compared to when they were prompted to attend locally. Experiment 4 demonstrated that this global facilitation effect was due, in part, to the facilitation of a hybrid’s low spatial frequencies. Manuscript 2 is presented in Chapter 3 and contains four experiments that investigated the hypothesis that distributed attention facilitates the extraction of a scene’s overall summary statistics, which in turn, facilitates the ability to rapidly categorize scenes (Evans & Treisman, 2005). This hypothesis was investigated by examining whether manipulations of attention affected scene categorization in the same fashion as the extraction of overall summary statistics. Experiment 1 replicated the result that extraction of a scene’s summary statistics is more compatible with distributed attention than focused attention (Chong & Treisman, 2005). Experiments 2 and 4 extended this finding by demonstrating that superordinate level categorization of both animals (e.g., detect the presence [or absence] of an animal, Experiment 2), and natural scenes (e.g., was the scene natural? Experiment 4), were more compatible with distributed than focused attention. However, Experiment 3 showed that there was no difference between the effects of distributed and focused attention on basic level categorization (e.g., was this a beach scene?). Together, the findings of this thesis demonstrate that visual attention is important in the rapid categorization of a natural scene, by facilitating the selection of scene information that is necessary to classify a scene category

HOAX DETECTION IN INDONESIA LANGUAGE USING LONG SHORT-TERM MEMORY MODEL

Nowadays, the internet and social media grow fast. This condition has positive and negative effects on society. They become media to communicate and share information without limitation. However, many people use that easiness to broadcast news or information which do not accurate with the facts and gather people's opinions to get benefits or we called a hoax. Therefore, we need to develop a system that can detect hoax. This research uses the neural network method with Long Short-Term Memory (LSTM) model. The process of the LSTM model to identify hoax has several steps, including dataset collection, pre-processing data, word embedding using pre-trained Word2Vec, built the LSTM model. Detection model performance measurement using precision, recall, and f1-measure matrix. This research results the highest average score of precision is 0.819, recall is 0.809, and f1-measure is 0.807.  These results obtained from the combination of the following parameters, i.e., Skip-gram Word2Vec Model Architecture, Hierarchical Softmax, 100 as vector dimension, max pooling, 0.5 as dropout value, and 0.001 of learning rate

DeepCL: Deep Change Feature Learning on Remote Sensing Images in the Metric Space

Change detection (CD) is an important yet challenging task in the Earth observation field for monitoring Earth surface dynamics. The advent of deep learning techniques has recently propelled automatic CD into a technological revolution. Nevertheless, deep learning-based CD methods are still plagued by two primary issues: 1) insufficient temporal relationship modeling and 2) pseudo-change misclassification. To address these issues, we complement the strong temporal modeling ability of metric learning with the prominent fitting ability of segmentation and propose a deep change feature learning (DeepCL) framework for robust and explainable CD. Firstly, we designed a hard sample-aware contrastive loss, which reweights the importance of hard and simple samples. This loss allows for explicit modeling of the temporal correlation between bi-temporal remote sensing images. Furthermore, the modeled temporal relations are utilized as knowledge prior to guide the segmentation process for detecting change regions. The DeepCL framework is thoroughly evaluated both theoretically and experimentally, demonstrating its superior feature discriminability, resilience against pseudo changes, and adaptability to a variety of CD algorithms. Extensive comparative experiments substantiate the quantitative and qualitative superiority of DeepCL over state-of-the-art CD approaches.Comment: 12 pages,7 figures, submitted to IEEE Transactions on Image Processin

Using fMRI to investigate speech-stream segregation and auditory attention in healthy adults and patients with memory complaints

Poor memory for recent conversations is the commonest presenting symptom in patients attending a cognitive neurology clinic. They also frequently have greater difficulty following and remembering conversations in the presence of background noise and/or unattended speech. While the ability to participate in and recall conversations depends on several cognitive functions (language-processing, attention, episodic and working memory), without the ability to perform auditory scene analysis, and more specifically speech-stream segregation, recall of verbal information will be impaired as a consequence of poor initial registration, over and above impaired encoding and subsequent retrieval. This thesis investigated auditory attention and speech-stream segregation in healthy participants (‘controls’) and patients presenting with ‘poor memory’, particularly a complaint of difficulty remembering recent verbal information. Although this resulted in the recruitment of many patients with possible or probable Alzheimer’s disease, it also included patients with mild cognitive impairment (MCI) of uncertain aetiology and a few with depression. Functional MRI data revealed brain activity involved in attention, working memory and speech-stream segregation as participants attended to a speaker in the absence and presence of background speech. The study on controls demonstrated that the right anterior insula, adjacent frontal operculum, left planum temporale and precuneus were more active when the attended speaker was partially masked by unattended speech. Analyses also revealed a central role for a right hemisphere system for successful attentive listening, a system that was not modulated by administration of a central cholinesterase inhibitor. Therefore, this study identified non-auditory higher-order regions in speech-stream segregation, and the demands on a right hemisphere system during attentive listening. Administration of a central cholinesterase inhibitor did not identify any benefit in the present patient group. However, my research has identified systems that might be therapeutic targets when attempting to modulate auditory attention and speech-stream segregation in patients with neurodegenerative disease.Open Acces

Deep Learning Methods for Remote Sensing

Remote sensing is a field where important physical characteristics of an area are exacted using emitted radiation generally captured by satellite cameras, sensors onboard aerial vehicles, etc. Captured data help researchers develop solutions to sense and detect various characteristics such as forest fires, flooding, changes in urban areas, crop diseases, soil moisture, etc. The recent impressive progress in artificial intelligence (AI) and deep learning has sparked innovations in technologies, algorithms, and approaches and led to results that were unachievable until recently in multiple areas, among them remote sensing. This book consists of sixteen peer-reviewed papers covering new advances in the use of AI for remote sensing