PCR: Proxy-based Contrastive Replay for Online Class-Incremental Continual Learning

Online class-incremental continual learning is a specific task of continual learning. It aims to continuously learn new classes from data stream and the samples of data stream are seen only once, which suffers from the catastrophic forgetting issue, i.e., forgetting historical knowledge of old classes. Existing replay-based methods effectively alleviate this issue by saving and replaying part of old data in a proxy-based or contrastive-based replay manner. Although these two replay manners are effective, the former would incline to new classes due to class imbalance issues, and the latter is unstable and hard to converge because of the limited number of samples. In this paper, we conduct a comprehensive analysis of these two replay manners and find that they can be complementary. Inspired by this finding, we propose a novel replay-based method called proxy-based contrastive replay (PCR). The key operation is to replace the contrastive samples of anchors with corresponding proxies in the contrastive-based way. It alleviates the phenomenon of catastrophic forgetting by effectively addressing the imbalance issue, as well as keeps a faster convergence of the model. We conduct extensive experiments on three real-world benchmark datasets, and empirical results consistently demonstrate the superiority of PCR over various state-of-the-art methods.Comment: To appear in CVPR 2023. 10 pages, 8 figures and 3 table

HPCR: Holistic Proxy-based Contrastive Replay for Online Continual Learning

Online continual learning (OCL) aims to continuously learn new data from a single pass over the online data stream. It generally suffers from the catastrophic forgetting issue. Existing replay-based methods effectively alleviate this issue by replaying part of old data in a proxy-based or contrastive-based replay manner. In this paper, we conduct a comprehensive analysis of these two replay manners and find they can be complementary. Inspired by this finding, we propose a novel replay-based method called proxy-based contrastive replay (PCR), which replaces anchor-to-sample pairs with anchor-to-proxy pairs in the contrastive-based loss to alleviate the phenomenon of forgetting. Based on PCR, we further develop a more advanced method named holistic proxy-based contrastive replay (HPCR), which consists of three components. The contrastive component conditionally incorporates anchor-to-sample pairs to PCR, learning more fine-grained semantic information with a large training batch. The second is a temperature component that decouples the temperature coefficient into two parts based on their impacts on the gradient and sets different values for them to learn more novel knowledge. The third is a distillation component that constrains the learning process to keep more historical knowledge. Experiments on four datasets consistently demonstrate the superiority of HPCR over various state-of-the-art methods.Comment: 18 pages, 11 figure

UER: A Heuristic Bias Addressing Approach for Online Continual Learning

Online continual learning aims to continuously train neural networks from a continuous data stream with a single pass-through data. As the most effective approach, the rehearsal-based methods replay part of previous data. Commonly used predictors in existing methods tend to generate biased dot-product logits that prefer to the classes of current data, which is known as a bias issue and a phenomenon of forgetting. Many approaches have been proposed to overcome the forgetting problem by correcting the bias; however, they still need to be improved in online fashion. In this paper, we try to address the bias issue by a more straightforward and more efficient method. By decomposing the dot-product logits into an angle factor and a norm factor, we empirically find that the bias problem mainly occurs in the angle factor, which can be used to learn novel knowledge as cosine logits. On the contrary, the norm factor abandoned by existing methods helps remember historical knowledge. Based on this observation, we intuitively propose to leverage the norm factor to balance the new and old knowledge for addressing the bias. To this end, we develop a heuristic approach called unbias experience replay (UER). UER learns current samples only by the angle factor and further replays previous samples by both the norm and angle factors. Extensive experiments on three datasets show that UER achieves superior performance over various state-of-the-art methods. The code is in https://github.com/FelixHuiweiLin/UER.Comment: 9 pages, 12 figures, ACM MM202

Parametric Estimation of Reference Signal Intensity for Semi-Quantification of Tau Deposition: A Flortaucipir and [18F]-APN-1607 Study

BackgroundTau positron emission tomography (PET) imaging can reveal the pathophysiology and neurodegeneration that occurs in Alzheimer’s disease (AD) in vivo. The standardized uptake value ratio (SUVR) is widely used for semi-quantification of tau deposition but is susceptible to disturbance from the reference region and the partial volume effect (PVE). To overcome this problem, we applied the parametric estimation of reference signal intensity (PERSI) method—which was previously evaluated for flortaucipir imaging—to two tau tracers, flortaucipir and [18F]-APN-1607.MethodsTwo cohorts underwent tau PET scanning. Flortaucipir PET imaging data for cohort I (65 healthy controls [HCs], 60 patients with mild cognitive impairment [MCI], and 12 AD patients) were from the AD Neuroimaging Initiative database. [18F]-APN-1607 ([18F]-PM-PBB3) PET imaging data were for Cohort II, which included 21 patients with a clinical diagnosis of amyloid PET-positive AD and 15 HCs recruited at Huashan Hospital. We used white matter (WM) postprocessed by PERSI (PERSI-WM) as the reference region and compared this with the traditional semi-quantification method that uses the whole cerebellum as the reference. SUVRs were calculated for regions of interest including the frontal, parietal, temporal, and occipital lobes; anterior and posterior cingulate; precuneus; and Braak I/II (entorhinal cortex and hippocampus). Receiver operating characteristic (ROC) curve analysis and effect sizes were used to compare the two methods in terms of ability to discriminate between different clinical groups.ResultsIn both cohorts, regional SUVR determined using the PERSI-WM method was superior to using the cerebellum as reference region for measuring tau retention in AD patients (e.g., SUVR of the temporal lobe: flortaucipir, 1.08 ± 0.17 and [18F]-APN-1607, 1.57 ± 0.34); and estimates of the effect size and areas under the ROC curve (AUC) indicated that it also increased between-group differences (e.g., AUC of the temporal lobe for HC vs AD: flortaucipir, 0.893 and [18F]-APN-1607: 0.949).ConclusionThe PERSI-WM method significantly improves diagnostic discrimination compared to conventional approach of using the cerebellum as a reference region and can mitigate the PVE; it can thus enhance the efficacy of semi-quantification of multiple tau tracers in PET scanning, making it suitable for large-scale clinical application

Brain Network and Abnormal Hemispheric Asymmetry Analyses to Explore the Marginal Differences in Glucose Metabolic Distributions Among Alzheimer's Disease, Parkinson's Disease Dementia, and Lewy Body Dementia

Facilitating accurate diagnosis and ensuring appropriate treatment of dementia subtypes, including Alzheimer's disease (AD), Parkinson's disease dementia (PDD), and Lewy body dementia (DLB), is clinically important. However, the differences in glucose metabolic distribution among these three dementia subtypes are minor, which can result in difficulties in diagnosis by visual assessment or traditional quantification methods. Here, we explored this issue using novel approaches, including brain network and abnormal hemispheric asymmetry analyses. We generated 18F-labeled fluorodeoxyglucose (18F-FDG) positron emission tomography (PET) images from patients with AD, PDD, and DLB, and healthy control (HC) subjects (n = 22, 18, 22, and 22, respectively) from Huashan hospital, Shanghai, China. Brain network properties were measured and between-group differences evaluated using graph theory. We also calculated and explored asymmetry indices for the cerebral hemispheres in the four groups, to explore whether differences between the two hemispheres were characteristic of each group. Our study revealed significant differences in the network properties of the HC and AD groups (small-world coefficient, 1.36 vs. 1.28; clustering coefficient, 1.48 vs. 1.59; characteristic path length, 1.57 vs. 1.64). In addition, differing hub regions were identified in the different dementias. We also identified rightward asymmetry in the hemispheric brain networks of patients with AD and DLB, and leftward asymmetry in the hemispheric brain networks of patients with PDD, which were attributable to aberrant topological properties in the corresponding hemispheres

Research Advances in Identifying Sulfate Contamination Sources of Water Environment by Using Stable Isotopes

As the main anion of groundwater, the content of sulfate affects the drinking water safety and ecological security directly. In recent years, with the acceleration of industrialization and urbanization development, the problem of sulfate pollution in water environments is becoming more and more serious. It is critical to effectively identify the sulfate sources of water environment to ensure human health and the benign evolution of water environment. Due to its “fingerprints” feature, the sulfur and oxygen isotopes of SO42− have been widely used to identify sources of sulfate contamination in water environment. However, research advances in tracing sulfate contamination sources of water environment by using stable isotopes are rarely reported. This paper reviewed the research advances of sulfate isotope technology domestically and abroad, which was used to trace the sources of sulfate pollution in water environment, compared different pre-treatment methods for analyzing the δ34S and δ18O of sulfate, and compiled the ranges of typical values of δ34S and δ18O from different potential sources of sulfate contamination. In this review, the limitation of the technique in traceability of sulfate pollution was also discussed, and the future traceability techniques of sulfate pollution were prospected

A time-varying study of Chinese investor sentiment, stock market liquidity and volatility: Based on deep learning BERT model and TVP-VAR model

Based on the commentary data of the Shenzhen Stock Index bar on the EastMoney website from January 1, 2018 to December 31, 2019. This paper extracts the embedded investor sentiment by using a deep learning BERT model and investigates the time-varying linkage between investment sentiment, stock market liquidity and volatility using a TVP-VAR model. The results show that the impact of investor sentiment on stock market liquidity and volatility is stronger. Although the inverse effect is relatively small, it is more pronounced with the state of the stock market. In all cases, the response is more pronounced in the short term than in the medium to long term, and the impact is asymmetric, with shocks stronger when the market is in a downward spiral.Comment: 25 pages, 7 figures, 8 tables, Funded by the National Student Innovation and Entrepreneurship Training Program (Project No. 202110561076

Rapid Urbanization Has Changed the Driving Factors of Groundwater Chemical Evolution in the Large Groundwater Depression Funnel Area of Northern China

With the rapid development of urbanization, the chemical evolution of groundwater has been significantly affected by human activities. However, the driving mechanisms of groundwater chemical evolution at different stages of urbanization are still unclear, which severely affects the implementation of groundwater protection. This study investigated the driving mechanisms of groundwater chemical evolution based on the long-term series (from 1985 to 2015) of hydrochemical data from 19 groundwater monitoring sites in rapidly urbanizing areas (Shijiazhuang, Hebei Province, China). The results show that the concentrations of various chemical components in groundwater gradually increase with the acceleration of the urbanization process, especially NO3−, which has increased from 13.7 mg/L in the primary stage of urbanization (PSU) to 65.1 mg/Lin the advanced stage of urbanization (ASU), exceeding the World Health Organization (WHO) drinking water standard (50 mg/L), indicating that the groundwater chemistry has been significantly affected by human activities. The main hydrochemical types have changed from the HCO3•SO4-Ca•Mg-type water in the primary stage of urbanization (PSU) to the SO4•HCO3-Ca•Mg-type water in the advanced stage of urbanization (ASU). It is worth noting that there are obvious differences in driving factors of groundwater chemical evolution at different urbanization stages. In the primary stage of urbanization (PSU), the driving factors were carbonate and rock salt dissolution, cation exchange, and industrial activities. However, in the intermediate stage and advanced stage, the driving factors were changed to carbonate and gypsum dissolution, groundwater over-exploitation, agricultural fertilization, and domestic sewage. Based on the above conclusions, it is suggested that future groundwater management should control the amount of agricultural fertilizers, apply scientific fertilization, and prohibit the discharge of various types of non-compliant sewage, while strengthening the supervision of groundwater extraction to reduce the impact of urbanization development on the groundwater chemical evolution process