Continual Learning of Neural Machine Translation within Low Forgetting Risk Regions

This paper considers continual learning of large-scale pretrained neural machine translation model without accessing the previous training data or introducing model separation. We argue that the widely used regularization-based methods, which perform multi-objective learning with an auxiliary loss, suffer from the misestimate problem and cannot always achieve a good balance between the previous and new tasks. To solve the problem, we propose a two-stage training method based on the local features of the real loss. We first search low forgetting risk regions, where the model can retain the performance on the previous task as the parameters are updated, to avoid the catastrophic forgetting problem. Then we can continually train the model within this region only with the new training data to fit the new task. Specifically, we propose two methods to search the low forgetting risk regions, which are based on the curvature of loss and the impacts of the parameters on the model output, respectively. We conduct experiments on domain adaptation and more challenging language adaptation tasks, and the experimental results show that our method can achieve significant improvements compared with several strong baselines.Comment: EMNLP 2022 Main Conference Long Pape

Dynamics of a model for the degradation mechanism of aggregated α-synuclein in Parkinson's disease

Accumulation of the misfolded synaptic protein α-synuclein (αSyn*) is a hallmark of neurodegenerative disease in Parkinson's disease (PD). Recent studies suggest that the autophagy lysosome pathway (ALP) including both the Beclin1-associated and mTOR-signaling pathways is involved in the αSyn* clearance mechanism. In this study, a mathematical model is proposed for the degradation of αSyn* by ALP with the crosstalk element of mTOR. Using codimension-1 bifurcation analysis, the tri-stability of αSyn* is surveyed under three different stress signals and, in addition, consideration is given to the regulatory mechanisms for the Beclin1- and mTOR-dependent rates on αSyn* degradation using the codimension-1 and−2 bifurcation diagrams. It was found that, especially under internal and external oxidative stresses (S1), the bistable switch of the aggregation of αSyn* can be transformed from an irreversible to a reversible condition through the ALP degradation pathways. Furthermore, the robustness of the tri-stable state for the stress S1 to the parameters related to mTOR-mediated ALP was probed. It was confirmed that mTOR-mediated ALP is important for maintaining the essential dynamic features of the tri-stable state. This study may provide a promising avenue for conducting further experiments and simulations of the degradation mechanism of dynamic modeling in PD

Intelligence Deficits in Chinese Patients with Brain Tumor: The Impact of Tumor Resection

Background. Intelligence is much important for brain tumor patients after their operation, while the reports about surgical related intelligence deficits are not frequent. It is not only theoretically important but also meaningful for clinical practice. Methods. Wechsler Adult Intelligence Scale was employed to evaluate the intelligence of 103 patients with intracranial tumor and to compare the intelligence quotient (IQ), verbal IQ (VIQ), and performance IQ (PIQ) between the intracerebral and extracerebral subgroups. Results. Although preoperative intelligence deficits appeared in all subgroups, IQ, VIQ, and PIQ were not found to have any significant difference between the intracerebral and extracerebral subgroups, but with VIQ lower than PIQ in all the subgroups. An immediate postoperative follow-up demonstrated a decline of IQ and PIQ in the extracerebral subgroup, but an improvement of VIQ in the right intracerebral subgroup. Pituitary adenoma resection exerted no effect on intelligence. In addition, age, years of education, and tumor size were found to play important roles. Conclusions. Brain tumors will impair IQ, VIQ, and PIQ. The extracerebral tumor resection can deteriorate IQ and PIQ. However, right intracerebral tumor resection is beneficial to VIQ, and transsphenoidal pituitary adenoma resection performs no effect on intelligence

Impacts of 319 wind farms on surface temperature and vegetation in the United States

The development of wind energy is essential for decarbonizing energy production. However, the construction of wind farms changes land surface temperature (LST) and vegetation by modifying land surface properties and disturbing land–atmosphere interactions. In this study, we used moderate resolution imaging spectroradiometer satellite data to quantify the impacts on local climate and vegetation of 319 wind farms in the United States. Our results indicated insignificant impacts on LST during the daytime but significant warming of 0.10 °C of annual mean nighttime LST averaged over all wind farms, and 0.36 °C for those 61% wind farms with warming. The nighttime LST impacts exhibited seasonal variations, with stronger warming in winter and autumn, up to 0.18 °C, but weaker effects in summer and spring. We observed a decrease in peak normalized difference vegetation index (NDVI) for 59% of wind farms due to infrastructure construction, with an average reduction of 0.0067 compared to non-wind farm areas. The impacts of wind farms depended on wind farm size, with winter LST impacts for large and small wind farms ranging from 0.21 °C to 0.14 °C, and peak NDVI impacts ranging from −0.009 to −0.006. The LST impacts declined with the increasing distance from the wind farm, with detectable impacts up to 10 km. In contrast, the vegetation impacts on NDVI were only evident within the wind farm locations. Wind farms built in grassland and cropland showed larger warming effects but weaker vegetation impact than those built on forests. Furthermore, spatial correlation analyses with environmental factors suggest limited geographical controls on the heterogeneous wind farm impacts and highlight the important role of local factors. Our analyses based on a large sample offer new evidence for wind farm impacts with improved representativeness compared to previous studies. This knowledge is important to fully understand the climatic and environmental implications of energy system decarbonization

Healthy cities initiative in China: Progress, challenges, and the way forward

Article discusses how China implemented the first phase of its National Healthy Cities pilot program from 2016-20. Authors recommend aligning the Healthy Cities initiative in China with strategic national and global level agendas such as Healthy China 2030 and the Sustainable Development Goals (SDGs) by providing an integrative governance framework to facilitate a coherent intersectoral program to systemically improve population health

A comprehensive quantification of global nitrous oxide sources and sinks

Nitrous oxide (N2O), like carbon dioxide, is a long-lived greenhouse gas that accumulates in the atmosphere. Over the past 150 years, increasing atmospheric N2O concentrations have contributed to stratospheric ozone depletion1 and climate change2, with the current rate of increase estimated at 2 per cent per decade. Existing national inventories do not provide a full picture of N2O emissions, owing to their omission of natural sources and limitations in methodology for attributing anthropogenic sources. Here we present a global N2O inventory that incorporates both natural and anthropogenic sources and accounts for the interaction between nitrogen additions and the biochemical processes that control N2O emissions. We use bottom-up (inventory, statistical extrapolation of flux measurements, process-based land and ocean modelling) and top-down (atmospheric inversion) approaches to provide a comprehensive quantification of global N2O sources and sinks resulting from 21 natural and human sectors between 1980 and 2016. Global N2O emissions were 17.0 (minimum–maximum estimates: 12.2–23.5) teragrams of nitrogen per year (bottom-up) and 16.9 (15.9–17.7) teragrams of nitrogen per year (top-down) between 2007 and 2016. Global human-induced emissions, which are dominated by nitrogen additions to croplands, increased by 30% over the past four decades to 7.3 (4.2–11.4) teragrams of nitrogen per year. This increase was mainly responsible for the growth in the atmospheric burden. Our findings point to growing N2O emissions in emerging economies—particularly Brazil, China and India. Analysis of process-based model estimates reveals an emerging N2O–climate feedback resulting from interactions between nitrogen additions and climate change. The recent growth in N2O emissions exceeds some of the highest projected emission scenarios3,4, underscoring the urgency to mitigate N2O emissions