Continual learning with direction-constrained optimization

This paper studies a new design of the optimization algorithm for training deep learning models with a fixed architecture of the classification network in a continual learning framework, where the training data is non-stationary and the non-stationarity is imposed by a sequence of distinct tasks. This setting implies the existence of a manifold of network parameters that correspond to good performance of the network on all tasks. Our algorithm is derived from the geometrical properties of this manifold. We first analyze a deep model trained on only one learning task in isolation and identify a region in network parameter space, where the model performance is close to the recovered optimum. We provide empirical evidence that this region resembles a cone that expands along the convergence direction. We study the principal directions of the trajectory of the optimizer after convergence and show that traveling along a few top principal directions can quickly bring the parameters outside the cone but this is not the case for the remaining directions. We argue that catastrophic forgetting in a continual learning setting can be alleviated when the parameters are constrained to stay within the intersection of the plausible cones of individual tasks that were so far encountered during training. Enforcing this is equivalent to preventing the parameters from moving along the top principal directions of convergence corresponding to the past tasks. For each task we introduce a new linear autoencoder to approximate its corresponding top forbidden principal directions. They are then incorporated into the loss function in the form of a regularization term for the purpose of learning the coming tasks without forgetting. We empirically demonstrate that our algorithm performs favorably compared to other state-of-art regularization-based continual learning methods, including EWC and SI

Ionic logic with highly asymmetric nanofluidic memristive switches

While most neuromorphic systems are based on nanoscale electronic devices, nature relies on ions for energy-efficient information processing. Therefore, finding memristive nanofluidic devices is a milestone toward realizing electrolytic computers mimicking the brain down to its basic principles of operations. Here, we present a nanofluidic device designed for circuit scale in-memory processing that combines single-digit nanometric confinement and large entrance asymmetry. Our fabrication process is scalable while the device operates at the second timescale with a twenty-fold conductance ratio. It displays a switching threshold due to the dynamics of an extended space charge. The combination of these features permits assembling logic circuits composed of two interactive nanofluidic devices and an ohmic resistor. These results open the way to design multi-component ionic machinery, such as nanofluidic neural networks, and implementing brain-inspired ionic computations

Macerals of lignite and the effect of alkali treatment on the structure and combustion performance of lignite

Suppressing the spontaneous combustion of lignite is of great significance for safe transportation and efficient utilization of lignite. Taking the Shengli lignite as the research object, two different macerals, inertinite and huminite, were selected by optical microscope, and treated with NaOH respectively to study the relationship between the structure and combustion reaction performance of different macerals and lignite treated with NaOH. The structure of the prepared coal samples was characterized by SEM-EDS, XPS, FT-IR, XRD and Raman, and the changes of the main functional groups were analyzed. The effect of NaOH treatment on the combustion performance of different maceral lignite was investigated by TGA. The results showed that the ignition temperature of huminite lignite was about 10 ℃ earlier than that of inertinite, but the comprehensive combustion characteristic index of inertinite lignite was slightly higher than that of huminite. After the NaOH treatment, the lignite of different macerals showed a hysteresis of combustion, there were two obvious weight losses in the range of 200−500 ℃ and 650−800 ℃, respectively, and the mass loss was mainly concentrated in the second weight loss, in particular, the effect of huminite lignite was more significant, and the temperature corresponding to the maximum combustion reaction rate was about 60 ℃ behind that of inertinite. The kinetic analysis of the combustion process of the coal samples showed that the activation energy of combustion reaction of lignite with different macerals significantly increased after the NaOH treatment, and the huminite lignite was higher than that of inertinite lignite. The XPS/FT-IR results revealed that the contents of carboxylic oxygen-containing functional groups in different macerals of lignite treated by NaOH decreased, the main reason is that in the process of NaOH treatment, Na+ interacted with the carboxylic oxygen-containing functional groups in lignite to form the sodium carboxylate structure, and the relative amount of the sodium carboxylate structure in huminite coal was relatively large. It is believed that the inhibitory effect on the combustion of lignite with different macerals is attributed to the stability of the sodium carboxylate structure, and the number of the sodium carboxylate structure formed by combining with Na is the main reason for the difference in its combustion performance. The XRD/Raman analysis indicates that the formation of the sodium carboxylate structure in lignite leads to the increase of the order degree of carbon microcrystalline structure, and the order degree of huminite lignite is higher than that in inertinite

Correlation Analysis Between Required Surgical Indexes and Complications in Patients With Coronary Heart Disease

A total of 215 patients with coronary heart disease (CHD) were analyzed with SPSS. Samples of different genders showed significance in the obtuse marginal branch of the left circumflex branch × 1, the diagonal branch D1 × 1, and the ms PV representation. Patients with left circumflex branch occlusion are more male and tend to be younger. Age displayed a positive correlation with left intima-media thickness (IMT) and right IMT. This indicated that as age increases, the values of left IMT and right IMT increase. Samples of different CHD types showed significance in the obtuse marginal branch of the left circumflex branch × 1, the middle part of RCA × 1, and the middle part of the left anterior descending branch × 1.5. For non-ST-segment elevation angina pectoris with acute total vascular occlusion, the left circumflex artery is the most common, followed by the right coronary artery and anterior descending branch. Ultrasound of carotid IMT in patients with CHD can predict changes in left ventricular function, but no specific correlation between left and right common carotid IMT was found. Samples with or without the medical history of ASCVD showed significance in the branch number of coronary vessel lesions. The value of the branch number of coronary vessel lesions in patients with atherosclerotic cardiovascular disease (ASCVD) was higher than in those without ASCVD. The occurrence of complication is significantly relative with the distance of left circumflex branch × 1, the middle segment of left anterior descending branch × 1.5, and the distance of left anterior descending branch × 1. For patients without complications, the values in the distal left circumflex branch × 1, the middle left anterior descending branch × 1.5, and the distal left anterior descending branch × 1 were higher than those for patients with complications. The VTE scores showed a positive correlation with the proximal part of RCA × 1, the branch number of coronary vessel lesions, the posterior descending branch of left circumflex branch × 1, the distal part of left circumflex branch × 1, and the middle part of left anterior descending branch × 1.5

Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans

Abstract: Machine learning methods offer great promise for fast and accurate detection and prognostication of coronavirus disease 2019 (COVID-19) from standard-of-care chest radiographs (CXR) and chest computed tomography (CT) images. Many articles have been published in 2020 describing new machine learning-based models for both of these tasks, but it is unclear which are of potential clinical utility. In this systematic review, we consider all published papers and preprints, for the period from 1 January 2020 to 3 October 2020, which describe new machine learning models for the diagnosis or prognosis of COVID-19 from CXR or CT images. All manuscripts uploaded to bioRxiv, medRxiv and arXiv along with all entries in EMBASE and MEDLINE in this timeframe are considered. Our search identified 2,212 studies, of which 415 were included after initial screening and, after quality screening, 62 studies were included in this systematic review. Our review finds that none of the models identified are of potential clinical use due to methodological flaws and/or underlying biases. This is a major weakness, given the urgency with which validated COVID-19 models are needed. To address this, we give many recommendations which, if followed, will solve these issues and lead to higher-quality model development and well-documented manuscripts

In vitro and in vivo functions of SARS-CoV-2 infection-enhancing and neutralizing antibodies

SARS-CoV-2 neutralizing antibodies (NAbs) protect against COVID-19. A concern regarding SARS-CoV-2 antibodies is whether they mediate disease enhancement. Here, we isolated NAbs against the receptor-binding domain (RBD) and the N-terminal domain (NTD) of SARS-CoV-2 spike from individuals with acute or convalescent SARS-CoV-2 or a history of SARS-CoV infection. Cryo-electron microscopy of RBD and NTD antibodies demonstrated function-specific modes of binding. Select RBD NAbs also demonstrated Fc receptor-g (FcgR)-mediated enhancement of virus infection in vitro, while five non-neutralizing NTD antibodies mediated FcgR-independent in vitro infection enhancement. However, both types of infection-enhancing antibodies protected from SARS-CoV-2 replication in monkeys and mice. Three of 46 monkeys infused with enhancing antibodies had higher lung inflammation scores compared to controls. One monkey had alveolar edema and elevated bronchoalveolar lavage inflammatory cytokines. Thus, while in vitro antibody-enhanced infection does not necessarily herald enhanced infection in vivo, increased lung inflammation can rarely occur in SARS-CoV-2 antibody-infused macaques

Effects of Digested Pig Slurry on Photosynthesis, Carbohydrate Metabolism and Yield of Tomato (Solanum lycopersicum L.)

Soilless cultivation of vegetables is widely used in production. It is also well accepted that digested slurry is frequently applied as a fertilizer in agricultural production. However, the effect of digested pig slurry on yield and quality of tomato soilless cultivation, as well as the yield and quality influenced by plant carbohydrate metabolism, remain unexplored. Here, the dual inputs of fertilizers (digested pig slurry (D) and mineral fertilizer (M)) and soilless substrates (peat substrate (P) and cinder substrate(C)) consisted of four treatments. The dry biomass and fruit yields, photosynthetic parameters, carbohydrate contents and metabolism enzymes in leaves and fruits were recorded during the experimental period. The highest fruit yields were obtained in DP and MP treatments. Although DP treatment significantly increased the fresh weight of single fruits by 18.0% compared to MP treatment, it reduced the number of ripe fruits. The photosynthetic efficiency and carbohydrate contents (sucrose, glucose and fructose) in leaves were generally higher in DP treatment compared to other treatments, as well as the activities of sucrose phosphate synthase and AGPase in leaves. The soluble sugar contents of fruits in DP and DC treatments were enhanced by 12.3% and 37.0%, respectively, compared to MP and MC treatments. Moreover, the current results showed that DP treatment significantly increased the activity of acid invertase in fruit by 36.3%, 31.3%, and 42.2%, respectively, compared to MP, DC, and MC treatments, and decreased the activity of AGPase by 24.2%, 16.0%, and 36.4%, respectively. The current results have demonstrated that DP treatment had better yield and quality, owing to digested pig slurry increasing the photosynthetic efficiency and source strength, and regulated the activities of carbohydrate metabolism enzymes

Invertible Autoencoder for Domain Adaptation

The unsupervised image-to-image translation aims at finding a mapping between the source ( A ) and target ( B ) image domains, where in many applications aligned image pairs are not available at training. This is an ill-posed learning problem since it requires inferring the joint probability distribution from marginals. Joint learning of coupled mappings F A B : A → B and F B A : B → A is commonly used by the state-of-the-art methods, like CycleGAN to learn this translation by introducing cycle consistency requirement to the learning problem, i.e., F A B ( F B A ( B ) ) ≈ B and F B A ( F A B ( A ) ) ≈ A . Cycle consistency enforces the preservation of the mutual information between input and translated images. However, it does not explicitly enforce F B A to be an inverse operation to F A B . We propose a new deep architecture that we call invertible autoencoder (InvAuto) to explicitly enforce this relation. This is done by forcing an encoder to be an inverted version of the decoder, where corresponding layers perform opposite mappings and share parameters. The mappings are constrained to be orthonormal. The resulting architecture leads to the reduction of the number of trainable parameters (up to 2 times). We present image translation results on benchmark datasets and demonstrate state-of-the art performance of our approach. Finally, we test the proposed domain adaptation method on the task of road video conversion. We demonstrate that the videos converted with InvAuto have high quality and show that the NVIDIA neural-network-based end-to-end learning system for autonomous driving, known as PilotNet, trained on real road videos performs well when tested on the converted ones