A Unified Continual Learning Framework with General Parameter-Efficient Tuning

The "pre-training $\rightarrow$ downstream adaptation" presents both new opportunities and challenges for Continual Learning (CL). Although the recent state-of-the-art in CL is achieved through Parameter-Efficient-Tuning (PET) adaptation paradigm, only prompt has been explored, limiting its application to Transformers only. In this paper, we position prompting as one instantiation of PET, and propose a unified CL framework with general PET, dubbed as Learning-Accumulation-Ensemble (LAE). PET, e.g., using Adapter, LoRA, or Prefix, can adapt a pre-trained model to downstream tasks with fewer parameters and resources. Given a PET method, our LAE framework incorporates it for CL with three novel designs. 1) Learning: the pre-trained model adapts to the new task by tuning an online PET module, along with our adaptation speed calibration to align different PET modules, 2) Accumulation: the task-specific knowledge learned by the online PET module is accumulated into an offline PET module through momentum update, 3) Ensemble: During inference, we respectively construct two experts with online/offline PET modules (which are favored by the novel/historical tasks) for prediction ensemble. We show that LAE is compatible with a battery of PET methods and gains strong CL capability. For example, LAE with Adaptor PET surpasses the prior state-of-the-art by 1.3% and 3.6% in last-incremental accuracy on CIFAR100 and ImageNet-R datasets, respectively. Code is available at \url{https://github.com/gqk/LAE}.Comment: Accepted to ICCV 202

Superconductivity with TcT_c up to 30.7 K in air-annealed CaFeAsF

Exploring new unconventional superconductors is of great value for both fundamental research and practical applications. It is a long-term challenge to develop and study more hole-doped superconductors in 1111 system of iron-based superconductors. However, fifteen years after the discovery of iron-based superconductors, it has become increasingly difficult to discover new members in this system by conventional means. Here we report the discovery of superconductivity with the critical transition temperature up to 30.7 K in the parent compound CaFeAsF by an annealing treatment in air atmosphere. The superconducting behaviors are verified in both the single-crystalline and polycrystalline samples by the resistance and magnetization measurements. The analysis by combining the depth-resolved time-of-flight secondary ion mass spectrometry (TOF-SIMS) and X-ray photoelectron spectroscopy (XPS) measurements show that the introduction of oxygen elements and the consequent changing in Fe valence by the annealing treatment may lead to the hole-type doping, which is the origin for the occurrence of superconductivity. Our results provide a new route to induce hole-doped superconductivity in Fe-based superconductors.Comment: 20 pages, 4 figures and 1 tabl

Multiparametric Cardiovascular Magnetic Resonance in Acute Myocarditis: Comparison of 2009 and 2018 Lake Louise Criteria With Endomyocardial Biopsy Confirmation.

Background: Cardiac magnetic resonance (CMR) has been shown to improve the diagnosis of myocarditis, but no systematic comparison of this technique is currently available. The purpose of this study was to compare the 2009 and 2018 Lake Louise Criteria (LLC) for the diagnosis of acute myocarditis using 3.0 T MRI with endomyocardial biopsy (EMB) as a reference and to provide the cutoff values for multiparametric CMR techniques. Methods: A total of 73 patients (32 ± 14 years, 71.2% men) with clinically suspected myocarditis undergoing EMB and CMR with 3.0 T were enrolled in the study. Patients were divided into two groups according to EMB results (EMB-positive and -negative groups). The CMR protocol consisted of cine-SSFP, T2 STIR, T2 mapping, early and late gadolinium enhancement (EGE, LGE), and pre- and post-contrast T1 mapping. Their potential diagnostic ability was assessed with receiver operating characteristic curves. Results: The myocardial T1 and T2 relaxation times were significantly higher in the EMB-positive group than in the EMB-negative group. Optimal cutoff values were 1,228 ms for T1 relaxation times and 58.5 ms for T2 relaxation times with sensitivities of 86.0 and 83.7% and specificities of 93.3 and 93.3%, respectively. The 2018 LLC had a better diagnostic performance than the 2009 LLC in terms of sensitivity, specificity, positive predictive value, and negative predictive value. T1 mapping + T2 mapping had the largest area under the curve (0.95) compared to other single or combined parameters (2018 LLC: 0.91; 2009 LLC: 0.76; T2 ratio: 0.71; EGEr: 0.67; LGE: 0.73; ). The diagnostic accuracy for the 2018 LLC was the highest (91.8%), followed by T1 mapping (89.0%) and T2 mapping (87.7%). Conclusion: Emerging technologies such as T1/ T2 mapping have significantly improved the diagnostic performance of CMR for the diagnosis of acute myocarditis. The 2018 LLC provided the overall best diagnostic performance in acute myocarditis compared to other single standard CMR parameters or combined parameters. There was no significant gain when 2018LLC is combined with the EGE sequence

Synthesis of Mesoporous Silica@Co–Al Layered Double Hydroxide Spheres: Layer-by-Layer Method and Their Effects on the Flame Retardancy of Epoxy Resins

Hierarchical mesoporous silica@Co–Al layered double hydroxide (m-SiO2@Co–Al LDH) spheres were prepared through a layer-by-layer assembly process, in order to integrate their excellent physical and chemical functionalities. TEM results depicted that, due to the electrostatic potential difference between m-SiO2 and Co–Al LDH, the synthetic m-SiO2@Co–Al LDH hybrids exhibited that m-SiO2 spheres were packaged by the Co–Al LDH nanosheets. Subsequently, the m-SiO2@Co–Al LDH spheres were incorporated into epoxy resin (EP) to prepare specimens for investigation of their flame-retardant performance. Cone results indicated that m-SiO2@Co–Al LDH incorporated obviously improved fire retardant of EP. A plausible mechanism of fire retardant was hypothesized based on the analyses of thermal conductivity, char residues, and pyrolysis fragments. Labyrinth effect of m-SiO2 and formation of graphitized carbon char catalyzed by Co–Al LDH play pivotal roles in the flame retardance enhancement

Proton-Boron Fusion Yield Increased by Orders of Magnitude with Foam Targets

A novel intense beam-driven scheme for high yield of the tri-alpha reaction 11B(p,{\alpha})2{\alpha} was investigated. We used a foam target made of cellulose triacetate (TAC, C_9H_{16}O_8) doped with boron. It was then heated volumetrically by soft X-ray radiation from a laser heated hohlraum and turned into a homogenous, and long living plasma. We employed a picosecond laser pulse to generate a high-intensity energetic proton beam via the well-known Target Normal Sheath Acceleration (TNSA) mechanism. We observed up to 10^{10}/sr {\alpha} particles per laser shot. This constitutes presently the highest yield value normalized to the laser energy on target. The measured fusion yield per proton exceeds the classical expectation of beam-target reactions by up to four orders of magnitude under high proton intensities. This enhancement is attributed to the strong electric fields and nonequilibrium thermonuclear fusion reactions as a result of the new method. Our approach shows opportunities to pursue ignition of aneutronic fusion

Transplantation of canine umbilical cord blood-derived mesenchymal stem cells in experimentally induced spinal cord injured dogs

This study was to determine the effects of allogenic umbilical cord blood (UCB)-derived mesenchymal stem cells (MSCs) and recombinant methionyl human granulocyte colony-stimulating factor (rmhGCSF) on a canine spinal cord injury model after balloon compression at the first lumbar vertebra. Twenty-five adult mongrel dogs were assigned to five groups according to treatment after a spinal cord injury: no treatment (CN); saline treatment (CP); rmhGCSF treatment (G); UCB-MSCs treatment (UCB-MSC); co-treatment (UCBG). The UCB-MSCs isolated from cord blood of canine fetuses were prepared as 106 cells/150 µl saline. The UCB-MSCs were directly injected into the injured site of the spinal cord and rmhGCSF was administered subcutaneously 1 week after the induction of spinal cord injury. The Olby score, magnetic resonance imaging, somatosensory evoked potentials and histopathological examinations were used to evaluate the functional recovery after transplantation. The Olby scores of all groups were zero at the 0-week evaluation. At 2 week after the transplantation, the Olby scores in the groups with the UCB-MSC and UCBG were significantly higher than in the CN and CP groups. However, there were no significant differences between the UCB-MSC and UCBG groups, and between the CN and CP groups. These comparisons remained stable at 4 and 8 week after transplantation. There was significant improvement in the nerve conduction velocity based on the somatosensory evoked potentials. In addition, a distinct structural consistency of the nerve cell bodies was noted in the lesion of the spinal cord of the UCB-MSC and UCBG groups. These results suggest that transplantation of the UCB-MSCs resulted in recovery of nerve function in dogs with a spinal cord injury and may be considered as a therapeutic modality for spinal cord injury