Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers

Following the major success of neural language models (LMs) such as BERT or GPT-2 on a variety of language understanding tasks, recent work focused on injecting (structured) knowledge from external resources into these models. While on the one hand, joint pretraining (i.e., training from scratch, adding objectives based on external knowledge to the primary LM objective) may be prohibitively computationally expensive, post-hoc fine-tuning on external knowledge, on the other hand, may lead to the catastrophic forgetting of distributional knowledge. In this work, we investigate models for complementing the distributional knowledge of BERT with conceptual knowledge from ConceptNet and its corresponding Open Mind Common Sense (OMCS) corpus, respectively, using adapter training. While overall results on the GLUE benchmark paint an inconclusive picture, a deeper analysis reveals that our adapter-based models substantially outperform BERT (up to 15-20 performance points) on inference tasks that require the type of conceptual knowledge explicitly present in ConceptNet and OMCS

Patients’ Radiation Doses During the Implantation of Stents in Carotid, Renal, Iliac, Femoral and Popliteal Arteries

AbstractObjectives and DesignThe aim of the study was to document the radiation doses to patients during the implantation of stents in various arteries and to discuss potential reasons for prolongation of radiological procedures.Materials and MethodsMeasurements of air kerma (Gy) and dose–area product (Gy cm2) (DAP) were carried out simultaneously on a sample of 345 patients, who underwent different interventional radiological procedures involving angioplasty with stenting of 73 carotid (21.5%), 22 renal (6.5%), 160 iliac (45%), 63 femoral (18.6%) and 27 popliteal (7.9%) arteries.ResultsThe highest mean air kerma values for fluoroscopy and exposure were found for renal angioplasty (340 and 420 mGy, respectively). With regard to total DAP values, the highest were obtained for renal (148 Gy cm2) and iliac/The Inter-Society Consensus for Management of Peripheral Arterial Disease (TASC) II C (199 Gy cm2) stent implantation. The lowest values were for carotid (53 Gy cm2), iliac/TASC II A (6.3 Gy cm2) and femoral/TASC II A (53 Gy cm2) arteries. For 3.5% of the patients, the air kerma was between 1 and 1.5 Gy and for 1.5%, it was between 1.5 and 2 Gy.ConclusionsIn procedures performed on the arteries of the lower limbs, a significantly higher dose was received by patients with TASC II C lesions. With regard to the number of stents implanted, the total DAP value was 50% higher for simultaneous three-stent implantation than for one or two stents

Folding mechanisms steer the amyloid fibril formation propensity of highly homologous proteins

Significant advances in the understanding of the molecular determinants of fibrillogenesis can be expected from comparative studies of the aggregation propensities of proteins with highly homologous structures but different folding pathways. Here, we fully characterize, by means of stopped-flow, T-jump, CD and DSC experiments, the unfolding mechanisms of three highly homologous proteins, zinc binding Ros87 and Ml153-149 and zinc-lacking Ml452-151. The results indicate that the three proteins significantly differ in terms of stability and (un)folding mechanisms. Particularly, Ros87 and Ml153-149 appear to be much more stable to guanidine denaturation and are characterized by folding mechanisms including the presence of an intermediate. On the other hand, metal lacking Ml452-151 folds according to a classic two-state model. Successively, we have monitored the capabilities of Ros87, Ml452-151 and Ml153-149 to form amyloid fibrils under native conditions. Particularly, we show, by CD, fluorescence, DLS, TEM and SEM experiments, that after 168 hours, amyloid formation of Ros87 has started, while Ml153-149 has formed only amorphous aggregates and Ml452-151 is still monomeric in solution. This study shows how metal binding can influence protein folding pathways and thereby control conformational accessibility to aggregation-prone states, which in turn changes aggregation kinetics, shedding light on the role of metal ions in the development of protein deposition diseases

A terahertz vibrational molecular clock with systematic uncertainty at the 10−1410^{-14} level

Neutral quantum absorbers in optical lattices have emerged as a leading platform for achieving clocks with exquisite spectroscopic resolution. However, the studies of these clocks and their systematic shifts have so far been limited to atoms. Here, we extend this architecture to an ensemble of diatomic molecules and experimentally realize an accurate lattice clock based on pure molecular vibration. We evaluate the leading systematics, including the characterization of nonlinear trap-induced light shifts, achieving a total systematic uncertainty of $4.6\times10^{-14}$. The absolute frequency of the vibrational splitting is measured to be 31 825 183 207 592.8(5.1) Hz, enabling the dissociation energy of our molecule to be determined with record accuracy. Our results represent an important milestone in molecular spectroscopy and THz-frequency standards, and may be generalized to other neutral molecular species with applications for fundamental physics, including tests of molecular quantum electrodynamics and the search for new interactions.Comment: 17 pages, 8 figure