Common Sense or World Knowledge? Investigating Adapter-Based Knowledge Injection into Pretrained Transformers

Following the major success of neural language models (LMs) such as BERT or GPT-2 on a variety of language understanding tasks, recent work focused on injecting (structured) knowledge from external resources into these models. While on the one hand, joint pretraining (i.e., training from scratch, adding objectives based on external knowledge to the primary LM objective) may be prohibitively computationally expensive, post-hoc fine-tuning on external knowledge, on the other hand, may lead to the catastrophic forgetting of distributional knowledge. In this work, we investigate models for complementing the distributional knowledge of BERT with conceptual knowledge from ConceptNet and its corresponding Open Mind Common Sense (OMCS) corpus, respectively, using adapter training. While overall results on the GLUE benchmark paint an inconclusive picture, a deeper analysis reveals that our adapter-based models substantially outperform BERT (up to 15-20 performance points) on inference tasks that require the type of conceptual knowledge explicitly present in ConceptNet and OMCS

Patients’ Radiation Doses During the Implantation of Stents in Carotid, Renal, Iliac, Femoral and Popliteal Arteries

AbstractObjectives and DesignThe aim of the study was to document the radiation doses to patients during the implantation of stents in various arteries and to discuss potential reasons for prolongation of radiological procedures.Materials and MethodsMeasurements of air kerma (Gy) and dose–area product (Gy cm2) (DAP) were carried out simultaneously on a sample of 345 patients, who underwent different interventional radiological procedures involving angioplasty with stenting of 73 carotid (21.5%), 22 renal (6.5%), 160 iliac (45%), 63 femoral (18.6%) and 27 popliteal (7.9%) arteries.ResultsThe highest mean air kerma values for fluoroscopy and exposure were found for renal angioplasty (340 and 420 mGy, respectively). With regard to total DAP values, the highest were obtained for renal (148 Gy cm2) and iliac/The Inter-Society Consensus for Management of Peripheral Arterial Disease (TASC) II C (199 Gy cm2) stent implantation. The lowest values were for carotid (53 Gy cm2), iliac/TASC II A (6.3 Gy cm2) and femoral/TASC II A (53 Gy cm2) arteries. For 3.5% of the patients, the air kerma was between 1 and 1.5 Gy and for 1.5%, it was between 1.5 and 2 Gy.ConclusionsIn procedures performed on the arteries of the lower limbs, a significantly higher dose was received by patients with TASC II C lesions. With regard to the number of stents implanted, the total DAP value was 50% higher for simultaneous three-stent implantation than for one or two stents

Folding mechanisms steer the amyloid fibril formation propensity of highly homologous proteins

Significant advances in the understanding of the molecular determinants of fibrillogenesis can be expected from comparative studies of the aggregation propensities of proteins with highly homologous structures but different folding pathways. Here, we fully characterize, by means of stopped-flow, T-jump, CD and DSC experiments, the unfolding mechanisms of three highly homologous proteins, zinc binding Ros87 and Ml153-149 and zinc-lacking Ml452-151. The results indicate that the three proteins significantly differ in terms of stability and (un)folding mechanisms. Particularly, Ros87 and Ml153-149 appear to be much more stable to guanidine denaturation and are characterized by folding mechanisms including the presence of an intermediate. On the other hand, metal lacking Ml452-151 folds according to a classic two-state model. Successively, we have monitored the capabilities of Ros87, Ml452-151 and Ml153-149 to form amyloid fibrils under native conditions. Particularly, we show, by CD, fluorescence, DLS, TEM and SEM experiments, that after 168 hours, amyloid formation of Ros87 has started, while Ml153-149 has formed only amorphous aggregates and Ml452-151 is still monomeric in solution. This study shows how metal binding can influence protein folding pathways and thereby control conformational accessibility to aggregation-prone states, which in turn changes aggregation kinetics, shedding light on the role of metal ions in the development of protein deposition diseases

The effect of protozoa on the bacterial composition and hydrolytic activity of the roe deer rumen.

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. The purpose of this study was to compare the effect of the presence of protozoa in the rumen of wild roe deer (Capreolus capreolus) on the bacteria composition and digestion rate of the main carbohydrates of forage. The research material involved rumen content and rumen fluid, which were collected in the autumn-winter season, from eight adult males of roe deer with an average body mass of 22.6 kg. The microscopic analysis demonstrated that there were only protozoa in 50% of the animals sampled. Qualitative analysis revealed the presence of protozoa belonging to the genus Entodinium. The density of protozoal population varied from 6.5 to 38.7 × 105/mL rumen fluid. The analysis of bacteria composition indicated that protozoa did not have an effect on bacterial diversity. Furthermore, the results of hydrolytic activity revealed that the fastest digestion of carbohydrates was for pectin, while the slowest was inulin. The pH and redox potential in the rumen varied from 5.9 to 6.1 and from −248.1 to −251.1 mV, respectively. In summary, the presence of protozoa in the rumen of wild roe deer does not have an effect on the bacterial population and has no effect on the digestion rate of carbohydrates in the rumen

A terahertz vibrational molecular clock with systematic uncertainty at the 10−1410^{-14} level

Neutral quantum absorbers in optical lattices have emerged as a leading platform for achieving clocks with exquisite spectroscopic resolution. However, the studies of these clocks and their systematic shifts have so far been limited to atoms. Here, we extend this architecture to an ensemble of diatomic molecules and experimentally realize an accurate lattice clock based on pure molecular vibration. We evaluate the leading systematics, including the characterization of nonlinear trap-induced light shifts, achieving a total systematic uncertainty of $4.6\times10^{-14}$. The absolute frequency of the vibrational splitting is measured to be 31 825 183 207 592.8(5.1) Hz, enabling the dissociation energy of our molecule to be determined with record accuracy. Our results represent an important milestone in molecular spectroscopy and THz-frequency standards, and may be generalized to other neutral molecular species with applications for fundamental physics, including tests of molecular quantum electrodynamics and the search for new interactions.Comment: 17 pages, 8 figure