On the Effectiveness of Compact Biomedical Transformers

Language models pre-trained on biomedical corpora, such as BioBERT, have recently shown promising results on downstream biomedical tasks. Many existing pre-trained models, on the other hand, are resource-intensive and computationally heavy owing to factors such as embedding size, hidden dimension, and number of layers. The natural language processing (NLP) community has developed numerous strategies to compress these models utilising techniques such as pruning, quantisation, and knowledge distillation, resulting in models that are considerably faster, smaller, and subsequently easier to use in practice. By the same token, in this paper we introduce six lightweight models, namely, BioDistilBERT, BioTinyBERT, BioMobileBERT, DistilBioBERT, TinyBioBERT, and CompactBioBERT which are obtained either by knowledge distillation from a biomedical teacher or continual learning on the Pubmed dataset via the Masked Language Modelling (MLM) objective. We evaluate all of our models on three biomedical tasks and compare them with BioBERT-v1.1 to create efficient lightweight models that perform on par with their larger counterparts. All the models will be publicly available on our Huggingface profile at https://huggingface.co/nlpie and the codes used to run the experiments will be available at https://github.com/nlpie-research/Compact-Biomedical-Transformers

Relaxations of fluorouracil tautomers by decorations of fullerene-like SiCs: DFT studies

Decorations of silicon carbide (SiC) fullerene-like nanoparticles by fluorouracil (FU) and its tautomers are investigated through density functional theory (DFT) calculations. Two models of fullerene-like particles including Si12C8 and Si8C12 are constructed to be counterparts of decorated hybrid structures, FU@Si12C8 and FU@Si8C12, respectively. The initial models including original FU and tautomeric structures and SiC nanoparticles are individually optimized and then combined for further optimizations in the hybrid forms. Covalent bonds are observed for FU@Si12C8 hybrids, whereas non-covalent interactions are seen for FU@Si8C12 ones. The obtained properties indicated that Si12C8 model could be considered as a better counterpart for interactions with FU structures than Si8C12 model. The results also showed significant effects of interactions on the properties of atoms close to the interacting regions in nanoparticles. Finally, the tautomeric structures show different behaviors in interactions with SiC nanoparticles, in which the SiC nanoparticles could be employed to detect the situations of tautomeric processes for FU structures. © 2016 Elsevier B.V

Increased risk of pre-eclampsia (PE) among women with the history of migraine

The Objective of this study was to assess possible association of history of migraine with pre-eclampsia (PE). This was a retrospective study to compare history of migraine in 90 women affected by PE with 90 women without PE as the control group. They recruited by a nonrandomized consecutive sampling method. Data were collected by a questionnaire including demographic, medical, obstetrics, and migraine assessment sections. Data were analyzed using SPSS. Results showed an increased risk of PE in women with history of migraine (odds ratio: 2.87; p < 0.05). Result demonstrated that migraine history in the case group is 144 and in control group is 56. Gestational age (GA) at delivery and weight of neonate (WN) were significantly lower compared to control (GA: 37.3 ± 2.6 vs. 38.7± 1.3 weeks T test; P < 0.01) (WN: 2930 ± 690 vs. 3330 ± 420; T test; P < 0.0). Cesarean section was more frequent in the PE group compared to the control group 37 (42) vs. 14 (15.6); chi square; p < 0.01. The association of migraine with PE is the result of some similar mechanism leading to endothelial dysfunction. Frequent reports of an association between migraine and PE in different populations suggest a history of migraine as a risk factor for PEgestational hypertension (GH). Copyright © Informa UK Ltd

A crowd of BashTheBug volunteers reproducibly and accurately measure the minimum inhibitory concentrations of 13 antitubercular drugs from photographs of 96-well broth microdilution plates

Tuberculosis is a respiratory disease that is treatable with antibiotics. An increasing prevalence of resistance means that to ensure a good treatment outcome it is desirable to test the susceptibility of each infection to different antibiotics. Conventionally, this is done by culturing a clinical sample and then exposing aliquots to a panel of antibiotics, each being present at a pre-determined concentration, thereby determining if the sample isresistant or susceptible to each sample. The minimum inhibitory concentration (MIC) of a drug is the lowestconcentration that inhibits growth and is a more useful quantity but requires each sample to be tested at a range ofconcentrations for each drug. Using 96-well broth micro dilution plates with each well containing a lyophilised pre-determined amount of an antibiotic is a convenient and cost-effective way to measure the MICs of several drugs at once for a clinical sample. Although accurate, this is still an expensive and slow process that requires highly-skilled and experienced laboratory scientists. Here we show that, through the BashTheBug project hosted on the Zooniverse citizen science platform, a crowd of volunteers can reproducibly and accurately determine the MICs for 13 drugs and that simply taking the median or mode of 11-17 independent classifications is sufficient. There is therefore a potential role for crowds to support (but not supplant) the role of experts in antibiotic susceptibility testing

Clinical and metabolic response to probiotic administration in people with Parkinson's disease: A randomized, double-blind, placebo-controlled trial

Background &amp; aims: The investigation was done to assess the impacts of probiotic supplementation on movement and metabolic parameters in individuals with Parkinson's disease (PD). Methods: The study is randomized, double-blind, placebo-controlled clinical trial, which was done in sixty people with PD. Individuals were randomly divided into two groups in order to take either 8 � 109 CFU/day probiotic or placebo (n = 30 each group) that lasted 12 weeks. The Movement Disorders Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS) was recorded at pre- and post-intervention. Results: Compared with the placebo, consuming probiotic decreased MDS-UPDRS (�4.8 ± 12.5 vs. +3.8 ± 13.0, P = 0.01). Probiotic supplementation also reduced high-sensitivity C-reactive protein (�1.6 ± 2.5 vs. +0.1 ± 0.3 mg/L, P &lt; 0.001) and malondialdehyde (�0.2 ± 0.3 vs. +0.1 ± 0.3 μmol/L, P = 0.006), and enhanced glutathione levels (+40.1 ± 81.5 vs. �12.1 ± 41.7 μmol/L, P = 0.03) in comparison with the placebo. Additionally, probiotic consumption resulted in a statistically significant reduction in insulin levels (�2.1 ± 3.4 vs. +1.5 ± 5.1 μIU/mL, P = 0.002) and insulin resistance (�0.5 ± 0.9 vs. +0.4 ± 1.2, P = 0.002), and a statistically significant rise in insulin sensitivity (+0.01 ± 0.02 vs. �0.006 ± 0.02, P = 0.01) in comparison with the placebo. Probiotic intake had no any significant impact on other metabolic profiles. Conclusions: Our study evidenced that 12 weeks of probiotic consumption by individuals with PD had useful impacts on MDS-UPDRS and few metabolic profiles. Registered under ClinicalTrials.gov Identifier no. http://www.irct.ir: IRCT2017082434497N4. © 2018 Elsevier Ltd and European Society for Clinical Nutrition and Metabolis

International Center for Partnered Pavement Preservation (ICP3): First Year Progress Report

0-6878The Accelerating Innovation in Partnered Pavement Preservation project was initiated to promote and streamline research in the area of pavement preservation and to optimize the use of Texas Department of Transportation's (TxDOT\u2019s) research and implementation resources by fostering cooperation and collaboration with the US DOT Center for Highway Pavement Preservation (CHPP). CHPP is a research and innovation partnership lead by Michigan State University which members include: The University of Texas at Austin, The University of Illinois at Urbana-Champaign, The University of Minnesota, The University of Hawaii at Manoa and North Carolina A&T University. This preliminary progress report summarizes the work performed during the first five months of the project, from April to August 2015. During this period two task orders were developed and the corresponding work was planned and initiated. This report also presents the initial findings of these two task orders. The two task orders are: 1) Determination of Field Performance of Thin Overlays Relative to Alternative Preservation Techniques and 2) Quantification of Highway Pavement Surface Micro- and Macro-Texture

Multiview classification and dimensionality reduction of scalp and intracranial EEG data through tensor factorisation

Electroencephalography (EEG) signals arise as a mixture of various neural processes that occur in different spatial, frequency and temporal locations. In classification paradigms, algorithms are developed that can distinguish between these processes. In this work, we apply tensor factorisation to a set of EEG data from a group of epileptic patients and factorise the data into three modes; space, time and frequency with each mode containing a number of components or signatures. We train separate classifiers on various feature sets corresponding to complementary combinations of those modes and components and test the classification accuracy of each set. The relative influence on the classification accuracy of the respective spatial, temporal or frequency signatures can then be analysed and useful interpretations can be made. Additionaly, we show that through tensor factorisation we can perform dimensionality reduction by evaluating the classification performance with regards to the number mode components and by rejecting components with insignificant contribution to the classification accuracy

Development of a Three-Dimensional Laser Scanning Prototype for Pavement Preservation Applications

0-6878The connection between the pavement friction and surface macro- and micro-texture makes pavement texture a vital topic for highway agencies and state Departments of Transportation to address. Accordingly, measuring surface texture is of prime importance in pavement preservation applications. Currently, most highway agencies rely on subjective measuring methods of pavement texture that are not reliable. With the recent advancements in laser technologies, highway agencies are gearing towards employing such developments that could potentially contribute towards better pavement surface texture characterization. With better surface texture characterization, they can better assess, monitor, and improve the pavement texture to provide better skid resistance for their highway network with the aim of ensuring safer roads for the public. This report proposes a laser scanning prototype for the characterization of the micro- and macro-texture of road surfaces. This prototype system has the advantage of capturing 3D data on pavement surfaces using an automatic, simple, and quick operation. This prototype is able to work in laboratory and in field. The development stages of this prototype are provided in this report. In addition, this report discusses different applications of the prototype created. The findings of this research are provided in three parts. Part I involves the feasibility of the prototype in characterizing the micro-texture of aggregates. Part II involves developing an algorithm to measure the mean texture depth of pavement surfaces. Part III involves finding the connection between the pavement friction and texture. Other applications of this prototype are in chip seal design. The results study and findings regarding the chip seal design are provided in the appendix