Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder

The sequence-to-sequence (seq2seq) task aims at generating the target sequence based on the given input source sequence. Traditionally, most of the seq2seq task is resolved by the Encoder-Decoder framework which requires an encoder to encode the source sequence and a decoder to generate the target text. Recently, a bunch of new approaches have emerged that apply decoder-only language models directly to the seq2seq task. Despite the significant advancements in applying language models to the seq2seq task, there is still a lack of thorough analysis on the effectiveness of the decoder-only language model architecture. This paper aims to address this gap by conducting a detailed comparison between the encoder-decoder architecture and the decoder-only language model framework through the analysis of a regularized encoder-decoder structure. This structure is designed to replicate all behaviors in the classical decoder-only language model but has an encoder and a decoder making it easier to be compared with the classical encoder-decoder structure. Based on the analysis, we unveil the attention degeneration problem in the language model, namely, as the generation step number grows, less and less attention is focused on the source sequence. To give a quantitative understanding of this problem, we conduct a theoretical sensitivity analysis of the attention output with respect to the source input. Grounded on our analysis, we propose a novel partial attention language model to solve the attention degeneration problem. Experimental results on machine translation, summarization, and data-to-text generation tasks support our analysis and demonstrate the effectiveness of our proposed model

Knowledge, practices, compliance and beliefs of university nursing students’ toward hand hygiene: A cross-sectional survey

Nursing students can contribute to healthcareassociated infections if having inadequate knowledge and noncompliance of hand hygiene. Effective hand hygiene is considered the most effective measure for preventing healthcare-associated infections. This study investigated the knowledge, practices, compliance, and beliefs of university nursing students toward hand hygiene. By using a well-validated, self-report hand hygiene questionnaire, a cross-sectional survey was performed. In the convenience sampling, 421 out of 506 questionnaires were returned (83.2%). There were 169 (40.14%), 170 (40.38%) and 82 (19.48%) nursing students from Years 1, 2 and 3, respectively. The respondents’ overall mean score for hand hygiene knowledge was 7.33/10 (SD = 1.52) with an increasing trend of knowledge performance along study year was observed. The hand hygiene practice inventory was 4.76/5 (SD = 0.303). The mean scores for respondents from Years 1, 2 and 3 were 4.75 (SD = 0.367), 4.82 (SD = 0.20) and 4.69 (SD = 0.290), respectively. The self-reported hand hygiene compliance rate was 88.17% (SD = 11.922) and the mean score on the health beliefs scale was 4.03/5 (SD = 0.34). The university nursing students demonstrated moderate knowledge, good practices, a high compliance rate and positive beliefs toward hand hygiene. More effort on educating healthcare professionals about alcohol-based hand rubs and the promotion of hand cream usage to prevent hand-hygiene-induced skin irritations are suggested. Regular training workshops or seminars could be arranged to maintain knowledge levels and cultivate a positive attitude toward hand hygiene

OPA1 analysis in an international series of probands with bilateral optic atrophy

Purpose To determine the molecular genetic cause in previously unreported probands with optic atrophy from the United Kingdom, Czech Republic and Canada. Methods OPA1 coding regions and flanking intronic sequences were screened by direct sequencing in 82 probands referred with a diagnosis of bilateral optic atrophy. Detected rare variants were assessed for pathogenicity by in silico analysis. Segregation of the identified variants was performed in available first degree relatives. Results A total of 29 heterozygous mutations evaluated as pathogenic were identified in 42 probands, of these seven were novel. In two probands, only variants of unknown significance were found. 76% of pathogenic mutations observed in 30 (71%) of 42 probands were evaluated to lead to unstable transcripts resulting in haploinsufficiency. Three probands with the following disease-causing mutations c.1230+1G>A, c.1367G>A and c.2965dup were documented to suffer from hearing loss and/or neurological impairment. Conclusions OPA1 gene screening in patients with bilateral optic atrophy is an important part of clinical evaluation as it may establish correct clinical diagnosis. Our study expands the spectrum of OPA1 mutations causing dominant optic atrophy and supports the fact that haploinsufficiency is the most common disease mechanism

Disparities of time trends and birth cohort effects on invasive breast cancer incidence in Shanghai and Hong Kong pre- and post-menopausal women

© 2017 The Author(s). Background: Breast cancer is the leading cause of cancer morbidity among Shanghai and Hong Kong women, which contributes to 20-25% of new female cancer incidents. This study aimed to describe the temporal trend of breast cancer and interpret the potential effects on the observed secular trends. Methods: Cancer incident data were obtained from the cancer registries. Age-standardized incidence rate was computed by the direct method using the World population of 2000. Average annual percentage change (AAPC) in incidence rate was estimated by the Joinpoint regression. Age, period and cohort effects were assessed by using a log-linear model with Poisson regression. Results: During 1976-2009, an increasing trend of breast cancer incidence was observed, with an AAPC of 1.73 [95% confidence interval (CI): 1.54-1.92)] for women in Hong Kong and 2.83 (95% CI, 2.26-3.40) in Shanghai. Greater upward trends were revealed in Shanghai women aged 50 years old or above (AAPC = 3.09; 95% CI, 1.48-4.73). Using age at 50 years old as cut-point, strong birth cohort effects were shown in both pre- and post-menopausal women, though a more remarkable effect was suggested in Shanghai post-menopausal women. No evidence for a period effect was indicated. Conclusions: Incidence rate of breast cancer has been more speedy in Shanghai post-menopausal women than that of the Hong Kong women over the past 30 years. Decreased birth rate and increasing environmental exposures (e.g., light-at-night) over successive generations may have constituted major impacts on the birth cohort effects, especially for the post-menopausal breast cancer; further analytic studies are warranted.Link_to_subscribed_fulltex

Possible Central Nervous System Infection by SARS Coronavirus

On day 22 of illness, generalized tonic-clonic convulsion developed in a 32-year-old woman with severe acute respiratory syndrome (SARS). Cerebrospinal fluid tested positive for SARS coronavirus (SARS-CoV) by reverse transcriptase–polymerase chain reaction. SARS-CoV may have caused an infection in the central nervous system in this patient

Cathelicidin preserves intestinal barrier function in polymicrobial sepsis.

ObjectivesThe intestinal epithelium compartmentalizes the sterile bloodstream and the commensal bacteria in the gut. Accumulating evidence suggests that this barrier is impaired in sepsis, aggravating systemic inflammation. Previous studies reported that cathelicidin is differentially expressed in various tissues in sepsis. However, its role in sepsis-induced intestinal barrier dysfunction has not been investigated.DesignTo examine the role of cathelicidin in polymicrobial sepsis, cathelicidin wild-(Cnlp+/+) and knockout (Cnlp-/-) mice underwent cecal-ligation and puncture (CLP) followed by the assessment of septic mortality and morbidity as well as histological, biochemical, immunological, and transcriptomic analyses in the ileal tissues. We also evaluated the prophylactic and therapeutic efficacies of vitamin D3 (an inducer of endogenous cathelicidin) in the CLP-induced murine polymicrobial sepsis model.ResultsThe ileal expression of cathelicidin was increased by three-fold after CLP, peaking at 4 h. Knockout of Cnlp significantly increased 7-day mortality and was associated with a higher murine sepsis score. Alcian-blue staining revealed a reduced number of mucin-positive goblet cells, accompanied by reduced mucin expression. Increased number of apoptotic cells and cleavage of caspase-3 were observed. Cnlp deletion increased intestinal permeability to 4kD fluorescein-labeled dextran and reduced the expression of tight junction proteins claudin-1 and occludin. Notably, circulating bacterial DNA load increased more than two-fold. Transcriptome analysis revealed upregulation of cytokine/inflammatory pathway. Depletion of Cnlp induced more M1 macrophages and neutrophils compared with the wild-type mice after CLP. Mice pre-treated with cholecalciferol (an inactive form of vitamin D3) or treated with 1alpha, 25-dihydroxyvitamin D3 (an active form of VD3) had decreased 7-day mortality and significantly less severe symptoms. Intriguingly, the administration of cholecalciferol after CLP led to worsened 7-day mortality and the associated symptoms.ConclusionsEndogenous cathelicidin promotes intestinal barrier integrity accompanied by modulating the infiltration of neutrophils and macrophages in polymicrobial sepsis. Our data suggested that 1alpha, 25-dihydroxyvitamin D3 but not cholecalciferol is a potential therapeutic agent for treating sepsis

Use of endobronchial one-way valves reveals questions on etiology of spontaneous pneumothorax: report of three cases

Spontaneous pneumothoraces are believed to arise when air from the supplying airway exit via a ruptured visceral pleural bleb into the pleural cavity. Endobronchial one-way valves (EBVs) allow air exit (but not entry) from individual segmental airways. Systematic deployment of EBVs was applied to three patients with secondary spontaneous pneumothoraces and persistent airleak. In all cases, balloon-catheter occlusion of the upper lobe bronchus stopped the airleak. EBVs applied to individual upper lobe segmental airways failed to terminate the airleak, which only stopped after placements of multiple EBVs to occlude all upper lobe segments. The observation questions the traditional belief of 'one-airway-one-bleb-one-leak' in spontaneous pneumothorax

A new vicious cycle involving glutamate excitotoxicity, oxidative stress and mitochondrial dynamics

Glutamate excitotoxicity leads to fragmented mitochondria in neurodegenerative diseases, mediated by nitric oxide and S-nitrosylation of dynamin-related protein 1, a mitochondrial outer membrane fission protein. Optic atrophy gene 1 (OPA1) is an inner membrane protein important for mitochondrial fusion. Autosomal dominant optic atrophy (ADOA), caused by mutations in OPA1, is a neurodegenerative disease affecting mainly retinal ganglion cells (RGCs). Here, we showed that OPA1 deficiency in an ADOA model influences N-methyl-D-aspartate (NMDA) receptor expression, which is involved in glutamate excitotoxicity and oxidative stress. Opa1enu/+ mice show a slow progressive loss of RGCs, activation of astroglia and microglia, and pronounced mitochondrial fission in optic nerve heads as found by electron tomography. Expression of NMDA receptors (NR1, 2A, and 2B) in the retina of Opa1enu/+ mice was significantly increased as determined by western blot and immunohistochemistry. Superoxide dismutase 2 (SOD2) expression was significantly decreased, the apoptotic pathway was activated as Bax was increased, and phosphorylated Bad and BcL-xL were decreased. Our results conclusively demonstrate that not only glutamate excitotoxicity and/or oxidative stress alters mitochondrial fission/fusion, but that an imbalance in mitochondrial fission/fusion in turn leads to NMDA receptor upregulation and oxidative stress. Therefore, we propose a new vicious cycle involved in neurodegeneration that includes glutamate excitotoxicity, oxidative stress, and mitochondrial dynamics

Coronavirus-positive Nasopharyngeal Aspirate as Predictor for Severe Acute Respiratory Syndrome Mortality

Severe acute respiratory syndrome (SARS) has caused a major epidemic worldwide. A novel coronavirus is deemed to be the causative agent. Early diagnosis can be made with reverse transcriptase-polymerase chain reaction (RT-PCR) of nasopharyngeal aspirate samples. We compared symptoms of 156 SARS-positive and 62 SARS-negative patients in Hong Kong; SARS was confirmed by RT-PCR. The RT-PCR–positive patients had significantly more shortness of breath, a lower lymphocyte count, and a lower lactate dehydrogenase level; they were also more likely to have bilateral and multifocal chest radiograph involvement, to be admitted to intensive care, to need mechanical ventilation, and to have higher mortality rates. By multivariate analysis, positive RT-PCR on nasopharyngeal aspirate samples was an independent predictor of death within 30 days