AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models

There are growing interests in adapting large-scale language models using parameter-efficient fine-tuning methods. However, accelerating the model itself and achieving better inference efficiency through model compression has not been thoroughly explored yet. Model compression could provide the benefits of reducing memory footprints, enabling low-precision computations, and ultimately achieving cost-effective inference. To combine parameter-efficient adaptation and model compression, we propose AlphaTuning consisting of post-training quantization of the pre-trained language model and fine-tuning only some parts of quantized parameters for a target task. Specifically, AlphaTuning works by employing binary-coding quantization, which factorizes the full-precision parameters into binary parameters and a separate set of scaling factors. During the adaptation phase, the binary values are frozen for all tasks, while the scaling factors are fine-tuned for the downstream task. We demonstrate that AlphaTuning, when applied to GPT-2 and OPT, performs competitively with full fine-tuning on a variety of downstream tasks while achieving >10x compression ratio under 4-bit quantization and >1,000x reduction in the number of trainable parameters.Comment: Findings of EMNLP 202

Effectiveness of mechanical thrombectomy in cancer-related stroke and associated factors with unfavorable outcome

Background The effectiveness of mechanical thrombectomy (MT) in cancer-related stroke (CRS) is largely unknown. This study aims to investigate the clinical and radiological outcomes of MT in CRS patients. We also explored the factors that independently affect functional outcomes of patients with CRS after MT. Methods We retrospectively reviewed 341 patients who underwent MT after acute ischemic stroke onset between May 2014 and May 2020. We classified the patients into CRS (n = 34) and control (n = 307) groups and compared their clinical details. Among CRS patients, we analyzed the groups with and without good outcomes (3-months modified Rankin scale [mRS] score 0, 1, 2). Multivariate analysis was performed to investigate the independent predictors of unfavorable outcomes in patients with CRS after MT. Results A total of 341 acute ischemic stroke patients received MT, of whom 34 (9.9%) had CRS. Although the baseline National institute of health stroke scale (NIHSS) score and the rate of successful recanalization was not significantly different between CRS patients and control group, CRS patients showed more any cerebral hemorrhage after MT (41.2% vs. controls 23.8%, p = 0.037) and unfavorable functional outcome at 3 months (CRS patients median 3-month mRS score 4, interquartile range [IQR] 2 to 5.25 vs. controls median 3-month mRS score 3, IQR 1 to 4, [p = 0.026]). In the patients with CRS, elevated serum D-dimer level and higher baseline NIHSS score were independently associated with unfavorable functional outcome at 3 months (adjusted odds ratio [aOR]: 1.524, 95% confidence interval [CI]: 1.043–2.226; aOR: 1.264, 95% CI: 1.010–1.582, respectively). Conclusions MT is an appropriate therapeutic treatment for revascularization in CRS patients. However, elevated serum D-dimer levels and higher baseline NIHSS scores were independent predictors of unfavorable outcome. Further research is warranted to evaluate the significance of these predictors.This research was supported by a fund (#2020ER620200) by the Korea Centers for Disease Control & Prevention and Clinical Research Society for Stroke, Republic of Korea

Regulation of synaptic Rac1 activity, long-term potentiation maintenance, and learning and memory by BCR and ABR Rac GTPase-activating proteins

Rho family small GTPases are important regulators of neuronal development. Defective Rho regulation causes nervous system dysfunctions including mental retardation and Alzheimer's disease. Rac1, a member of the Rho family, regulates dendritic spines and excitatory synapses, but relatively little is known about how synaptic Rac1 is negatively regulated. Breakpoint cluster region (BCR) is a Rac GTPase-activating protein known to form a fusion protein with the c-Abl tyrosine kinase in Philadelphia chromosome-positive chronic myelogenous leukemia. Despite the fact that BCR mRNAs are abundantly expressed in the brain, the neural functions of BCR protein have remained obscure. We report here that BCR and its close relative active BCR-related (ABR) localize at excitatory synapses and directly interact with PSD-95, an abundant postsynaptic scaffolding protein. Mice deficient for BCR or ABR show enhanced basal Rac1 activity but only a small increase in spine density. Importantly, mice lacking BCR or ABR exhibit a marked decrease in the maintenance, but not induction, of long-term potentiation, and show impaired spatial and object recognition memory. These results suggest that BCR and ABR have novel roles in the regulation of synaptic Rac1 signaling, synaptic plasticity, and learning and memory, and that excessive Rac1 activity negatively affects synaptic and cognitive functions.This work was supported by the National Creative Research Initiative Program of the Korean Ministry of Education, Science and Technology (E.K.), Neuroscience Program Grant 2009-0081468 (S.-Y.C.), 21st Century Frontier R&D Program in Neuroscience Grant 2009K001284 (H.K.), Basic Science Research Program Grant R13-2008-009-01001-0 (Y.C.B.), and United States Public Health Service Grants HL071945 (J.G.) and HL060231 (J.G., N.H.)

Impact of onset-to-door time on outcomes and factors associated with late hospital arrival in patients with acute ischemic stroke.

Background and purposePrevious studies have reported that early hospital arrival improves clinical outcomes in patients with acute ischemic stroke; however, whether early arrival is associated with favorable outcomes regardless of reperfusion therapy and the type of stroke onset time is unclear. Thus, we investigated the impact of onset-to-door time on outcomes and evaluated the predictors of pre-hospital delay after ischemic stroke.MethodsConsecutive acute ischemic stroke patients who arrived at the hospital within five days of onset from September 2019 to May 2020 were selected from the prospective stroke registries of Seoul National University Hospital and Chung-Ang University Hospital of Seoul, Korea. Patients were divided into early (onset-to-door time, ≤4.5 h) and late (>4.5 h) arrivers. Multivariate analyses were performed to assess the effect of early arrival on clinical outcomes and predictors of late arrival.ResultsAmong the 539 patients, 28.4% arrived early and 71.6% arrived late. Early hospital arrival was significantly associated with favorable outcomes (three-month modified Rankin Scale [mRS]: 0-2, adjusted odds ratio [aOR]: 2.03, 95% confidence interval: [CI] 1.04-3.96) regardless of various confounders, including receiving reperfusion therapy and type of stroke onset time. Furthermore, a lower initial National Institute of Health Stroke Scale (NIHSS) score (aOR: 0.94, 95% CI: 0.90-0.97), greater pre-stroke mRS score (aOR: 1.58, 95% CI: 1.18-2.13), female sex (aOR: 1.71, 95% CI: 1.14-2.58), unclear onset time, and ≤6 years of schooling (aOR: 1.76, 95% CI: 1.03-3.00 compared to >12 years of schooling) were independent predictors of late arrival.ConclusionsThus, the onset-to-door time of≤4.5 h is crucial for better clinical outcome, and lower NIHSS score, greater pre-stroke mRS score, female sex, unclear onset times, and ≤6 years of schooling were independent predictors of late arrival. Therefore, educating about the importance of early hospital arrival after acute ischemic stroke should be emphasized. More strategic efforts are needed to reduce the prehospital delay by understanding the predictors of late arrival

Serum CYFRA 21.1 Level Predicts Disease Course in Thyroid Cancer with Distant Metastasis

Background: Serum Cyfra 21.1, the soluble fragment of CK19, has been used as a prognostic tumor marker in various cancers, indicating poor tumor differentiation and increased metastasis. Methods: We analyzed the serum Cyfra 21.1 level in 51 consecutive patients with thyroid cancer manifesting distant metastasis treated with prior total thyroidectomy. Serum Cyfra 21.1 levels of 26 thyroid cancer patients without metastasis and 50 healthy individuals were used for comparison. Results: Higher serum Cyfra 21.1 levels were detected in thyroid cancer patients with distant metastasis compared with healthy subjects and thyroid cancer patients without metastasis (p = 0.012). Serum Cyfra 21.1 levels were significantly increased in patients with positive BRAF V600E mutation (p = 0.019), undergoing Tyrosine Kinase Inhibitor (TKI) therapy (p = 0.008), with radioiodine-refractory status (p = 0.047), and in disease progression compared with those manifesting stable disease (p = 0.007). In progressive disease with undetectable or unmonitored thyroglobulin because of thyroglobulin antibody, serum Cyfra 21.1 was useful as a biomarker for follow-up of disease course. Conclusion: Serum Cyfra 21.1 in thyroid cancer patients might represent an alternative biomarker predicting tumor progression, especially in cases not associated with serum Tg levels

Enhanced NMDA Receptor-Mediated Synaptic Transmission, Enhanced Long-Term Potentiation, and Impaired Learning and Memory in Mice Lacking IRSp53

IRSp53 is an adaptor protein that acts downstream of Rac and Cdc42 small GTPases and is implicated in the regulation of membrane deformation and actin filament assembly. In neurons, IRSp53 is an abundant postsynaptic protein and regulates actin-rich dendritic spines; however, its in vivo functions have not been explored. We characterized transgenic mice deficient of IRSp53 expression. Unexpectedly, IRSp53–/– neurons do not show significant changes in the density and ultrastructural morphologies of dendritic spines. Instead, IRSp53–/– neurons exhibit reduced AMPA/NMDA ratio of excitatory synaptic transmission and a selective increase in NMDA but not AMPA receptor-mediated transmission. IRSp53–/– hippocampal slices show a markedly enhanced long-term potentiation (LTP) with no changes in long-term depression. LTP-inducing theta burst stimulation enhances NMDA receptor-mediated transmission. Spatial learning and novel object recognition are impaired in IRSp53–/– mice. These results suggest that IRSp53 is involved in the regulation of NMDA receptor-mediated excitatory synaptic transmission, LTP, and learning and memory behaviors

Glycated Albumin, a Novel Biomarker for Short-Term Functional Outcomes in Acute Ischemic Stroke

Background: There is growing interest in the use of new biomarkers such as glycated albumin (GA), but data are limited in acute ischemic stroke. We explored the impact of GA on short-term functional outcomes as measured using the modified Rankin Scale (mRS) at 3 months compared to glycated hemoglobin (HbA1c). Methods: A total of 1163 AIS patients from two hospitals between 2016 and 2019 were included. Patients were divided into two groups according to GA levels (GA Results: A total of 518 patients (44.5%) were included in the GA ≥ 16% group. After adjusting for multiple covariates, the higher GA group (GA ≥ 16%) had a 1.4-fold risk of having unfavorable mRS (95% CI 1.02–1.847). However, HbA1c was not significantly associated with 3-month mRS. In addition, GA ≥ 16% was independently associated with unfavorable short-term outcomes only in patients without diabetes. Conclusions: In light of these results, GA level might be a novel prognostic biomarker compared to HbA1c for short-term stroke outcome. Although the impact of GA is undervalued in the current stroke guidelines, GA monitoring should be considered in addition to HbA1c monitoring

Glycated Albumin, a Novel Biomarker for Short-Term Functional Outcomes in Acute Ischemic Stroke

Background: There is growing interest in the use of new biomarkers such as glycated albumin (GA), but data are limited in acute ischemic stroke. We explored the impact of GA on short-term functional outcomes as measured using the modified Rankin Scale (mRS) at 3 months compared to glycated hemoglobin (HbA1c). Methods: A total of 1163 AIS patients from two hospitals between 2016 and 2019 were included. Patients were divided into two groups according to GA levels (GA < 16% versus GA ≥ 16%). Results: A total of 518 patients (44.5%) were included in the GA ≥ 16% group. After adjusting for multiple covariates, the higher GA group (GA ≥ 16%) had a 1.4-fold risk of having unfavorable mRS (95% CI 1.02–1.847). However, HbA1c was not significantly associated with 3-month mRS. In addition, GA ≥ 16% was independently associated with unfavorable short-term outcomes only in patients without diabetes. Conclusions: In light of these results, GA level might be a novel prognostic biomarker compared to HbA1c for short-term stroke outcome. Although the impact of GA is undervalued in the current stroke guidelines, GA monitoring should be considered in addition to HbA1c monitoring