AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models

There are growing interests in adapting large-scale language models using parameter-efficient fine-tuning methods. However, accelerating the model itself and achieving better inference efficiency through model compression has not been thoroughly explored yet. Model compression could provide the benefits of reducing memory footprints, enabling low-precision computations, and ultimately achieving cost-effective inference. To combine parameter-efficient adaptation and model compression, we propose AlphaTuning consisting of post-training quantization of the pre-trained language model and fine-tuning only some parts of quantized parameters for a target task. Specifically, AlphaTuning works by employing binary-coding quantization, which factorizes the full-precision parameters into binary parameters and a separate set of scaling factors. During the adaptation phase, the binary values are frozen for all tasks, while the scaling factors are fine-tuned for the downstream task. We demonstrate that AlphaTuning, when applied to GPT-2 and OPT, performs competitively with full fine-tuning on a variety of downstream tasks while achieving >10x compression ratio under 4-bit quantization and >1,000x reduction in the number of trainable parameters.Comment: Findings of EMNLP 202

Effectiveness of mechanical thrombectomy in cancer-related stroke and associated factors with unfavorable outcome

Background The effectiveness of mechanical thrombectomy (MT) in cancer-related stroke (CRS) is largely unknown. This study aims to investigate the clinical and radiological outcomes of MT in CRS patients. We also explored the factors that independently affect functional outcomes of patients with CRS after MT. Methods We retrospectively reviewed 341 patients who underwent MT after acute ischemic stroke onset between May 2014 and May 2020. We classified the patients into CRS (n = 34) and control (n = 307) groups and compared their clinical details. Among CRS patients, we analyzed the groups with and without good outcomes (3-months modified Rankin scale [mRS] score 0, 1, 2). Multivariate analysis was performed to investigate the independent predictors of unfavorable outcomes in patients with CRS after MT. Results A total of 341 acute ischemic stroke patients received MT, of whom 34 (9.9%) had CRS. Although the baseline National institute of health stroke scale (NIHSS) score and the rate of successful recanalization was not significantly different between CRS patients and control group, CRS patients showed more any cerebral hemorrhage after MT (41.2% vs. controls 23.8%, p = 0.037) and unfavorable functional outcome at 3 months (CRS patients median 3-month mRS score 4, interquartile range [IQR] 2 to 5.25 vs. controls median 3-month mRS score 3, IQR 1 to 4, [p = 0.026]). In the patients with CRS, elevated serum D-dimer level and higher baseline NIHSS score were independently associated with unfavorable functional outcome at 3 months (adjusted odds ratio [aOR]: 1.524, 95% confidence interval [CI]: 1.043–2.226; aOR: 1.264, 95% CI: 1.010–1.582, respectively). Conclusions MT is an appropriate therapeutic treatment for revascularization in CRS patients. However, elevated serum D-dimer levels and higher baseline NIHSS scores were independent predictors of unfavorable outcome. Further research is warranted to evaluate the significance of these predictors.This research was supported by a fund (#2020ER620200) by the Korea Centers for Disease Control & Prevention and Clinical Research Society for Stroke, Republic of Korea

Seawater Battery-Based Wireless Marine Buoy System with Battery Degradation Prediction and Multiple Power Optimization Capabilities

This paper presents a wireless marine buoy system based on the seawater battery (SWB), providing self-powered operation, power-efficient management, and degradation prediction and fault detection. Since conventional open circuit voltage (OCV) methods cannot be applied due to inherent cell characteristics of SWB, the coulomb counting (CC) method is adopted for the state of charge (SOC) monitoring. For the state of health (SOH), a variance-based detection scheme is proposed to provide degradation prediction and fault detection of the SWB. The self-powered operation is augmented by two proposed power optimization schemes such as multiple power management and three-step LED light control. A wireless buoy system prototype is manufactured, and its functional feasibility is experimentally verified, where its location and SOC are periodically monitored in a smartphone-based wireless platform

Regulation of Dendritic Spine Morphogenesis by Insulin Receptor Substrate 53, a Downstream Effector of Rac1 and Cdc42 Small GTPases

The small GTPases Rac1 and Cdc42 are key regulators of the morphogenesis of actin-rich dendritic spines in neurons. However, little is known about how activated Rac1/Cdc42 regulates dendritic spines. Insulin receptor substrate 53 (IRSp53), which is highly expressed in the postsynaptic density (PSD), is known to link activated Rac1/Cdc42 to downstream effectors for actin regulation in non-neural cells. Here, we report that IRSp53 interacts with two specific members of the PSD-95 family, PSD-95 and chapsyn-110/PSD-93, in brain. An IRSp53 mutant lacking the C-terminal PSD-95-binding motif shows significant loss of synaptic localization in cultured neurons. Overexpression of IRSp53 in cultured neurons increases the density of dendritic spines but does not affect their length or width. Conversely, short-interfering RNA-mediated knock-down of IRSp53 reduces the density, length, and width of spines. In addition, the density and size of spines are decreased by a dominant-negative IRSp53 with a point mutation in the Src homology 3 (SH3) domain and a dominant-negative proline-rich region of WAVE2 (Wiskott-Aldrich syndrome protein family Verprolin-homologous protein), a downstream effector of IRSp53 that binds to the SH3 domain of IRSp53. These results suggest that PSD-95 interaction is an important determinant of synaptic IRSp53 localization and that the SH3 domain of IRSp53 links activated Rac1/Cdc42 to downstream effectors for the regulation of spine morphogenesis

Regulation of synaptic Rac1 activity, long-term potentiation maintenance, and learning and memory by BCR and ABR Rac GTPase-activating proteins

Rho family small GTPases are important regulators of neuronal development. Defective Rho regulation causes nervous system dysfunctions including mental retardation and Alzheimer's disease. Rac1, a member of the Rho family, regulates dendritic spines and excitatory synapses, but relatively little is known about how synaptic Rac1 is negatively regulated. Breakpoint cluster region (BCR) is a Rac GTPase-activating protein known to form a fusion protein with the c-Abl tyrosine kinase in Philadelphia chromosome-positive chronic myelogenous leukemia. Despite the fact that BCR mRNAs are abundantly expressed in the brain, the neural functions of BCR protein have remained obscure. We report here that BCR and its close relative active BCR-related (ABR) localize at excitatory synapses and directly interact with PSD-95, an abundant postsynaptic scaffolding protein. Mice deficient for BCR or ABR show enhanced basal Rac1 activity but only a small increase in spine density. Importantly, mice lacking BCR or ABR exhibit a marked decrease in the maintenance, but not induction, of long-term potentiation, and show impaired spatial and object recognition memory. These results suggest that BCR and ABR have novel roles in the regulation of synaptic Rac1 signaling, synaptic plasticity, and learning and memory, and that excessive Rac1 activity negatively affects synaptic and cognitive functions.This work was supported by the National Creative Research Initiative Program of the Korean Ministry of Education, Science and Technology (E.K.), Neuroscience Program Grant 2009-0081468 (S.-Y.C.), 21st Century Frontier R&D Program in Neuroscience Grant 2009K001284 (H.K.), Basic Science Research Program Grant R13-2008-009-01001-0 (Y.C.B.), and United States Public Health Service Grants HL071945 (J.G.) and HL060231 (J.G., N.H.)

Joint Network-wide Opportunistic Scheduling and Power Control in Multi-cell Networks

We present a unified analytical framework that maximizes generalized utilities of a wireless network by networkwide opportunistic scheduling and power control. That is, base stations in the network jointly decide mobile stations to be served at the same time as the transmission powers of base stations are coordinated to mitigate the mutually interfering effect. Although the maximization at the first glance appears to be a mixed, twofold and nonlinear optimization requiring excessive computational complexity, we show that the maximization can be transformed into a pure binary optimization with much lower complexity. To be exact, it is proven that binary power control of base stations is necessary and sufficient for maximizing the network-wide utilities under a physical layer regime where the channel capacity is linear in the signal-to-interference-noise ratio. To further reduce the complexity of the problem, a distributed heuristic algorithm is proposed that performs much better than existing opportunistic algorithms. Through extensive simulations, it becomes clear that network-wide opportunistic scheduling and power control is most suitable for fairness-oriented networks and underloaded networks. We believe that our work will serve as a cornerstone for network-wide scheduling approaches from theoretical and practical standpoints