AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models

There are growing interests in adapting large-scale language models using parameter-efficient fine-tuning methods. However, accelerating the model itself and achieving better inference efficiency through model compression has not been thoroughly explored yet. Model compression could provide the benefits of reducing memory footprints, enabling low-precision computations, and ultimately achieving cost-effective inference. To combine parameter-efficient adaptation and model compression, we propose AlphaTuning consisting of post-training quantization of the pre-trained language model and fine-tuning only some parts of quantized parameters for a target task. Specifically, AlphaTuning works by employing binary-coding quantization, which factorizes the full-precision parameters into binary parameters and a separate set of scaling factors. During the adaptation phase, the binary values are frozen for all tasks, while the scaling factors are fine-tuned for the downstream task. We demonstrate that AlphaTuning, when applied to GPT-2 and OPT, performs competitively with full fine-tuning on a variety of downstream tasks while achieving >10x compression ratio under 4-bit quantization and >1,000x reduction in the number of trainable parameters.Comment: Findings of EMNLP 202

Roles of peroxiredoxin II in the regulation of proinflammatory responses to LPS and protection against endotoxin-induced lethal shock

Mammalian 2-Cys peroxiredoxin II (Prx II) is a cellular peroxidase that eliminates endogenous H2O2. The involvement of Prx II in the regulation of lipopolysaccharide (LPS) signaling is poorly understood. In this report, we show that LPS induces substantially enhanced inflammatory events, which include the signaling molecules nuclear factor κB and mitogen-activated protein kinase (MAPK), in Prx II–deficient macrophages. This effect of LPS was mediated by the robust up-regulation of the reactive oxygen species (ROS)–generating nicotinamide adenine dinucleotide phosphate (NADPH) oxidases and the phosphorylation of p47phox. Furthermore, challenge with LPS induced greater sensitivity to LPS-induced lethal shock in Prx II–deficient mice than in wild-type mice. Intravenous injection of Prx II–deficient mice with the adenovirus-encoding Prx II gene significantly rescued mice from LPS-induced lethal shock as compared with the injection of a control virus. The administration of catalase mimicked the reversal effects of Prx II on LPS-induced inflammatory responses in Prx II–deficient cells, which suggests that intracellular H2O2 is attributable, at least in part, to the enhanced sensitivity to LPS. These results indicate that Prx II is an essential negative regulator of LPS-induced inflammatory signaling through modulation of ROS synthesis via NADPH oxidase activities and, therefore, is crucial for the prevention of excessive host responses to microbial products

Modifications of T-Scores by Quantitative Ultrasonography for the Diagnosis of Osteoporosis in Koreans

To identify a proper T-score threshold for the diagnosis of osteoporosis in Koreans using quantitative ultrasonography (QUS), normative data from 240 females and 238 males (ages 20-29 yr) were newly generated. Then, the osteoporosis prevalence estimate for men and women over 50 yr of age was analyzed using previous World Health Organization (WHO) methods and heel QUS. T-scores were calculated from the normative data. There were definite negative correlations between age and all of the QUS parameters, such as speed of sound (SOS), broadband ultrasound attenuation (BUA), and estimated heel bone mineral density (BMD) (p<0.0001). After applying the recently determined prevalence of incident vertebral fracture in Koreans over 50 yr of age (11.6% and 9.1%, female vs male, respectively) to the diagnosis of osteoporosis by T-scores from heel BMD as measured by QUS, it was revealed that applicable T-scores for women and men were -2.25 and -1.85, respectively. These data suggest that simply using a T-score of -2.5, the classical WHO threshold for osteoporosis, underestimates the true prevalence when using peripheral QUS. Further prospective study of the power of QUS in predicting the absolute risk of fracture is needed

Clinical outcomes of balloon-occluded retrograde transvenous obliteration for the treatment of gastric variceal hemorrhage in Korean patients with liver cirrhosis: a retrospective multicenter study

Background/AimsThis study evaluated the clinical outcomes of balloon-occluded retrograde transvenous obliteration (BRTO) for the treatment of hemorrhage from gastric varices (GV) in Korean patients with liver cirrhosis (LC).MethodsWe retrospectively analyzed data from 183 LC patients who underwent BRTO for GV bleeding in 6 university-based hospitals between January 2001 and December 2010.ResultsOf the 183 enrolled patients, 49 patients had Child-Pugh (CP) class A LC, 105 had CP class B, and 30 had CP class C at the time of BRTO. BRTO was successfully performed in 177 patients (96.7%). Procedure-related complications (e.g., pulmonary thromboembolism and renal infarction) occurred in eight patients (4.4%). Among 151 patients who underwent follow-up examinations of GV, 79 patients (52.3%) achieved eradication of GV, and 110 patients (72.8%) exhibited marked shrinkage of the treated GV to grade 0 or I. Meanwhile, new-appearance or aggravation of esophageal varices (EV) occurred in 54 out of 136 patients who underwent follow-up endoscopy (41.2%). During the 36.0±29.2 months (mean±SD) of follow-up, 39 patients rebled (hemorrhage from GV in 7, EV in 18, nonvariceal origin in 4, and unknown in 10 patients). The estimated 3-year rebleeding-free rate was 74.8%, and multivariate analysis showed that CP class C was associated with rebleeding (odds ratio, 2.404; 95% confidence-interval, 1.013-5.704; P=0.047).ConclusionsBRTO can be performed safely and effectively for the treatment of GV bleeding. However, aggravation of EV or bleeding from EV is not uncommon after BRTO; thus, periodic endoscopy to follow-up of EV with or without prophylactic treatment might be necessary in LC patients undergoing BRTO