多結晶ニッケル超合金Inconel 713合金の組織形成に及ぼすイットリウム添加の影響

Tohoku University千葉晶彦課

Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantization

Large language models (LLMs) face the challenges in fine-tuning and deployment due to their high memory demands and computational costs. While parameter-efficient fine-tuning (PEFT) methods aim to reduce the memory usage of the optimizer state during fine-tuning, the inherent size of pre-trained LLM weights continues to be a pressing concern. Even though quantization techniques are widely proposed to ease memory demands and accelerate LLM inference, most of these techniques are geared towards the deployment phase. To bridge this gap, this paper presents Parameter-Efficient and Quantization-aware Adaptation (PEQA) - a simple yet effective method that combines the advantages of PEFT with quantized LLMs. By updating solely the quantization scales, PEQA can be directly applied to quantized LLMs, ensuring seamless task transitions. Parallel to existing PEFT methods, PEQA significantly reduces the memory overhead associated with the optimizer state. Furthermore, it leverages the advantages of quantization to substantially reduce model sizes. Even after fine-tuning, the quantization structure of a PEQA-tuned LLM remains intact, allowing for accelerated inference on the deployment stage. We employ PEQA-tuning for task-specific adaptation on LLMs with up to 65 billion parameters. To assess the logical reasoning and language comprehension of PEQA-tuned LLMs, we fine-tune low-bit quantized LLMs using a instruction dataset. Our results show that even when LLMs are quantized to below 4-bit precision, their capabilities in language modeling, few-shot in-context learning, and comprehension can be resiliently restored to (or even improved over) their full-precision original performances with PEQA.Comment: Published at NeurIPS 2023. Camera-ready versio

Building a Private LoRaWAN Platform

LoRaWAN technology has been here for several years as one of LPWAN technologies. It consists of various components such as end nodes, a gateway, a network server, and an application server at the minimum. The servers have been exclusive products of commercial companies, and not many experimental or academic ones are available. Recently one such software has been developed. However, few fully functional academic ones have been reported. In this study, we implement a fully functional private independent LoRaWAN platform for the academic research of LPWAN Internet of Things (IoT) and demonstrate that our platform can support not only end-to-end LoRaWAN communication but also graphical user interface on an embedded and limited computing power system

AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models

There are growing interests in adapting large-scale language models using parameter-efficient fine-tuning methods. However, accelerating the model itself and achieving better inference efficiency through model compression has not been thoroughly explored yet. Model compression could provide the benefits of reducing memory footprints, enabling low-precision computations, and ultimately achieving cost-effective inference. To combine parameter-efficient adaptation and model compression, we propose AlphaTuning consisting of post-training quantization of the pre-trained language model and fine-tuning only some parts of quantized parameters for a target task. Specifically, AlphaTuning works by employing binary-coding quantization, which factorizes the full-precision parameters into binary parameters and a separate set of scaling factors. During the adaptation phase, the binary values are frozen for all tasks, while the scaling factors are fine-tuned for the downstream task. We demonstrate that AlphaTuning, when applied to GPT-2 and OPT, performs competitively with full fine-tuning on a variety of downstream tasks while achieving >10x compression ratio under 4-bit quantization and >1,000x reduction in the number of trainable parameters.Comment: Findings of EMNLP 202

A Case of Acute Ventricular Capture Threshold Rise Associated with Flecainide Acetate

Antiarrhythmic agents may increase capture threshold, but this is rarely of clinical significance. Flecainide acetate, a class IC agent, is reported to have a significant effect on the myocardial capture threshold. In this presentation, we report the case of a 72-year-old male, with a previously implanted VVI pacemaker due to sick sinus syndrome, who was treated with flecainide acetate for paroxysmal atrial arrhythmia control. During the fifteenth day of treatment, an abrupt rise in the ventricular capture threshold with ventricular pacing failure was noted. The capture threshold decreased two days after discontinuation of flecainide acetate

Serum Levels of Advanced Glycation End Products Are Associated with In-Stent Restenosis in Diabetic Patients

The formation of advanced glycation end products (AGEs), in various tissues has been known to enhance immunoinflammatory reactions and local oxidant stresses in long standing diabetes. Recently, AGEs have been reported to play a role in neointimal formation in animal models of arterial injury. We attempted to determine whether the serum levels of AGEs are associated with coronary restenosis in diabetic patients. Blood samples were collected from diabetic patients with coronary artery disease undergoing stent implantation and the serum levels of AGEs were analyzed by the fluorescent intensity method. The development of in-stent restenosis (ISR) was evaluated by a 6-month follow-up coronary angiography. A total of 263 target lesions were evaluated, in 203 patients. The ISR rate in the high-AGE (>170 U/ml) group (40.1%) was significantly higher than in the low-AGE group (≤170 U/ml) (19.6%) (p<0.001). Furthermore, multivariate analysis revealed that a high level of serum AGEs is an independent risk factor for the development of ISR (odds ratio, 2.659; 95% CI, 1.431-4.940; p=0.002). The serum levels of AGEs constitute an excellent predictive factor for ISR, and should be one of the guidelines for medical therapy and interventional strategy to prevent ISR in diabetic patients