AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models

There are growing interests in adapting large-scale language models using parameter-efficient fine-tuning methods. However, accelerating the model itself and achieving better inference efficiency through model compression has not been thoroughly explored yet. Model compression could provide the benefits of reducing memory footprints, enabling low-precision computations, and ultimately achieving cost-effective inference. To combine parameter-efficient adaptation and model compression, we propose AlphaTuning consisting of post-training quantization of the pre-trained language model and fine-tuning only some parts of quantized parameters for a target task. Specifically, AlphaTuning works by employing binary-coding quantization, which factorizes the full-precision parameters into binary parameters and a separate set of scaling factors. During the adaptation phase, the binary values are frozen for all tasks, while the scaling factors are fine-tuned for the downstream task. We demonstrate that AlphaTuning, when applied to GPT-2 and OPT, performs competitively with full fine-tuning on a variety of downstream tasks while achieving >10x compression ratio under 4-bit quantization and >1,000x reduction in the number of trainable parameters.Comment: Findings of EMNLP 202

Strain Engineering a 4a×3a4a\times\sqrt{3}a Charge Density Wave Phase in Transition Metal Dichalcogenide 1T-VSe2_2

We report a rectangular charge density wave (CDW) phase in strained 1T-VSe$_2$ thin films synthesized by molecular beam epitaxy on c-sapphire substrates. The observed CDW structure exhibits an unconventional rectangular 4a{\times}{\sqrt{3a}} periodicity, as opposed to the previously reported hexagonal $4a\times4a$ structure in bulk crystals and exfoliated thin layered samples. Tunneling spectroscopy shows a strong modulation of the local density of states of the same $4a\times\sqrt{3}a$ CDW periodicity and an energy gap of $2\Delta_{CDW}=(9.1\pm0.1)$ meV. The CDW energy gap evolves into a full gap at temperatures below 500 mK, indicating a transition to an insulating phase at ultra-low temperatures. First-principles calculations confirm the stability of both $4a\times4a$ and $4a\times\sqrt{3}a$ structures arising from soft modes in the phonon dispersion. The unconventional structure becomes preferred in the presence of strain, in agreement with experimental findings

Physical activity and reduced risk of fracture in thyroid cancer patients after thyroidectomy — a nationwide cohort study

ObjectivesLevothyroxine suppressive therapy following thyroidectomy for thyroid cancer patients is considered as a risk factor for osteoporosis and fragility fractures. We evaluated the association of regular exercise and exercise habit change with fracture risk in adults older than 40 years who underwent thyroidectomy for thyroid cancer.MethodsWe enrolled the patients who underwent thyroidectomy for thyroid cancer older than 40 years between 2010 and 2016 from the Korean National Health Insurance Service data, and they were followed through 2019. Based on the questionnaire of health examination within 2 years before and after surgery, whether regular exercise once a week was evaluated. The reference group for the statistical analysis was the continuing lack of physical activity group that did not exercise before or after surgery. For fractures newly diagnosed during the follow-up period, univariate and multivariate Cox regression analyses were performed for risk evaluation.ResultsWe evaluated 74,774 subjects, of whom 2,924 (3.9%) experienced any fractures during a median follow-up of 4.5 years. Compared with the group consistently lack of physical activity, the group that exercised before and after surgery showed a significant decrease in the risk of any fracture, vertebral fracture, and hip fracture: adjusted hazard ratio 0.848 (95% Confidence Interval 0.771–0.932), 0.703 (0.591–0.836), and 0.405 (0.224–0.732), respectively. For vertebral fracture, a significant reduction in fracture risk was confirmed even in patients who started their regular exercise after surgery: adjusted hazard ratio 0.779 (0.648–0.936). The risk reduction for vertebral fractures upon the initiation of exercise was found to be significant in the high-risk groups of patients: women and total thyroidectomy patients.ConclusionWe suggest that maintaining or starting regular exercise after surgery may help prevent fractures in thyroid cancer patients older than 40 years who have undergone thyroidectomy

2023 Korean Endocrine Society Consensus Guidelines for the Diagnosis and Management of Primary Aldosteronism

Primary aldosteronism (PA) is a common, yet underdiagnosed cause of secondary hypertension. It is characterized by an overproduction of aldosterone, leading to hypertension and/or hypokalemia. Despite affecting between 5.9% and 34% of patients with hypertension, PA is frequently missed due to a lack of clinical awareness and systematic screening, which can result in significant cardiovascular complications. To address this, medical societies have developed clinical practice guidelines to improve the management of hypertension and PA. The Korean Endocrine Society, drawing on a wealth of research, has formulated new guidelines for PA. A task force has been established to prepare PA guidelines, which encompass epidemiology, pathophysiology, clinical presentation, diagnosis, treatment, and follow-up care. The Korean clinical guidelines for PA aim to deliver an evidence-based protocol for PA diagnosis, treatment, and patient monitoring. These guidelines are anticipated to ease the burden of this potentially curable condition

Calcification Patterns in Papillary Thyroid Carcinoma are Associated with Changes in Thyroid Hormones and Coronary Artery Calcification

Recent studies suggested that a lower serum thyroid hormone level is associated with more vascular calcification. However, it has been rarely evaluated whether lower thyroid hormone levels affect the calcification of thyroid cancer and there is a relationship between calcification patterns of papillary thyroid carcinoma (PTC) and coronary artery calcification (CAC). The study was divided into two groups: First, we retrospectively reviewed 182 PTC patients and examined the correlation between PTC calcification patterns and CAC by coronary computed tomography (CT). Second, the correlation between the calcification pattern of PTC and thyroid hormone concentration was investigated (n = 354). The calcification pattern of PTC was evaluated by thyroid ultrasonography and classified into four groups: no-calcification, microcalcification, macrocalcification, and mixed-calcification. In PTC patients with microcalcification and mixed calcification, more CAC was observed and coronary calcium score (CCS) was higher. Lower free T4 and higher thyroid-stimulating hormone (TSH) levels were associated with microcalcification and mixed calcification, not with macrocalcification and no calcification. PTC with microcalcification and mixed calcification showed more aggressive phenotypes like lymph node metastasis and more advanced TNM (tumor, node, and metastasis) stage than those with no calcification and macrocalcification. Calcification patterns of PTC showed close association with thyroid hormone levels and CAC. Further research is needed to determine how these findings are related to cardiovascular risk and disease-specific mortality