Continually Updating Generative Retrieval on Dynamic Corpora

Generative retrieval has recently been gaining a lot of attention from the research community for its simplicity, high performance, and the ability to fully leverage the power of deep autoregressive models. However, prior work on generative retrieval has mostly investigated on static benchmarks, while realistic retrieval applications often involve dynamic environments where knowledge is temporal and accumulated over time. In this paper, we introduce a new benchmark called STREAMINGIR, dedicated to quantifying the generalizability of retrieval methods to dynamically changing corpora derived from StreamingQA, that simulates realistic retrieval use cases. On this benchmark, we conduct an in-depth comparative evaluation of bi-encoder and generative retrieval in terms of performance as well as efficiency under varying degree of supervision. Our results suggest that generative retrieval shows (1) detrimental performance when only supervised data is used for fine-tuning, (2) superior performance over bi-encoders when only unsupervised data is available, and (3) lower performance to bi-encoders when both unsupervised and supervised data is used due to catastrophic forgetting; nevertheless, we show that parameter-efficient measures can effectively mitigate the issue and result in competitive performance and efficiency with respect to the bi-encoder baseline. Our results open up a new potential for generative retrieval in practical dynamic environments. Our work will be open-sourced.Comment: Work in progres

Knowledge Unlearning for Mitigating Privacy Risks in Language Models

Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pretraining, including information that may violate the privacy of personal lives and identities. Previous work addressing privacy issues for language models has mostly focused on data preprocessing and differential privacy methods, both requiring re-training the underlying LM. We propose knowledge unlearning as an alternative method to reduce privacy risks for LMs post hoc. We show that simply applying the unlikelihood training objective to target token sequences is effective at forgetting them with little to no degradation of general language modeling performances; it sometimes even substantially improves the underlying LM with just a few iterations. We also find that sequential unlearning is better than trying to unlearn all the data at once and that unlearning is highly dependent on which kind of data (domain) is forgotten. By showing comparisons with a previous data preprocessing method known to mitigate privacy risks for LMs, we show that unlearning can give a stronger empirical privacy guarantee in scenarios where the data vulnerable to extraction attacks are known a priori while being orders of magnitude more computationally efficient. We release the code and dataset needed to replicate our results at https://github.com/joeljang/knowledge-unlearning

A hybrid decision support model to discover informative knowledge in diagnosing acute appendicitis

BACKGROUND: The aim of this study is to develop a simple and reliable hybrid decision support model by combining statistical analysis and decision tree algorithms to ensure high accuracy of early diagnosis in patients with suspected acute appendicitis and to identify useful decision rules. METHODS: We enrolled 326 patients who attended an emergency medical center complaining mainly of acute abdominal pain. Statistical analysis approaches were used as a feature selection process in the design of decision support models, including the Chi-square test, Fisher's exact test, the Mann-Whitney U-test (p < 0.01), and Wald forward logistic regression (entry and removal criteria of 0.01 and 0.05, or 0.05 and 0.10, respectively). The final decision support models were constructed using the C5.0 decision tree algorithm of Clementine 12.0 after pre-processing. RESULTS: Of 55 variables, two subsets were found to be indispensable for early diagnostic knowledge discovery in acute appendicitis. The two subsets were as follows: (1) lymphocytes, urine glucose, total bilirubin, total amylase, chloride, red blood cell, neutrophils, eosinophils, white blood cell, complaints, basophils, glucose, monocytes, activated partial thromboplastin time, urine ketone, and direct bilirubin in the univariate analysis-based model; and (2) neutrophils, complaints, total bilirubin, urine glucose, and lipase in the multivariate analysis-based model. The experimental results showed that the model with univariate analysis (80.2%, 82.4%, 78.3%, 76.8%, 83.5%, and 80.3%) outperformed models using multivariate analysis (71.6%, 69.3%, 73.7%, 69.7%, 73.3%, and 71.5% with entry and removal criteria of 0.01 and 0.05; 73.5%, 66.0%, 80.0%, 74.3%, 72.9%, and 73.0% with entry and removal criteria of 0.05 and 0.10) in terms of accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and area under ROC curve, during a 10-fold cross validation. A statistically significant difference was detected in the pairwise comparison of ROC curves (p < 0.01, 95% CI, 3.13-14.5; p < 0.05, 95% CI, 1.54-13.1). The larger induced decision model was more effective for identifying acute appendicitis in patients with acute abdominal pain, whereas the smaller induced decision tree was less accurate with the test data. CONCLUSIONS: The decision model developed in this study can be applied as an aid in the initial decision making of clinicians to increase vigilance in cases of suspected acute appendicitis

Computed Tomographic Image Analysis Based on FEM Performance Comparison of Segmentation on Knee Joint Reconstruction

The demand for an accurate and accessible image segmentation to generate 3D models from CT scan data has been increasing as such models are required in many areas of orthopedics. In this paper, to find the optimal image segmentation to create a 3D model of the knee CT data, we compared and validated segmentation algorithms based on both objective comparisons and finite element (FE) analysis. For comparison purposes, we used 1 model reconstructed in accordance with the instructions of a clinical professional and 3 models reconstructed using image processing algorithms (Sobel operator, Laplacian of Gaussian operator, and Canny edge detection). Comparison was performed by inspecting intermodel morphological deviations with the iterative closest point (ICP) algorithm, and FE analysis was performed to examine the effects of the segmentation algorithm on the results of the knee joint movement analysis

Quantum Artificial Intelligence on Cryptanalysis

With the recent development of quantum computers, various studies on quantum artificial intelligence technology are being conducted. Quantum artificial intelligence can improve performance in terms of accuracy and memory usage compared to deep learning on classical computers. In this work, we proposed an attack technique that recovers keys by learning patterns in cryptographic algorithms by applying quantum artificial intelligence to cryptanalysis. Cryptanalysis was performed in the current practically usable quantum computer environment, and this is the world\u27s first study to the best of our knowledge. As a result, we reduced 70 epochs and reduced the parameters by 19.6%. In addition, higher average BAP (Bit Accuracy Probability) was achieved despite using fewer epochs and parameters. For the same epoch, the method using a quantum neural network achieved a 2.8% higher BAP with fewer parameters. In our approach, quantum advantages in accuracy and memory usage were obtained with quantum neural networks. It is expected that the cryptanalysis proposed in this work will be better utilized if a larger-scale stable quantum computer is developed in the future

Linguistic, visuospatial, and kinematic writing characteristics in cognitively impaired patients with beta-amyloid deposition

IntroductionBeta-amyloid (Aβ) deposition, a hallmark of Alzheimer’s disease (AD), begins before dementia and is an important factor in mild cognitive impairment (MCI). Aβ deposition is a recognized risk factor for various cognitive impairments and has been reported to affect motor performance as well. This study aimed to identify the linguistic, visuospatial, and kinematic characteristics evident in the writing performance of patients with cognitive impairment (CI) who exhibit Aβ deposition.MethodsA total of 31 patients diagnosed with amnestic mild cognitive impairment (aMCI) with Aβ deposition, 26 patients with Alzheimer’s-type dementia, and 33 healthy control (HC) participants without deposition were administered tasks involving dictation of 60 regular words, irregular words, and non-words consisting of 1–4 syllables. Responses from all participants were collected and analyzed through digitized writing tests and analysis tools.ResultsIn terms of linguistic aspects, as cognitive decline progressed, performance in the dictation of irregular words decreased, with errors observed in substituting the target grapheme with other graphemes. The aMCI group frequently exhibited corrective aspects involving letter rewriting during the task. In terms of visuospatial aspects, the AD group displayed more errors in grapheme combination compared to the HC group. Lastly, in the kinematic aspects, both the aMCI group and the AD group exhibited slower writing speeds compared to the HC group.DiscussionThe findings suggest that individuals in the CI group exhibited lower performance in word dictation tasks than those in the HC group, and these results possibly indicate complex cognitive-language-motor deficits resulting from temporal-parietal lobe damage, particularly affecting spelling processing. These results provide valuable clinical insights into understanding linguistic-visuospatial-kinematic aspects that contribute to the early diagnosis of CI with Aβ deposition

MMP9 Processing of HSPB1 Regulates Tumor Progression

Matrix metalloproteinases regulate pathophysiological events by processing matrix proteins and secreted proteins. Previously, we demonstrated that soluble heat shock protein B1 (HSPB1) is released primarily from endothelial cells (ECs) and regulates angiogenesis via direct interaction with vascular endothelial growth factor (VEGF). Here we report that MMP9 can cleave HSPB1 and release anti-angiogenic fragments, which play a key role in tumorprogression. We mapped the cleavage sites and explored their physiological relevance during these processing events. HSPB1 cleavage by MMP9 inhibited VEGF-induced ECs activation and the C-terminal HSPB1 fragment exhibited more interaction with VEGF than did full-length HSPB1. HSPB1 cleavage occurs during B16F10 lung progression in wild-type mice. Also, intact HSPB1 was more detected on tumor endothelium of MMP9 null mice than wild type mice. Finally, we confirmed that secretion of C-terminal HSPB1 fragment was significantly inhibited lung and liver tumor progression of B16F10 melanoma cells and lung tumor progression of CT26 colon carcinoma cells, compared to full-length HSPB1. These data suggest that in vivo MMP9-mediated processing of HSPB1 acts to regulate VEGF-induced ECs activation for tumor progression, releasing anti-angiogenic HSPB1 fragments. Moreover, these findings potentially explain an anti-target effect for the failure of MMP inhibitors in clinical trials, suggesting that MMP inhibitors may have pro-tumorigenic effects by reducing HSPB1 fragmentation

Choice of recipient vessels in muscle-sparing transverse rectus abdominis myocutaneous flap breast reconstruction: A comparative study

Background Thoracodorsal vessels (TDVs) and internal mammary vessels (IMVs) have both been widely employed as recipient vessels for use in free muscle-sparing transverse rectus abdominis myocutaneous (MS-TRAM) flaps. However, whether TDVs or IMVs are preferable as recipient vessels for autologous breast reconstruction with a free MS-TRAM flap remains controversial. The purpose of this study was to compare the clinical outcomes when TDVs were used as recipient vessels to those obtained when IMVs were used as recipient vessels for autologous breast reconstruction with a free MS-TRAM flap. Methods A retrospective matched-cohort study was performed. We retrospectively reviewed data collected from patients who underwent a free MS-TRAM flap for autologous breast reconstructions after mastectomy between March 2003 and June 2013. After a one-to-one matching using age, 100 autologous breast reconstructions were selected in this study. Of the 100 breast reconstructions, 50 flaps were anastomosed to TDVs and 50 to IMVs. Patient demographics and clinical outcomes including operation time, length of hospital stay, postoperative complications, and aesthetic score were compared between the two groups. Results No statistically significant differences were found between the two groups in patient demographics and clinical outcomes, including the complication rates and aesthetic scores. There were no major complications such as total or partial flap loss in either group. Conclusions The results of our study demonstrate that both TDVs and IMVs were safe and efficient as recipient vessels in terms of the complication rates and aesthetic outcomes