Bandit Parameter Estimation

Contextual bandit is useful algorithm for the recommendation task in many applications such as NETFLEX, Amazon Echo, etc. Many algorithms are researched and showed a good result in terms of high total reward or low regret. However, when user wants to receive a recommendation in the new task, these algorithms do not use information that learned from before task. We suggest new topic, Bandit Parameter Estimation, to solve that inefficient problem. In the same setting with Contextual bandit, we consider as user’s latent profile. And then we propose some algorithms to estimate as fast as possible. We conducted to experiment to verify algorithms that we proposed in two case by using a synthetic dataset. As a result of experiment, we found that our algorithm estimates parameters faster than other algorithms in Contextual bandit. ⓒ 2017 DGISTopenⅠ. Introduction 1-- 1.1 Overview 1-- 1.2 Background 2-- 1.2.1 Multi-Armed bandit 2-- 1.2.2 K-armed (Linear) Contextual bandit 3-- 1.3 Related work 4-- 1.3.1 algorithm 4-- 1.3.2 UCB 5-- 1.3.3 LinUCB 6-- Ⅱ. Materials 8-- 2.1 Problem setting for Bandit Parameter Estimation 8-- 2.2 The uncertainty ellipsoid of _(*) 9-- 2.2.1 (. ) 10-- 2.2.2 ((Σ_(t))) 11-- 2.2.3 ((Σ_(t)^(-1))) 11-- 2.2.4 Max(Det(Σ_(t))) 12Ⅲ. Method 13-- 3.1 Generating synthetic data 13-- 3.2 The experiment process 13-- Ⅳ. Experimental result 14-- 4.1 The experiment case 1 : Various k, fixed d 14-- 4.2 The experiment case 2 : Various d, fixed k 15-- Ⅴ. Discussion 17-- 5.1 Conclusion 17-- 5.2 Future work 17-- Reference 18-- Summary (Korean) 19최근 많은 애플리케이션에서 사용자 맞춤형 추천을 제공하고 있다. 이때 주로 사용되는 알고리즘은 Contextual Bandit이라는 형태로 이미 많은 연구가 진행되어 좋은 결과를 보여주고 있다. 하지만 이 알고리즘들은 특정 유저에 대해서 하나의 Task에서는 빠르게 사용자에게 맞는 추천을 제공하고 있으나 만약 새로운 Task에 대해 추천을 제공해야 할 때, 이전 Task에서 학습한 정보를 이용하지 못하고 Task 별로 독립적으로 다시 학습해야 하므로 효율적이지 않다. 이러한 점에서 동기를 얻어 Contextual Bandit과 같은 환경에서 최근 사용자의 프로필을 학습하기 위한 Bandit Parameter Estimation이라는 형태의 새로운 문제를 제시하였다. 빠른 학습을 위하여 The uncertainty ellipsoid을 수축하기 위한 몇 가지 알고리즘을 제시하였고 실험을 위해 만든 데이터 셋에서 제시한 알고리즘이 기존의 Contextual bandit 알고리즘보다 빠르게 Parameter Estimation을 수행하는 것을 확인했다. 또한 향후 연구 주제로 본 논문을 통해 확인된 알고리즘을 실제 데이터에 적용하여 알고리즘을 검증하는 것 그리고 학습된 사용자의 프로필을 추가적으로 이용하여 Contextual Bandit에 사용되는 알고리즘을 사용했을 때 프로필을 사용하지 않았을 때 보다 더 빠르게 좋은 추천을 제공하는지 확인하는 연구가 필요하다는 것을 제시하였다.MasterdCollectio

DoPAMINE: Double-sided Masked CNN for Pixel Adaptive Multiplicative Noise Despeckling

We propose DoPAMINE, a new neural network based multiplicative noise despeckling algorithm. Our algorithm is inspired by Neural AIDE (N-AIDE), which is a recently proposed neural adaptive image denoiser. While the original N-AIDE was designed for the additive noise case, we show that the same framework, i.e., adaptively learning a network for pixel-wise affine denoisers by minimizing an unbiased estimate of MSE, can be applied to the multiplicative noise case as well. Moreover, we derive a double-sided masked CNN architecture which can control the variance of the activation values in each layer and converge fast to high denoising performance during supervised training. In the experimental results, we show our DoPAMINE possesses high adaptivity via fine-tuning the network parameters based on the given noisy image and achieves significantly better despeckling results compared to SAR-DRN, a state-of-the-art CNN-based algorithm.Comment: AAAI 2019 Camera Ready Versio

Sy-CON: Symmetric Contrastive Loss for Continual Self-Supervised Representation Learning

We introduce a novel and general loss function, called Symmetric Contrastive (Sy-CON) loss, for effective continual self-supervised learning (CSSL). We first argue that the conventional loss form of continual learning which consists of single task-specific loss (for plasticity) and a regularizer (for stability) may not be ideal for contrastive loss based CSSL that focus on representation learning. Our reasoning is that, in contrastive learning based methods, the task-specific loss would suffer from decreasing diversity of negative samples and the regularizer may hinder learning new distinctive representations. To that end, we propose Sy-CON that consists of two losses (one for plasticity and the other for stability) with symmetric dependence on current and past models' negative sample embeddings. We argue our model can naturally find good trade-off between the plasticity and stability without any explicit hyperparameter tuning. We validate the effectiveness of our approach through extensive experiments, demonstrating that MoCo-based implementation of Sy-CON loss achieves superior performance compared to other state-of-the-art CSSL methods.Comment: Preprin

Knowledge Unlearning for Mitigating Privacy Risks in Language Models

Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pretraining, including information that may violate the privacy of personal lives and identities. Previous work addressing privacy issues for language models has mostly focused on data preprocessing and differential privacy methods, both requiring re-training the underlying LM. We propose knowledge unlearning as an alternative method to reduce privacy risks for LMs post hoc. We show that simply applying the unlikelihood training objective to target token sequences is effective at forgetting them with little to no degradation of general language modeling performances; it sometimes even substantially improves the underlying LM with just a few iterations. We also find that sequential unlearning is better than trying to unlearn all the data at once and that unlearning is highly dependent on which kind of data (domain) is forgotten. By showing comparisons with a previous data preprocessing method known to mitigate privacy risks for LMs, we show that unlearning can give a stronger empirical privacy guarantee in scenarios where the data vulnerable to extraction attacks are known a priori while being orders of magnitude more computationally efficient. We release the code and dataset needed to replicate our results at https://github.com/joeljang/knowledge-unlearning

Identification of the antibacterial action mechanism of diterpenoids through transcriptome profiling

Effective antibacterial substances of Aralia continentalis have anti-biofilm and bactericidal activity to the oral pathogen Streptococcus mutans. In this study, three compounds extracted from A. continentalis were identified as acanthoic acid, continentalic acid, and kaurenoic acid by NMR and were further investigated how these diterpenoids affect the physiology of the S. mutans. When S. mutans was exposed to individual or mixed fraction of diterpenoids, severe growth defects and unique morphology were observed. The proportion of unsaturated fatty acids in the cell membrane was increased compared to that of saturated fatty acids in the presence of diterpenoids. Genome-wide gene expression profiles with RNA-seq were compared to reveal the mode of action of diterpenoids. Streptococcus mutans commonly enhanced the expression of 176 genes in the presence of the individual diterpenoids, whereas the expression of 232 genes was considerably reduced. The diterpenoid treatment modulated the expression of genes or operon(s) involved in cell membrane synthesis, cell division, and carbohydrate metabolism of S. mutans. Collectively, these findings provide novel insights into the antibacterial effect of diterpenoids to control S. mutans infection, which causes human dental caries

Phase Variation of Biofilm Formation in Staphylococcus aureus by IS256 Insertion and Its Impact on the Capacity Adhering to Polyurethane Surface

While ica gene of Staphylococcus epidermidis is known to undergo phase variation by insertion of IS256, the phenomenon in Staphylococcus aureus has not been evaluated. Six biofilm-positive strains were tested for the presence of biofilm-negative phase-variant strains by Congo red agar test. For potential phase-variant strains, pulsed-field gel electrophoresis was done to exclude the possibility of contamination. To investigate the mechanism of the biofilm-negative phase variation, PCR for each ica genes were done. Changes of ica genes detected by PCR were confirmed by southern hybridization, and their nucleotides were analyzed by DNA sequencing. Influence of ica genes and biofilm formation on capacity for adherence to biomedical material was evaluated by comparing the ability of adhering to polyurethane surface among a biofilm-negative phase-variant strain and its parent strain. A biofilm-negative phase-variant S. aureus strain was detected from 6 strains tested. icaC gene of the phase-variant strain was found to be inactivated by insertion of additional gene segment, IS256. The biofilm-negative phase-variant strain showed lower adhering capacity to polyurethane than its parent strain. This study shows that phase variation of ica gene occurs in S. aureus by insertion of IS256 also, and this biofilm-negative phase variation reduces adhering capacity of the bacteria