Online Class-Incremental Continual Learning with Adversarial Shapley Value

As image-based deep learning becomes pervasive on every device, from cell phones to smart watches, there is a growing need to develop methods that continually learn from data while minimizing memory footprint and power consumption. While memory replay techniques have shown exceptional promise for this task of continual learning, the best method for selecting which buffered images to replay is still an open question. In this paper, we specifically focus on the online class-incremental setting where a model needs to learn new classes continually from an online data stream. To this end, we contribute a novel Adversarial Shapley value scoring method that scores memory data samples according to their ability to preserve latent decision boundaries for previously observed classes (to maintain learning stability and avoid forgetting) while interfering with latent decision boundaries of current classes being learned (to encourage plasticity and optimal learning of new class boundaries). Overall, we observe that our proposed ASER method provides competitive or improved performance compared to state-of-the-art replay-based continual learning methods on a variety of datasets.Comment: Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI-21

Conservative Bayesian Model-Based Value Expansion for Offline Policy Optimization

Offline reinforcement learning (RL) addresses the problem of learning a performant policy from a fixed batch of data collected by following some behavior policy. Model-based approaches are particularly appealing in the offline setting since they can extract more learning signals from the logged dataset by learning a model of the environment. However, the performance of existing model-based approaches falls short of model-free counterparts, due to the compounding of estimation errors in the learned model. Driven by this observation, we argue that it is critical for a model-based method to understand when to trust the model and when to rely on model-free estimates, and how to act conservatively w.r.t. both. To this end, we derive an elegant and simple methodology called conservative Bayesian model-based value expansion for offline policy optimization (CBOP), that trades off model-free and model-based estimates during the policy evaluation step according to their epistemic uncertainties, and facilitates conservatism by taking a lower bound on the Bayesian posterior value estimate. On the standard D4RL continuous control tasks, we find that our method significantly outperforms previous model-based approaches: e.g., MOPO by $116.4$%, MOReL by $23.2$% and COMBO by $23.7$%. Further, CBOP achieves state-of-the-art performance on $11$ out of $18$ benchmark datasets while doing on par on the remaining datasets

Growth and atomically resolved polarization mapping of ferroelectric Bi2WO6Bi_2WO_6 thin film

Aurivillius ferroelectric $Bi_2WO_6$ (BWO) encompasses a broad range of functionalities, including robust fatigue-free ferroelectricity, high photocatalytic activity, and ionic conductivity. Despite these promising characteristics, an in-depth study on the growth of BWO thin films and ferroelectric characterization, especially at the atomic scale, is still lacking. Here, we report pulsed laser deposition (PLD) of BWO thin films on (001) $SrTiO_3$ substrates and characterization of ferroelectricity using the scanning transmission electron microscopy (STEM) and piezoresponse force microscopy (PFM) techniques. We show that the background oxygen gas pressure used during PLD growth mainly determines the phase stability of BWO films, whereas the influence of growth temperature is comparatively minor. Atomically resolved STEM study of a fully strained BWO film revealed collective in-plane polar off-centering displacement of W atoms. We estimated the spontaneous polarization value based on polar displacement mapping to be about 54 $\pm$ 4 ${\mu}C cm^{-2}$, which is in good agreement with the bulk polarization value. Furthermore, we found that pristine film is composed of type-I and type-II domains, with mutually orthogonal polar axes. Complementary PFM measurements further elucidated that the coexisting type-I and type-II domains formed a multidomain state that consisted of 90$\deg$ domain walls (DWs) alongside multiple head-to-head and tail-to-tail 180$\deg$ DWs. Application of an electrical bias led to in-plane 180$\deg$ polarization switching and 90$\deg$ polarization rotation, highlighting a unique aspect of domain switching, which is immune to substrate-induced strain.Comment: This document is the Accepted Manuscript version of a Published Work that appeared in final form in ACS Applied Electronic Materials, \copyright American Chemical Society after peer review and technical editing by the publisher. To access the final edited and published work see: https://pubs.acs.org/doi/full/10.1021/acsaelm.1c00005 .This submission contains 34 page

Factual and Personalized Recommendations using Language Models and Reinforcement Learning

Recommender systems (RSs) play a central role in connecting users to content, products, and services, matching candidate items to users based on their preferences. While traditional RSs rely on implicit user feedback signals, conversational RSs interact with users in natural language. In this work, we develop a comPelling, Precise, Personalized, Preference-relevant language model (P4LM) that recommends items to users while putting emphasis on explaining item characteristics and their relevance. P4LM uses the embedding space representation of a user's preferences to generate compelling responses that are factually-grounded and relevant w.r.t. the user's preferences. Moreover, we develop a joint reward function that measures precision, appeal, and personalization, which we use as AI-based feedback in a reinforcement learning-based language model framework. Using the MovieLens 25M dataset, we demonstrate that P4LM delivers compelling, personalized movie narratives to users

Gaussian Quantum Illumination via Monotone Metrics

Quantum illumination is to discern the presence or absence of a low reflectivity target, where the error probability decays exponentially in the number of copies used. When the target reflectivity is small so that it is hard to distinguish target presence or absence, the exponential decay constant falls into a class of objects called monotone metrics. We evaluate monotone metrics restricted to Gaussian states in terms of first-order moments and covariance matrix. Under the assumption of a low reflectivity target, we explicitly derive analytic formulae for decay constant of an arbitrary Gaussian input state. Especially, in the limit of large background noise and low reflectivity, there is no need of symplectic diagonalization which usually complicates the computation of decay constants. First, we show that two-mode squeezed vacuum (TMSV) states are the optimal probe among pure Gaussian states with fixed signal mean photon number. Second, as an alternative to preparing TMSV states with high mean photon number, we show that preparing a TMSV state with low mean photon number and displacing the signal mode is a more experimentally feasible setup without degrading the performance that much. Third, we show that it is of utmost importance to prepare an efficient idler memory to beat coherent states and provide analytic bounds on the idler memory transmittivity in terms of signal power, background noise, and idler memory noise. Finally, we identify the region of physically possible correlations between the signal and idler modes that can beat coherent states.Comment: 16 pages, 6 figure

Combined Analysis of the Time-Resolved Transcriptome and Proteome of Plant Pathogen Xanthomonas oryzae pv. oryzae

Xanthomonas oryzae pv. oryzae (Xoo) is a plant pathogen responsible for causing bacterial blight in rice. The immediate alterations in Xoo upon initial contact with rice are essential for pathogenesis. We studied time-resolved genome-wide gene expression in pathogenicity-activated Xoo cells at the transcriptome and proteome levels. The early response genes of Xoo include genes related to cell motility, inorganic ion transport, and effectors. The alteration of gene expression is initiated as early as few minutes after the initial interaction and changes with time. The time-resolved comparison of the transcriptome and proteome shows the differences between transcriptional and translational expression peaks in many genes, although the overall expression pattern of mRNAs and proteins is conserved. The discrepancy suggests an important role of translational regulation in Xoo at the early stages of pathogenesis. The gene expression analysis using time-resolved transcriptome and proteome provides unprecedented valuable information regarding Xoo pathogenesis