Local Metric Learning for Off-Policy Evaluation in Contextual Bandits with Continuous Actions

We consider local kernel metric learning for off-policy evaluation (OPE) of deterministic policies in contextual bandits with continuous action spaces. Our work is motivated by practical scenarios where the target policy needs to be deterministic due to domain requirements, such as prescription of treatment dosage and duration in medicine. Although importance sampling (IS) provides a basic principle for OPE, it is ill-posed for the deterministic target policy with continuous actions. Our main idea is to relax the target policy and pose the problem as kernel-based estimation, where we learn the kernel metric in order to minimize the overall mean squared error (MSE). We present an analytic solution for the optimal metric, based on the analysis of bias and variance. Whereas prior work has been limited to scalar action spaces or kernel bandwidth selection, our work takes a step further being capable of vector action spaces and metric optimization. We show that our estimator is consistent, and significantly reduces the MSE compared to baseline OPE methods through experiments on various domains

Activation of AMP-activated Protein Kinase Is Essential for Lysophosphatidic Acid-induced Cell Migration in Ovarian Cancer Cells

Lysophosphatidic acid (LPA) is a bioactive phospholipid that affects various biological functions, such as cell proliferation, migration, and survival, through LPA receptors. Among them, the motility of cancer cells is an especially important activity for invasion and metastasis. Recently, AMP-activated protein kinase (AMPK), an energy-sensing kinase, was shown to regulate cell migration. However, the specific role of AMPK in cancer cell migration is unknown. The present study investigated whether LPA could induce AMPK activation and whether this process was associated with cell migration in ovarian cancer cells. We found that LPA led to a striking increase in AMPK phosphorylation in pathways involving the phospholipase C-beta 3 (PLC-beta 3) and calcium/calmodulin-dependent protein kinase kinase beta (CaMKK beta) in SKOV3 ovarian cancer cells. siRNA-mediated knockdown of AMPK beta 1, PLC-beta 3, or (CaMKK beta) impaired the stimulatory effects of LPA on cell migration. Furthermore, we found that knockdown of AMPK beta 1 abrogated LPA-induced activation of the small GTPase RhoA and ezrin/radixin/moesin proteins regulating membrane dynamics as membrane-cytoskeleton linkers. In ovarian cancer xenograft models, knockdown of AMPK significantly decreased peritoneal dissemination and lung metastasis. Taken together, our results suggest that activation of AMPK by LPA induces cell migration through the signaling pathway to cytoskeletal dynamics and increases tumor metastasis in ovarian cancer.close161

Dynamic relocalization of NHERF1 mediates chemotactic migration of ovarian cancer cells toward lysophosphatidic acid stimulation

NHERF1/EBP50 (Na+/H+ exchanger regulating factor 1; Ezrin-binding phosphoprotein of 50 kDa) organizes stable protein complexes beneath the apical membrane of polar epithelial cells. By contrast, in cancer cells without any fixed polarity, NHERF1 often localizes in the cytoplasm. The regulation of cytoplasmic NHERF1 and its role in cancer progression remain unclear. In this study, we found that, upon lysophosphatidic acid (LPA) stimulation, cytoplasmic NHERF1 rapidly translocated to the plasma membrane, and subsequently to cortical protrusion structures, of ovarian cancer cells. This movement depended on direct binding of NHERF1 to C-terminally phosphorylated ERM proteins (cpERMs). Moreover, NHERF1 depletion downregulated cpERMs and further impaired cpERM-dependent remodeling of the cell cortex, suggesting reciprocal regulation between these proteins. The LPA-induced protein complex was highly enriched in migratory pseudopodia, whose formation was impaired by overexpression of NHERF1 truncation mutants. Consistent with this, NHERF1 depletion in various types of cancer cells abolished chemotactic cell migration toward a LPA gradient. Taken together, our findings suggest that the high dynamics of cytosolic NHERF1 provide cancer cells with a means of controlling chemotactic migration. This capacity is likely to be essential for ovarian cancer progression in tumor microenvironments containing LPA

Reward Shaping for Model-Based Bayesian Reinforcement Learning

Bayesian reinforcement learning (BRL) provides a formal framework for optimal exploration-exploitation tradeoff in reinforcement learning. Unfortunately, it is generally intractable to find the Bayes-optimal behavior except for restricted cases. As a consequence, many BRL algorithms, model-based approaches in particular, rely on approximated models or real-time search methods. In this paper, we present potential-based shaping for improving the learning performance in model-based BRL. We propose a number of potential functions that are particularly well suited for BRL, and are domain-independent in the sense that they do not require any prior knowledge about the actual environment. By incorporating the potential function into real-time heuristic search, we show that we can significantly improve the learning performance in standard benchmark domains

Default mode network connectivity is associated with long-term clinical outcome in patients with schizophrenia

This study investigated whether resting-state functional connectivity is associated with long-term clinical outcomes of patients with schizophrenia. Resting-state brain images were obtained from 79 outpatients with schizophrenia and 30 healthy controls (HC), using a 3 T-MRI scanner. All patients were 20–50 years old with >3 years' duration of illness and appeared clinically stable. We assessed their psychopathology using the 18-item Brief Psychiatric Rating Scale (BPRS-18) and divided them into “good,” “moderate,” and “poor” outcome (SZ-GO, SZ-MO, and SZ-PO) groups depending on BPRS-18 total score. We obtained individual functional connectivity maps between a seed region of the bilateral posterior cingulate cortex (PCC) and all other brain regions and compared the functional connectivity of the default mode network (DMN) among the HC and 3 schizophrenia outcome groups, with a voxel-wise threshold of P < .001 within a cluster-extent threshold of 114 voxels. Additionally, we assessed correlations between functional connectivity and BPRS-18 scores. The SZ-MO and SZ-PO groups showed decreased functional connectivity between PCC and right ventromedial prefrontal cortex (vmPFC), left middle cingulate cortex, and left frontopolar cortex (FPC) compared to the SZ-GO and HC groups. DMN connectivity in the right vmPFC and left FPC negatively correlated with subscale scores of the BPRS-18, except the negative symptoms subscale. In this study, poorer clinical outcomes in patients with schizophrenia were associated with decreased DMN connectivity. In particular, the decreased functional connectivity might be related to the severity of positive and mood symptoms rather than negative symptoms. Keywords: Schizophrenia, Outcome, Functional connectivity, Default mode network, Resting-stat

Common gray and white matter abnormalities in schizophrenia and bipolar disorder.

This study aimed to investigate abnormalities in the gray matter and white matter (GM and WM, respectively) that are shared between schizophrenia (SZ) and bipolar disorder (BD). We used 3T-magnetic resonance imaging to examine patients with SZ, BD, or healthy control (HC) subjects (aged 20-50 years, N = 65 in each group). We generated modulated GM maps through voxel-based morphometry (VBM) for T1-weighted images and skeletonized fractional anisotropy, mean diffusion, and radial diffusivity maps through tract-based special statistics (TBSS) methods for diffusion tensor imaging (DTI) data. These data were analyzed using a generalized linear model with pairwise comparisons between groups with a family-wise error corrected P < 0.017. The VBM analysis revealed widespread decreases in GM volume in SZ compared to HC, but patients with BD showed GM volume deficits limited to the right thalamus and left insular lobe. The TBSS analysis showed alterations of DTI parameters in widespread WM tracts both in SZ and BD patients compared to HC. The two disorders had WM alterations in the corpus callosum, superior longitudinal fasciculus, internal capsule, external capsule, posterior thalamic radiation, and fornix. However, we observed no differences in GM volume or WM integrity between SZ and BD. The study results suggest that GM volume deficits in the thalamus and insular lobe along with widespread disruptions of WM integrity might be the common neural mechanisms underlying the pathologies of SZ and BD

Prediction of hemorrhagic transformation in acute ischemic stroke: role of diffusion-weighted imaging and early parenchymal enhancement

BACKGROUND AND PURPOSE: MR imaging may help in predicting hemorrhagic transformation (HT) in acute ischemic stroke. Our purpose was to determine whether the lesion volumes on diffusion-weighted (DW) imaging, apparent diffusion coefficient (ADC) values, and early parenchymal enhancement are predictive of HT and to investigate the mechanism of the enhancement. METHODS: We retrospectively examined 55 patients with acute ischemic stroke who underwent gadolinium-enhanced MR imaging within 6 hours of symptom onset and follow-up CT or MR imaging within 72 hours. Intravenous thrombolysis was performed in 15 patients. DW imaging lesion volumes and ADC values were compared between patients with and those without HT. ADCs and perfusion parameters were compared between lesions with and those without parenchymal enhancement. RESULTS: Nineteen (34.5%) patients had HT (14 with hemorrhagic infarction, five with parenchymal hematoma). Patients with HT had decreased mean ADCs and large lesion volumes on DW imaging, but differences were not significant (P > .05). HT occurred in five patients (100%) with parenchymal enhancement, which corresponded to the site of HT. In enhancing lesions, the ADC ratio (0.76 +/- 0.06) was slightly higher and the delay in time to peak (0.10 +/- 2.79) was less than respective values in the rest of the ischemic lesion (0.66 +/- 0.06 and 8.79 +/- 4.86, respectively; P = .068). CONCLUSION: Early parenchymal enhancement is highly specific for HT and may be associated with early reperfusion and damage to the blood-brain barrier in ischemic tissue. DW imaging lesion volumes and ADC values had no strong relationship with HT