Solving Non-Stationary Bandit Problems by Random Sampling from Sibling Kalman Filters

The multi-armed bandit problem is a classical optimization problem where an agent sequentially pulls one of multiple arms attached to a gambling machine, with each pull resulting in a random reward. The reward distributions are unknown, and thus, one must balance between exploiting existing knowledge about the arms, and obtaining new information. Dynamically changing (non-stationary) bandit problems are particularly challenging because each change of the reward distributions may progressively degrade the performance of any fixed strategy. Although computationally intractable in many cases, Bayesian methods provide a standard for optimal decision making. This paper proposes a novel solution scheme for bandit problems with non-stationary normally distributed rewards. The scheme is inherently Bayesian in nature, yet avoids computational intractability by relying simply on updating the hyper parameters of sibling Kalman Filters, and on random sampling from these posteriors. Furthermore, it is able to track the better actions, thus supporting non-stationary bandit problems. Extensive experiments demonstrate that our scheme outperforms recently proposed bandit playing algorithms, not only in non-stationary environments, but in stationary environments also. Furthermore, our scheme is robust to inexact parameter settings. We thus believe that our methodology opens avenues for obtaining improved novel solutions

In vivo imaging of pancreatic tumours and liver metastases using 7 Tesla MRI in a murine orthotopic pancreatic cancer model and a liver metastases model

<p>Abstract</p> <p>Background</p> <p>Pancreatic cancer is the fourth leading cause of tumour death in the western world. However, appropriate tumour models are scarce. Here we present a syngeneic murine pancreatic cancer model using 7 Tesla MRI and evaluate its clinical relevance and applicability.</p> <p>Methods</p> <p>6606PDA murine pancreatic cancer cells were orthotopically injected into the pancreatic head. Liver metastases were induced through splenic injection. Animals were analyzed by MRI three and five weeks following injection. Tumours were detected using T2-weighted high resolution sequences. Tumour volumes were determined by callipers and MRI. Liver metastases were analyzed using gadolinium-EOB-DTPA and T1-weighted 3D-Flash sequences. Tumour blood flow was measured using low molecular gadobutrol and high molecular gadolinium-DTPA.</p> <p>Results</p> <p>MRI handling and applicability was similar to human systems, resolution as low as 0.1 mm. After 5 weeks tumour volumes differed significantly (p < 0.01) when comparing calliper measurments (n = 5, mean 1065 mm³+/-243 mm³) with MRI (mean 918 mm³+/-193 mm³) with MRI being more precise. Histology (n = 5) confirmed MRI tumour measurements (mean size MRI 38.5 mm²+/-22.8 mm²versus 32.6 mm²+/-22.6 mm²(histology), p < 0,0004) with differences due to fixation and processing of specimens. After splenic injection all mice developed liver metastases with a mean of 8 metastases and a mean volume of 173.8 mm³+/-56.7 mm³after 5 weeks. Lymphnodes were also easily identified. Tumour accumulation of gadobutrol was significantly (p < 0.05) higher than gadolinium-DTPA. All imaging experiments could be done repeatedly to comply with the 3R-principle thus reducing the number of experimental animals.</p> <p>Conclusions</p> <p>This model permits monitoring of tumour growth and metastasis formation in longitudinal non-invasive high-resolution MR studies including using contrast agents comparable to human pancreatic cancer. This multidisciplinary environment enables radiologists, surgeons and physicians to further improve translational research and therapies of pancreatic cancer.</p

Electrical and Mechanical Ventricular Activation During Left Bundle Branch Block and Resynchronization

Cardiac resynchronization therapy (CRT) aims to treat selected heart failure patients suffering from conduction abnormalities with left bundle branch block (LBBB) as the culprit disease. LBBB remained largely underinvestigated until it became apparent that the amount of response to CRT was heterogeneous and that the therapy and underlying pathology were thus incompletely understood. In this review, current knowledge concerning activation in LBBB and during biventricular pacing will be explored and applied to current CRT practice, highlighting novel ways to better measure and treat the electrical substrate

Defining the Functional Domain of Programmed Cell Death 10 through Its Interactions with Phosphatidylinositol-3,4,5-Trisphosphate

Cerebral cavernous malformations (CCM) are vascular abnormalities of the central nervous system predisposing blood vessels to leakage, leading to hemorrhagic stroke. Three genes, Krit1 (CCM1), OSM (CCM2), and PDCD10 (CCM3) are involved in CCM development. PDCD10 binds specifically to PtdIns(3,4,5)P3 and OSM. Using threading analysis and multi-template modeling, we constructed a three-dimensional model of PDCD10. PDCD10 appears to be a six-helical-bundle protein formed by two heptad-repeat-hairpin structures (α1–3 and α4–6) sharing the closest 3D homology with the bacterial phosphate transporter, PhoU. We identified a stretch of five lysines forming an amphipathic helix, a potential PtdIns(3,4,5)P3 binding site, in the α5 helix. We generated a recombinant wild-type (WT) and three PDCD10 mutants that have two (Δ2KA), three (Δ3KA), and five (Δ5KA) K to A mutations. Δ2KA and Δ3KA mutants hypothetically lack binding residues to PtdIns(3,4,5)P3 at the beginning and the end of predicted helix, while Δ5KA completely lacks all predicted binding residues. The WT, Δ2KA, and Δ3KA mutants maintain their binding to PtdIns(3,4,5)P3. Only the Δ5KA abolishes binding to PtdIns(3,4,5)P3. Both Δ5KA and WT show similar secondary and tertiary structures; however, Δ5KA does not bind to OSM. When WT and Δ5KA are co-expressed with membrane-bound constitutively-active PI3 kinase (p110-CAAX), the majority of the WT is co-localized with p110-CAAX at the plasma membrane where PtdIns(3,4,5)P3 is presumably abundant. In contrast, the Δ5KA remains in the cytoplasm and is not present in the plasma membrane. Combining computational modeling and biological data, we propose that the CCM protein complex functions in the PI3K signaling pathway through the interaction between PDCD10 and PtdIns(3,4,5)P3