Improving the Knowledge Gradient Algorithm

The knowledge gradient (KG) algorithm is a popular policy for the best arm identification (BAI) problem. It is built on the simple idea of always choosing the measurement that yields the greatest expected one-step improvement in the estimate of the best mean of the arms. In this research, we show that this policy has limitations, causing the algorithm not asymptotically optimal. We next provide a remedy for it, by following the manner of one-step look ahead of KG, but instead choosing the measurement that yields the greatest one-step improvement in the probability of selecting the best arm. The new policy is called improved knowledge gradient (iKG). iKG can be shown to be asymptotically optimal. In addition, we show that compared to KG, it is easier to extend iKG to variant problems of BAI, with the $\epsilon$-good arm identification and feasible arm identification as two examples. The superior performances of iKG on these problems are further demonstrated using numerical examples.Comment: 32 pages, 42 figure

Bayesian Active Edge Evaluation on Expensive Graphs

Robots operate in environments with varying implicit structure. For instance, a helicopter flying over terrain encounters a very different arrangement of obstacles than a robotic arm manipulating objects on a cluttered table top. State-of-the-art motion planning systems do not exploit this structure, thereby expending valuable planning effort searching for implausible solutions. We are interested in planning algorithms that actively infer the underlying structure of the valid configuration space during planning in order to find solutions with minimal effort. Consider the problem of evaluating edges on a graph to quickly discover collision-free paths. Evaluating edges is expensive, both for robots with complex geometries like robot arms, and for robots with limited onboard computation like UAVs. Until now, this challenge has been addressed via laziness i.e. deferring edge evaluation until absolutely necessary, with the hope that edges turn out to be valid. However, all edges are not alike in value - some have a lot of potentially good paths flowing through them, and some others encode the likelihood of neighbouring edges being valid. This leads to our key insight - instead of passive laziness, we can actively choose edges that reduce the uncertainty about the validity of paths. We show that this is equivalent to the Bayesian active learning paradigm of decision region determination (DRD). However, the DRD problem is not only combinatorially hard, but also requires explicit enumeration of all possible worlds. We propose a novel framework that combines two DRD algorithms, DIRECT and BISECT, to overcome both issues. We show that our approach outperforms several state-of-the-art algorithms on a spectrum of planning problems for mobile robots, manipulators and autonomous helicopters

Feasibility and oncological safety of axillary reverse mapping in early breast cancer, using premixed serum and indocyanine green dye flourescence technique and an in-house near-infrared fluorescence imaging system and methylene blue dye: ARM study

OBJECTIVES : To determine the metastatic rate and compare the detection rates of arm lymphatics and arm nodes, between serum and indocyanine green (ICG) dye, using an in-house near infrared (NIR) fluorescent imaging system and methylene blue dye, in patients with early breast cancer. METHODS: This IRB approved study included 52 patients with early breast cancer, undergoing ALND, equally allocated into two groups. In one group standardized solution of serum and ICG was injected intradermally posterior to the proximal part of the arm inter-muscular groove and in-house NIR imaging system was used and 2ml of methylene blue was injected at the same site in the other group. The identified ARM node is sent for histo-pathological examination to detect metastasis. RESULTS: After identifying the accurate site of injection, the identification rate of arm lymphatics and arm lymph node using serum and ICG and methylene blue were comparable. Metastatic rate in the arm node was low (5.8%). Thus ARM technique is feasible and safe in patients with early breast cancer. Keywords : Indocyanine green, axillary lymph node dissection(ALND), fluorescent imaging, axillary reverse mapping CONCLUSIONS: 1. Identification rate of arm lymphatics and lateral group of lymph node using indocyanine green dye were 96.2%, which is comparable to the reported identification rate in various studies. 2. There were no complications following the injection of ICG. None of the patients had any pain, swelling or skin tattooing at the site of the injection. 3. Identification rate of lymphatic channel and lateral group of lymph node using methylene blue were 100%, following the identification of the accurate site of injection of the dye in ARM technique. 4. Though the identification rates were high with methylene blue, 57.7% of patients developed local complications following methylene blue, which increased the post-operative morbidity. 5. Metastatic rate in the lateral group of lymph nodes in patients with early breast cancer is only 5.8%.. This low rate of metastasis , early breast cancer, could be treated with adjuvant chemotherapy or radiotherapy, which is already planned as a the course of treatment in these patients. This may allow the lateral group of lymph nodes to be preserved in axillary lymph node dissection, to prevent secondary lymphedema of the ipsilateral arm. 6. Axillary reverse mapping technique is not only feasible but it can also be oncologically safe in patients with early breast cancer

Comparative Analysis of Molecular Clouds in M31, M33 and the Milky Way

We present BIMA observations of a 2\arcmin field in the northeastern spiral arm of M31. In this region we find six giant molecular clouds that have a mean diameter of 57$\pm$13 pc, a mean velocity width of 6.5$\pm$1.2 \kms, and a mean molecular mass of 3.0 $\pm$ 1.6 $\times$ 10$^5$\Msun. The peak brightness temperature of these clouds ranges from 1.6--4.2 K. We compare these clouds to clouds in M33 observed by \citet{wilson90} using the OVRO millimeter array, and some cloud complexes in the Milky Way observed by \cite{dame01} using the CfA 1.2m telescope. In order to properly compare the single dish data to the spatially filtered interferometric data, we project several well-known Milky Way complexes to the distance of Andromeda and simulate their observation with the BIMA interferometer. We compare the simulated Milky Way clouds with the M31 and M33 data using the same cloud identification and analysis technique and find no significant differences in the cloud properties in all three galaxies. Thus we conclude that previous claims of differences in the molecular cloud properties between these galaxies may have been due to differences in the choice of cloud identification techniques. With the upcoming CARMA array, individual molecular clouds may be studied in a variety of nearby galaxies. With ALMA, comprehensive GMC studies will be feasible at least as far as the Virgo cluster. With these data, comparative studies of molecular clouds across galactic disks of all types and between different galaxy disks will be possible. Our results emphasize that interferometric observations combined with the use of a consistent cloud identification and analysis technique will be essential for such forthcoming studies that will compare GMCs in the Local Group galaxies to galaxies in the Virgo cluster.Comment: Accepted for Publication in the Astrophysical Journa

Bayesian Best-Arm Identification for Selecting Influenza Mitigation Strategies

Pandemic influenza has the epidemic potential to kill millions of people. While various preventive measures exist (i.a., vaccination and school closures), deciding on strategies that lead to their most effective and efficient use remains challenging. To this end, individual-based epidemiological models are essential to assist decision makers in determining the best strategy to curb epidemic spread. However, individual-based models are computationally intensive and it is therefore pivotal to identify the optimal strategy using a minimal amount of model evaluations. Additionally, as epidemiological modeling experiments need to be planned, a computational budget needs to be specified a priori. Consequently, we present a new sampling technique to optimize the evaluation of preventive strategies using fixed budget best-arm identification algorithms. We use epidemiological modeling theory to derive knowledge about the reward distribution which we exploit using Bayesian best-arm identification algorithms (i.e., Top-two Thompson sampling and BayesGap). We evaluate these algorithms in a realistic experimental setting and demonstrate that it is possible to identify the optimal strategy using only a limited number of model evaluations, i.e., 2-to-3 times faster compared to the uniform sampling method, the predominant technique used for epidemiological decision making in the literature. Finally, we contribute and evaluate a statistic for Top-two Thompson sampling to inform the decision makers about the confidence of an arm recommendation