Algorithms for Differentially Private Multi-Armed Bandits

We present differentially private algorithms for the stochastic Multi-Armed Bandit (MAB) problem. This is a problem for applications such as adaptive clinical trials, experiment design, and user-targeted advertising where private information is connected to individual rewards. Our major contribution is to show that there exist $(\epsilon, \delta)$ differentially private variants of Upper Confidence Bound algorithms which have optimal regret, $O(\epsilon^{-1} + \log T)$. This is a significant improvement over previous results, which only achieve poly-log regret $O(\epsilon^{-2} \log^{2} T)$, because of our use of a novel interval-based mechanism. We also substantially improve the bounds of previous family of algorithms which use a continual release mechanism. Experiments clearly validate our theoretical bounds

Probabilistic inverse reinforcement learning in unknown environments

We consider the problem of learning by demonstration from agents acting in unknown stochastic Markov environments or games. Our aim is to estimate agent preferences in order to construct improved policies for the same task that the agents are trying to solve. To do so, we extend previous probabilistic approaches for inverse reinforcement learning in known MDPs to the case of unknown dynamics or opponents. We do this by deriving two simplified probabilistic models of the demonstrator's policy and utility. For tractability, we use maximum a posteriori estimation rather than full Bayesian inference. Under a flat prior, this results in a convex optimisation problem. We find that the resulting algorithms are highly competitive against a variety of other methods for inverse reinforcement learning that do have knowledge of the dynamics.Comment: Appears in Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI2013

Generalised Entropy MDPs and Minimax Regret

Bayesian methods suffer from the problem of how to specify prior beliefs. One interesting idea is to consider worst-case priors. This requires solving a stochastic zero-sum game. In this paper, we extend well-known results from bandit theory in order to discover minimax-Bayes policies and discuss when they are practical.Comment: 7 pages, NIPS workshop "From bad models to good policies

Phoneme and sentence-level ensembles for speech recognition

We address the question of whether and how boosting and bagging can be used for speech recognition. In order to do this, we compare two different boosting schemes, one at the phoneme level and one at the utterance level, with a phoneme-level bagging scheme. We control for many parameters and other choices, such as the state inference scheme used. In an unbiased experiment, we clearly show that the gain of boosting methods compared to a single hidden Markov model is in all cases only marginal, while bagging significantly outperforms all other methods. We thus conclude that bagging methods, which have so far been overlooked in favour of boosting, should be examined more closely as a potentially useful ensemble learning technique for speech recognition

Expected loss analysis of thresholded authentication protocols in noisy conditions

A number of authentication protocols have been proposed recently, where at least some part of the authentication is performed during a phase, lasting $n$ rounds, with no error correction. This requires assigning an acceptable threshold for the number of detected errors. This paper describes a framework enabling an expected loss analysis for all the protocols in this family. Furthermore, computationally simple methods to obtain nearly optimal value of the threshold, as well as for the number of rounds is suggested. Finally, a method to adaptively select both the number of rounds and the threshold is proposed.Comment: 17 pages, 2 figures; draf

A sports headlight retrofitted on magnifying loupes: A simple and cheap method for daily use

Medical professionals such as doctors, nurses and paramedics often use headlight to examine or to perform surgical intervention in the patients. However, there are concerns related to its use such as comfort for the user, mobility and asepsis for the cable, availability in the departments plus cost effectiveness. The concept of a retrofitted 1-watt sports headlight (adjusted on magnifying loupes) would give quick access to a light source, be available and reliable at any place, save vital funds and would be environmentally friendly as the battery can be replaced. The same concept can be applied to pre-hospital emergency care and disaster medicine as well. BACKGROUND Headlights with fibre optic cables have being used for two decades as an adjunct to the operating theatre lighting. The cable-powered headlights pose, to our experience, some limitations for the operating team: Smooth personnel circulation around the operating field is hindered by repeated unplugging and re-plugging of the cable when surgeon and assistants change sides. Protocols for draping and asepsis have to accommodate the cumbersome cable and the light source and in addition are time consuming and arising issues of flexibility. The weight of the headlight and cable may cause health issues for the bearer (head ache, low back pain) [1]. Portable surgical headlights have also been available for the last decade for a not negligible cost. They are powered by a battery pack, attached to the torso/waist and connected to the headlight by a shorter cable. They are priced at hundreds of pounds. METHOD As an alternative to cumbersome cables and expensive ‘ad hoc’ designs, we use a retrofitted 1-watt sports headlight with a weight of 100 grams. We acquired that for $ 14.99 (approximately £10) from an outdoor specialist retailer (Petzl America, Clearfield, Utah, USA). The headlight is powered by three 1.5 Volt AAA batteries and provides 60 lumen of luminous flux (Fig.1). We have wrapped the elastic bands of the headlight around the corresponding horizontal (axial circumferential) and sagittal elements of the headband, where the magnifying loupes are mounted (Keeler Ltd., Clewer Hill Road, Windsor SL4 4AA). The headlight can be aimed by tilting the housing (Fig.1, 2). DISCUSSION The luminous flux from our headlight according to our experience in cardiothoracic surgery is adequate for a variety of procedures: femoral and axillary arterial access, harvesting internal thoracic (mammary) arteries, open pulmonary resections, valve surgery. Being fully portable without cable, light source or pouches, it is especially handy outside the operating suite (ITU, A&E, wards) for emergency re-explorations for bleeding, secondary wound closures, application of vacuum therapy dressings, trauma, for ECMO work etc. Finally, we have had no evidence of thermal injury, as has being reported from strong xenon beams [2]. This simple affordable headlight system can be easily adapted to the needs of the entire spectrum of surgical specialties, especially those using magnifying loupes. Therefore, can be part of basic life support kits for use in prehospital emergency care, disaster and military medicine [3]. The device has the following advantages: 1. ‘‘Two-in-one’’ function of Loupes and Torch. 2. Battery can be changed (so no need to throw away the item) and is environmentally friendly 3. No need for asepsis 4. Cost effective 5. Availability everywhere In conclusion, we believe this is a practical medical device