Generalised Entropy MDPs and Minimax Regret

Bayesian methods suffer from the problem of how to specify prior beliefs. One interesting idea is to consider worst-case priors. This requires solving a stochastic zero-sum game. In this paper, we extend well-known results from bandit theory in order to discover minimax-Bayes policies and discuss when they are practical.Comment: 7 pages, NIPS workshop "From bad models to good policies

A sports headlight retrofitted on magnifying loupes: A simple and cheap method for daily use

Medical professionals such as doctors, nurses and paramedics often use headlight to examine or to perform surgical intervention in the patients. However, there are concerns related to its use such as comfort for the user, mobility and asepsis for the cable, availability in the departments plus cost effectiveness. The concept of a retrofitted 1-watt sports headlight (adjusted on magnifying loupes) would give quick access to a light source, be available and reliable at any place, save vital funds and would be environmentally friendly as the battery can be replaced. The same concept can be applied to pre-hospital emergency care and disaster medicine as well. BACKGROUND Headlights with fibre optic cables have being used for two decades as an adjunct to the operating theatre lighting. The cable-powered headlights pose, to our experience, some limitations for the operating team: Smooth personnel circulation around the operating field is hindered by repeated unplugging and re-plugging of the cable when surgeon and assistants change sides. Protocols for draping and asepsis have to accommodate the cumbersome cable and the light source and in addition are time consuming and arising issues of flexibility. The weight of the headlight and cable may cause health issues for the bearer (head ache, low back pain) [1]. Portable surgical headlights have also been available for the last decade for a not negligible cost. They are powered by a battery pack, attached to the torso/waist and connected to the headlight by a shorter cable. They are priced at hundreds of pounds. METHOD As an alternative to cumbersome cables and expensive ‘ad hoc’ designs, we use a retrofitted 1-watt sports headlight with a weight of 100 grams. We acquired that for $ 14.99 (approximately £10) from an outdoor specialist retailer (Petzl America, Clearfield, Utah, USA). The headlight is powered by three 1.5 Volt AAA batteries and provides 60 lumen of luminous flux (Fig.1). We have wrapped the elastic bands of the headlight around the corresponding horizontal (axial circumferential) and sagittal elements of the headband, where the magnifying loupes are mounted (Keeler Ltd., Clewer Hill Road, Windsor SL4 4AA). The headlight can be aimed by tilting the housing (Fig.1, 2). DISCUSSION The luminous flux from our headlight according to our experience in cardiothoracic surgery is adequate for a variety of procedures: femoral and axillary arterial access, harvesting internal thoracic (mammary) arteries, open pulmonary resections, valve surgery. Being fully portable without cable, light source or pouches, it is especially handy outside the operating suite (ITU, A&E, wards) for emergency re-explorations for bleeding, secondary wound closures, application of vacuum therapy dressings, trauma, for ECMO work etc. Finally, we have had no evidence of thermal injury, as has being reported from strong xenon beams [2]. This simple affordable headlight system can be easily adapted to the needs of the entire spectrum of surgical specialties, especially those using magnifying loupes. Therefore, can be part of basic life support kits for use in prehospital emergency care, disaster and military medicine [3]. The device has the following advantages: 1. ‘‘Two-in-one’’ function of Loupes and Torch. 2. Battery can be changed (so no need to throw away the item) and is environmentally friendly 3. No need for asepsis 4. Cost effective 5. Availability everywhere In conclusion, we believe this is a practical medical device

The Reinforcement Learning Competition 2014

Reinforcement learning is one of the most general problems in artificial intelligence. It has been used to model problems in automated experiment design, control, economics, game playing, scheduling and telecommunications. The aim of the reinforcement learning competition is to encourage the development of very general learning agents for arbitrary reinforcement learning problems and to provide a test-bed for the unbiased evaluation of algorithms

An intuitive surgical handle design for robotic neurosurgery.

PURPOSE: The expanded endoscopic endonasal approach, a representative example of keyhole brain surgery, allows access to the pituitary gland and surrounding areas through the nasal and sphenoid cavities. Manipulating rigid instruments through these constrained spaces makes this approach technically challenging, and thus, a handheld robotic instrument could expand the surgeon's capabilities. In this study, we present an intuitive handle prototype for such a robotic instrument. METHODS: We have designed and fabricated a surgical instrument handle prototype that maps the surgeon's wrist directly to the robot joints. To alleviate the surgeon's wrist of any excessive strain and fatigue, the tool is mounted on the surgeon's forearm, making it parallel with the instrument's shaft. To evaluate the handle's performance and limitations, we constructed a surgical task simulator and compared our novel handle with a standard neurosurgical tool, with the tasks being performed by a consultant neurosurgeon. RESULTS: While using the proposed handle, the surgeon's average success rate was [Formula: see text], compared to [Formula: see text] when using a conventional tool. Additionally, the surgeon's body posture while using the suggested prototype was deemed acceptable by the Rapid Upper Limb Assessment ergonomic survey, while early results indicate the absence of a learning curve. CONCLUSIONS: Based on these preliminary results, the proposed handle prototype could offer an improvement over current neurosurgical tools and procedural ergonomics. By redirecting forces applied during the procedure to the forearm of the surgeon, and allowing for intuitive surgeon wrist to robot-joints movement mapping without compromising the robotic end effector's expanded workspace, we believe that this handle could prove a substantial step toward improved neurosurgical instrumentation

A spherical joint robotic end-effector for the Expanded Endoscopic Endonasal Approach

The endonasal transsphenoidal approach allows surgeons to access the pituitary gland through the natural orifice of the nose. Recently, surgeons have also described an Expanded Endoscopic Endonasal Approach (EEEA) for the treatment of other tumours around the base of the brain. However, operating in this way with nonarticulated tools is technically very difficult and not widely adopted. The goal of this study is to develop an articulated end-effector for a novel handheld robotic tool for the EEEA. We present a design and implementation of a 3.6mm diameter, three degrees-of-freedom, tendon-driven robotic end-effector that, contrary to rigid instruments which operate under fulcrum, will give the surgeon the ability to reach areas on the surface of the brain that were previously inaccessible. We model the end-effector kinematics in simulation to study the theoretical workspace it can achieve prior to implementing a test-bench device to validate the efficacy of the end-effector. We find promising repeatability of the proposed robotic end-effector of 0.42mm with an effective workspace with limits of ±30∘, which is greater than conventional neurosurgical tools. Additionally, although the tool’s end-effector has a small enough diameter to operate through the narrow nasal access path and the constrained workspace of EEEA, it showcased promising structural integrity and was able to support approximately a 6N load, despite a large deflection angle the limiting of which is scope of future work. These preliminary results indicate the end-effector is a promising first step towards developing appropriate handheld robotic instrumentation to drive EEEA adoption

Fast Reinforcement Learning with Large Action Sets Using Error-Correcting Output Codes for MDP Factorization

International audienceThe use of Reinforcement Learning in real-world scenarios is strongly limited by issues of scale. Most RL learning algorithms are unable to deal with problems composed of hundreds or sometimes even dozens of possible actions, and therefore cannot be applied to many real-world problems. We consider the RL problem in the supervised classification framework where the optimal policy is obtained through a multiclass classifier, the set of classes being the set of actions of the problem. We introduce error-correcting output codes (ECOCs) in this setting and propose two new methods for reducing complexity when using rollouts-based approaches. The first method consists in using an ECOC-based classifier as the multiclass classifier, reducing the learning complexity from O(A2) to O(Alog(A)) . We then propose a novel method that profits from the ECOC's coding dictionary to split the initial MDP into O(log(A)) separate two-action MDPs. This second method reduces learning complexity even further, from O(A2) to O(log(A)) , thus rendering problems with large action sets tractable. We finish by experimentally demonstrating the advantages of our approach on a set of benchmark problems, both in speed and performance

Rollout Sampling Approximate Policy Iteration

Several researchers have recently investigated the connection between reinforcement learning and classification. We are motivated by proposals of approximate policy iteration schemes without value functions which focus on policy representation using classifiers and address policy learning as a supervised learning problem. This paper proposes variants of an improved policy iteration scheme which addresses the core sampling problem in evaluating a policy through simulation as a multi-armed bandit machine. The resulting algorithm offers comparable performance to the previous algorithm achieved, however, with significantly less computational effort. An order of magnitude improvement is demonstrated experimentally in two standard reinforcement learning domains: inverted pendulum and mountain-car.Comment: 18 pages, 2 figures, to appear in Machine Learning 72(3). Presented at EWRL08, to be presented at ECML 200

APLIKASI E-TILANG KENDARAAN BERMOTOR BERBASIS ANDROID DI POLSEK SUKOMORO

Sistem tilang selama ini masih menggunakan media kertas yang berwarna merah dan biru. Sidang yang dilakukan di pengadilan tidak terlepas dari oknum calo dan oknum pengadilan yang meminta pemungutan liar terhadap pelanggaran yang telah dilakukan pelanggar lalu lintas. Pelanggaran dalam berkendara di Nganjuk saat ini masih banyak pengendara yang mengabaikan peraturan lalu lintas, sehingga kasus pelanggaran dan resiko kecelakaan yang tinggi maka harus dilakukan tindakan lebih lanjut. Berbagai jenis teknologi semakin berkembang dan tidak terhitung jumlahkanya akan sangat mudah ditemukan pada era modern saat ini salah satunya Android. Saat ini hampir semua orang memiliki Android yang difungsikan sebagai alat komunikasi. Tilang yang dilakukan kepolisian sangat sering jumpai di jalan raya. Hampir setiap hari di setiap daerah tilang kendaraan selalu dilakukan. Tilang yang yang dilakukan dengan cara manual di masa digital seperti ini kurang efektif dijalankan. Tilang secara manual memerlukan waktu yang banyak untuk mendata ulang serta data tersebut sampai kejaksaan. Oleh karena itu, diperlukan system berbasis Android yang dapat mempermudah apparat kepolisian untuk mendata pelanggaran yang terjadi di jalan raya oleh pengendara kendaraan bermotor khususnya di kota Nganjuk. Pengujian aplikasi e-tilang ini dilakukan dengan menggunakan pengujian black box dan memberikan hasil yang sesuai dengan perancangan system

Electrochemical integration of graphene with light absorbing copper-based thin films

We present an electrochemical route for the integration of graphene with light sensitive copper-based alloys used in optoelectronic applications. Graphene grown using chemical vapor deposition (CVD) transferred to glass is found to be a robust substrate on which photoconductive Cu_{x}S films of 1-2 um thickness can be deposited. The effect of growth parameters on the morphology and photoconductivity of Cu_{x}S films is presented. Current-voltage characterization and photoconductivity decay experiments are performed with graphene as one contact and silver epoxy as the other