Robust Losses for Learning Value Functions

Most value function learning algorithms in reinforcement learning are based on the mean squared (projected) Bellman error. However, squared errors are known to be sensitive to outliers, both skewing the solution of the objective and resulting in high-magnitude and high-variance gradients. To control these high-magnitude updates, typical strategies in RL involve clipping gradients, clipping rewards, rescaling rewards, or clipping errors. While these strategies appear to be related to robust losses -- like the Huber loss -- they are built on semi-gradient update rules which do not minimize a known loss. In this work, we build on recent insights reformulating squared Bellman errors as a saddlepoint optimization problem and propose a saddlepoint reformulation for a Huber Bellman error and Absolute Bellman error. We start from a formalization of robust losses, then derive sound gradient-based approaches to minimize these losses in both the online off-policy prediction and control settings. We characterize the solutions of the robust losses, providing insight into the problem settings where the robust losses define notably better solutions than the mean squared Bellman error. Finally, we show that the resulting gradient-based algorithms are more stable, for both prediction and control, with less sensitivity to meta-parameters.Comment: IEEE Transactions on Pattern Analysis and Machine Intelligence (2022

A Bayesian tool for epilepsy diagnosis in the resource-poor world: Development and early validation

AbstractPurposeThe epilepsy treatment gap in resource-poor countries is so large that existing numbers of doctors are unlikely to be able to close it. Other health workers are likely to be needed but they will need help. The diagnosis of an attack as epileptic or not is an essential step in the management of epilepsy. It should be possible to devise a tool to give the probability of episodes being epileptic based on a Bayesian analysis of the results of history taking.MethodWe asked about the nature of episodes in patients referred to epilepsy camps in Nepal. Answers were then compared to the final clinical diagnosis of epilepsy and the likelihood ratio (LR) of the episode being epileptic obtained for each answer. The most informative LRs, tested sequentially, formed the basis for a tool which was validated in a different Nepalese population.ResultsData was obtained from 67 patients. The pre-test probability of having epilepsy was 0.76. Answers to 11 questions with the most informative LRs were then combined into a tool. This was tested on 14 different patients. Post-test probability scores in those with epilepsy ranged from 0.88 to 1 and for those with non-epilepsy from 0.07 to 0.42.ConclusionIt is possible to devise a tool based on simple clinical information using Bayesian principles. Initial validation suggests that this has the potential to enable health workers to diagnose episodes as epileptic or not. This now needs to be tested in different populations. The tool is easily converted to a mobile phone app

A web-based telemedicine system for low-resource settings 13 years on: insights from referrers and specialists

Background: One way to tackle health inequalities in resource-poor settings is to establish links between doctors and health professionals there and specialists elsewhere using web-based telemedicine. One such system run by the Swinfen Charitable Trust has been in existence for 13 years which is an unusually long time for such systems. Objective: We wanted to gain some insights into whether and how this system might be improved. Methods: We carried out a survey by questionnaire of referrers and specialists over a six months period. Results: During the study period, a total of 111 cases were referred from 35 different practitioners, of whom 24% were not doctors. Survey replies were received concerning 67 cases, a response rate of 61 per cent. Eighty-seven per cent of the responding referrers found the telemedicine advice useful, and 78% were able to follow the advice provided. As a result of the advice received, the diagnosis was changed in 22% of all cases and confirmed in a further 18 per cent. Patient management was changed in 33 per cent. There was no substantial difference between doctors and non-doctors. During the study period, the 111 cases were responded to by 148 specialists, from whom 108 replies to the questionnaire were received, a response rate of 73 per cent. About half of the specialists (47%) felt that their advice had improved the management of the patients. There were 62 cases where it was possible to match up the opinions of the referrer and the consultants about the value of a specific teleconsultation. In 34 cases (55%) the referrers and specialists agreed about the value. However, in 28 cases (45%) they did not: specialists markedly underestimated the value of a consultation compared to referrers. Both referrers and specialist were extremely positive about the system which appears to be working well. Minor changes such as a clearer referral template and an improved web interface for specialists may improve it

Revenue Implications of Nigeria’S Tax System

This is a study of the properties of the Nigeria’s tax system particularly the bases of the company income tax, valueadded tax and personal income tax. The results indicate that their bases are not stable (not persistent and volatile).However, while the bases of the company income tax and personal income tax are more sensitive to cyclical swings(current state of the economy over time), that of the value added tax (VAT) is not. The policy implications of thesefindings support the recent government tax policy reform of a shift in focus in the tax system from direct taxation toindirect taxation. With the tax base of VAT being insensitive to the current state of the economy, the revenuetherefrom will not drop sharply when the economy slows down. It will also shield the government from budgetaryshortfalls as it will likely cushion against sharp declines in aggregate tax revenues.Keywords: Tax System, Company Income Tax, Value Added Tax, Personal Income, Tax Policy, Nigeria

Modelling the Long Run Determinants of Foreign Portfolio Investment in Nigeria

This study tries to ascertain the long run determinants of foreign portfolio investment (FPI) in Nigeria such thatappropriate policies will be pursued to attract same in the long run. FPI has grown recently in proportion relative toother types of capital inflows to Nigeria before the wake of global financial crisis. Incidentally, there is no empiricalregularity regarding the determinants of FPI. This study tries to add to the stock of knowledge by modelling thelong-run determinants of FPI in Nigeria over the period of 1981-2010 converted into quarterly series. The variablesconsidered are, market capitalization, real exchange rate, real interest rate, real gross domestic product and tradeopenness. The study applies time series analysis specifically the finite distributed lag model and discovers that FPIhas a positive long-run relationship with market capitalization, and trade openness in Nigeria. Ongoing effortstherefore to sanitize the capital market should be vigorously pursued.Keywords: Nigeria, Foreign Portfolio Investment, macroeconomic variables

An all-Ireland epidemiological study of MND, 2004-2005

Background and methods: We conducted an all-Ireland population-based prospective epidemiological survey of motor neurone disease (MND) using the Northern Ireland and Republic of Ireland MND registers to examine the incidence and prevalence of the disease over the period 2004–2005.Results and conclusions: Incidence of MND was 1.9 per 100 000 person-years and rates were comparable in both the north and south of Ireland. Prevalence of MND was 5.0 per 100 000 population. When compared with previous published surveys of MND performed in the Republic of Ireland over the last 10 years, rates of disease have remained relatively constant. When standardized to the 1990 US population, the incidence of MND in Ireland was found to be consistent with other European prospective surveys of MND

Bringing UMAP Closer to the Speed of Light with GPU Acceleration

The Uniform Manifold Approximation and Projection (UMAP) algorithm has become widely popular for its ease of use, quality of results, and support for exploratory, unsupervised, supervised, and semi-supervised learning. While many algorithms can be ported to a GPU in a simple and direct fashion, such efforts have resulted in inefficient and inaccurate versions of UMAP. We show a number of techniques that can be used to make a faster and more faithful GPU version of UMAP, and obtain speedups of up to 100x in practice. Many of these design choices/lessons are general purpose and may inform the conversion of other graph and manifold learning algorithms to use GPUs. Our implementation has been made publicly available as part of the open source RAPIDS cuML library (https://github.com/rapidsai/cuml)