You are the only possible oracle: Effective test selection for end users of interactive machine learning systems

How do you test a program when only a single user, with no expertise in software testing, is able to determine if the program is performing correctly? Such programs are common today in the form of machine-learned classifiers. We consider the problem of testing this common kind of machine-generated program when the only oracle is an end user: e.g., only you can determine if your email is properly filed. We present test selection methods that provide very good failure rates even for small test suites, and show that these methods work in both large-scale random experiments using a “gold standard” and in studies with real users. Our methods are inexpensive and largely algorithm-independent. Key to our methods is an exploitation of properties of classifiers that is not possible in traditional software testing. Our results suggest that it is plausible for time-pressured end users to interactively detect failures—even very hard-to-find failures—without wading through a large number of successful (and thus less useful) tests. We additionally show that some methods are able to find the arguably most difficult-to-detect faults of classifiers: cases where machine learning algorithms have high confidence in an incorrect result

Radioiodine therapy (RAI) for Graves' disease (GD) and the effect on ophthalmopathy: a systematic review

An association between radioiodine therapy (RAI) for Graves' disease (GD) and the development or worsening of Graves' ophthalmopathy (GO) is widely quoted but there has been no systematic review of the evidence.We undertook a systematic review of randomized controlled trials (RCTs) to assess whether RAI for GD is associated with increased risk of ophthalmopathy compared with antithyroid drugs (ATDs) or surgery. We also assessed the efficacy of glucocorticoid prophylaxis in the prevention of occurrence or progression of ophthalmopathy, when used with RAI.We identified RCTs regardless of language or publication status by searching six databases and trial registries. Dual, blinded data abstraction and quality assessment were undertaken. Random effects meta-analyses were used to combine the study data. Ten RCTs involving 1136 patients permitted 13 comparisons. Two RCTs compared RAI with ATD. Two RCTs compared RAI with thyroidectomy. Four RCTs compared the use of adjunctive ATD with RAI vs. RAI. Five RCTs examined the use of glucocorticoid prophylaxis with RAI.RAI was associated with an increased risk of ophthalmopathy compared with ATD [relative risk (RR) 4.23; 95% confidence interval (CI): 2.04-8.77] but compared with thyroidectomy, there was no statistically significant increased risk (RR 1.59, 95% CI 0.89-2.81). The risk of severe GO was also increased with RAI compared with ATD (RR 4.35; 95% CI 1.28-14.73). Prednisolone prophylaxis for RAI was highly effective in preventing the progression of GO in patients with pre-existing GO (RR 0.03; 95% CI 0.00-0.24). The use of adjunctive ATD with RAI was not associated with any significant benefit on the course of GO.RAI for GD is associated with a small but definite increased risk of development or worsening of Graves' ophthalmopathy compared with ATDs. Steroid prophylaxis is beneficial for patients with pre-existing GO.</p

Psychotherapy in India

The articles that appeared in Indian Journal of Psychiatry were related to different areas of psychotherapy. Case reports dealt with a wide variety of cases. The review papers focused on the suitability of psychotherapy in the Indian context, different approaches in psychotherapy, psychotherapy training and supervision. Psychotherapy has been viewed very close to faith orientation. There were attempts to identify the indigenous concepts that are applicable to psychotherapy. Empirical studies are low in number. Concerted effort is needed to generate interest in psychotherapy, conduct more research on evidence-based therapies as well as on psychotherapeutic process variables