Logics and practices of transparency and opacity in real-world applications of public sector machine learning

Machine learning systems are increasingly used to support public sector decision-making across a variety of sectors. Given concerns around accountability in these domains, and amidst accusations of intentional or unintentional bias, there have been increased calls for transparency of these technologies. Few, however, have considered how logics and practices concerning transparency have been understood by those involved in the machine learning systems already being piloted and deployed in public bodies today. This short paper distils insights about transparency on the ground from interviews with 27 such actors, largely public servants and relevant contractors, across 5 OECD countries. Considering transparency and opacity in relation to trust and buy-in, better decision-making, and the avoidance of gaming, it seeks to provide useful insights for those hoping to develop socio-technical approaches to transparency that might be useful to practitioners on-the-ground. An extended, archival version of this paper is available as Veale M., Van Kleek M., & Binns R. (2018). `Fairness and accountability design needs for algorithmic support in high-stakes public sector decision-making' Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI'18), http://doi.org/10.1145/3173574.3174014.Comment: 5 pages, 0 figures, presented as a talk at the 2017 Workshop on Fairness, Accountability, and Transparency in Machine Learning (FAT/ML 2017), Halifax, Canada, August 14, 201

Eavesdropping Whilst You're Shopping: Balancing Personalisation and Privacy in Connected Retail Spaces

Physical retailers, who once led the way in tracking with loyalty cards and `reverse appends', now lag behind online competitors. Yet we might be seeing these tables turn, as many increasingly deploy technologies ranging from simple sensors to advanced emotion detection systems, even enabling them to tailor prices and shopping experiences on a per-customer basis. Here, we examine these in-store tracking technologies in the retail context, and evaluate them from both technical and regulatory standpoints. We first introduce the relevant technologies in context, before considering privacy impacts, the current remedies individuals might seek through technology and the law, and those remedies' limitations. To illustrate challenging tensions in this space we consider the feasibility of technical and legal approaches to both a) the recent `Go' store concept from Amazon which requires fine-grained, multi-modal tracking to function as a shop, and b) current challenges in opting in or out of increasingly pervasive passive Wi-Fi tracking. The `Go' store presents significant challenges with its legality in Europe significantly unclear and unilateral, technical measures to avoid biometric tracking likely ineffective. In the case of MAC addresses, we see a difficult-to-reconcile clash between privacy-as-confidentiality and privacy-as-control, and suggest a technical framework which might help balance the two. Significant challenges exist when seeking to balance personalisation with privacy, and researchers must work together, including across the boundaries of preferred privacy definitions, to come up with solutions that draw on both technology and the legal frameworks to provide effective and proportionate protection. Retailers, simultaneously, must ensure that their tracking is not just legal, but worthy of the trust of concerned data subjects.Comment: 10 pages, 1 figure, Proceedings of the PETRAS/IoTUK/IET Living in the Internet of Things Conference, London, United Kingdom, 28-29 March 201

Slave to the Algorithm? Why a \u27Right to an Explanation\u27 Is Probably Not the Remedy You Are Looking For

Algorithms, particularly machine learning (ML) algorithms, are increasingly important to individuals’ lives, but have caused a range of concerns revolving mainly around unfairness, discrimination and opacity. Transparency in the form of a “right to an explanation” has emerged as a compellingly attractive remedy since it intuitively promises to open the algorithmic “black box” to promote challenge, redress, and hopefully heightened accountability. Amidst the general furore over algorithmic bias we describe, any remedy in a storm has looked attractive. However, we argue that a right to an explanation in the EU General Data Protection Regulation (GDPR) is unlikely to present a complete remedy to algorithmic harms, particularly in some of the core “algorithmic war stories” that have shaped recent attitudes in this domain. Firstly, the law is restrictive, unclear, or even paradoxical concerning when any explanation-related right can be triggered. Secondly, even navigating this, the legal conception of explanations as “meaningful information about the logic of processing” may not be provided by the kind of ML “explanations” computer scientists have developed, partially in response. ML explanations are restricted both by the type of explanation sought, the dimensionality of the domain and the type of user seeking an explanation. However, “subject-centric explanations (SCEs) focussing on particular regions of a model around a query show promise for interactive exploration, as do explanation systems based on learning a model from outside rather than taking it apart (pedagogical versus decompositional explanations) in dodging developers\u27 worries of intellectual property or trade secrets disclosure. Based on our analysis, we fear that the search for a “right to an explanation” in the GDPR may be at best distracting, and at worst nurture a new kind of “transparency fallacy.” But all is not lost. We argue that other parts of the GDPR related (i) to the right to erasure ( right to be forgotten ) and the right to data portability; and (ii) to privacy by design, Data Protection Impact Assessments and certification and privacy seals, may have the seeds we can use to make algorithms more responsible, explicable, and human-centered

Mg II flux and profile variability of hybrid-chromosphere stars

Results from an IUE investigation of the variability seen in the Mg II emission line fluxes and wind absorption components of hybrid-chromosphere stars are presented. Real Mg II flux variability is shown with most of the variation being long-term. This variability appears to be due to global changes in the Mg II surface activity. Major changes in the wind absorption component are not seen

Algorithms that Remember: Model Inversion Attacks and Data Protection Law

Many individuals are concerned about the governance of machine learning systems and the prevention of algorithmic harms. The EU's recent General Data Protection Regulation (GDPR) has been seen as a core tool for achieving better governance of this area. While the GDPR does apply to the use of models in some limited situations, most of its provisions relate to the governance of personal data, while models have traditionally been seen as intellectual property. We present recent work from the information security literature around `model inversion' and `membership inference' attacks, which indicate that the process of turning training data into machine learned systems is not one-way, and demonstrate how this could lead some models to be legally classified as personal data. Taking this as a probing experiment, we explore the different rights and obligations this would trigger and their utility, and posit future directions for algorithmic governance and regulation.Comment: 15 pages, 1 figur

Sexual Functioning and Behavior of Men with Body Dysmorphic Disorder Concerning Penis Size Compared with Men Anxious about Penis Size and with Controls: A Cohort Study

Introduction: Little is known about the sexual functioning and behavior of men anxious about the size of their penis and the means that they might use to try to alter the size of their penis. Aim: To compare sexual functioning and behavior in men with body dysmorphic disorder (BDD) concerning penis size and in men with small penis anxiety (SPA without BDD) and in a control group of men who do not have any concerns. Methods: An opportunistic sample of 90 men from the community were recruited and divided into three groups: BDD (n = 26); SPA (n = 31) and controls (n = 33). Main Outcome Measures: The Index of Erectile Function (IEF), sexual identity and history; and interventions to alter the size of their penis. Results: Men with BDD compared with controls had reduced erectile dysfunction, orgasmic function, intercourse satisfaction and overall satisfaction on the IEF. Men with SPA compared with controls had reduced intercourse satisfaction. There were no differences in sexual desire, the frequency of intercourse or masturbation across any of the three groups. Men with BDD and SPA were more likely than the controls to attempt to alter the shape or size of their penis (for example jelqing, vacuum pumps or stretching devices) with poor reported success. Conclusion: Men with BDD are more likely to have erectile dysfunction and less satisfaction with intercourse than controls but maintain their libido. Further research is required to develop and evaluate a psychological intervention for such men with adequate outcome measures

Teaching and Professional Fellowship Report 2007-2008 : Make the Numbers Count, Improving students' learning experiences through an analysis of Library and Learning Resources data

Feedback on Library and Learning Resources (LLR) services and support at the University of the Arts London (UAL) is received from a very extensive range of sources. However, although a considerable quantity of information is available for analysis, it remains very difficult for LLR staff to extract meaningful data from these numerous sources, which can reveal, in depth, the true, individual student experience of LLR services. LLR is aware that there is a lot that is not known, or is not being asked of our students, about their individual experience, which could prove extremely helpful for effective decision-making and service design and delivery. In particular, LLR is interested in identifying and resolving issues of access and support for students currently studying at UAL who are from under represented groups in higher education. Make the numbers counts has enabled the Fellowship Team to move beyond looking at usage figures, gate counts, numbers of issue and renewal transactions, borrower numbers and percentage satisfaction levels, and to extend and add to our knowledge-base of what students at UAL really think and feel about academic libraries. It has enabled us to explore how individual students make use of the services and support provided by LLR. The qualitative data collected through this project has been compared to existing sources of information and to staff experience and understanding of the issues raised, to see if the findings of our research challenge or correlate with other evidence which relates to LLR use and non-use. The focus of the Fellowship has been to explore the particular experience of students who are the first in their family to go to university and who are currently studying at Camberwell, Chelsea and Wimbledon (CCW) and the London College of Communication. Research has shown that this group of students is important amongst the range of students who are now entering higher education through the expansion of participation in university and tertiary-level study. Talking to first generation students was a key element of the project and the greatest source of learning for the Project Team

Comparing Simple Quasar Demographics Models

This paper explores several simple model variations for the connections among quasars, galaxies, and dark matter halos for redshifts 1 < z < 6. A key component of these models is that we enforce a self-consistent black hole (BH) history by tracking both BH mass and BH growth rate at all redshifts. We connect objects across redshift with a simple constant-number-density procedure, and choose a fiducial model with a relationship between BH and galaxy growth rates that is linear and evolves in a simple way with redshift. Within this fiducial model, we find the quasar luminosity function (QLF) by calculating an "intrinsic" luminosity based on either the BH mass or BH growth rate, and then choosing a model of quasar variability with either a lognormal or truncated power-law distribution of instantaneous luminosities. This gives four model variations, which we fit to the observed QLF at each redshift. With the best-fit models in hand, we undertake a detailed comparison of the four fiducial models, and explore changes to our fiducial model of the BH-galaxy relationship. Each model variation can successfully fit the observed QLF, the shape of which is generally set by the "intrinsic" luminosity at the faint end and by the scatter due to variability at the bright end. We focus on accounting for the reasons that physically different models can make such similar predictions, and on identifying what observational data or physical arguments are most essential in breaking the degeneracies among models.Comment: 14 pages, 8 figures, 1 tabl

Swimming capabilities of stoats and the threat to inshore sanctuaries

Stoats (Mustela erminea) are small carnivorous mammals which were introduced into New Zealand in the late 19th century, and have now become widespread invasive pests. Stoats have long been known to be capable of swimming to islands 1-1.5 km offshore. Islands further out have usually been assumed to be safe from invasion, therefore routine stoat monitoring on them has been considered un-necessary. Recent incursions, including a stoat found on Rangitoto Island (3 km offshore) in 2010, and another which was deduced to have reached Kapiti (5 km offshore) in 2009, along with distribution modelling and genetic studies, strongly support the proposition that stoats can swim much further than 1.5 km. Acceptance of this hypothesis depends on estimating the probability that such small animals could indeed swim so far unaided. This paper reports the results of a project designed to assist this debate by recording the paddling action, speed and minimal endurance of nine stoats observed (once each) swimming against an endless current in a flume at the Aquatic Research Centre, University of Waikato. Four of the five males and two of the four females could hold a position for at least five minutes against the maximum current available, averaging 1.36 ± 0.336 km/h. In steady swimming against a current of c. 1 km/hr, they all used a rapid quadripedal paddling action (averaging 250 strokes/min, stronger with the spread forepaws). Four of the nine swam strongly for >1 h, including one female who covered 1.8 km in nearly 2 h non- stop. Results from such artificial conditions cannot be conclusive, but support suggestions that wild stoats could indeed swim much further than 1.5 km, hence we conclude that the “risk zone” for stoat reinvasions of inshore islands has been seriously under-estimated

Children bereaved by suicide: evaluation of a group intervention

1. This report describes a longitudinal evaluation of a therapeutic groupwork intervention for children aged 8-12 years bereaved by the suicide of a parent or relative, offered through the Daughters of Charity Child & Family Service. In 2001, a steering group on suicide prevention was established by the Northern Area Health Board. A primary recommendation of the group was that a specialist service for children bereaved by suicide should be established as an urgent priority. 2. Empirical evidence now strongly indicates that parental suicide is a risk factor for offspring mental health difficulties and even suicide. Research findings suggest that suicide bereaved children were more likely to experience anger and shame and less acceptance of the death, one year after bereavement than non-suicide bereaved children. Age at bereavement by suicide is emerging as a significant predictor of later emotional and behavioural problems as recent research findings show that parental suicide affects children more profoundly than young adults (Wilcox et al, 2010; Sørensen et al., 2009) … 8. Four years on, some of the former group participants have taken leadership roles in their schools by working with teachers and mental health professionals to set up projects or organise talks on suicide, suicide bereavement and suicide prevention. Thus the project has had an unanticipated multiplier effect in reaching children affected by suicide