348 research outputs found
Runaway Feedback Loops in Predictive Policing
Predictive policing systems are increasingly used to determine how to
allocate police across a city in order to best prevent crime. Discovered crime
data (e.g., arrest counts) are used to help update the model, and the process
is repeated. Such systems have been empirically shown to be susceptible to
runaway feedback loops, where police are repeatedly sent back to the same
neighborhoods regardless of the true crime rate.
In response, we develop a mathematical model of predictive policing that
proves why this feedback loop occurs, show empirically that this model exhibits
such problems, and demonstrate how to change the inputs to a predictive
policing system (in a black-box manner) so the runaway feedback loop does not
occur, allowing the true crime rate to be learned. Our results are
quantitative: we can establish a link (in our model) between the degree to
which runaway feedback causes problems and the disparity in crime rates between
areas. Moreover, we can also demonstrate the way in which \emph{reported}
incidents of crime (those reported by residents) and \emph{discovered}
incidents of crime (i.e. those directly observed by police officers dispatched
as a result of the predictive policing algorithm) interact: in brief, while
reported incidents can attenuate the degree of runaway feedback, they cannot
entirely remove it without the interventions we suggest.Comment: Extended version accepted to the 1st Conference on Fairness,
Accountability and Transparency, 2018. Adds further treatment of reported as
well as discovered incident
Beyond Ads: Sequential Decision-Making Algorithms in Law and Public Policy
We explore the promises and challenges of employing sequential
decision-making algorithms - such as bandits, reinforcement learning, and
active learning - in law and public policy. While such algorithms have
well-characterized performance in the private sector (e.g., online
advertising), their potential in law and the public sector remains largely
unexplored, due in part to distinct methodological challenges of the policy
setting. Public law, for instance, can pose multiple objectives, necessitate
batched and delayed feedback, and require systems to learn rational, causal
decision-making policies, each of which presents novel questions at the
research frontier. We highlight several applications of sequential
decision-making algorithms in regulation and governance, and discuss areas for
needed research to render such methods policy-compliant, more widely
applicable, and effective in the public sector. We also note the potential
risks of such deployments and describe how sequential decision systems can also
facilitate the discovery of harms. We hope our work inspires more investigation
of sequential decision making in law and public policy, which provide unique
challenges for machine learning researchers with tremendous potential for
social benefit.Comment: Version 1 presented at Causal Inference Challenges in Sequential
Decision Making: Bridging Theory and Practice, a NeurIPS 2021 Worksho
Steps Towards Value-Aligned Systems
Algorithmic (including AI/ML) decision-making artifacts are an established
and growing part of our decision-making ecosystem. They are indispensable tools
for managing the flood of information needed to make effective decisions in a
complex world. The current literature is full of examples of how individual
artifacts violate societal norms and expectations (e.g. violations of fairness,
privacy, or safety norms). Against this backdrop, this discussion highlights an
under-emphasized perspective in the literature on assessing value misalignment
in AI-equipped sociotechnical systems. The research on value misalignment has a
strong focus on the behavior of individual tech artifacts. This discussion
argues for a more structured systems-level approach for assessing
value-alignment in sociotechnical systems. We rely primarily on the research on
fairness to make our arguments more concrete. And we use the opportunity to
highlight how adopting a system perspective improves our ability to explain and
address value misalignments better. Our discussion ends with an exploration of
priority questions that demand attention if we are to assure the value
alignment of whole systems, not just individual artifacts.Comment: Original version appeared in Proceedings of the 2020 AAAI ACM
Conference on AI, Ethics, and Society (AIES '20), February 7-8, 2020, New
York, NY, USA. 5 pages, 2 figures. Corrected some typos in this versio
A Penalized Likelihood Method for Balancing Accuracy and Fairness in Predictive Policing
Racial bias of predictive policing algorithms has been the focus of recent research and, in the case of Hawkes processes, feedback loops are possible where biased arrests are amplified through self-excitation, leading to hotspot formation and further arrests of minority populations. In this article we develop a penalized likelihood approach for introducing fairness into point process models of crime. In particular, we add a penalty term to the likelihood function that encourages the amount of police patrol received by each of several demographic groups to be proportional to the representation of that group in the total population. We apply our model to historical crime incident data in Indianapolis and measure the fairness and accuracy of the two approaches across several crime categories. We show that fairness can be introduced into point process models of crime so that patrol levels proportionally match demographics, though at a cost of reduced accuracy of the algorithms
Data analytics and algorithms in policing in England and Wales: Towards a new policy framework
RUSI was commissioned by the Centre for Data Ethics and Innovation (CDEI) to conduct an independent study into the use of data analytics by police forces in England and Wales, with a focus on algorithmic bias. The primary purpose of the project is to inform CDEI’s review of bias in algorithmic decision-making, which is focusing on four sectors, including policing, and working towards a draft framework for the ethical development and deployment of data analytics tools for policing.
This paper focuses on advanced algorithms used by the police to derive insights, inform operational decision-making or make predictions. Biometric technology, including live facial recognition, DNA analysis and fingerprint matching, are outside the direct scope of this study, as are covert surveillance capabilities and digital forensics technology, such as mobile phone data extraction and computer forensics. However, because many of the policy issues discussed in this paper stem from general underlying data protection and human rights frameworks, these issues will also be relevant to other police technologies, and their use must be considered in parallel to the tools examined in this paper.
The project involved engaging closely with senior police officers, government officials, academics, legal experts, regulatory and oversight bodies and civil society organisations. Sixty nine participants took part in the research in the form of semi-structured interviews, focus groups and roundtable discussions. The project has revealed widespread concern across the UK law enforcement community regarding the lack of official national guidance for the use of algorithms in policing, with respondents suggesting that this gap should be addressed as a matter of urgency.
Any future policy framework should be principles-based and complement existing police guidance in a ‘tech-agnostic’ way. Rather than establishing prescriptive rules and standards for different data technologies, the framework should establish standardised processes to ensure that data analytics projects follow recommended routes for the empirical evaluation of algorithms within their operational context and evaluate the project against legal requirements and ethical standards. The new guidance should focus on ensuring multi-disciplinary legal, ethical and operational input from the outset of a police technology project; a standard process for model development, testing and evaluation; a clear focus on the human–machine interaction and the ultimate interventions a data driven process may inform; and ongoing tracking and mitigation of discrimination risk
POTs: Protective Optimization Technologies
Algorithmic fairness aims to address the economic, moral, social, and
political impact that digital systems have on populations through solutions
that can be applied by service providers. Fairness frameworks do so, in part,
by mapping these problems to a narrow definition and assuming the service
providers can be trusted to deploy countermeasures. Not surprisingly, these
decisions limit fairness frameworks' ability to capture a variety of harms
caused by systems.
We characterize fairness limitations using concepts from requirements
engineering and from social sciences. We show that the focus on algorithms'
inputs and outputs misses harms that arise from systems interacting with the
world; that the focus on bias and discrimination omits broader harms on
populations and their environments; and that relying on service providers
excludes scenarios where they are not cooperative or intentionally adversarial.
We propose Protective Optimization Technologies (POTs). POTs provide means
for affected parties to address the negative impacts of systems in the
environment, expanding avenues for political contestation. POTs intervene from
outside the system, do not require service providers to cooperate, and can
serve to correct, shift, or expose harms that systems impose on populations and
their environments. We illustrate the potential and limitations of POTs in two
case studies: countering road congestion caused by traffic-beating
applications, and recalibrating credit scoring for loan applicants.Comment: Appears in Conference on Fairness, Accountability, and Transparency
(FAT* 2020). Bogdan Kulynych and Rebekah Overdorf contributed equally to this
work. Version v1/v2 by Seda G\"urses, Rebekah Overdorf, and Ero Balsa was
presented at HotPETS 2018 and at PiMLAI 201
- …