74 research outputs found
The Intuitive Appeal of Explainable Machines
Algorithmic decision-making has become synonymous with inexplicable decision-making, but what makes algorithms so difficult to explain? This Article examines what sets machine learning apart from other ways of developing rules for decision-making and the problem these properties pose for explanation. We show that machine learning models can be both inscrutable and nonintuitive and that these are related, but distinct, properties. Calls for explanation have treated these problems as one and the same, but disentangling the two reveals that they demand very different responses. Dealing with inscrutability requires providing a sensible description of the rules; addressing nonintuitiveness requires providing a satisfying explanation for why the rules are what they are. Existing laws like the Fair Credit Reporting Act (FCRA), the Equal Credit Opportunity Act (ECOA), and the General Data Protection Regulation (GDPR), as well as techniques within machine learning, are focused almost entirely on the problem of inscrutability. While such techniques could allow a machine learning system to comply with existing law, doing so may not help if the goal is to assess whether the basis for decision-making is normatively defensible. In most cases, intuition serves as the unacknowledged bridge between a descriptive account and a normative evaluation. But because machine learning is often valued for its ability to uncover statistical relationships that defy intuition, relying on intuition is not a satisfying approach. This Article thus argues for other mechanisms for normative evaluation. To know why the rules are what they are, one must seek explanations of the process behind a model’s development, not just explanations of the model itself
Privacy Dependencies
This Article offers a comprehensive survey of privacy dependencies—the many ways that our privacy depends on the decisions and disclosures of other people. What we do and what we say can reveal as much about others as it does about ourselves, even when we don’t realize it or when we think we’re sharing information about ourselves alone. We identify three bases upon which our privacy can depend: our social ties, our similarities to others, and our differences from others. In a tie-based dependency, an observer learns about one person by virtue of her social relationships with others—family, friends, or other associates. In a similarity-based dependency, inferences about our unrevealed attributes are drawn from our similarities to others for whom that attribute is known. And in difference- based dependencies, revelations about ourselves demonstrate how we are different from others—by showing, for example, how we “break the mold” of normal behavior or establishing how we rank relative to others with respect to some desirable attribute. We elaborate how these dependencies operate, isolating the relevant mechanisms and providing concrete examples of each mechanism in practice, the values they implicate, and the legal and technical interventions that may be brought to bear on them. Our work adds to a growing chorus demonstrating that privacy is neither an individual choice nor an individual value— but it is the first to systematically demonstrate how different types of dependencies can raise very different normative concerns, implicate different areas of law, and create different challenges for regulation
Language (Technology) is Power: A Critical Survey of "Bias" in NLP
We survey 146 papers analyzing "bias" in NLP systems, finding that their
motivations are often vague, inconsistent, and lacking in normative reasoning,
despite the fact that analyzing "bias" is an inherently normative process. We
further find that these papers' proposed quantitative techniques for measuring
or mitigating "bias" are poorly matched to their motivations and do not engage
with the relevant literature outside of NLP. Based on these findings, we
describe the beginnings of a path forward by proposing three recommendations
that should guide work analyzing "bias" in NLP systems. These recommendations
rest on a greater recognition of the relationships between language and social
hierarchies, encouraging researchers and practitioners to articulate their
conceptualizations of "bias"---i.e., what kinds of system behaviors are
harmful, in what ways, to whom, and why, as well as the normative reasoning
underlying these statements---and to center work around the lived experiences
of members of communities affected by NLP systems, while interrogating and
reimagining the power relations between technologists and such communities
A Critical Look at Decentralized Personal Data Architectures
While the Internet was conceived as a decentralized network, the most widely
used web applications today tend toward centralization. Control increasingly
rests with centralized service providers who, as a consequence, have also
amassed unprecedented amounts of data about the behaviors and personalities of
individuals.
Developers, regulators, and consumer advocates have looked to alternative
decentralized architectures as the natural response to threats posed by these
centralized services. The result has been a great variety of solutions that
include personal data stores (PDS), infomediaries, Vendor Relationship
Management (VRM) systems, and federated and distributed social networks. And
yet, for all these efforts, decentralized personal data architectures have seen
little adoption.
This position paper attempts to account for these failures, challenging the
accepted wisdom in the web community on the feasibility and desirability of
these approaches. We start with a historical discussion of the development of
various categories of decentralized personal data architectures. Then we survey
the main ideas to illustrate the common themes among these efforts. We tease
apart the design characteristics of these systems from the social values that
they (are intended to) promote. We use this understanding to point out numerous
drawbacks of the decentralization paradigm, some inherent and others
incidental. We end with recommendations for designers of these systems for
working towards goals that are achievable, but perhaps more limited in scope
and ambition
Racial categories in machine learning
Controversies around race and machine learning have sparked debate among
computer scientists over how to design machine learning systems that guarantee
fairness. These debates rarely engage with how racial identity is embedded in
our social experience, making for sociological and psychological complexity.
This complexity challenges the paradigm of considering fairness to be a formal
property of supervised learning with respect to protected personal attributes.
Racial identity is not simply a personal subjective quality. For people labeled
"Black" it is an ascribed political category that has consequences for social
differentiation embedded in systemic patterns of social inequality achieved
through both social and spatial segregation. In the United States, racial
classification can best be understood as a system of inherently unequal status
categories that places whites as the most privileged category while signifying
the Negro/black category as stigmatized. Social stigma is reinforced through
the unequal distribution of societal rewards and goods along racial lines that
is reinforced by state, corporate, and civic institutions and practices. This
creates a dilemma for society and designers: be blind to racial group
disparities and thereby reify racialized social inequality by no longer
measuring systemic inequality, or be conscious of racial categories in a way
that itself reifies race. We propose a third option. By preceding group
fairness interventions with unsupervised learning to dynamically detect
patterns of segregation, machine learning systems can mitigate the root cause
of social disparities, social segregation and stratification, without further
anchoring status categories of disadvantage
- …