DJ-MC: A Reinforcement-Learning Agent for Music Playlist Recommendation

In recent years, there has been growing focus on the study of automated recommender systems. Music recommendation systems serve as a prominent domain for such works, both from an academic and a commercial perspective. A fundamental aspect of music perception is that music is experienced in temporal context and in sequence. In this work we present DJ-MC, a novel reinforcement-learning framework for music recommendation that does not recommend songs individually but rather song sequences, or playlists, based on a model of preferences for both songs and song transitions. The model is learned online and is uniquely adapted for each listener. To reduce exploration time, DJ-MC exploits user feedback to initialize a model, which it subsequently updates by reinforcement. We evaluate our framework with human participants using both real song and playlist data. Our results indicate that DJ-MC's ability to recommend sequences of songs provides a significant improvement over more straightforward approaches, which do not take transitions into account.Comment: -Updated to the most recent and completed version (to be presented at AAMAS 2015) -Updated author list. in Autonomous Agents and Multiagent Systems (AAMAS) 2015, Istanbul, Turkey, May 201

Biased but Reasonable: Bias Under the Cover of Standard of Care

Inequities in the distribution of healthcare are widely acknowledged to plague the United States healthcare system. Controversies as to whether anti-discrimination law allows individuals to bring lawsuits with respect to implicit rather than intentional bias render negligence law an important avenue for redressing harms caused by implicit bias in medical care. Yet, as this Article argues, the focus of negligence law on medical standards of care to define the boundaries of healthcare providers’ legal duty of care prevents the law from adequately deterring implicit bias and leaves patients harmed by biased treatment decisions without redress for their losses, so long as those decisions fall within the range of medically accepted practices. I term this the problem of “biased-but-reasonable” decision-making.In medical malpractice, the duty of care is set according to standards of real-world practice, which typically recognize more than one course of treatment as acceptable for a given medical condition. Provided that a physician’s choice of treatment for a particular patient falls within the range of those accepted by the professional community, she is perceived as acting reasonably, even if her decision was influenced by implicit bias. In this way, biased-but-reasonable treatment evades the radar of negligence law.After revealing the concept of biased-but-reasonable, the Article examines the normative problems it creates, particularly with respect to deterrence. Negligence law’s failure to assign liability to physicians, whose treatment decisions are influenced by bias but who nonetheless act within the bounds of professional standards, creates a situation in which some patients are less costly to treat—and therefore less costly to harm—than others. As long as the architecture of the negligence doctrine enables biased choices to hide under the veil of reasonable care, health care providers will remain disincentivized to eliminate it.Finally, the Article provides a normative framework that identifies biased treatment choices as negligent, even when they fall within the range of what is considered medically reasonable. It then confronts the evidentiary difficulties that prevent patients, harmed by biased choices of treatment, from establishing their entitlement to damages on a theory of negligence. Specifically, it demonstrates that a key element of such a claim—proof by a preponderance of the evidence that their treatment was chosen based on bias rather than objective medical judgment—places an insurmountable burden on most victims of implicitly biased treatment. The Article argues that the loss of chance doctrine can be harnessed to contend with this evidentiary hurdle and illustrates how the use of this doctrine can both incentivize healthcare providers to eliminate biased judgments and provide redress for victims of biased medical care

Relational Patterns

Information Systems Working Papers Serie

Loser Takes All: Multiple Claimants & Probabilistic Restitution

Consider these two seemingly unrelated recent scandals: The publicized fall from grace of cyclist Lance Armstrong, and the truly ruinous Madoff pyramid scheme. These cases (as well as a plethora of more mundane scenarios discussed throughout this Article) share a common feature, hitherto scantly discussed by courts and legal scholars: causal ambiguity in restitution claims involving multiple claimants. In such cases, a wrongdoer was enriched at the expense of others—sometimes a great many others—and it is therefore difficult to determine exactly which possible victim is indeed the source of the wrongdoer’s enrichment. In such cases, it can be near impossible to preponderantly prove the identity of the claimant at whose expense the wrongdoer was enriched. This Article is the first to identify this problem as a reoccurring pattern in restitutionary claims. By making this novel contribution, the Article fills an important gap in the literature and identifies a new paradigm within the law of restitution, that of causal ambiguity in multiple-claimant cases. This vacuum in the literature on restitutionary claims is especially striking, considering the vast scholarship on a closely related topic, namely causal ambiguity in multiple-defendant tort cases. The Article argues and demonstrates that the existing rules of the law of restitution do not provide appropriate solutions in multiple-claimant cases. Under existing law, many deserving claimants—sometimes all of them—can be left with no remedy, thereby denied of their rights and not compensated for harms they suffered at the hands of a wrongdoer. Drawing on the more developed literature on causal ambiguity in tort law, we propose a solution for this injustice by presenting, for the first time, a new concept of probabilistic restitution. The Article shows that the proposed regime can lead to just and efficient outcomes, serving the goals of both interpersonal justice and deterrence

Decision-centric Active Learning of Binary-Outcome Models

It can be expensive to acquire the data required for businesses to employ data-driven predictive modeling, for example to model consumer preferences to optimize targeting. Prior research has introduced “active learning” policies for identifying data that are particularly useful for model induction, with the goal of decreasing the statistical error for a given acquisition cost (error-centric approaches). However, predictive models are used as part of a decision-making process, and costly improvements in model accuracy do not always result in better decisions. This paper introduces a new approach for active data acquisition that targets decision-making specifically. The new decision-centric approach departs from traditional active learning by placing emphasis on acquisitions that are more likely to affect decision-making. We describe two different types of decision-centric techniques. Next, using direct-marketing data, we compare various data-acquisition techniques. We demonstrate that strategies for reducing statistical error can be wasteful in a decision-making context, and show that one decision-centric technique in particular can improve targeting decisions significantly. We also show that this method is robust in the face of decreasing quality of utility estimations, eventually converging to uniform random sampling, and that it can be extended to situations where different data acquisitions have different costs. The results suggest that businesses should consider modifying their strategies for acquiring information through normal business transactions. For example, a firm such as Amazon.com that models consumer preferences for customized marketing may accelerate learning by proactively offering recommendations—not merely to induce immediate sales, but for improving recommendations in the future.NYU, Stern School of Business, IOMS Department, Center for Digital Economy Researc

Active Sampling for Class Probability Estimation and Ranking

In many cost-sensitive environments class probability estimates are used by decision makers to evaluate the expected utility from a set of alternatives. Supervised learning can be used to build class probability estimates; however, it often is very costly to obtain training data with class labels. Active sampling acquires data incrementally, at each phase identifying especially useful additional data for labeling, and can be used to economize on examples needed for learning. We outline the critical features for an active sampling approach and present an active sampling method for estimating class probabilities and ranking. BOOTSTRAP-LV identifies particularly informative new data for learning based on the variance in probability estimates, and by accounting for a particular data item's informative value for the rest of the input space. We show empirically that the method reduces the number of data items that must be obtained and labeled, across a wide variety of domains. We investigate the contribution of the components of the algorithm and show that each provides valuable information to help identify informative examples. We also compare BOOTSTRAP-LV with UNCERTAINTY SAMPLING,a n existing active sampling method designed to maximize classification accuracy. The results show that BOOTSTRAP-LV uses fewer examples to exhibit a certain class probability estimation accuracy and provide insights on the behavior of the algorithms. Finally, to further our understanding of the contributions made by the elements of BOOTSTRAP-LV, we experiment with a new active sampling algorithm drawing from both UNCERTAINIY SAMPLING and BOOTSTRAP-LV and show that it is significantly more competitive with BOOTSTRAP-LV compared to UNCERTAINTY SAMPLING. The analysis suggests more general implications for improving existing active sampling algorithms for classification.Information Systems Working Papers Serie

Active Learning for Decision Making

This paper addresses focused information acquisition for predictive data mining. As businesses strive to cater to the preferences of individual consumers, they often employ predictive models to customize marketing efforts. Building accurate models requires information about consumer preferences that often is costly to acquire. Prior research has introduced many â active learningâ policies for identifying information that is particularly useful for model induction, the goal being to reduce the acquisition cost necessary to induce a model with a given accuracy. However, predictive models often are used as part of a decision-making process, and costly improvements in model accuracy do not always result in better decisions. This paper develops a new approach for active information acquisition that targets decision-making specifically. The method we introduce departs from the traditional error-reducing paradigm and places emphasis on acquisitions that are more likely to affect decision-making. Empirical evaluations with direct marketing data demonstrate that for a fixed information acquisition cost the method significantly improves the targeting decisions. The method is designed to be genericâ not based on a single model or induction algorithmâ and we show that it can be applied effectively to various predictive modeling techniques.Information Systems Working Papers Serie

Majority is not Needed: A Counterstrategy to Selfish Mining

In the last few years several papers investigated selfish mine attacks, most of which assumed that every miner that is not part of the selfish mine pool will continue to mine honestly. However, in reality, remaining honest is not always incentivized, particularly when another pool is employing selfish mining or other deviant strategies. In this work we explore the scenario in which a large enough pool capitalises on another selfish pool to gain 100\% of the profit and commit double spending attacks. We show that this counterstrategy can effectively counter any deviant strategy, and that even the possibility of it discourages other pools from implementing deviant strategies