Distill-and-Compare: Auditing Black-Box Models Using Transparent Model Distillation

Black-box risk scoring models permeate our lives, yet are typically proprietary or opaque. We propose Distill-and-Compare, a model distillation and comparison approach to audit such models. To gain insight into black-box models, we treat them as teachers, training transparent student models to mimic the risk scores assigned by black-box models. We compare the student model trained with distillation to a second un-distilled transparent model trained on ground-truth outcomes, and use differences between the two models to gain insight into the black-box model. Our approach can be applied in a realistic setting, without probing the black-box model API. We demonstrate the approach on four public data sets: COMPAS, Stop-and-Frisk, Chicago Police, and Lending Club. We also propose a statistical test to determine if a data set is missing key features used to train the black-box model. Our test finds that the ProPublica data is likely missing key feature(s) used in COMPAS.Comment: Camera-ready version for AAAI/ACM AIES 2018. Data and pseudocode at https://github.com/shftan/auditblackbox. Previously titled "Detecting Bias in Black-Box Models Using Transparent Model Distillation". A short version was presented at NIPS 2017 Symposium on Interpretable Machine Learnin

The use of intellectual capital information by sell-side analysts in company valuation

This paper investigates the role of intellectual capital information (ICI) in sell-side analysts’ fundamental analysis and valuation of companies. Using in-depth semi-structured interviews, it penetrates the black box of analysts’ valuation decision-making by identifying and conceptualising the mechanisms and rationales by which ICI is integrated within their valuation decision processes. We find that capital market participants are not ambivalent to ICI, and ICI is used: (1) to form analysts’ perceptions of the overall quality, strengths and future prospects of companies; (2) in deriving valuation model inputs; (3) in setting price targets and making investment recommendations; and (4) as an important and integral element in analyst–client communications. We show that: there is a ‘pecking order’ of mechanisms for incorporating ICI in valuations, based on quantifiability; IC valuation is grounded in valuation theory; there are designated entry points in the valuation process for ICI; and a number of factors affect analysts’ ICI use in valuation. We also identify a need to redefine ‘value-relevant’ ICI to include non-price-sensitive information; acknowledge the boundedness and contextuality of analysts’ rationality and motives of their ICI use; and the important role of analyst–client meetings for ICI communication

Axiomatic Characterization of Data-Driven Influence Measures for Classification

We study the following problem: given a labeled dataset and a specific datapoint x, how did the i-th feature influence the classification for x? We identify a family of numerical influence measures - functions that, given a datapoint x, assign a numeric value phi_i(x) to every feature i, corresponding to how altering i's value would influence the outcome for x. This family, which we term monotone influence measures (MIM), is uniquely derived from a set of desirable properties, or axioms. The MIM family constitutes a provably sound methodology for measuring feature influence in classification domains; the values generated by MIM are based on the dataset alone, and do not make any queries to the classifier. While this requirement naturally limits the scope of our framework, we demonstrate its effectiveness on data

Model-agnostic auditing: a lost cause?

Tools for interpretable machine learning (IML) or explainable artificial intelligence (xAI) can be used to audit algorithms for fairness or other desiderata. In a black-box setting without access to the algorithm’s internal structure an auditor may be limited to methods that are model-agnostic. These methods have severe limitations with important consequences for outcomes such as fairness. Among model-agnostic IML methods, visualizations such as the partial dependence plot (PDP) or individual conditional expectation (ICE) plots are popular and useful for displaying qualitative relationships. Although we focus on fairness auditing with PDP/ICE plots, the consequences we highlight generalize to other auditing or IML/xAI applications. This paper questions the validity of auditing in high-stakes settings with contested values or conflicting interests if the audit methods are model-agnostic