Retrospective Loss: Looking Back to Improve Training of Deep Neural Networks

Deep neural networks (DNNs) are powerful learning machines that have enabled breakthroughs in several domains. In this work, we introduce a new retrospective loss to improve the training of deep neural network models by utilizing the prior experience available in past model states during training. Minimizing the retrospective loss, along with the task-specific loss, pushes the parameter state at the current training step towards the optimal parameter state while pulling it away from the parameter state at a previous training step. Although a simple idea, we analyze the method as well as to conduct comprehensive sets of experiments across domains - images, speech, text, and graphs - to show that the proposed loss results in improved performance across input domains, tasks, and architectures.Comment: Accepted at KDD 2020; The first two authors contributed equall

DeepSearch: A Simple and Effective Blackbox Attack for Deep Neural Networks

Although deep neural networks have been very successful in image-classification tasks, they are prone to adversarial attacks. To generate adversarial inputs, there has emerged a wide variety of techniques, such as black- and whitebox attacks for neural networks. In this paper, we present DeepSearch, a novel fuzzing-based, query-efficient, blackbox attack for image classifiers. Despite its simplicity, DeepSearch is shown to be more effective in finding adversarial inputs than state-of-the-art blackbox approaches. DeepSearch is additionally able to generate the most subtle adversarial inputs in comparison to these approaches

Motivational Drivers for Serial Position Effects in High-Stakes Legal Decisions

Experts and employees in many domains make multiple similar but independent decisions in sequence. Often, the serial position of the case in the sequence influences the decision. Explanations for these serial position effects focus on the role of decision makers’ fatigue, but these effects emerge also when fatigue is unlikely. Here, we suggest that serial position effects can emerge due to decision makers’ motivation to be or appear to be consistent. For example, to avoid having inconsistencies revealed, decisions may become more favorable towards the side that is more likely to put a decision under scrutiny. As a context, we focus on the legal domain in which many high-stakes decisions are made in sequence and in which there are clear institutional processes of decision scrutiny. We analyze two field datasets: 386,109 US immigration judges’ decisions on asylum requests and 20,796 jury decisions in 18th century London criminal court. We distinguish between five mechanisms that can drive serial position effects and examine their predictions in these settings. We find that consistent with motivation-based explanations of serial position effects, but inconsistent with fatigue-based explanations, decisions become more lenient as a function of serial position, and the effect persists over breaks. We further find, as is predicted by motivational accounts, that the leniency effect is stronger among more experienced decision makers. By elucidating the different drivers of serial position effects, our investigation clarifies why they are common, when they are expected, and how to reduce them

Cycle-consistent Conditional Adversarial Transfer Networks

Domain adaptation investigates the problem of cross-domain knowledge transfer where the labeled source domain and unlabeled target domain have distinctive data distributions. Recently, adversarial training have been successfully applied to domain adaptation and achieved state-of-the-art performance. However, there is still a fatal weakness existing in current adversarial models which is raised from the equilibrium challenge of adversarial training. Specifically, although most of existing methods are able to confuse the domain discriminator, they cannot guarantee that the source domain and target domain are sufficiently similar. In this paper, we propose a novel approach named cycle-consistent conditional adversarial transfer networks (3CATN) to handle this issue. Our approach takes care of the domain alignment by leveraging adversarial training. Specifically, we condition the adversarial networks with the cross-covariance of learned features and classifier predictions to capture the multimodal structures of data distributions. However, since the classifier predictions are not certainty information, a strong condition with the predictions is risky when the predictions are not accurate. We, therefore, further propose that the truly domain-invariant features should be able to be translated from one domain to the other. To this end, we introduce two feature translation losses and one cycle-consistent loss into the conditional adversarial domain adaptation networks. Extensive experiments on both classical and large-scale datasets verify that our model is able to outperform previous state-of-the-arts with significant improvements