Early processing of consonance and dissonance in human auditory cortex

Pitch is the perceptual correlate of sound's periodicity and a fundamental property of the auditory sensation. The interaction of two or more pitches gives rise to a sensation that can be characterized by its degree of consonance or dissonance. In the current study, we investigated the neuromagnetic representations of consonant and dissonant musical dyads using a new model of cortical activity, in an effort to assess the possible involvement of pitch-specific neural mechanisms in consonance processing at early cortical stages. In the first step of the study, we developed a novel model of cortical pitch processing designed to explain the morphology of the pitch onset response (POR), a pitch-specific subcomponent of the auditory evoked N100 component in the human auditory cortex. The model explains the neural mechanisms underlying the generation of the POR and quantitatively accounts for the relation between its peak latency and the perceived pitch. Next, we applied magnetoencephalography (MEG) to record the POR as elicited by six consonant and dissonant dyads. The peak latency of the POR was strongly modulated by the degree of consonance within the stimuli; specifically, the most dissonant dyad exhibited a POR with a latency that was about 30ms longer than that of the most consonant dyad, an effect that greatly exceeds the expected latency difference induced by a single pitch sound. Our model was able to predict the POR latency pattern observed in the neuromagnetic data, and to generalize this prediction to additional dyads. These results indicate that the neural mechanisms responsible for pitch processing exhibit an intrinsic differential response to concurrent consonant and dissonant pitch combinations, suggesting that the perception of consonance and dissonance might be an emergent property of the pitch processing system in human auditory cortex

WAIC, but Why? Generative Ensembles for Robust Anomaly Detection

Machine learning models encounter Out-of-Distribution (OoD) errors when the data seen at test time are generated from a different stochastic generator than the one used to generate the training data. One proposal to scale OoD detection to high-dimensional data is to learn a tractable likelihood approximation of the training distribution, and use it to reject unlikely inputs. However, likelihood models on natural data are themselves susceptible to OoD errors, and even assign large likelihoods to samples from other datasets. To mitigate this problem, we propose Generative Ensembles, which robustify density-based OoD detection by way of estimating epistemic uncertainty of the likelihood model. We present a puzzling observation in need of an explanation -- although likelihood measures cannot account for the typical set of a distribution, and therefore should not be suitable on their own for OoD detection, WAIC performs surprisingly well in practice

Joint learning of interpretation and distillation

The extra trust brought by the model interpretation has made it an indispensable part of machine learning systems. But to explain a distilled model's prediction, one may either work with the student model itself, or turn to its teacher model. This leads to a more fundamental question: if a distilled model should give a similar prediction for a similar reason as its teacher model on the same input? This question becomes even more crucial when the two models have dramatically different structure, taking GBDT2NN for example. This paper conducts an empirical study on the new approach to explaining each prediction of GBDT2NN, and how imitating the explanation can further improve the distillation process as an auxiliary learning task. Experiments on several benchmarks show that the proposed methods achieve better performance on both explanations and predictions

Onset of coherent attitude layers in a population of sports fans

The aim of this paper was to empirically investigate the behavior of fans, globally coupled to a common environmental source of information. The environmental stimuli were given in a form of referee's decisions list. The sample of fans had to respond on each stimulus by associating points signifying his/her own opinion, emotion and action that referee's decisions provoke. Data were fitted by the Brillouin function which was a solution of an adapted model of quantum statistical physics to social phenomena. Correlation and a principal component analysis were performed in order to detect any collective behavior of the social ensemble of fans. Results showed that fans behaved as a system subject to a phase transition where the neutral state in the opinion, emotional and action space has been destabilized and a new stable state of coherent attitudes was formed. The enhancement of fluctuations and the increase of social susceptibility (responsiveness) to referee's decisions were connected to the first few decisions. The subsequent reduction of values in these parameters signified the onset of coherent layering within the attitude space of the social ensemble of fans. In the space of opinions fan coherence was maximal as only one layer of coherence emerged. In the emotional and action spaces the number of coherent levels was 2 and 4 respectively. The principal component analysis revealed a strong collective behavior and a high degree of integration within and between the opinion, emotional and action spaces of the sample of fans. These results point to one possible way of how different proto-groups, violent and moderate, may be formed as a consequence of global coupling to a common source of information.Comment: Paper 4 for the Complex Systems in Sports Workshop 2011 (CS-Sports 2011) Adaptation and Self-Organizing System

Improving Adversarial Robustness via Promoting Ensemble Diversity

Though deep neural networks have achieved significant progress on various tasks, often enhanced by model ensemble, existing high-performance models can be vulnerable to adversarial attacks. Many efforts have been devoted to enhancing the robustness of individual networks and then constructing a straightforward ensemble, e.g., by directly averaging the outputs, which ignores the interaction among networks. This paper presents a new method that explores the interaction among individual networks to improve robustness for ensemble models. Technically, we define a new notion of ensemble diversity in the adversarial setting as the diversity among non-maximal predictions of individual members, and present an adaptive diversity promoting (ADP) regularizer to encourage the diversity, which leads to globally better robustness for the ensemble by making adversarial examples difficult to transfer among individual members. Our method is computationally efficient and compatible with the defense methods acting on individual networks. Empirical results on various datasets verify that our method can improve adversarial robustness while maintaining state-of-the-art accuracy on normal examples.Comment: ICML 201

Bridging Knowledge Gaps in Neural Entailment via Symbolic Models

Most textual entailment models focus on lexical gaps between the premise text and the hypothesis, but rarely on knowledge gaps. We focus on filling these knowledge gaps in the Science Entailment task, by leveraging an external structured knowledge base (KB) of science facts. Our new architecture combines standard neural entailment models with a knowledge lookup module. To facilitate this lookup, we propose a fact-level decomposition of the hypothesis, and verifying the resulting sub-facts against both the textual premise and the structured KB. Our model, NSnet, learns to aggregate predictions from these heterogeneous data formats. On the SciTail dataset, NSnet outperforms a simpler combination of the two predictions by 3% and the base entailment model by 5%.Comment: EMNLP 201

Countering Adversarial Images using Input Transformations

This paper investigates strategies that defend against adversarial-example attacks on image-classification systems by transforming the inputs before feeding them to the system. Specifically, we study applying image transformations such as bit-depth reduction, JPEG compression, total variance minimization, and image quilting before feeding the image to a convolutional network classifier. Our experiments on ImageNet show that total variance minimization and image quilting are very effective defenses in practice, in particular, when the network is trained on transformed images. The strength of those defenses lies in their non-differentiable nature and their inherent randomness, which makes it difficult for an adversary to circumvent the defenses. Our best defense eliminates 60% of strong gray-box and 90% of strong black-box attacks by a variety of major attack methodsComment: 12 pages, 6 figures, submitted to ICLR 201

Segmentation of Skin Lesions and their Attributes Using Multi-Scale Convolutional Neural Networks and Domain Specific Augmentations

Computer-aided diagnosis systems for classification of different type of skin lesions have been an active field of research in recent decades. It has been shown that introducing lesions and their attributes masks into lesion classification pipeline can greatly improve the performance. In this paper, we propose a framework by incorporating transfer learning for segmenting lesions and their attributes based on the convolutional neural networks. The proposed framework is based on the encoder-decoder architecture which utilizes a variety of pre-trained networks in the encoding path and generates the prediction map by combining multi-scale information in decoding path using a pyramid pooling manner. To address the lack of training data and increase the proposed model generalization, an extensive set of novel domain-specific augmentation routines have been applied to simulate the real variations in dermoscopy images. Finally, by performing broad experiments on three different data sets obtained from International Skin Imaging Collaboration archive (ISIC2016, ISIC2017, and ISIC2018 challenges data sets), we show that the proposed method outperforms other state-of-the-art approaches for ISIC2016 and ISIC2017 segmentation task and achieved the first rank on the leader-board of ISIC2018 attribute detection task.Comment: 18 page

Modeling brand choice using boosted and stacked neural networks

The brand choice problem in marketing has recently been addressed with methods from computational intelligence such as neural networks. Another class of methods from computational intelligence, the so-called ensemble methods such as boosting and stacking have never been applied to the brand choice problem, as far as we know. Ensemble methods generate a number of models for the same problem using any base method and combine the outcomes of these different models. It is well known that in many cases the predictive performance of ensemble methods significantly exceeds the predictive performance of the their base methods. In this report we use boosting and stacking of neural networks and apply this to a scanner dataset that is a benchmark dataset in the marketing literature. Using these methods, we find a significant improvement in predictive performance on this dataset.

Neural translation and automated recognition of ICD10 medical entities from natural language

The recognition of medical entities from natural language is an ubiquitous problem in the medical field, with applications ranging from medical act coding to the analysis of electronic health data for public health. It is however a complex task usually requiring human expert intervention, thus making it expansive and time consuming. The recent advances in artificial intelligence, specifically the raise of deep learning methods, has enabled computers to make efficient decisions on a number of complex problems, with the notable example of neural sequence models and their powerful applications in natural language processing. They however require a considerable amount of data to learn from, which is typically their main limiting factor. However, the C\'epiDc stores an exhaustive database of death certificates at the French national scale, amounting to several millions of natural language examples provided with their associated human coded medical entities available to the machine learning practitioner. This article investigates the applications of deep neural sequence models to the medical entity recognition from natural language problem