The scientific evaluation of music content analysis systems: Valid empirical foundations for future real-world impact

We discuss the problem of music content analysis within the formal framework of experimental design

Machine Learning for Software Engineering: Models, Methods, and Applications

Machine Learning (ML) is the discipline that studies methods for automatically inferring models from data. Machine learning has been successfully applied in many areas of software engineering ranging from behaviour extraction, to testing, to bug fixing. Many more applications are yet be defined. However, a better understanding of ML methods, their assumptions and guarantees would help software engineers adopt and identify the appropriate methods for their desired applications. We argue that this choice can be guided by the models one seeks to infer. In this technical briefing, we review and reflect on the applications of ML for software engineering organised according to the models they produce and the methods they use. We introduce the principles of ML, give an overview of some key methods, and present examples of areas of software engineering benefiting from ML. We also discuss the open challenges for reaching the full potential of ML for software engineering and how ML can benefit from software engineering methods

Replica conditional sequential monte carlo

© 2019 International Machine Learning Society (IMLS). We propose a Markov chain Monte Carlo (MCMC) scheme to perform state inference in non-linear non-Gaussian state-space models. Current state-of-the-art methods to address this problem rely on particle MCMC techniques and its variants, such as the iterated conditional Sequential Monte Carlo (cSMC) scheme, which uses a Sequential Monte Carlo (SMC) type proposal within MCMC. A deficiency of standard SMC proposals is that they only use observations up to time t to propose states at time t when an entire observation sequence is available. More sophisticated SMC based on lookahead techniques could be used but they can be difficult to put in practice. We propose here replica cSMC where we build SMC proposals for one replica using information from the entire observation sequence by conditioning on the states of the other replicas. This approach is easily parallelizable and we demonstrate its excellent empirical performance when compared to the standard iterated cSMC scheme at fixed computational complexity

Validating generic metrics of fairness in game-based resource allocation scenarios with crowdsourced annotations

Being able to effectively measure the notion of fairness is of vital importance as it can provide insight into the formation and evolution of complex patterns and phenomena, such as social preferences, collaboration, group structures and social conflicts. This paper presents a comparative study for quantitatively modelling the notion of fairness in one-to-many resource allocation scenarios - i.e. one provider agent has to allocate resources to multiple receiver agents. For this purpose, we investigate the efficacy of six metrics and cross-validate them on crowdsourced human ranks of fairness annotated through a computer game implementation of the one-to-many resource allocation scenario. Four of the fairness metrics examined are well-established metrics of data dispersion, namely standard deviation, normalised entropy, the Gini coefficient and the fairness index. The fifth metric, proposed by the authors, is an ad-hoc context-based measure which is based on key aspects of distribution strategies. The sixth metric, finally, is machine learned via ranking support vector machines (SVMs) on the crowdsourced human perceptions of fairness. Results suggest that all ad-hoc designed metrics correlate well with the human notion of fairness, and the context-based metrics we propose appear to have a predictability advantage over the other ad-hoc metrics. On the other hand, the normalised entropy and fairness index metrics appear to be the most expressive and generic for measuring fairness for the scenario adopted in this study and beyond. The SVM model can automatically model fairness more accurately than any ad-hoc metric examined (with an accuracy of 81.86%) but it is limited by its expressivity and generalisability.Being able to effectively measure the notion of fairness is of vital importance as it can provide insight into the formation and evolution of complex patterns and phenomena, such as social preferences, collaboration, group structures and social conflicts. This paper presents a comparative study for quantitatively modelling the notion of fairness in one-to-many resource allocation scenarios - i.e. one provider agent has to allocate resources to multiple receiver agents. For this purpose, we investigate the efficacy of six metrics and cross-validate them on crowdsourced human ranks of fairness annotated through a computer game implementation of the one-to-many resource allocation scenario. Four of the fairness metrics examined are well-established metrics of data dispersion, namely standard deviation, normalised entropy, the Gini coefficient and the fairness index. The fifth metric, proposed by the authors, is an ad-hoc context-based measure which is based on key aspects of distribution strategies. The sixth metric, finally, is machine learned via ranking support vector machines (SVMs) on the crowdsourced human perceptions of fairness. Results suggest that all ad-hoc designed metrics correlate well with the human notion of fairness, and the context-based metrics we propose appear to have a predictability advantage over the other ad-hoc metrics. On the other hand, the normalised entropy and fairness index metrics appear to be the most expressive and generic for measuring fairness for the scenario adopted in this study and beyond. The SVM model can automatically model fairness more accurately than any ad-hoc metric examined (with an accuracy of 81.86%) but it is limited by its expressivity and generalisability.peer-reviewe

Using machine learning techniques for sentiment analysis

The Natural language processing is the discipline that studies how to make the machines read and interpret the language that the people use, the natural language. But in the machines world, the words not exist and they are represented by sequences of numbers that the machine represents with a character when displaying them on screen. The Sentiment Analysis is the name of the problem that with a sentence or text the machine gets capable to analyze and predict with the maximum precision possible the sentiment that will be obtained by a person when reads it or the contextual opinion related to something. This document wants to show what we can obtain using the most used machine learning tools.El processament de llenguatge natural és la disciplina que estudia com fer que les màquines aprenguin a llegir i interpretar el llenguatge que usem les persones, el llenguatge natural. Però, en el món de la computació, les paraules no existeixen i són representades per seqüències de números que a l'hora de mostrar-los per pantalla són convertits en lletres. L'anàlisi de sentiments és el nom que obté el problema que donada una sentència o text una computadora sigui capaç d'avaluar-lo i predir amb la màxima precisió possible el sentiment que obtindria una persona en llegir-lo o l'opinió contextual en vers a alguna cosa. Aquest article pretén mostrar el que es pot obtenir, en aquest àmbit, usant les eines d'aprenentatge automàtic més usades.El procesamiento de lenguaje natural es la disciplina que estudia cómo hacer que las máquinas aprendan a leer e interpretar el lenguaje que usamos las personas, el lenguaje natural. Pero, en el mundo de la computación, las palabras no existen y son representadas por secuencias de números que a la hora de mostrarlos por pantalla son convertidos en letras. El análisis de sentimientos es el nombre que obtiene el problema que dada una sentencia o texto una computadora sea capaz de evaluarlo y predecir con la máxima precisión posible el sentimiento que obtendría una persona en leerlo o la opinión contextual hacia algo. Este artículo pretende mostrar lo que se puede obtener, en este ámbito, usando las herramientas de aprendizaje automático más usadas

Detecting cover songs with pitch class key-invariant networks

Deep Learning (DL) has recently been applied successfully to the task of Cover Song Identification (CSI). Meanwhile, neural networks that consider music signal data structure in their design have been developed. In this paper, we propose a Pitch Class Key-Invariant Network, PiCKINet, for CSI. Like some other CSI networks, PiCKINet inputs a Constant-Q Transform (CQT) pitch feature. Unlike other such networks, large multi-octave kernels produce a latent representation with pitch class dimensions that are maintained throughout PiCKINet by key-invariant convolutions. PiCKINet is seen to be more effective, and efficient, than other CQT-based networks. We also propose an extended variant, PiCKINet+, that employs a centre loss penalty, squeeze and excite units, and octave swapping data augmentation. PiCKINet+ shows an improvement of ~17% MAP relative to the well-known CQTNet when tested on a set of ~16K tracks