Affective Music Information Retrieval

Much of the appeal of music lies in its power to convey emotions/moods and to evoke them in listeners. In consequence, the past decade witnessed a growing interest in modeling emotions from musical signals in the music information retrieval (MIR) community. In this article, we present a novel generative approach to music emotion modeling, with a specific focus on the valence-arousal (VA) dimension model of emotion. The presented generative model, called \emph{acoustic emotion Gaussians} (AEG), better accounts for the subjectivity of emotion perception by the use of probability distributions. Specifically, it learns from the emotion annotations of multiple subjects a Gaussian mixture model in the VA space with prior constraints on the corresponding acoustic features of the training music pieces. Such a computational framework is technically sound, capable of learning in an online fashion, and thus applicable to a variety of applications, including user-independent (general) and user-dependent (personalized) emotion recognition and emotion-based music retrieval. We report evaluations of the aforementioned applications of AEG on a larger-scale emotion-annotated corpora, AMG1608, to demonstrate the effectiveness of AEG and to showcase how evaluations are conducted for research on emotion-based MIR. Directions of future work are also discussed.Comment: 40 pages, 18 figures, 5 tables, author versio

An intelligent fuzzy regression approach for affective product design that captures nonlinearity and fuzziness

Affective product design aims at incorporating customers’ affective needs into design variables of a new product so as to optimise customers’ affective satisfaction. Faced with fierce competition in marketplaces, companies try to determine the settings in order to maximise customers’ affective satisfaction with products. To achieve this, a set of customer survey data is required in order to develop a model which relates customers’ affective responses to the design variables of a new product. Customer survey data are usually fuzzy since human feeling is usually fuzzy, and the relationship between customers’ affective responses and design variables is usually nonlinear. However, previous research on modelling the relationship between affective response and design variables has not addressed the development of explicit models involving either nonlinearity or fuzziness. In this paper, an intelligent fuzzy regression approach is proposed to generate models which represent this nonlinear and fuzzy relationship between affective responses and design variables. In order to do this, we extend the existing work on fuzzy regression by first utilising an evolutionary algorithm to construct branches of a tree representing structures of a model where the nonlinearity of the model can be addressed. The fuzzy regression algorithm is then used to determine the fuzzy coefficients of the model. The models thus developed are explicit, and consist of fuzzy, nonlinear terms which relate affective responses to design variables. A case study of affective product design of mobile phones is used to illustrate the proposed method

Emotional Design: An Overview

Emotional design has been well recognized in the domain of human factors and ergonomics. In this chapter, we reviewed related models and methods of emotional design. We are motivated to encourage emotional designers to take multiple perspectives when examining these models and methods. Then we proposed a systematic process for emotional design, including affective-cognitive needs elicitation, affective-cognitive needs analysis, and affective-cognitive needs fulfillment to support emotional design. Within each step, we provided an updated review of the representative methods to support and offer further guidance on emotional design. We hope researchers and industrial practitioners can take a systematic approach to consider each step in the framework with care. Finally, the speculations on the challenges and future directions can potentially help researchers across different fields to further advance emotional design.http://deepblue.lib.umich.edu/bitstream/2027.42/163319/1/Emotional_Design_Manuscript_Final.pdfSEL

A guided search genetic algorithm using mined rules for optimal affective product design

Affective design is an important aspect of new product development, especially for consumer products, to achieve a competitive edge in the marketplace. It can help companies to develop new products that can better satisfy the emotional needs of customers. However, product designers usually encounter difficulties in determining the optimal settings of the design attributes for affective design. In this article, a novel guided search genetic algorithm (GA) approach is proposed to determine the optimal design attribute settings for affective design. The optimization model formulated based on the proposed approach applied constraints and guided search operators, which were formulated based on mined rules, to guide the GA search and to achieve desirable solutions. A case study on the affective design of mobile phones was conducted to illustrate the proposed approach and validate its effectiveness. Validation tests were conducted, and the results show that the guided search GA approach outperforms the GA approach without the guided search strategy in terms of GA convergence and computational time. In addition, the guided search optimization model is capable of improving GA to generate good solutions for affective design

Understanding farmers’ uptake of organic farming: An application of the theory of planned behaviour

Whilst the adoption of agricultural techniques has received considerable attention in the literature, the ability and willingness of potential adopters to change their current farming system is often overlooked. This paper is concerned with the intention of conventional farmers to convert to organic farming by using the social-psychology theory of planned behaviour. Drivers and barriers of conversion to organic farming are identified by applying a belief based concept, which is confirmed using principal component analysis. In addition, accounting for heterogeneity regarding farmers‟ environmental attitudes masks considerable differences, notably at intention, attitudes and control perceptions. Overall, results reveal that conversion is indeed affected by attitudes of the farmer, perceived social pressure and ability to convert.Organic farming, Theory of planned behaviour, Principal component analysis, Heterogeneity, Production Economics,

Time- and value-continuous explainable affect estimation in-the-wild

Today, the relevance of Affective Computing, i.e., of making computers recognise and simulate human emotions, cannot be overstated. All technology giants (from manufacturers of laptops to mobile phones to smart speakers) are in a fierce competition to make their devices understand not only what is being said, but also how it is being said to recognise user’s emotions. The goals have evolved from predicting the basic emotions (e.g., happy, sad) to now the more nuanced affective states (e.g., relaxed, bored) real-time. The databases used in such research too have evolved, from earlier featuring the acted behaviours to now spontaneous behaviours. There is a more powerful shift lately, called in-the-wild affect recognition, i.e., taking the research out of the laboratory, into the uncontrolled real-world. This thesis discusses, for the very first time, affect recognition for two unique in-the-wild audiovisual databases, GRAS2 and SEWA. The GRAS2 is the only database till date with time- and value-continuous affect annotations for Labov effect-free affective behaviours, i.e., without the participant’s awareness of being recorded (which otherwise is known to affect the naturalness of one’s affective behaviour). The SEWA features participants from six different cultural backgrounds, conversing using a video-calling platform. Thus, SEWA features in-the-wild recordings further corrupted by unpredictable artifacts, such as the network-induced delays, frame-freezing and echoes. The two databases present a unique opportunity to study time- and value-continuous affect estimation that is truly in-the-wild. A novel ‘Evaluator Weighted Estimation’ formulation is proposed to generate a gold standard sequence from several annotations. An illustration is presented demonstrating that the moving bag-of-words (BoW) representation better preserves the temporal context of the features, yet remaining more robust against the outliers compared to other statistical summaries, e.g., moving average. A novel, data-independent randomised codebook is proposed for the BoW representation; especially useful for cross-corpus model generalisation testing when the feature-spaces of the databases differ drastically. Various deep learning models and support vector regressors are used to predict affect dimensions time- and value-continuously. Better generalisability of the models trained on GRAS2 , despite the smaller training size, makes a strong case for the collection and use of Labov effect-free data. A further foundational contribution is the discovery of the missing many-to-many mapping between the mean square error (MSE) and the concordance correlation coefficient (CCC), i.e., between two of the most popular utility functions till date. The newly invented cost function |MSE_{XY}/σ_{XY}| has been evaluated in the experiments aimed at demystifying the inner workings of a well-performing, simple, low-cost neural network effectively utilising the BoW text features. Also proposed herein is the shallowest-possible convolutional neural network (CNN) that uses the facial action unit (FAU) features. The CNN exploits sequential context, but unlike RNNs, also inherently allows data- and process-parallelism. Interestingly, for the most part, these white-box AI models have shown to utilise the provided features consistent with the human perception of emotion expression