Personalised aesthetics with residual adapters

The use of computational methods to evaluate aesthetics in photography has gained interest in recent years due to the popularization of convolutional neural networks and the availability of new annotated datasets. Most studies in this area have focused on designing models that do not take into account individual preferences for the prediction of the aesthetic value of pictures. We propose a model based on residual learning that is capable of learning subjective, user specific preferences over aesthetics in photography, while surpassing the state-of-the-art methods and keeping a limited number of user-specific parameters in the model. Our model can also be used for picture enhancement, and it is suitable for content-based or hybrid recommender systems in which the amount of computational resources is limited.Comment: 12 pages, 4 figures. In Iberian Conference on Pattern Recognition and Image Analysis proceeding

Estimating Beauty Ratings of Videos using Supervoxels

The major low-level perceptual components that influence the beauty ratings of video are color, contrast, and motion. To estimate the beauty ratings of the NHK dataset, we propose to extract these features based on supervoxels, which are a group of pixels that share similar color and spatial information through the temporal domain. Recent beauty methods use frame-level processing for visual features and disregard the spatio-temporal aspect of beauty. In this paper, we explicitly model this property by introducing supervoxel-based visual and motion features. In order to create a beauty estimator, we first identify 60 videos (either beautiful or not beautiful) in the NHK dataset. We then train a neural network regressor using the supervoxel-based features and binary beauty ratings. We rate the 1000 videos in the NHK dataset and rank them according to their ratings. When comparing our rankings with the actual rankings of the NHK dataset, we obtain a Spearman correlation coefficient of 0.42

Hierarchical aesthetic quality assessment using deep convolutional neural networks

Aesthetic image analysis has attracted much attention in recent years. However, assessing the aesthetic quality and assigning an aesthetic score are challenging problems. In this paper, we propose a novel framework for assessing the aesthetic quality of images. Firstly, we divide the images into three categories: “scene”, “object” and “texture”. Each category has an associated convolutional neural network (CNN) which learns the aesthetic features for the category in question. The object CNN is trained using the whole images and a salient region in each image. The texture CNN is trained using small regions in the original images. Furthermore, an A & C CNN is developed to simultaneously assess the aesthetic quality and identify the category for overall images. For each CNN, classification and regression models are developed separately to predict aesthetic class (high or low) and to assign an aesthetic score. Experimental results on a recently published large-scale dataset show that the proposed method can outperform the state-of-the-art methods for each category

COMPUTATIONAL MODELLING OF HUMAN AESTHETIC PREFERENCES IN THE VISUAL DOMAIN: A BRAIN-INSPIRED APPROACH

Following the rise of neuroaesthetics as a research domain, computational aesthetics has also known a regain in popularity over the past decade with many works using novel computer vision and machine learning techniques to evaluate the aesthetic value of visual information. This thesis presents a new approach where low-level features inspired from the human visual system are extracted from images to train a machine learning-based system to classify visual information depending on its aesthetics, regardless of the type of visual media. Extensive tests are developed to highlight strengths and weaknesses of such low-level features while establishing good practices in the domain of study of computational aesthetics. The aesthetic classification system is not only tested on the most widely used dataset of photographs, called AVA, on which it is trained initially, but also on other photographic datasets to evaluate the robustness of the learnt aesthetic preferences over other rating communities. The system is then assessed in terms of aesthetic classification on other types of visual media to investigate whether the learnt aesthetic preferences represent photography rules or more general aesthetic rules. The skill transfer from aesthetic classification of photos to videos demonstrates a satisfying correct classification rate of videos without any prior training on the test set created by Tzelepis et al. Moreover, the initial photograph classifier can also be used on feature films to investigate the classifier’s learnt visual preferences, due to films providing a large number of frames easily labellable. The study on aesthetic classification of videos concludes with a case study on the work by an online content creator. The classifier recognised a significantly greater percentage of aesthetically high frames in videos filmed in studios than on-the-go. The results obtained across datasets containing videos of diverse natures manifest the extent of the system’s aesthetic knowledge. To conclude, the evolution of low-level visual features is studied in popular culture such as in paintings and brand logos. The work attempts to link aesthetic preferences during contemplation tasks such as aesthetic rating of photographs with preferred low-level visual features in art creation. It questions whether favoured visual features usage varies over the life of a painter, implicitly showing a relationship with artistic expertise. Findings display significant changes in use of universally preferred features over influential vi abstract painters’ careers such an increase in cardinal lines and the colour blue; changes that were not observed in landscape painters. Regarding brand logos, only a few features evolved in a significant manner, most of them being colour-related features. Despite the incredible amount of data available online, phenomena developing over an entire life are still complicated to study. These computational experiments show that simple approaches focusing on the fundamentals instead of high-level measures allow to analyse artists’ visual preferences, as well as extract a community’s visual preferences from photos or videos while limiting impact from cultural and personal experiences

Can you believe what you see? A qualitative study about the determinants affecting the perceived credibility of video eWOM

Thesis Purpose: With the increasing popularity of social networks like YouTube, and the increasing accessibility of consumers to devices able to make and edit videos, video reviews are facing a remarkable growth. Hitherto, the research in eWOM has paid more attention to written reviews, leaving a notable gap of research in video eWOM. The purpose of this study was to improve the knowledge about video eWOM, studying the determinants affecting the perceived credibility of video reviews. Indeed, credibility is a crucial factor, playing a significant role in the consumer’s attitude toward a brand or product, and the consequent purchase intention. The study reached an answer to the following question: What are the determinants affecting the perceived credibility of video-eWOM reviews? Theoretical Perspective: The research model, adopted in this study, built upon three (3) main theoretical areas, due to a lack of research in video eWOM and in order to formulate an adequate background, suitable for the interpretation and analysis of data. First, the study drew upon the determinants of perceived credibility, found by prior research in the field of written eWOM. With the purpose of facilitating the use of this theory, it was built a comprehensive model, summarising all the determinants. Second, some theories about the influence of video features on the audience were implemented in order to fully grasp the potentiality of video reviews. Finally, the third area was focused on the peculiar features of nonverbal communication, involved in video reviews through the adoption of images, motion and sound to convey the message. Methodology: The empirical research was performed through a qualitative study based on a variation grounded theory. The data was collected at one point in time, and the sample consisted in 12 female participants who interacted with five (5) video reviews on YouTube. The participants’ opinions were collected performing semi-structured interviews, supported by the techniques of photo elicitation and list of thoughts. The empirical data was then analysed through a sequence of definite steps, based on grounded analysis. Results: The results of the empirical research were summarised in a new model, encompassing all the determinants observed to exert an influence in the current study. Two (2) new determinants of perceived credibility – visual evidence and testing – were revealed along with one (1) new moderator, first impression. Besides, the determinants and moderators, corresponding to the ones of written eWOM, were assessed by the participants through the adoption of more cues, including the reviewer’s appearance, facial expressions, tone of voice, and video features (e.g. setting). Keywords: Video eWOM, Video reviews, Electronic word-of-mouth, Credibility, YouTube