Personalised aesthetics with residual adapters

The use of computational methods to evaluate aesthetics in photography has gained interest in recent years due to the popularization of convolutional neural networks and the availability of new annotated datasets. Most studies in this area have focused on designing models that do not take into account individual preferences for the prediction of the aesthetic value of pictures. We propose a model based on residual learning that is capable of learning subjective, user specific preferences over aesthetics in photography, while surpassing the state-of-the-art methods and keeping a limited number of user-specific parameters in the model. Our model can also be used for picture enhancement, and it is suitable for content-based or hybrid recommender systems in which the amount of computational resources is limited.Comment: 12 pages, 4 figures. In Iberian Conference on Pattern Recognition and Image Analysis proceeding

Exploiting visual saliency for assessing the impact of car commercials upon viewers

Content based video indexing and retrieval (CBVIR) is a lively area of research which focuses on automating the indexing, retrieval and management of videos. This area has a wide spectrum of promising applications where assessing the impact of audiovisual productions emerges as a particularly interesting and motivating one. In this paper we present a computational model capable to predict the impact (i.e. positive or negative) upon viewers of car advertisements videos by using a set of visual saliency descriptors. Visual saliency provides information about parts of the image perceived as most important, which are instinctively targeted by humans when looking at a picture or watching a video. For this reason we propose to exploit visual information, introducing it as a new feature which reflects high-level semantics objectively, to improve the video impact categorization results. The suggested salience descriptors are inspired by the mechanisms that underlie the attentional abilities of the human visual system and organized into seven distinct families according to different measurements over the identified salient areas in the video frames, namely population, size, location, geometry, orientation, movement and photographic composition. Proposed approach starts by computing saliency maps for all the video frames, where two different visual saliency detection frameworks have been considered and evaluated: the popular graph based visual saliency (GBVS) algorithm, and a state-of-the-art DNN-based approach.This work has been partially supported by the National Grants RTC-2016-5305-7 and TEC2014-53390-P of the Spanish Ministry of Economy and Competitiveness.Publicad

Autonomous Quadcopter Videographer

In recent years, the interest in quadcopters as a robotics platform for autonomous photography has increased. This is due to their small size and mobility, which allow them to reach places that are difficult or even impossible for humans. This thesis focuses on the design of an autonomous quadcopter videographer, i.e. a quadcopter capable of capturing good footage of a specific subject. In order to obtain this footage, the system needs to choose appropriate vantage points and control the quadcopter. Skilled human videographers can easily spot good filming locations where the subject and its actions can be seen clearly in the resulting video footage, but translating this knowledge to a robot can be complex. We present an autonomous system implemented on a commercially available quadcopter that achieves this using only the monocular information and an accelerometer. Our system has two vantage point selection strategies: 1) a reactive approach, which moves the robot to a fixed location with respect to the human and 2) the combination of the reactive approach and a POMDP planner that considers the target\u27s movement intentions. We compare the behavior of these two approaches under different target movement scenarios. The results show that the POMDP planner obtains more stable footage with less quadcopter motion

Estimation automatique des impressions véhiculées par une photographie de visage

Picture selection is a time-consuming task for humans and a real challenge for machines, which have to retrieve complex and subjective information from image pixels. An automated system that infers human feelings from digital portraits would be of great help for profile picture selection, photo album creation or photo editing. In this work, several models of facial pictures evaluation are defined. The first one predicts the overall aesthetic quality of a facial image by computing 15 features that encode low-level statistics in different image regions (face, eyes and mouth). Relevant features are automatically selected by a feature ranking technique, and the outputs of 4 learning algorithms are fused in order to make a robust and accurate prediction of the image quality. Results are compared with recent works and the proposed algorithm obtains the best performance. The same pipeline is then considered to evaluate the likability and competence induced by a facial picture, with the difference that the estimation is based on high-level attributes such as gender, age and smile. Performance of these attributes is compared with previous techniques that mostly rely on facial keypoints positions, and it is shown that it is possible to obtain predictions that are close to human perception. Finally, a combination of both models that selects a likable facial image of good aesthetic quality for a given person is described.Avec le développement des appareils photos numériques et des sites de partage de photos, nous passons une part croissante de notre temps à observer, sélectionner et partager des images, parmi lesquelles figurent un grand nombre de photos de visage. Dans cette thèse, nous nous proposons de créer un premier système entièrement automatique renvoyant une estimation de la pertinence d'une photo de visage pour son utilisation dans la création d'un album de photos, la sélection de photos pour un réseau social ou professionnel, etc. Pour cela, nous créons plusieurs modèles d'estimation de la pertinence d'une photo de visage en fonction de son utilisation. Dans un premier temps, nous adaptons les modèles d'estimation de la qualité esthétique d'une photo au cas particulier des photos de visage. Nous montrons que le fait de calculer 15 caractéristiques décrivant différents aspects de l'image (texture, illumination, couleurs) dans des régions spécifiques de l'image (le visage, les yeux, la bouche) améliore significativement la précision des estimations par rapport aux modèles de l'état de l'art. La précision de ce modèle est renforcée par la sélection de caractéristiques adaptées à notre problème, ainsi que par la fusion des prédictions de 4 algorithmes d'apprentissage. Dans un second temps, nous proposons d'enrichir l'évaluation automatique d'une photo de visage en définissant des modèles d'estimation associés à des critères tels que le degré de sympathie ou de compétence dégagé par une photo de visage. Ces modèles reposent sur l'utilisation d'attributs de haut niveau (présence de sourire, ouverture des yeux, expressions faciales), qui se montrent plus efficaces que les caractéristiques de bas niveau utilisées dans l'état de l'art (filtres de Gabor, position des points de repère du visage). Enfin, nous fusionnons ces modèles afin de sélectionner automatiquement des photos de bonne qualité esthétique et appropriées à une utilisation donnée : photos inspirant de la sympathie à partager en famille, photos dégageant une impression de compétence sur un réseau professionnel

Autonomous Evolutionary Art

Eiben, A.E. [Promotor