How to measure the relevance of a retargeting approach?

International audienceMost cell phones today can receive and display video content. Nonetheless, we are still significantly behind the point where premium made for mobile content is mainstream, largely available, and affordable. Significant issues must be overcome. The small screen size is one of them. Indeed, the direct transfer of conventional contents (not specifically shot for mobile devices) will provide a video in which the main characters or objects of interest may become indistinguishable from the rest of the scene. Therefore, it is required to retarget the content. Different solutions exist, either based on distortion of the image, on removal of redundant areas, or cropping. The most efficient ones are based on dynamic adaptation of the cropping window. They significantly improve the viewing experience by zooming in the regions of interest. Currently, there is no common agreement on how to compare different solutions. A retargeting metric is proposed in order to gauge its quality. Eye-tracking experiments, zooming effect through coverage ratio and temporal consistency are introduced and discussed

The MediaEval 2016 Emotional Impact of Movies Task

Volume: 1739 Host publication title: MediaEval 2016 Multimedia Benchmark Workshop Host publication sub-title: Working Notes Proceedings of the MediaEval 2016 WorkshopNon peer reviewe

The MediaEval 2016 Emotional Impact of Movies Task

ABSTRACT This paper provides a description of the MediaEval 2016 "Emotional Impact of Movies" task. It continues builds on previous years' editions of the Affect in Multimedia Task: Violent Scenes Detection. However, in this year's task, participants are expected to create systems that automatically predict the emotional impact that video content will have on viewers, in terms of valence and arousal scores. Here we provide insights on the use case, task challenges, dataset and ground truth, task run requirements and evaluation metrics

Harmonie des couleurs : modélisation expérimentale et algorithmique

Since the consumption of digital media exploded in the last decade, making aesthetic pictures quickly - with or without artistic expertise - is more than ever a research topic. Different axis of investigations remain possible: high resolution, high dynamic range or wide color gamut. Additionally to these objective image properties, more perceptual and artistic insights could be of benefit to any user manipulating pictures. In such context, this thesis deals with the topic of Color Harmony. The literature related to this topic is limited, but involves many different scientific areas: color science, image processing and psychology and so on. The validity of collected data is questionable due to their limitation to two- or three-colors patches. The models fitted from these data remain non-exploitable on natural pictures. Other models depicting rules or areas on color wheel lack scientific guidelines for their utilization. Nonetheless, some algorithms employing color harmony theory and models as a core concept showed up in the literature, but suffered from being quantitatively tested and validated. In this thesis, two views are put in perspective in order to respond to the previous statements: an experimental and a computational approaches. The conducted experiment allowed observing some effects with an eye-tracking protocol, never applied before with a task on color harmony assessment. From the collected data of our experimental work, we designed a method to generate a ground truth, which would serve to the validation of the two proposed computational methods. First, we improved an existing architecture for automatic color harmonization and demonstrated exhaustively the benefit of our approach. As a second computational contribution, a novel quality metric is introduced that integrates the concepts of visual masking and color harmony. Thus, we may predict which areas would be perceived harmonious regarding its neighborhood and then the potential masking effects. As a last contribution, two editing tools made accessible the color harmony theory through a hidden formulation of it and a user-friendly and intuitive interface.Comme la consommation de médias numériques a explosé ces dernières années, faire des photos esthétiques, avec ou sans expertise artistique, est plus que jamais un sujet de recherche. Plusieurs axes peuvent être explorés: la haute définition, la luminance ou contraste étendue, les gamut couleur étendus. En plus de ces propriétés intrinsèques de l'image, des connaissances perceptuelles et/ou artistiques seraient de grande valeur pour tout utilisateur manipulant le contenu des images. Cette thèse propose d'aborder le thème de l'harmonie des couleurs. La littérature en lien avec ce sujet se retrouve à travers diverses disciplines : la science des couleurs, le traitement d'image, la psychologie… Ces expériences menées en science des couleurs privilégient la mesure de patchs combinant deux ou trois couleurs, rendant l'extrapolation à des images naturelles impossibles. D'autres approches ont défini des lois empiriques dictant l'arrangement des couleurs sur la roue des teintes. Le cadre applicatif de ces modèles géométriques manque de rigueur quant à leur utilisation. Malgré cela, des algorithmes en traitement d'image employant ces modèles ont vus le jour. Si les résultats semblent qualitativement agréables, ces algorithmes méritent une validation plus quantitative et objective, faisant intervenir une base de données appropriée. Dans cette thèse, deux approches sont mises en perspective: un travail expérimental et une partie algorithmique. Une expérience a été menée à l'aide d'un oculomètre avec une tâche dédiée à l'analyse de l'harmonie des couleurs, permettant de mesurer des effets dans le déploiement de l'attention visuelle. A partir de ces données, une vérité terrain a été extrapolée, permettant la validation des méthodes algorithmiques ensuite proposées. En premier, nous avons amélioré l'état de l'art sur l'harmonisation automatique des images au travers de diverses contributions et avons démontré de façon exhaustive le gain de notre approche. En deuxième contribution algorithmique, nous avons introduit une nouvelle sorte de métrique de qualité qui combine les concepts de masquage visuel et d'harmonie des couleurs. Ainsi, nous pouvons prédire quelles zones de l'image seront perçues harmonieuses, au vue de leur voisinage et donc des effets de masquages potentiels. Enfin, une dernière contribution, nous a amené à dériver deux outils d'édition incorporant les deux techniques précédentes, permettant de rendre accessible les concepts d'harmonie des couleurs à travers une formulation cachée et intuitive

Descriptor-based Image Colorization and Regularization

International audienceWe propose a new, fully automatic method for example-based image colorization and a robust color artifact regularization solution. To determine correspondences between the two images, we supplement the PatchMatch algorithm with rich statistical image descriptors. Based on detected matches, our method transfers colors from the reference to the target grayscale image. In addition, we propose a general regularization scheme that can smooth artifacts typical to color manipulation algorithms. Our regularization approach propagates the major colors in image regions, as determined through superpixel-based seg-mentation of the original image. We evaluate the effectiveness of our colorization for a varied set of images and demonstrate our regulariza-tion scheme for both colorization and color transfer applications

From crowdsourced rankings to affective ratings

International audienceAutomatic prediction of emotions requires reliably annotated data which can be achieved using scoring or pairwise ranking. But can we predict an emotional score using a ranking-based annotation approach? In this paper, we propose to answer this question by describing a regression analysis to map crowdsourced rankings into affective scores in the induced valence- arousal emotional space. This process takes advantages of the Gaussian Processes for regression that can take into account the variance of the ratings and thus the subjectivity of emotions. Regression models successfully learn to fit input data and provide valid predictions. Two distinct experiments were realized using a small subset of the publicly available LIRIS-ACCEDE affective video database for which crowdsourced ranks, as well as affective ratings, are available for arousal and valence. It allows to enrich LIRIS-ACCEDE by providing absolute video ratings for the whole database in addition to video rankings that are already available

A Protocol for Cross-Validating Large Crowdsourced Data

International audienceRecently, we released a large affective video dataset, namely LIRIS-ACCEDE, which was annotated through crowdsourcing along both induced valence and arousal axes using pairwise comparisons. In this paper, we design an annotation protocol which enables the scoring of induced affective feelings for cross-validating the annotations of the LIRIS-ACCEDE dataset and identifying any potential bias. We have collected in a controlled setup the ratings from 28 users on a subset of video clips carefully selected from the dataset by computing the inter-observer reliabilities on the crowdsourced data. On contrary to crowdsourced rankings gathered in unconstrained environments, users were asked to rate each video through the Self-Assessment Manikin tool. The significant correlation between crowdsourced rankings and controlled ratings validates the reliability of the dataset for future uses in affective video analysis and paves the way for the automatic generation of ratings over the whole dataset

Affective Video Content Analysis: A Multidisciplinary Insight

International audienceIn our present society, the cinema has become one of the major forms of entertainment providing unlimited contexts of emotion elicitation for the emotional needs of human beings. Since emotions are universal and shape all aspects of our interpersonal and intellectual experience, they have proved to be a highly multidisciplinary research field, ranging from psychology, sociology, neuroscience, etc., to computer science. However, affective multimedia content analysis work from the computer science community benefits but little from the progress achieved in other research fields. In this paper, a multidisciplinary state-of-the-art for affective movie content analysis is given, in order to promote and encourage exchanges between researchers from a very wide range of fields. In contrast to other state-of-the-art papers on affective video content analysis, this work confronts the ideas and models of psychology, sociology, neuroscience, and computer science. The concepts of aesthetic emotions and emotion induction, as well as the different representations of emotions are introduced, based on psychological and sociological theories. Previous global and continuous affective video content analysis work, including video emotion recognition and violence detection, are also presented in order to point out the limitations of affective video content analysis work