Computational Media Aesthetics for Media Synthesis

Ph.DDOCTOR OF PHILOSOPH

Exploiting visual saliency for assessing the impact of car commercials upon viewers

Content based video indexing and retrieval (CBVIR) is a lively area of research which focuses on automating the indexing, retrieval and management of videos. This area has a wide spectrum of promising applications where assessing the impact of audiovisual productions emerges as a particularly interesting and motivating one. In this paper we present a computational model capable to predict the impact (i.e. positive or negative) upon viewers of car advertisements videos by using a set of visual saliency descriptors. Visual saliency provides information about parts of the image perceived as most important, which are instinctively targeted by humans when looking at a picture or watching a video. For this reason we propose to exploit visual information, introducing it as a new feature which reflects high-level semantics objectively, to improve the video impact categorization results. The suggested salience descriptors are inspired by the mechanisms that underlie the attentional abilities of the human visual system and organized into seven distinct families according to different measurements over the identified salient areas in the video frames, namely population, size, location, geometry, orientation, movement and photographic composition. Proposed approach starts by computing saliency maps for all the video frames, where two different visual saliency detection frameworks have been considered and evaluated: the popular graph based visual saliency (GBVS) algorithm, and a state-of-the-art DNN-based approach.This work has been partially supported by the National Grants RTC-2016-5305-7 and TEC2014-53390-P of the Spanish Ministry of Economy and Competitiveness.Publicad

Gazo bunseki to kanren joho o riyoshita gazo imi rikai ni kansuru kenkyu

制度:新 ; 報告番号:甲3514号 ; 学位の種類:博士(国際情報通信学) ; 授与年月日:2012/2/8 ; 早大学位記番号:新585

Improving the aesthetic and other experiential design aspects of bicycle paths in Western Australia

Governments around Australia are in the process of promoting cycling as both a sustainable form of transport that can be a viable alternative to the motor vehicle, particularly for shorter trips, and as a healthy recreational pursuit that can play an important role in addressing the growing problem of obesity and illnesses associated with a sedentary lifestyle in the community. As part of this initiative, the development of effective and efficient infrastructure for bicycles is seen as a vital step for achieving higher participation rates. A major component of the nation’s bicycle infrastructure is the growing networks of paved paths and natural surface trails located in both urban and regional areas. A well-designed path or trail must meet agreed standards related to safety and function and, in order to achieve maximum usage, it must also create a desirable riding experience. While requirements for safety and function are well understood by path and trail planners, little empirical information has been produced to enable these planning professionals to understand the elements that impact upon an individual’s riding experience and to then incorporate them into the design process. Accordingly, the overall aim of this research was to investigate how the aesthetic, cultural and other experiential design aspects of bicycle paths and trails can enhance the perceived riding experience. A secondary objective of the project involved a determination of the procedural factors guiding the local path and trail design protocols and process. A third objective was to gain an insight into the most effective method of communicating the benefits of these riding environments to important target groups. Following the establishment of a theoretical framework incorporating the psychophysical nature of cycling, the effect of landscape and current design practices, the research progressed through several stages beginning with an autoethnography examining the researcher’s extensive experience in the promotion of cycling in Western Australia, augmented by in-depth discussions with leading key informants. This was followed by a mix of quantitative and qualitative methodology to gauge perception of various elements of in-situ and photo-surrogate path-riding environments among the general population in Perth, Western Australia. The findings indicate that there are specific experiential design aspects related to the riding environment, surrounding landscape or associated features that can directly influence a person’s decision to use a particular path, trail or route. The research also identified preferred communication strategies and found deficiencies in the current design process that if addressed, could lead to the development of better received and patronised riding environments. It is intended that the outcome of this research will be to provide a design framework to guide path and trail planners in the development of facilities that enhance the overall riding experience. A number of agencies responsible for developing bicycle infrastructure, or design standards, have indicated a desire to access parts of this research project for use in the decision-making process, thus achieving a better balance between safety, functional and experiential aspects

Blood in the Corridor

This article examines the significance of the digital aesthetic of violence in the uniquely contemporary action-film “hero run” shoot-out sequences. By using the case studies of Kick-Ass (2010) and Wanted (2008), the article focuses on how the particular stylistic tendencies of these sequences display a link between the onscreen, digital-enabled mastery of the shooter with the offscreen digital mastery of the visual-effects artist

Integrating virtual reality and GIS tools for geological mapping, data collection and analysis: an example from Metaxa Mine, Santorini (Greece)

In the present work we highlight the effectiveness of integrating different techniques and tools for better surveying, mapping and collecting data in volcanic areas. We use an Immersive Virtual Reality (IVR) approach for data collection, integrated with Geographic Information System (GIS) analysis in a well-known volcanological site in Santorini (Metaxa mine), a site where volcanic processes influenced the island’s industrial development, especially with regard to pumice mining. Specifically, we have focused on: (i) three-dimensional (3D) high-resolution IVR scenario building, based on Structure from Motion photogrammetry (SfM) modeling; (ii) subsequent geological survey, mapping and data collection using IVR; (iii) data analysis, e.g., calculation of extracted volumes, as well as production of new maps in a GIS environment using input data directly from the IVR survey; and finally, (iv) presentation of new outcomes that highlight the importance of the Metaxa Mine as a key geological and volcanological geosite

Data-driven visual quality estimation using machine learning

Heutzutage werden viele visuelle Inhalte erstellt und sind zugänglich, was auf Verbesserungen der Technologie wie Smartphones und das Internet zurückzuführen ist. Es ist daher notwendig, die von den Nutzern wahrgenommene Qualität zu bewerten, um das Erlebnis weiter zu verbessern. Allerdings sind nur wenige der aktuellen Qualitätsmodelle speziell für höhere Auflösungen konzipiert, sagen mehr als nur den Mean Opinion Score vorher oder nutzen maschinelles Lernen. Ein Ziel dieser Arbeit ist es, solche maschinellen Modelle für höhere Auflösungen mit verschiedenen Datensätzen zu trainieren und zu evaluieren. Als Erstes wird eine objektive Analyse der Bildqualität bei höheren Auflösungen durchgeführt. Die Bilder wurden mit Video-Encodern komprimiert, hierbei weist AV1 die beste Qualität und Kompression auf. Anschließend werden die Ergebnisse eines Crowd-Sourcing-Tests mit einem Labortest bezüglich Bildqualität verglichen. Weiterhin werden auf Deep Learning basierende Modelle für die Vorhersage von Bild- und Videoqualität beschrieben. Das auf Deep Learning basierende Modell ist aufgrund der benötigten Ressourcen für die Vorhersage der Videoqualität in der Praxis nicht anwendbar. Aus diesem Grund werden pixelbasierte Videoqualitätsmodelle vorgeschlagen und ausgewertet, die aussagekräftige Features verwenden, welche Bild- und Bewegungsaspekte abdecken. Diese Modelle können zur Vorhersage von Mean Opinion Scores für Videos oder sogar für anderer Werte im Zusammenhang mit der Videoqualität verwendet werden, wie z.B. einer Bewertungsverteilung. Die vorgestellte Modellarchitektur kann auf andere Videoprobleme angewandt werden, wie z.B. Videoklassifizierung, Vorhersage der Qualität von Spielevideos, Klassifikation von Spielegenres oder der Klassifikation von Kodierungsparametern. Ein wichtiger Aspekt ist auch die Verarbeitungszeit solcher Modelle. Daher wird ein allgemeiner Ansatz zur Beschleunigung von State-of-the-Art-Videoqualitätsmodellen vorgestellt, der zeigt, dass ein erheblicher Teil der Verarbeitungszeit eingespart werden kann, während eine ähnliche Vorhersagegenauigkeit erhalten bleibt. Die Modelle sind als Open Source veröffentlicht, so dass die entwickelten Frameworks für weitere Forschungsarbeiten genutzt werden können. Außerdem können die vorgestellten Ansätze als Bausteine für neuere Medienformate verwendet werden.Today a lot of visual content is accessible and produced, due to improvements in technology such as smartphones and the internet. This results in a need to assess the quality perceived by users to further improve the experience. However, only a few of the state-of-the-art quality models are specifically designed for higher resolutions, predict more than mean opinion score, or use machine learning. One goal of the thesis is to train and evaluate such machine learning models of higher resolutions with several datasets. At first, an objective evaluation of image quality in case of higher resolutions is performed. The images are compressed using video encoders, and it is shown that AV1 is best considering quality and compression. This evaluation is followed by the analysis of a crowdsourcing test in comparison with a lab test investigating image quality. Afterward, deep learning-based models for image quality prediction and an extension for video quality are proposed. However, the deep learning-based video quality model is not practically usable because of performance constrains. For this reason, pixel-based video quality models using well-motivated features covering image and motion aspects are proposed and evaluated. These models can be used to predict mean opinion scores for videos, or even to predict other video quality-related information, such as a rating distributions. The introduced model architecture can be applied to other video problems, such as video classification, gaming video quality prediction, gaming genre classification or encoding parameter estimation. Furthermore, one important aspect is the processing time of such models. Hence, a generic approach to speed up state-of-the-art video quality models is introduced, which shows that a significant amount of processing time can be saved, while achieving similar prediction accuracy. The models have been made publicly available as open source so that the developed frameworks can be used for further research. Moreover, the presented approaches may be usable as building blocks for newer media formats

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task