Representations and representation learning for image aesthetics prediction and image enhancement

With the continual improvement in cell phone cameras and improvements in the connectivity of mobile devices, we have seen an exponential increase in the images that are captured, stored and shared on social media. For example, as of July 1st 2017 Instagram had over 715 million registered users which had posted just shy of 35 billion images. This represented approximately seven and nine-fold increase in the number of users and photos present on Instagram since 2012. Whether the images are stored on personal computers or reside on social networks (e.g. Instagram, Flickr), the sheer number of images calls for methods to determine various image properties, such as object presence or appeal, for the purpose of automatic image management and curation. One of the central problems in consumer photography centers around determining the aesthetic appeal of an image and motivates us to explore questions related to understanding aesthetic preferences, image enhancement and the possibility of using such models on devices with constrained resources. In this dissertation, we present our work on exploring representations and representation learning approaches for aesthetic inference, composition ranking and its application to image enhancement. Firstly, we discuss early representations that mainly consisted of expert features, and their possibility to enhance Convolutional Neural Networks (CNN). Secondly, we discuss the ability of resource-constrained CNNs, and the different architecture choices (inputs size and layer depth) in solving various aesthetic inference tasks: binary classification, regression, and image cropping. We show that if trained for solving fine-grained aesthetics inference, such models can rival the cropping performance of other aesthetics-based croppers, however they fall short in comparison to models trained for composition ranking. Lastly, we discuss our work on exploring and identifying the design choices in training composition ranking functions, with the goal of using them for image composition enhancement

can we measure the beauty of an image

n/

複雑さに関連した特徴を用いた主観的印象予測

学位の種別: 課程博士審査委員会委員 : （主査）東京大学准教授 山﨑 俊彦, 東京大学教授 相澤 清晴, 国立情報学研究所教授 佐藤 真一, 東京大学教授 佐藤 洋一, 東京大学教授 苗村 健University of Tokyo(東京大学

VRSC 2021 Conference Proceedings

The biennial conference aims to catalyze ideas and innovation between academia, practice, NGOs and government agencies who work to address analysis, planning, valuation, design and management of visual resources. The aim of the 2021 Virtual Conference is to share ideas and discuss the issues associated with the assessment and protection of visual resources in an era of major landscape change - regionally, national and globally

Timeline design for visualising cultural heritage data

This thesis is concerned with the design of data visualisations of digitised museum, archive and library collections, in timelines. As cultural institutions digitise their collections—converting texts, objects, and artworks to electronic records—the volume of cultural data available grows. There is a growing perception, though, that we need to get more out of this data. Merely digitising does not automatically make collections accessible, discoverable and comprehensible, and standard interfaces do not necessarily support the types of interactions users wish to make. Data visualisations—this thesis focuses on interactive visual representations of data created with software—allow us to see an overview of, observe patterns in, and showcase the richness of, digitised collections. Visualisation can support analysis, exploration and presentation of collections for different audiences: research, collection administration, and the general public. The focus here is on visualising cultural data by time: a fundamental dimension for making sense of historical data, but also one with unique strangeness. Through cataloguing, cultural institutions define the meaning and value of items in their collections and the structure within which to make sense of them. By visualising threads in cataloguing data through time, can historical narratives be made visible? And is the data alone enough to tell the stories that people wish to tell? The intended audience for this research is cultural heritage institutions. This work sits at the crossroads between design, cultural heritage (particularly museology), and computing—drawing on the fields of digital humanities, information visualisation and human computer-interaction which also live in these overlapping spaces. This PhD adds clarity around the question of what cultural visualisation is (and can be) for, and highlights issues in the visualisation of qualitative or nominal data. The first chapter lays out the background, characterising cultural data and its visualisation. Chapter two walks through examples of existing cultural timeline visualisations, from the most handcrafted displays to automated approaches. At this point, the research agenda and methodology are set out. The next five chapters document a portfolio of visualisation projects, designing and building novel prototype timeline visualisations with data from the Wellcome Library and Victoria & Albert Museum, London, Cooper Hewitt Smithsonian Design Museum, New York City, and the Nordic Museum, Stockholm. In the process, a range of issues are identified for further discussion. The final chapters reflect on these projects, arguing that automated timeline visualisation can be a productive way to explore and present historical narratives in collection data, but a range of factors govern what is possible and useful. Trust in cultural data visualisation is also discussed. This research argues that visualising cultural data can add value to the data both for users and for data-holding institutions. However, that value is likely to be best achieved by customising a visualisation design to the dataset, audience and use case. Keywords: cultural heritage data; historical data; cultural analytics; cultural informatics; humanities visualisation; generous interfaces; digital humanities; design; information design; interface design; data visualisation; information visualisation; time; timeline; history; historiography; museums; museology; archives; chronographics