3,394 research outputs found
Representations and representation learning for image aesthetics prediction and image enhancement
With the continual improvement in cell phone cameras and improvements in the connectivity of mobile devices, we have seen an exponential increase in the images that are captured, stored and shared on social media. For example, as of July 1st 2017 Instagram had over 715 million registered users which had posted just shy of 35 billion images. This represented approximately seven and nine-fold increase in the number of users and photos present on Instagram since 2012. Whether the images are stored on personal computers or reside on social networks (e.g. Instagram, Flickr), the sheer number of images calls for methods to determine various image properties, such as object presence or appeal, for the purpose of automatic image management and curation. One of the central problems in consumer photography centers around determining the aesthetic appeal of an image and motivates us to explore questions related to understanding aesthetic preferences, image enhancement and the possibility of using such models on devices with constrained resources.
In this dissertation, we present our work on exploring representations and representation learning approaches for aesthetic inference, composition ranking and its application to image enhancement. Firstly, we discuss early representations that mainly consisted of expert features, and their possibility to enhance Convolutional Neural Networks (CNN). Secondly, we discuss the ability of resource-constrained CNNs, and the different architecture choices (inputs size and layer depth) in solving various aesthetic inference tasks: binary classification, regression, and image cropping. We show that if trained for solving fine-grained aesthetics inference, such models can rival the cropping performance of other aesthetics-based croppers, however they fall short in comparison to models trained for composition ranking. Lastly, we discuss our work on exploring and identifying the design choices in training composition ranking functions, with the goal of using them for image composition enhancement
Crowdsourcing in Computer Vision
Computer vision systems require large amounts of manually annotated data to
properly learn challenging visual concepts. Crowdsourcing platforms offer an
inexpensive method to capture human knowledge and understanding, for a vast
number of visual perception tasks. In this survey, we describe the types of
annotations computer vision researchers have collected using crowdsourcing, and
how they have ensured that this data is of high quality while annotation effort
is minimized. We begin by discussing data collection on both classic (e.g.,
object recognition) and recent (e.g., visual story-telling) vision tasks. We
then summarize key design decisions for creating effective data collection
interfaces and workflows, and present strategies for intelligently selecting
the most important data instances to annotate. Finally, we conclude with some
thoughts on the future of crowdsourcing in computer vision.Comment: A 69-page meta review of the field, Foundations and Trends in
Computer Graphics and Vision, 201
Design and Evaluation of Product Aesthetics: A Human-Machine Hybrid Approach
Aesthetics are critically important to market acceptance in many product
categories. In the automotive industry in particular, an improved aesthetic
design can boost sales by 30% or more. Firms invest heavily in designing and
testing new product aesthetics. A single automotive "theme clinic" costs
between \$100,000 and \$1,000,000, and hundreds are conducted annually. We use
machine learning to augment human judgment when designing and testing new
product aesthetics. The model combines a probabilistic variational autoencoder
(VAE) and adversarial components from generative adversarial networks (GAN),
along with modeling assumptions that address managerial requirements for firm
adoption. We train our model with data from an automotive partner-7,000 images
evaluated by targeted consumers and 180,000 high-quality unrated images. Our
model predicts well the appeal of new aesthetic designs-38% improvement
relative to a baseline and substantial improvement over both conventional
machine learning models and pretrained deep learning models. New automotive
designs are generated in a controllable manner for the design team to consider,
which we also empirically verify are appealing to consumers. These results,
combining human and machine inputs for practical managerial usage, suggest that
machine learning offers significant opportunity to augment aesthetic design
- …