19 research outputs found
Image Aesthetics Assessment Using Composite Features from off-the-Shelf Deep Models
Deep convolutional neural networks have recently achieved great success on
image aesthetics assessment task. In this paper, we propose an efficient method
which takes the global, local and scene-aware information of images into
consideration and exploits the composite features extracted from corresponding
pretrained deep learning models to classify the derived features with support
vector machine. Contrary to popular methods that require fine-tuning or
training a new model from scratch, our training-free method directly takes the
deep features generated by off-the-shelf models for image classification and
scene recognition. Also, we analyzed the factors that could influence the
performance from two aspects: the architecture of the deep neural network and
the contribution of local and scene-aware information. It turns out that deep
residual network could produce more aesthetics-aware image representation and
composite features lead to the improvement of overall performance. Experiments
on common large-scale aesthetics assessment benchmarks demonstrate that our
method outperforms the state-of-the-art results in photo aesthetics assessment.Comment: Accepted by ICIP 201
Some like it hot - visual guidance for preference prediction
For people first impressions of someone are of determining importance. They
are hard to alter through further information. This begs the question if a
computer can reach the same judgement. Earlier research has already pointed out
that age, gender, and average attractiveness can be estimated with reasonable
precision. We improve the state-of-the-art, but also predict - based on
someone's known preferences - how much that particular person is attracted to a
novel face. Our computational pipeline comprises a face detector, convolutional
neural networks for the extraction of deep features, standard support vector
regression for gender, age and facial beauty, and - as the main novelties -
visual regularized collaborative filtering to infer inter-person preferences as
well as a novel regression technique for handling visual queries without rating
history. We validate the method using a very large dataset from a dating site
as well as images from celebrities. Our experiments yield convincing results,
i.e. we predict 76% of the ratings correctly solely based on an image, and
reveal some sociologically relevant conclusions. We also validate our
collaborative filtering solution on the standard MovieLens rating dataset,
augmented with movie posters, to predict an individual's movie rating. We
demonstrate our algorithms on howhot.io which went viral around the Internet
with more than 50 million pictures evaluated in the first month.Comment: accepted for publication at CVPR 201
Revisiting Image Aesthetic Assessment via Self-Supervised Feature Learning
Visual aesthetic assessment has been an active research field for decades.
Although latest methods have achieved promising performance on benchmark
datasets, they typically rely on a large number of manual annotations including
both aesthetic labels and related image attributes. In this paper, we revisit
the problem of image aesthetic assessment from the self-supervised feature
learning perspective. Our motivation is that a suitable feature representation
for image aesthetic assessment should be able to distinguish different
expert-designed image manipulations, which have close relationships with
negative aesthetic effects. To this end, we design two novel pretext tasks to
identify the types and parameters of editing operations applied to synthetic
instances. The features from our pretext tasks are then adapted for a one-layer
linear classifier to evaluate the performance in terms of binary aesthetic
classification. We conduct extensive quantitative experiments on three
benchmark datasets and demonstrate that our approach can faithfully extract
aesthetics-aware features and outperform alternative pretext schemes. Moreover,
we achieve comparable results to state-of-the-art supervised methods that use
10 million labels from ImageNet.Comment: AAAI Conference on Artificial Intelligence, 2020, accepte
IMAGE AESTHETIC PREDICTORS BASED ON WEIGHTED CNNS
Convolutional Neural Networks (CNNs) have been widely adopted for many imaging applications. For image aesthetics prediction, state-of-the-art algorithms train CNNs on a recently-published large-scale dataset, AVA. However, the distribution of the aesthetic scores on this dataset is extremely unbalanced, which limits the prediction capability of existing methods. We overcome such limitation by using weighted CNNs. We train a regression model that improves the prediction accuracy of the aesthetic scores over state-of-the-art algorithms. In addition, we propose a novel histogram prediction model that not only predicts the aesthetic score, but also estimates the difficulty of performing aesthetics assessment for an input image. We further show an image enhancement application where we obtain an aesthetically pleasing crop of an input image using our regression model