45 research outputs found
Learning Visual Importance for Graphic Designs and Data Visualizations
Knowing where people look and click on visual designs can provide clues about
how the designs are perceived, and where the most important or relevant content
lies. The most important content of a visual design can be used for effective
summarization or to facilitate retrieval from a database. We present automated
models that predict the relative importance of different elements in data
visualizations and graphic designs. Our models are neural networks trained on
human clicks and importance annotations on hundreds of designs. We collected a
new dataset of crowdsourced importance, and analyzed the predictions of our
models with respect to ground truth importance and human eye movements. We
demonstrate how such predictions of importance can be used for automatic design
retargeting and thumbnailing. User studies with hundreds of MTurk participants
validate that, with limited post-processing, our importance-driven applications
are on par with, or outperform, current state-of-the-art methods, including
natural image saliency. We also provide a demonstration of how our importance
predictions can be built into interactive design tools to offer immediate
feedback during the design process
Image Cropping with Composition and Saliency Aware Aesthetic Score Map
Aesthetic image cropping is a practical but challenging task which aims at
finding the best crops with the highest aesthetic quality in an image.
Recently, many deep learning methods have been proposed to address this
problem, but they did not reveal the intrinsic mechanism of aesthetic
evaluation. In this paper, we propose an interpretable image cropping model to
unveil the mystery. For each image, we use a fully convolutional network to
produce an aesthetic score map, which is shared among all candidate crops
during crop-level aesthetic evaluation. Then, we require the aesthetic score
map to be both composition-aware and saliency-aware. In particular, the same
region is assigned with different aesthetic scores based on its relative
positions in different crops. Moreover, a visually salient region is supposed
to have more sensitive aesthetic scores so that our network can learn to place
salient objects at more proper positions. Such an aesthetic score map can be
used to localize aesthetically important regions in an image, which sheds light
on the composition rules learned by our model. We show the competitive
performance of our model in the image cropping task on several benchmark
datasets, and also demonstrate its generality in real-world applications.Comment: Accepted by AAAI 2
Image Cropping under Design Constraints
Image cropping is essential in image editing for obtaining a compositionally
enhanced image. In display media, image cropping is a prospective technique for
automatically creating media content. However, image cropping for media
contents is often required to satisfy various constraints, such as an aspect
ratio and blank regions for placing texts or objects. We call this problem
image cropping under design constraints. To achieve image cropping under design
constraints, we propose a score function-based approach, which computes scores
for cropped results whether aesthetically plausible and satisfies design
constraints. We explore two derived approaches, a proposal-based approach, and
a heatmap-based approach, and we construct a dataset for evaluating the
performance of the proposed approaches on image cropping under design
constraints. In experiments, we demonstrate that the proposed approaches
outperform a baseline, and we observe that the proposal-based approach is
better than the heatmap-based approach under the same computation cost, but the
heatmap-based approach leads to better scores by increasing computation cost.
The experimental results indicate that balancing aesthetically plausible
regions and satisfying design constraints is not a trivial problem and requires
sensitive balance, and both proposed approaches are reasonable alternatives.Comment: ACMMM Asia accepte