Weakly- and Self-Supervised Learning for Content-Aware Deep Image Retargeting

This paper proposes a weakly- and self-supervised deep convolutional neural network (WSSDCNN) for content-aware image retargeting. Our network takes a source image and a target aspect ratio, and then directly outputs a retargeted image. Retargeting is performed through a shift map, which is a pixel-wise mapping from the source to the target grid. Our method implicitly learns an attention map, which leads to a content-aware shift map for image retargeting. As a result, discriminative parts in an image are preserved, while background regions are adjusted seamlessly. In the training phase, pairs of an image and its image-level annotation are used to compute content and structure losses. We demonstrate the effectiveness of our proposed method for a retargeting application with insightful analyses.Comment: 10 pages, 11 figures. To appear in ICCV 2017, Spotlight Presentatio

Real-time content-aware video retargeting on the Android platform for tunnel vision assistance

As mobile devices continue to rise in popularity, advances in overall mobile device processing power lead to further expansion of their capabilities. This, coupled with the fact that many people suffer from low vision, leaves substantial room for advancing mobile development for low vision assistance. Computer vision is capable of assisting and accommodating individuals with blind spots or tunnel vision by extracting the necessary information and presenting it to the user in a manner they are able to visualize. Such a system would enable individuals with low vision to function with greater ease. Additionally, offering assistance on a mobile platform allows greater access. The objective of this thesis is to develop a computer vision application for low vision assistance on the Android mobile device platform. Specifically, the goal of the application is to reduce the effects tunnel vision inflicts on individuals. This is accomplished by providing an in-depth real-time video retargeting model that builds upon previous works and applications. Seam carving is a content-aware retargeting operator which defines 8-connected paths, or seams, of pixels. The optimality of these seams is based on a specific energy function. Discrete removal of these seams permits changes in the aspect ratio while simultaneously preserving important regions. The video retargeting model incorporates spatial and temporal considerations to provide effective image and video retargeting. Data reduction techniques are utilized in order to generate an efficient model. Additionally, a minimalistic multi-operator approach is constructed to diminish the disadvantages experienced by individual operators. In the event automated techniques fail, interactive options are provided that allow for user intervention. Evaluation of the application and its video retargeting model is based on its comparison to existing standard algorithms and its ability to extend itself to real-time. Performance metrics are obtained for both PC environments and mobile device platforms for comparison

A deep evaluator for image retargeting quality by geometrical and contextual interaction

An image is compressed or stretched during the multidevice displaying, which will have a very big impact on perception quality. In order to solve this problem, a variety of image retargeting methods have been proposed for the retargeting process. However, how to evaluate the results of different image retargeting is a very critical issue. In various application systems, the subjective evaluation method cannot be applied on a large scale. So we put this problem in the accurate objective-quality evaluation. Currently, most of the image retargeting quality assessment algorithms use simple regression methods as the last step to obtain the evaluation result, which are not corresponding with the perception simulation in the human vision system (HVS). In this paper, a deep quality evaluator for image retargeting based on the segmented stacked AutoEnCoder (SAE) is proposed. Through the help of regularization, the designed deep learning framework can solve the overfitting problem. The main contributions in this framework are to simulate the perception of retargeted images in HVS. Especially, it trains two separated SAE models based on geometrical shape and content matching. Then, the weighting schemes can be used to combine the obtained scores from two models. Experimental results in three well-known databases show that our method can achieve better performance than traditional methods in evaluating different image retargeting results