Image Aesthetics Assessment Based on Multi-stream CNN Architecture and Saliency Features

In this paper, we explore how higher-level perceptual information based on visual attention can be used for aesthetic assessment of images. We assume that visually dominant subjects in a photograph influence stronger aesthetic interest. In other words, visual attention may be a key to predicting photographic aesthetics. Our proposed aesthetic assessment method, which is based on multi-stream and multi-task convolutional neural networks (CNNs), extracts global features and saliency features from an input image. These provide higher-level visual information such as the quality of the photo subject and the subject–background relationship. Results from our experiments support the effectiveness of our approach

Using a two-stage convolutional neural network to rapidly identify tiny herbivorous beetles in the field

Recently, deep convolutional neural networks (CNN) have been adopted to help non-experts identify insect species from field images. However, the application of these methods on the rapid identification of tiny congeneric species moving across heterogeneous background remains difficult. To improve rapid and automatic identification in the field, we customized an existing CNN-based method for a field video involving two Phyllotreta beetles. We first performed data augmentation using transformations, syntheses, and random erasing of the original images. We then proposed a two-stage method for the detection and identification of small insects based on CNN, where YOLOv4 and EfficientNet were used as a detector and a classifier, respectively. Evaluation of the model revealed that one-step object detection by YOLOv4 alone was not precise (Precision = 0.55) when classifying two species of flea beetles and background objects. In contrast, the two-step CNNs improved the precision (Precision = 0.89) with moderate accuracy (F-measure = 0.55) and acceptable speed (ca. 5 frames per second for full HD images) of detection and identification of insect species in the field. Although real-time identification of tiny insects remains a challenge in the field, our method aids in improving small object detection on a heterogeneous background

Anomaly Detection Using Siamese Network with Attention Mechanism for Few-Shot Learning

Automated inspection using deep-learning has been attracting attention for visual inspection at the manufacturing site. However, the inability to obtain sufficient abnormal product data for training deep- learning models is a problem in practical application. This study proposes an anomaly detection method based on the Siamese network with an attention mechanism for a small dataset. Moreover, attention branch loss (ABL) is proposed for Siamese network to render more task-specific attention maps from attention mechanism. Experimental results confirm that the proposed method with the attention mechanism and ABL is effective even with limited abnormal data

Food Constituent Estimation for Lifestyle Disease Prevention by Multi-Task CNN

Unbalanced nutrition due to an unhealthy diet may increase the risk of developing lifestyle diseases. Many mobile applications have been released to record everyday meals for the health-conscious to enable them to improve their dietary habits. Most of these applications only base their food classification on an image of the food, requiring the user to manually input information about the ingredients such as the calories and salinity. To address this problem, food ingredient estimation from food images has been attracting increasing attention. Automatic ingredient estimation could possibly strongly alleviate the process of food-intake estimation and dietary assessment. In this paper, we propose an automatic food ingredient estimation method from food images by using multi-task CNN. We focus on classification of the food category and estimation of the calorie content and salinity for lifestyle disease prevention. Two-stage transfer learning using a large number of food category recognition image databases is applied to train our multi-task CNN for improved ingredient estimation. We experimentally analyze the relationship between the food category and salinity by using multi-task CNN