Multiclass insect counting through deep learning-based density maps estimation

The use of digital technologies and artificial intelligence techniques for the automation of some visual assessment processes in agriculture is currently a reality. Image-based, and recently deep learning-based systems are being used in several applications. Main challenge of these applications is to achieve a correct performance in real field conditions over images that are usually acquired with mobile devices and thus offer limited quality. Plagues control is a problem to be tackled in the field. Pest management strategies relies on the identification of the level of infestation. This degree of infestation is established through a counting task manually done by the field researcher so far. Current models were not able to appropriately count due to the small size of the insects and on the last year we presented a density map based algorithm that superseded state of the art methods for a single insect type. In this paper, we extend previous work into a multiclass and multi-stadia approach. Concretely, the proposed algorithm has been tested in two use cases: on the one hand, it counts five different types of adult individuals over multiple crop leaves; and on the other hand, it identifies four different stages for immatures over 2-cm leaf disks. In these leaf disks, some of the species are in different stadia being some of them micron size and difficult to be identified even for the non-expert user. The proposed method achieves good results in both cases. The model for counting adult insects in a leaf achieves a RMSE ranging from 0.89 to 4.47, MAE ranging from 0.40 to 2.15, and R2 ranging from 0.86 to 0.91 for 4 different species in its adult phase (BEMITA, FRANOC, MYZUPE and APHIGO) that may appear together in the same leaf. Besides, for FRANOC, two stadia nymphs and adults are considered. The model developed for counting BEMITA immatures in 2-cm disks obtains R2 values up to 0.98 for big nymphs. This solution was embedded in a docker and can be accessed through an app via REST service in mobile devices. It has been tested in the wild under real conditions in different locations worldwide and over 14 different crops.The authors would like to thank all field researchers that generated the dataset, carried out the annotation process, performed the validation in the wild, and in general, supported the work in Tecnalia and BASF specially to Javier Romero, Carlos Javier Jim ́enez, Amaia Ortiz, Aitor Alvarez and Jone Echazarra

Insect counting through deep learning-based density maps estimation

Digitalization and automation of assessments in field trials are established practice for farming product development. The use of image-based methods has provided good results in different applications. Although these models can leverage some problems, they still perform poorly under real field conditions using mobile devices on complex applications. Among these applications, insect counting and detection is necessary for integrated pest management strategies in order to apply specific treatments at early infection stages to reduce economic losses and minimize chemical usage. Currently the counting task for the assessment of the degree of infestation is done manually by the farmer. Current state of the art object counting methods do not provide accurate counting in crowded images with overlapped or touching objects which is the case for insect counting images. This makes necessary to define novel approaches for insect counting. In this work, we propose a novel solution based on deep learning density map estimation to tackle insects counting in wild conditions. To this end, a fully convolutional regression network has been designed to accurately estimate a probabilistic density map for the counting regression problem. The estimated density map is then used for counting whiteflies in eggplant leaves. The proposed method was compared with a baseline based on candidate object selection and classification approach. The results for alive adult whitefly counting by means of density map estimation provided R2 = 0.97 for the counted insects in the main leaf of the image, that outperforms by far the baseline algorithm (R2 = 0.85) based on image processing methods for feature extraction and candidate selection and deep learning-based classifier. This solution was embedded to be used in mobile devices, and it has been gone for exhaustive validation tests, with diverse illumination conditions and background variability, over leaves taken at different heights, with different perspectives and even unfocused images, for the analyzed pest under real conditions