Convolutional Neural Network on Three Orthogonal Planes for Dynamic Texture Classification

Dynamic Textures (DTs) are sequences of images of moving scenes that exhibit certain stationarity properties in time such as smoke, vegetation and fire. The analysis of DT is important for recognition, segmentation, synthesis or retrieval for a range of applications including surveillance, medical imaging and remote sensing. Deep learning methods have shown impressive results and are now the new state of the art for a wide range of computer vision tasks including image and video recognition and segmentation. In particular, Convolutional Neural Networks (CNNs) have recently proven to be well suited for texture analysis with a design similar to a filter bank approach. In this paper, we develop a new approach to DT analysis based on a CNN method applied on three orthogonal planes x y , xt and y t . We train CNNs on spatial frames and temporal slices extracted from the DT sequences and combine their outputs to obtain a competitive DT classifier. Our results on a wide range of commonly used DT classification benchmark datasets prove the robustness of our approach. Significant improvement of the state of the art is shown on the larger datasets.Comment: 19 pages, 10 figure

Improving Transfer Learning for Use in Multi-Spectral Data

Recently Nasa as well as the European Space Agency have made observational satellites images public. The main reason behind opening it to public is to foster research among university students and corporations alike. Sentinel is a program by the European Space Agency which has plans to release a series of seven satellites in lower earth orbit for observing land and sea patterns. Recently huge datasets have been made public by the Sentinel program. Many advancements have been made in the field of computer vision in the last decade. Krizhevsky, Sutskever & Hinton, 2012, revolutionized the field of image analysis by training deep neural nets and introduced the idea of using convolutions to obtain a high accuracy value on coloured image dataset of more than one million images known as Imagenet ILSVRC. Convolutional Neural Network, or CNN architecture has undergone much improvement since then. One CNN model known as Resnet or Residual Network architecture (He, Zhang, Ren & Sun, 2015) has seen mass acceptance in particular owing to it processing speed and high accuracy. Resnet is widely used for applying features it learned in Imagenet ILSVRC tasks into other image classification or object detection tasks. This concept, in the domain of deep learning, is known as Transfer learning, where a classifier is trained on a bigger more complex task and then learning is transferred to a smaller, more specific task. Transfer learning can often lead to good performance on new smaller tasks and this approach has given state of the art results in several problem domains of image classification and even in object detection (Dai, Li, He, & Sun, 2016). The real problem is that not all the problems in computer vision field belongs to regular RGB images or images consisting of only Red, Green, and Blue band set. For example, a field like medical image analysis has most of the images belonging to greyscale color space, while most of the Remote sensing images collected by satellites belong to multispectral bands of light. Transferring features learned from Imagenet ILSVRC tasks to these fields might give you higher accuracy than training from scratch, but it is a problem of fundamentally incorrect approach. Thus, there is a need to create network models that can learn from single channel or multispectral images iv and can transfer features seamlessly to similar domains with smaller datasets.This thesis presents a study in multispectral image analysis using multiple ways of feature transfer. In this study, Transfer Learning of features is done using a Resnet50 model which is trained on RGB images, and another Resnet50 model which is trained on Greyscale images alone. The dataset used to pretrain these models is a combination of images from ImageNet (Deng, Dong, Socher, Li, Li, & Fei-Fei, 2009) and Eurosat (Helber, Bischke, Dengel, & Borth. 2017). The idea behind choosing Resnet50 is that it has been doing really well in image processing and transfer learning and has outperformed all the other traditional techniques, while still not being computationally prohibitive to train in the context of this work. An attempt is made to classify different land-cover classes in multispectral images taken up by Sentinel 2A satellite. The dataset used here has a key challenge of a smaller number of samples, which means a CNN classifier trained from scratch on these small number of samples will be highly inaccurate and overfitted. This thesis focuses on improving the accuracies of this classifier using transfer learning, and the performance is measured after fine-tuning the baseline above Resnet50 model. The experiment results show that fine-tuning the Greyscale or single channel based Resnet50 model helps in improving the accuracy a bit more than using a RGB trained Resnet50 model for fine tuning, though it haven\u27t achieved great result due to the limitation of lesser computational power and smaller dataset to train a large computer vision network like Resnet50. This work is a contribution towards improving classification in domain of multispectral images usually taken up by satellites. There is no baseline model available right now, which can be used to transfer features to single or multispectral domains like the rest of RGB image field has. The contribution of this work is to build such a classifier for multispectral domain and to extend the state of the art in such computer vision domains

Rapid and low-cost insect detection for analysing species trapped on yellow sticky traps

While insect monitoring is a prerequisite for precise decision-making regarding integrated pest management (IPM), it is time- and cost-intensive. Low-cost, time-saving and easy-to-operate tools for automated monitoring will therefore play a key role in increased acceptance and application of IPM in practice. In this study, we tested the differentiation of two whitefly species and their natural enemies trapped on yellow sticky traps (YSTs) via image processing approaches under practical conditions. Using the bag of visual words (BoVW) algorithm, accurate differentiation between both natural enemies and the Trialeurodes vaporariorum and Bemisia tabaci species was possible, whereas the procedure for B. tabaci could not be used to differentiate this species from T. vaporariorum. The decay of species was considered using fresh and aged catches of all the species on the YSTs, and different pooling scenarios were applied to enhance model performance. The best performance was reached when fresh and aged individuals were used together and the whitefly species were pooled into one category for model training. With an independent dataset consisting of photos from the YSTs that were placed in greenhouses and consequently with a naturally occurring species mixture as the background, a differentiation rate of more than 85% was reached for natural enemies and whiteflies

Learning visual representations of style

Learning Visual Representations of Style Door Nanne van Noord De stijl van een kunstenaar is zichtbaar in zijn/haar werk, onafhankelijk van de vorm of het onderwerp van een kunstwerk kunnen kunstexperts deze stijl herkennen. Of het nu om een landschap of een portret gaat, het connaisseurschap van kunstexperts stelt hen in staat om de stijl van de kunstenaar te herkennen. Het vertalen van dit vermogen tot connaisseurschap naar een computer, zodat de computer in staat is om de stijl van een kunstenaar te herkennen, en om kunstwerken te (re)produceren in de stijl van de kunstenaar, staat centraal in dit onderzoek. Voor visuele analyseren van kunstwerken maken computers gebruik van beeldverwerkingstechnieken. Traditioneel gesproken bestaan deze technieken uit door computerwetenschappers ontwikkelde algoritmes die vooraf gedefinieerde visuele kernmerken kunnen herkennen. Omdat deze kenmerken zijn ontwikkelt voor de analyse van de inhoud van foto’s zijn ze beperkt toepasbaar voor de analyse van de stijl van visuele kunst. Daarnaast is er ook geen definitief antwoord welke visuele kenmerken indicatief zijn voor stijl. Om deze beperkingen te overkomen maken we in dit onderzoek gebruik van Deep Learning, een methodologie die het beeldverwerking onderzoeksveld in de laatste jaren enorm heeft gerevolutionaliseerd. De kracht van Deep Learning komt voort uit het zelflerende vermogen, in plaats van dat we afhankelijk zijn van vooraf gedefinieerde kenmerken, kan de computer zelf leren wat de juiste kenmerken zijn. In dit onderzoek hebben we algoritmes ontwikkelt met het doel om het voor de computer mogelijk te maken om 1) zelf te leren om de stijl van een kunstenaar te herkennen, en 2) nieuwe afbeeldingen te genereren in de stijl van een kunstenaar. Op basis van het in het proefschrift gepresenteerde werk kunnen we concluderen dat de computer inderdaad in staat is om te leren om de stijl van een kunstenaar te herkennen, ook in een uitdagende setting met duizenden kunstwerken en enkele honderden kunstenaars. Daarnaast kunnen we concluderen dat het mogelijk is om, op basis van bestaande kunstwerken, nieuwe kunstwerken te generen in de stijl van de kunstenaar. Namelijk, een kleurloze afbeeldingen van een kunstwerk kan ingekleurd worden in de stijl van de kunstenaar, en wanneer er delen missen uit een kunstwerk is het mogelijk om deze missende stukken in te vullen (te retoucheren). Alhoewel we nog niet in staat zijn om volledig nieuwe kunstwerken te generen, is dit onderzoek een grote stap in die richting. Bovendien zijn de in dit onderzoek ontwikkelde technieken en methodes veelbelovend als digitale middelen ter ondersteuning van kunstexperts en restauratoren