A 4D Light-Field Dataset and CNN Architectures for Material Recognition

We introduce a new light-field dataset of materials, and take advantage of the recent success of deep learning to perform material recognition on the 4D light-field. Our dataset contains 12 material categories, each with 100 images taken with a Lytro Illum, from which we extract about 30,000 patches in total. To the best of our knowledge, this is the first mid-size dataset for light-field images. Our main goal is to investigate whether the additional information in a light-field (such as multiple sub-aperture views and view-dependent reflectance effects) can aid material recognition. Since recognition networks have not been trained on 4D images before, we propose and compare several novel CNN architectures to train on light-field images. In our experiments, the best performing CNN architecture achieves a 7% boost compared with 2D image classification (70% to 77%). These results constitute important baselines that can spur further research in the use of CNNs for light-field applications. Upon publication, our dataset also enables other novel applications of light-fields, including object detection, image segmentation and view interpolation.Comment: European Conference on Computer Vision (ECCV) 201

Rice crop classification and yield estimation using multi-temporal sentinel-2 data: a case study of Terai districts of Nepal

Dissertation submitted in partial fulfilment of the requirements for the degree of Master of Science in Geospatial TechnologiesCrop monitoring, especially in developing countries, can improve food production, address food security issues, and support sustainable development goals. Crop type mapping and yield estimation are the two major aspects of crop monitoring that remain challenging due to the problem of timely and adequate data availability. Existing approaches rely on ground-surveys and traditional means which are time-consuming and costly. In this context, we introduce the use of freely available Sentinel-2 (S2) imagery with high spatial, spectral and temporal resolution to classify crop and estimate its yield through a deep learning approach. In particular, this study uses patch-based 2D and 3D Convolutional Neural Network (CNN) algorithms to map rice crop and predict its yield in the Terai districts of Nepal. Firstly, the study reviews the existing state-of-art technologies in this field and selects suitable CNN architectures. Secondly, the selected architectures are implemented and trained using S2 imagery, groundtruth and auxiliary data in addition for yield estimation.We also introduce a variation in the chosen 3D CNN architecture to enhance its performance in estimating rice yield. The performance of the models is validated and then evaluated using performance metrics namely overall accuracy and F1-score for classification and Root Mean Squared Error (RMSE) for yield estimation. In consistency with the existing works, the results demonstrate recommendable performance of the models with remarkable accuracy, indicating the suitability of S2 data for crop mapping and yield estimation in developing countries. Reproducibility self-assessment (https://osf.io/j97zp/): 2, 2, 2, 1, 2 (input data, preprocessing, methods, computational environment, results)

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin