Deep learning in food category recognition

Integrating artificial intelligence with food category recognition has been a field of interest for research for the past few decades. It is potentially one of the next steps in revolutionizing human interaction with food. The modern advent of big data and the development of data-oriented fields like deep learning have provided advancements in food category recognition. With increasing computational power and ever-larger food datasets, the approach’s potential has yet to be realized. This survey provides an overview of methods that can be applied to various food category recognition tasks, including detecting type, ingredients, quality, and quantity. We survey the core components for constructing a machine learning system for food category recognition, including datasets, data augmentation, hand-crafted feature extraction, and machine learning algorithms. We place a particular focus on the field of deep learning, including the utilization of convolutional neural networks, transfer learning, and semi-supervised learning. We provide an overview of relevant studies to promote further developments in food category recognition for research and industrial applicationsMRC (MC_PC_17171)Royal Society (RP202G0230)BHF (AA/18/3/34220)Hope Foundation for Cancer Research (RM60G0680)GCRF (P202PF11)Sino-UK Industrial Fund (RP202G0289)LIAS (P202ED10Data Science Enhancement Fund (P202RE237)Fight for Sight (24NN201);Sino-UK Education Fund (OP202006)BBSRC (RM32G0178B8

A review of technical factors to consider when designing neural networks for semantic segmentation of Earth Observation imagery

Semantic segmentation (classification) of Earth Observation imagery is a crucial task in remote sensing. This paper presents a comprehensive review of technical factors to consider when designing neural networks for this purpose. The review focuses on Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), and transformer models, discussing prominent design patterns for these ANN families and their implications for semantic segmentation. Common pre-processing techniques for ensuring optimal data preparation are also covered. These include methods for image normalization and chipping, as well as strategies for addressing data imbalance in training samples, and techniques for overcoming limited data, including augmentation techniques, transfer learning, and domain adaptation. By encompassing both the technical aspects of neural network design and the data-related considerations, this review provides researchers and practitioners with a comprehensive and up-to-date understanding of the factors involved in designing effective neural networks for semantic segmentation of Earth Observation imagery.Comment: 145 pages with 32 figure

Characterizing Dryland Ecosystems Using Remote Sensing and Dynamic Global Vegetation Modeling

Drylands include all terrestrial regions where the production of crops, forage, wood and other ecosystem services are limited by water. These ecosystems cover approximately 40% of the earth terrestrial surface and accommodate more than 2 billion people (Millennium Ecosystem Assessment, 2005). Moreover, the interannual variability of the global carbon budget is strongly regulated by vegetation dynamics in drylands. Understanding the dynamics of such ecosystems is significant for assessing the potential for and impacts of natural or anthropogenic disturbances and mitigation planning, and a necessary step toward enhancing the economic and social well-being of dryland communities in a sustainable manner (Global Drylands: A UN system-wide response, 2011). In this research, a combination of remote sensing, field data collection, and ecosystem modeling were used to establish an integrated framework for semi-arid ecosystems dynamics monitoring. Foliar nitrogen (N) plays an important role in vegetation processes such as photosynthesis and there is wide interest in retrieving this variable from hyperspectral remote sensing data. In this study, I used the theory of canopy spectral invariants (AKA p-theory) to understand the role of canopy structure and soil in the retrieval of foliar N from hyperspectral data and machine learning techniques. The results of this study showed the inconsistencies among different machine learning techniques used for estimating N. Using p-theory, I demonstrated that soil can contribute up to 95% to the total radiation budget of the canopy. I suggested an alternative approach to study photosynthesis is the use of dynamic global vegetation models (DGVMs). Gross primary production (GPP) is the apparent ecosystem scale photosynthesis that can be estimated using DGVMs. In this study, I performed a thorough sensitivity analysis and calibrated the Ecosystem Demography (EDv2.2) model along an elevation gradient in a dryland study area. I investigated the GPP capacity and activity by comparing the EDv2.2 GPP with flux towers and remote sensing products. The overall results showed that EDv2.2 performed well in capturing GPP capacity and its long term trend at lower elevation sites within the study area; whereas the model performed worse at higher elevations likely due to the change in vegetation community. I discussed that adding more heterogeneity and modifying ecosystem processes such as phenology and plant hydraulics in ED.v2.2 will improve its application to higher elevation ecosystems where there is more vegetation production. And finally, I developed an integrated hyperspectral-lidar framework for regional mapping of xeric and mesic vegetation in the study area. I showed that by considering spectral shape and magnitude, canopy structure and landscape features (riparian zone), we can develop a straightforward algorithm for vegetation mapping in drylands. This framework is simple, easy to interpret and consistent with our ecological understanding of vegetation distribution in drylands over large areas. Collectively, the results I present in this dissertation demonstrate the potential for advanced remote sensing and modeling to help us better understand ecosystem processes in drylands

Deep Vision in Optical Imagery: From Perception to Reasoning

Deep learning has achieved extraordinary success in a wide range of tasks in computer vision field over the past years. Remote sensing data present different properties as compared to natural images/videos, due to their unique imaging technique, shooting angle, etc. For instance, hyperspectral images usually have hundreds of spectral bands, offering additional information, and the size of objects (e.g., vehicles) in remote sensing images is quite limited, which brings challenges for detection or segmentation tasks. This thesis focuses on two kinds of remote sensing data, namely hyper/multi-spectral and high-resolution images, and explores several methods to try to find answers to the following questions: - In comparison with natural images or videos in computer vision, the unique asset of hyper/multi-spectral data is their rich spectral information. But what this “additional” information brings for learning a network? And how do we take full advantage of these spectral bands? - Remote sensing images at high resolution have pretty different characteristics, bringing challenges for several tasks, for example, small object segmentation. Can we devise tailored networks for such tasks? - Deep networks have produced stunning results in a variety of perception tasks, e.g., image classification, object detection, and semantic segmentation. While the capacity to reason about relations over space is vital for intelligent species. Can a network/module with the capacity of reasoning benefit to parsing remote sensing data? To this end, a couple of networks are devised to figure out what a network learns from hyperspectral images and how to efficiently use spectral bands. In addition, a multi-task learning network is investigated for the instance segmentation of vehicles from aerial images and videos. Finally, relational reasoning modules are designed to improve semantic segmentation of aerial images