Evaluation of low-cost Raspberry Pi sensors for structure-from-motion reconstructions of glacier calving fronts

Glacier calving fronts are highly dynamic environments that are becoming ubiquitous as glaciers recede and, in many cases, develop proglacial lakes. Monitoring of calving fronts is necessary to fully quantify the glacier ablation budget and to warn nearby communities of the threat of hazards, such as glacial lake outburst floods (GLOFs), tsunami waves, and iceberg collapses. Time-lapse camera arrays, with structure-from-motion photogrammetry, can produce regular 3D models of glaciers to monitor changes in the ice but are seldom incorporated into monitoring systems owing to the high cost of equipment. In this proof-of-concept study at Fjallsjökull, Iceland, we present and test a low-cost, highly adaptable camera system based on Raspberry Pi computers and compare the resulting point cloud data to a reference cloud generated using an unoccupied aerial vehicle (UAV). The mean absolute difference between the Raspberry Pi and UAV point clouds is found to be 0.301 m with a standard deviation of 0.738 m. We find that high-resolution point clouds can be robustly generated from cameras positioned up to 1.5 km from the glacier (mean absolute difference 0.341 m, standard deviation 0.742 m). Combined, these experiments suggest that for monitoring calving events in glaciers, Raspberry Pi cameras are an affordable, flexible, and practical option for future scientific research. Owing to the connectivity capabilities of Raspberry Pi computers, this opens the possibility for real-time structure-from-motion reconstructions of glacier calving fronts for deployment as an early warning system to calving-triggered GLOFs.</p

Standardized spectral and radiometric calibration of consumer cameras

Consumer cameras, particularly onboard smartphones and UAVs, are now commonly used as scientific instruments. However, their data processing pipelines are not optimized for quantitative radiometry and their calibration is more complex than that of scientific cameras. The lack of a standardized calibration methodology limits the interoperability between devices and, in the ever-changing market, ultimately the lifespan of projects using them. We present a standardized methodology and database (SPECTACLE) for spectral and radiometric calibrations of consumer cameras, including linearity, bias variations, read-out noise, dark current, ISO speed and gain, flat-field, and RGB spectral response. This includes golden standard ground-truth methods and do-it-yourself methods suitable for non-experts. Applying this methodology to seven popular cameras, we found high linearity in RAW but not JPEG data, inter-pixel gain variations >400% correlated with large-scale bias and read-out noise patterns, non-trivial ISO speed normalization functions, flat-field correction factors varying by up to 2.79 over the field of view, and both similarities and differences in spectral response. Moreover, these results differed wildly between camera models, highlighting the importance of standardization and a centralized database

Development of an accurate low cost NDVI imaging system for assessing plant health.

BACKGROUND: Spectral imaging is a key method for high throughput phenotyping that can be related to a large variety of biological parameters. The Normalised Difference Vegetation Index (NDVI), uses specific wavelengths to compare crop health and performance. Increasing the accessibility of spectral imaging systems through the development of small, low cost, and easy to use platforms will generalise its use for precision agriculture. We describe a method for using a dual camera system connected to a Raspberry Pi to produce NDVI imagery, referred to as NDVIpi. Spectral reference targets were used to calibrate images into values of reflectance, that are then used to calculated NDVI with improved accuracy compared with systems that use single references/standards. RESULTS: NDVIpi imagery showed strong performance against standard spectrometry, as an accurate measurement of leaf NDVI. The NDVIpi was also compared to a relatively more expensive commercial camera (Micasense RedEdge), with both cameras having a comparable performance in measuring NDVI. There were differences between the NDVI values of the NDVIpi and the RedEdge, which could be attributed to the measurement of different wavelengths for use in the NDVI calculation by each camera. Subsequently, the wavelengths used by the NDVIpi show greater sensitivity to changes in chlorophyll content than the RedEdge. CONCLUSION: We present a methodology for a Raspberry Pi based NDVI imaging system that utilizes low cost, off-the-shelf components, and a robust multi-reference calibration protocols that provides accurate NDVI measurements. When compared with a commercial system, comparable NDVI values were obtained, despite the fact that our system was a fraction of the cost. Our results also highlight the importance of the choice of red wavelengths in the calculation of NDVI, which resulted in differences in sensitivity between camera systems

Using a new generation of remote sensing to monitor Peru’s mountain glaciers

Remote sensing technologies are integral to monitoring mountain glaciers in a warming world. Tropical glaciers, of which around 70% are located in Peru, are particularly at risk as a result of climate warming. Satellite missions and field-based platforms have transformed understanding of the processes driving mountain glacier dynamics and the associated emergence of hazards (e.g. avalanches, floods, landslides), yet are seldom specialised to overcome the unique challenges of acquiring data in mountainous environments. A ‘new generation’ of remote sensing, marked by open access to powerful cloud computing and large datasets, high resolution satellite missions, and low-cost science-grade field sensors, looks to revolutionise the way we monitor the mountain cryosphere. In this thesis, three novel remote sensing techniques and their applicability towards monitoring the glaciers of the Peruvian Cordillera Vilcanota are examined. Using novel processing chains and image archives generated by the ASTER satellite, the first mass balance estimate of the Cordillera Vilcanota is calculated (-0.48 ± 0.07 m w.e. yr-1) and ELA change of up to 32.8 m per decade in the neighbouring Cordillera Vilcabamba is quantified. The performance of new satellite altimetry missions, Sentinel-3 and ICESat-2, are assessed, with the tracking mode of Sentinel-3 being a key limitation of the potential for its use over mountain environments. Although currently limited in its ability to extract widespread mass balance measurements over mountain glaciers, other applications for ICESat-2 in long-term monitoring of mountain glaciers include quantifying surface elevation change, identifying large accumulation events, and monitoring lake bathymetry. Finally, a novel low-cost method of performing timelapse photogrammetry using Raspberry Pi camera sensors is created and compared to 3D models generated by a UAV. Mean difference between the Raspberry Pi and UAV sensors is 0.31 ± 0.74 m, giving promise to the use of these sensors for long-term monitoring of recession and short-term warning of hazards at glacier calving fronts. Together, this ‘new generation’ of remote sensing looks to provide new glaciological insights and opportunities for regular monitoring of data-scarce mountainous regions. The techniques discussed in this thesis could benefit communities and societal programmes in rapidly deglaciating environments, including across the Cordillera Vilcanota

Just-in-time Pastureland Trait Estimation for Silage Optimization, under Limited Data Constraints

To ensure that pasture-based farming meets production and environmental targets for a growing population under increasing resource constraints, producers need to know pastureland traits. Current proximal pastureland trait prediction methods largely rely on vegetation indices to determine biomass and moisture content. The development of new techniques relies on the challenging task of collecting labelled pastureland data, leading to small datasets. Classical computer vision has already been applied to weed identification and recognition of fruit blemishes using morphological features, but machine learning algorithms can parameterise models without the provision of explicit features, and deep learning can extract even more abstract knowledge although typically this is assumed to be based around very large datasets. This work hypothesises that through the advantages of state-of-the-art deep learning systems, pastureland crop traits can be accurately assessed in a just-in-time fashion, based on data retrieved from an inexpensive sensor platform, under the constraint of limited amounts of labelled data. However the challenges to achieve this overall goal are great, and for applications such as just-in-time yield and moisture estimation for farm-machinery, this work must bring together systems development, knowledge of good pastureland practice, and also techniques for handling low-volume datasets in a machine learning context. Given these challenges, this thesis makes a number of contributions. The first of these is a comprehensive literature review, relating pastureland traits to ruminant nutrient requirements and exploring trait estimation methods, from contact to remote sensing methods, including details of vegetation indices and the sensors and techniques required to use them. The second major contribution is a high-level specification of a platform for collecting and labelling pastureland data. This includes the collection of four-channel Blue, Green, Red and NIR (VISNIR) images, narrowband data, height and temperature differential, using inexpensive proximal sensors and provides a basis for holistic data analysis. Physical data platforms built around this specification were created to collect and label pastureland data, involving computer scientists, agricultural, mechanical and electronic engineers, and biologists from academia and industry, working with farmers. Using the developed platform and a set of protocols for data collection, a further contribution of this work was the collection of a multi-sensor multimodal dataset for pastureland properties. This was made up of four-channel image data, height data, thermal data, Global Positioning System (GPS) and hyperspectral data, and is available and labelled with biomass (Kg/Ha) and percentage dry matter, ready for use in deep learning. However, the most notable contribution of this work was a systematic investigation of various machine learning methods applied to the collected data in order to maximise model performance under the constraints indicated above. The initial set of models focused on collected hyperspectral datasets. However, due to their relative complexity in real-time deployment, the focus was instead on models that could best leverage image data. The main body of these models centred on image processing methods and, in particular, the use of the so-called Inception Resnet and MobileNet models to predict fresh biomass and percentage dry matter, enhancing performance using data fusion, transfer learning and multi-task learning. Images were subdivided to augment the dataset, using two different patch sizes, resulting in around 10,000 small patches of size 156 x 156 pixels and around 5,000 large patches of size 240 x 240 pixels. Five-fold cross validation was used in all analysis. Prediction accuracy was compared to older mechanisms, albeit using hyperspectral data collected, with no provision made for lighting, humidity or temperature. Hyperspectral labelled data did not produce accurate results when used to calculate Normalized Difference Vegetation Index (NDVI), or to train a neural network (NN), a 1D Convolutional Neural Network (CNN) or Long Short Term Memory (LSTM) models. Potential reasons for this are discussed, including issues around the use of highly sensitive devices in uncontrolled environments. The most accurate prediction came from a multi-modal hybrid model that concatenated output from an Inception ResNet based model, run on RGB data with ImageNet pre-trained RGB weights, output from a residual network trained on NIR data, and LiDAR height data, before fully connected layers, using the small patch dataset with a minimum validation MAPE of 28.23% for fresh biomass and 11.43% for dryness. However, a very similar prediction accuracy resulted from a model that omitted NIR data, thus requiring fewer sensors and training resources, making it more sustainable. Although NIR and temperature differential data were collected and used for analysis, neither improved prediction accuracy, with the Inception ResNet model’s minimum validation MAPE rising to 39.42% when NIR data was added. When both NIR data and temperature differential were added to a multi-task learning Inception ResNet model, it yielded a minimum validation MAPE of 33.32%. As more labelled data are collected, the models can be further trained, enabling sensors on mowers to collect data and give timely trait information to farmers. This technology is also transferable to other crops. Overall, this work should provide a valuable contribution to the smart agriculture research space

Engineering a Low-Cost Remote Sensing Capability for Deep-Space Applications

Systems engineering (SE) has been a useful tool for providing objective processes to breaking down complex technical problems to simpler tasks, while concurrently generating metrics to provide assurance that the solution is fit-for-purpose. Tailored forms of SE have also been used by cubesat mission designers to assist in reducing risk by providing iterative feedback and key artifacts to provide managers with the evidence to adjust resources and tasking for success. Cubesat-sized spacecraft are being planned, built and in some cases, flown to provide a lower-cost entry point for deep-space exploration. This is particularly important for agencies and countries with lower space exploration budgets, where specific mission objectives can be used to develop tailored payloads within tighter constraints, while also returning useful scientific results or engineering data. In this work, a tailored SE tradespace approach was used to help determine how a 6 unit (6U) cubesat could be built from commercial-off-the-shelf (COTS)-based components and undertake remote sensing missions near Mars or near-Earth Asteroids. The primary purpose of these missions is to carry a hyperspectral sensor sensitive to 600-800nm wavelengths (hereafter defined as “red-edge”), that will investigate mineralogy characteristics commonly associated with oxidizing and hydrating environments in red-edge. Minerals of this type remain of high interest for indicators of present or past habitability for life, or active geologic processes. Implications of operating in a deep-space environment were considered as part of engineering constraints of the design, including potential reduction of available solar energy, changes in thermal environment and background radiation, and vastly increased communications distances. The engineering tradespace analysis identified realistic COTS options that could satisfy mission objectives for the 6U cubesat bus while also accommodating a reasonable degree of risk. The exception was the communication subsystem, in which case suitable capability was restricted to one particular option. This analysis was used to support an additional trade investigation into the type of sensors that would be most suitable for building the red-edge hyperspectral payload. This was in part constrained by ensuring not only that readily available COTS sensors were used, but that affordability, particularly during a geopolitical environment that was affecting component supply surety and access to manufacturing facilities, was optimized. It was found that a number of sensor options were available for designing a useful instrument, although the rapid development and life-of-type issues with COTS sensors restricted the ability to obtain useful metrics on their performance in the space environment. Additional engineering testing was conducted by constructing hyperspectral sensors using sensors popular in science, technology, engineering and mathematics (STEM) contexts. Engineering and performance metrics of the payload containing the sensors was conducted; and performance of these sensors in relevant analogous environments. A selection of materials exhibiting spectral phenomenology in the red-edge portion of the spectrum was used to produce metrics on the performance of the sensors. It was found that low-cost cameras were able to distinguish between most minerals, although they required a wider spectral range to do so. Additionally, while Raspberry Pi cameras have been popular with scientific applications, a low-cost camera without a Bayer filter markedly improved spectral sensitivity. Consideration for space-environment testing was also trialed in additional experiments using high-altitude balloons to reach the near-space environment. The sensor payloads experienced conditions approximating the surface of Mars, and results were compared with Landsat 7, a heritage Earth sensing satellite, using a popular vegetation index. The selected Raspberry Pi cameras were able to provide useful results from near-space that could be compared with space imagery. Further testing incorporated comparative analysis of custom-built sensors using readily available Raspberry Pi and astronomy cameras, and results from Mastcam and Mastcam/z instruments currently on the surface of Mars. Two sensor designs were trialed in field settings possessing Mars-analogue materials, and a subset of these materials were analysed using a laboratory-grade spectro-radiometer. Results showed the Raspberry Pi multispectral camera would be best suited for broad-scale indications of mineralogy that could be targeted by the pushbroom sensor. This sensor was found to possess a narrower spectral range than the Mastcam and Mastcam/z but was sensitive to a greater number of bands within this range. The pushbroom sensor returned data on spectral phenomenology associated with attributes of Minerals of the type found on Mars. The actual performance of the payload in appropriate conditions was important to provide critical information used to risk reduce future designs. Additionally, the successful outcomes of the trials reduced risk for their application in a deep space environment. The SE and practical performance testing conducted in this thesis could be developed further to design, build and fly a hyperspectral sensor, sensitive to red-edge wavelengths, on a deep-space cubesat mission. Such a mission could be flown at reasonable cost yet return useful scientific and engineering data

Automated license plate recognition for resource-constrained environments

The incorporation of deep-learning techniques in embedded systems has enhanced the capabilities of edge computing to a great extent. However, most of these solutions rely on high-end hardware and often require a high processing capacity, which cannot be achieved with resource-constrained edge computing. This study presents a novel approach and a proof of concept for a hardware-efficient automated license plate recognition system for a constrained environment with limited resources. The proposed solution is purely implemented for low-resource edge devices and performed well for extreme illumination changes such as day and nighttime. The generalisability of the proposed models has been achieved using a novel set of neural networks for different hardware configurations based on the computational capabilities and low cost. The accuracy, energy efficiency, communication, and computational latency of the proposed models are validated using different license plate datasets in the daytime and nighttime and in real time. Meanwhile, the results obtained from the proposed study have shown competitive performance to the state-of-the-art server-grade hardware solutions as well

Use of a Raspberry-Pi Video Camera for Coastal Flooding Vulnerability Assessment: The Case of Riccione (Italy)

Coastal monitoring is strategic for the correct assessment of nearshore morphodynamics, to verify the effects of anthropogenic interventions for the purpose of coastal protection and for the rapid assessment of flooding vulnerability due to severe events. Remote sensing and field surveys are among the main approaches that have been developed to meet these necessities. Key parameters in the assessment and prevision of coastal flooding extensions, beside meteomarine characteristics, are the topography and slope of beaches, which can be extremely dynamic. The use of continuous monitoring through orthorectified video images allows for the rapid detection of the intertidal bathymetry and flooding threshold during severe events. The aim of this work was to present a comparison of different monitoring strategies and methodologies that have been integrated into repeated surveys in order to evaluate the performance of a new camera system. We used a low-cost camera based on Raspberry Pi called VISTAE (Video monitoring Intelligent STAtion for Environmental applications) for long-term remote observations and GNSS-laser tools for field measurements. The case study was a coastal tract in Riccione, Italy (Northern Adriatic Sea), which is the seat of nourishment interventions and of different types of underwater protection structures to combat coastal erosion. We performed data acquisition and analysis of the emerged beach and of the swash zone in terms of the intertidal bathymetry and shoreline. The results show a generally good agreement between the field and remote measurements through image processing, with a small discrepancy of the order of ≈0.05 m in the vertical and ≈1.5 m in the horizontal in terms of the root mean square error (RMSE). These values are comparable with that of current video monitoring instruments, but the VISTAE has the advantages of its low-cost, programmability and automatized analyses. This result, together with the possibility of continuous monitoring during daylight hours, supports the advantages of a combined approach in coastal flooding vulnerability assessment through integrated and complementary techniques

Application of CMOS sensors in biology

S vývojem CMOS technologie dochází ke stálému zmenšování velikosti pixelu CMOS obrazových senzorů a ke snižování ceny senzoru, což umožňuje řadu nových využití v oblasti biomedicinského zobrazování. Díky těmto pokrokům dosahují metody bezčočkového zobrazování dostatečného rozlišení pro jejich aplikace namísto klasických optických systémů, s výhodami nižší ceny, zobrazování s rozlišením hloubky, většího zorného pole a vysoké přizpůsobitelnosti. Tato práce představuje řadu bezčočkových zobrazovacích metod a jejich aplikací, popisuje základní teorii holografie, metod holografické rekonstrukce a popisuje postupný návrh bezčočkového digitálního holografického mikroskopu v on-chip konfiguraci.ObhájenoWith the advances in CMOS technology, the pixel size of CMOS image sensors is getting smaller and the sensor price lower, allowing for many applications in biomedical imaging. Following these advances, lensless imaging techniques are reaching sufficient resolution capabilities that enable their use instead of classical lens-based optical systems with the advantage of lower cost, dept-resolved imaging, large Field-of-View and high adaptability. This thesis introduces various lensless imaging methods and their applications, describes the basic holographic theory, reconstruction methods and step-by-step design of a digital lensless holographic microscope in an on-chip configuration