TractorEYE: Vision-based Real-time Detection for Autonomous Vehicles in Agriculture

Agricultural vehicles such as tractors and harvesters have for decades been able to navigate automatically and more efficiently using commercially available products such as auto-steering and tractor-guidance systems. However, a human operator is still required inside the vehicle to ensure the safety of vehicle and especially surroundings such as humans and animals. To get fully autonomous vehicles certified for farming, computer vision algorithms and sensor technologies must detect obstacles with equivalent or better than human-level performance. Furthermore, detections must run in real-time to allow vehicles to actuate and avoid collision.This thesis proposes a detection system (TractorEYE), a dataset (FieldSAFE), and procedures to fuse information from multiple sensor technologies to improve detection of obstacles and to generate a map. TractorEYE is a multi-sensor detection system for autonomous vehicles in agriculture. The multi-sensor system consists of three hardware synchronized and registered sensors (stereo camera, thermal camera and multi-beam lidar) mounted on/in a ruggedized and water-resistant casing. Algorithms have been developed to run a total of six detection algorithms (four for rgb camera, one for thermal camera and one for a Multi-beam lidar) and fuse detection information in a common format using either 3D positions or Inverse Sensor Models. A GPU powered computational platform is able to run detection algorithms online. For the rgb camera, a deep learning algorithm is proposed DeepAnomaly to perform real-time anomaly detection of distant, heavy occluded and unknown obstacles in agriculture. DeepAnomaly is -- compared to a state-of-the-art object detector Faster R-CNN -- for an agricultural use-case able to detect humans better and at longer ranges (45-90m) using a smaller memory footprint and 7.3-times faster processing. Low memory footprint and fast processing makes DeepAnomaly suitable for real-time applications running on an embedded GPU. FieldSAFE is a multi-modal dataset for detection of static and moving obstacles in agriculture. The dataset includes synchronized recordings from a rgb camera, stereo camera, thermal camera, 360-degree camera, lidar and radar. Precise localization and pose is provided using IMU and GPS. Ground truth of static and moving obstacles (humans, mannequin dolls, barrels, buildings, vehicles, and vegetation) are available as an annotated orthophoto and GPS coordinates for moving obstacles. Detection information from multiple detection algorithms and sensors are fused into a map using Inverse Sensor Models and occupancy grid maps. This thesis presented many scientific contribution and state-of-the-art within perception for autonomous tractors; this includes a dataset, sensor platform, detection algorithms and procedures to perform multi-sensor fusion. Furthermore, important engineering contributions to autonomous farming vehicles are presented such as easily applicable, open-source software packages and algorithms that have been demonstrated in an end-to-end real-time detection system. The contributions of this thesis have demonstrated, addressed and solved critical issues to utilize camera-based perception systems that are essential to make autonomous vehicles in agriculture a reality

Monitoring, modelling and quantification of accumulation of damage on masonry structures due to recursive loads

The use of induced seismicity is gaining in popularity, particularly in Northern Europe, as people strive to increase local energy supplies. Τhe local building stock, comprising mainly of low-rise domestic masonry structures without any aseismic design, has been found susceptible to these induced tremors. Induced seismicity is generally characterized by frequent small-to-medium magnitude earthquakes in which structural and non-structural damage have been reported. Since the induced earthquakes are caused by third parties liability issues arise and a damage claim mechanism is activated. Typically, any damage are evaluated by visual inspections. This damage assessment process has been found rather cumbersome since visual inspections are laborious, slow and expensive while the identification of the cause of any light damage is a challenging task rendering essential the development of a more reliable approach. The aim of this PhD study is to gain a better understanding of the monitoring, modelling and quantification of accumulation of damage in masonry structures due to recursive loads. Fraeylemaborg, the most emblematic monument in the Groningen region dating back to the 14 th century, has experienced damage due to the induced seismic activity in the region in recent years. A novel monitoring approach is proposed to detect damage accumulation due to induced seismicity on the monument. Results of the monitoring, in particular the monitoring of the effects of induced seismic activity,, as well as the usefulness and need of various monitoring data for similar cases are discussed. A numerical model is developed and calibrated based on experimental findings and different loading scenarios are compared with the actual damage patterns observed on the structure. Vision-based techniques are developed for the detection of damage accumulation in masonry structures in an attempt to enhance effectiveness of the inspection process. In particular, an artificial intelligence solution is proposed for the automatic detection of cracks on masonry structures. A dataset with photographs from masonry structures is produced containing complex backgrounds and various crack types and sizes. Moreover, different convolutional neural networks are evaluated on their efficacy to automatically detect cracks. Furthermore, computer vision and photogrammetry methods are considered along with novel invisible markers for monitoring cracks. The proposed method shifts the marker reflection and its contrast with the background into the invisible wavelength of light (i.e. to the near-infrared) so that the markers are not easily distinguishable. The method is thus particularly vi suitable for monitoring historical buildings where it is important to avoid any interventions or disruption to the authenticity of the basic fabric of construction.. Further on, the quantification and modelling of damage in masonry structures are attempted by taking into consideration the initiation and propagation of damage due to earthquake excitations. The evaluation of damage in masonry structures due to (induced) earthquakes represents a challenging task. Cumulative damage due to subsequent ground motions is expected to have an effect on the seismic capacity of a structure. Crack patterns obtained from experimental campaigns from the literature are investigated and their correlation with damage propagation is examined. Discontinuous modelling techniques are able to reliably reproduce damage initiation and propagation by accounting for residual cracks even for low intensity loading. Detailed models based on the Distinct Element Method and Finite Element Model analysis are considered to capture and quantify the cumulative damage in micro level in masonry subjected to seismic loads. Finally, an experimental campaign is undertaken to investigate the accumulation of damage in masonry structure under repetitive load. Six wall specimens resembling the configuration of a spandrel element are tested under three-point in-plane bending considering different loading protocols. The walls were prepared adopting materials and practices followed in the Groningen region. Different numerical approaches are researched for their efficacy to reproduce the experimental response and any limitations are highlighted

Remote Sensing of the Aquatic Environments

The book highlights recent research efforts in the monitoring of aquatic districts with remote sensing observations and proximal sensing technology integrated with laboratory measurements. Optical satellite imagery gathered at spatial resolutions down to few meters has been used for quantitative estimations of harmful algal bloom extent and Chl-a mapping, as well as winds and currents from SAR acquisitions. The knowledge and understanding gained from this book can be used for the sustainable management of bodies of water across our planet

Texture and Colour in Image Analysis

Research in colour and texture has experienced major changes in the last few years. This book presents some recent advances in the field, specifically in the theory and applications of colour texture analysis. This volume also features benchmarks, comparative evaluations and reviews

Integrated Applications of Geo-Information in Environmental Monitoring

This book focuses on fundamental and applied research on geo-information technology, notably optical and radar remote sensing and algorithm improvements, and their applications in environmental monitoring. This Special Issue presents ten high-quality research papers covering up-to-date research in land cover change and desertification analyses, geo-disaster risk and damage evaluation, mining area restoration assessments, the improvement and development of algorithms, and coastal environmental monitoring and object targeting. The purpose of this Special Issue is to promote exchanges, communications and share the research outcomes of scientists worldwide and to bridge the gap between scientific research and its applications for advancing and improving society

Sustainable Agriculture and Advances of Remote Sensing (Volume 1)

Agriculture, as the main source of alimentation and the most important economic activity globally, is being affected by the impacts of climate change. To maintain and increase our global food system production, to reduce biodiversity loss and preserve our natural ecosystem, new practices and technologies are required. This book focuses on the latest advances in remote sensing technology and agricultural engineering leading to the sustainable agriculture practices. Earth observation data, in situ and proxy-remote sensing data are the main source of information for monitoring and analyzing agriculture activities. Particular attention is given to earth observation satellites and the Internet of Things for data collection, to multispectral and hyperspectral data analysis using machine learning and deep learning, to WebGIS and the Internet of Things for sharing and publishing the results, among others

Improving Urban Cooling in the Semi-arid Phoenix Metropolis: Land System Science, Landscape Ecology and Urban Climatology Approaches

abstract: The global increase in urbanization has raised questions about urban sustainability to which multiple research communities have entered. Those communities addressing interest in the urban heat island (UHI) effect and extreme temperatures include land system science, urban/landscape ecology, and urban climatology. General investigations of UHI have focused primarily on land surface and canopy layer air temperatures. The surface temperature is of prime importance to UHI studies because of its central rule in the surface energy balance, direct effects on air temperature, and outdoor thermal comfort. Focusing on the diurnal surface temperature variations in Phoenix, Arizona, especially on the cool (green space) island effect and the surface heat island effect, the dissertation develops three research papers that improve the integration among the abovementioned sub-fields. Specifically, these papers involve: (1) the quantification and modeling of the diurnal cooling benefits of green space; (2) the optimization of green space locations to reduce the surface heat island effect in daytime and nighttime; and, (3) an evaluation of the effects of vertical urban forms on land surface temperature using Google Street View. These works demonstrate that the pattern of new green spaces in central Phoenix could be optimized such that 96% of the maximum daytime and nighttime cooling benefits would be achieved, and that Google Street View data offers an alternative to other data, providing the vertical dimensions of land-cover for addressing surface temperature impacts, increasing the model accuracy over the use of horizontal land-cover data alone. Taken together, the dissertation points the way towards the integration of research directions to better understand the consequences of detailed land conditions on temperatures in urban areas, providing insights for urban designs to alleviate these extremes.Dissertation/ThesisDoctoral Dissertation Geography 201