Path planning for automatic recharging system for steep-slope vineyard robots

Develop cost-effective ground robots for crop monitoring in steep slope vineyards is a complex challenge. The terrain presents harsh conditions for mobile robots and most of the time there is no one available to give support to the robots. So, a fully autonomous steep-slope robot requires a robust automatic recharging system. This work proposes a multilevel system that monitors a vineyard robot autonomy, to plan off-line the trajectory to the nearest recharging point and dock the robot on that recharging point considering visual tags. The proposed system called VineRecharge was developed to be deployed into a cost-effective robot with low computational power. Besides, this paper benchmarks several visual tags and detectors and integrates the best one into the VineRecharge system.This work is financed by the ERDF – European Regional Development Fund through the Operational Programme for Competitiveness and Internationalisation - COMPETE 2020 Programme within project “POCI-01-0145-FEDER-006961”, and by National Funds through the FCT – Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) as part of project UID/EEA/50014/2013.info:eu-repo/semantics/publishedVersio

Local Motion Planner for Autonomous Navigation in Vineyards with a RGB-D Camera-Based Algorithm and Deep Learning Synergy

With the advent of agriculture 3.0 and 4.0, researchers are increasingly focusing on the development of innovative smart farming and precision agriculture technologies by introducing automation and robotics into the agricultural processes. Autonomous agricultural field machines have been gaining significant attention from farmers and industries to reduce costs, human workload, and required resources. Nevertheless, achieving sufficient autonomous navigation capabilities requires the simultaneous cooperation of different processes; localization, mapping, and path planning are just some of the steps that aim at providing to the machine the right set of skills to operate in semi-structured and unstructured environments. In this context, this study presents a low-cost local motion planner for autonomous navigation in vineyards based only on an RGB-D camera, low range hardware, and a dual layer control algorithm. The first algorithm exploits the disparity map and its depth representation to generate a proportional control for the robotic platform. Concurrently, a second back-up algorithm, based on representations learning and resilient to illumination variations, can take control of the machine in case of a momentaneous failure of the first block. Moreover, due to the double nature of the system, after initial training of the deep learning model with an initial dataset, the strict synergy between the two algorithms opens the possibility of exploiting new automatically labeled data, coming from the field, to extend the existing model knowledge. The machine learning algorithm has been trained and tested, using transfer learning, with acquired images during different field surveys in the North region of Italy and then optimized for on-device inference with model pruning and quantization. Finally, the overall system has been validated with a customized robot platform in the relevant environment

Odometria visual monocular em robôs para a agricultura com camara(s) com lentes "olho de peixe"

One of the main challenges in robotics is to develop accurate localization methods that achieve acceptable runtime performances.One of the most common approaches is to use Global Navigation Satellite System such as GPS to localize robots.However, satellite signals are not full-time available in some kind of environments.The purpose of this dissertation is to develop a localization system for a ground robot.This robot is inserted in a project called RoMoVi and is intended to perform tasks like crop monitoring and harvesting in steep slope vineyards.This vineyards are localized in the Douro region which are characterized by the presence of high hills.Thus, the context of RoMoVi is not prosperous for the use of GPS-based localization systems.Therefore, the main goal of this work is to create a reliable localization system based on vision techniques and low cost sensors.To do so, a Visual Odometry system will be used.The concept of Visual Odometry is equivalent to wheel odometry but it has the advantage of not suffering from wheel slip which is present in these kind of environments due to the harsh terrain conditions.Here, motion is tracked computing the homogeneous transformation between camera frames, incrementally.However, this approach also presents some open issues.Most of the state of art methods, specially those who present a monocular camera system, don't perform good motion estimations in pure rotations.In some of them, motion even degenerates in these situations.Also, computing the motion scale is a difficult task that is widely investigated in this field.This work is intended to solve these issues.To do so, fisheye lens cameras will be used in order to achieve wide vision field of views

Towards autonomous mapping in agriculture: A review of supportive technologies for ground robotics

This paper surveys the supportive technologies currently available for ground mobile robots used for autonomous mapping in agriculture. Unlike previous reviews, we describe state-of-the-art approaches and technologies aimed at extracting information from agricultural environments, not only for navigation purposes but especially for mapping and monitoring. The state-of-the-art platforms and sensors, the modern localization techniques, the navigation and path planning approaches, as well as the potentialities of artificial intelligence towards autonomous mapping in agriculture are analyzed. According to the findings of this review, many examples of recent mobile robots provide full navigation and autonomous mapping capability. Significant resources are currently devoted to this research area, in order to further improve mobile robot capabilities in this complex and challenging field

Cost-effective robot for steep slope crops monitoring

This project aims to develop a low cost, simple and robust robot able to autonomously monitorcrops using simple sensors. It will be required do develop robotic sub-systems and integrate them with pre-selected mechanical components, electrical interfaces and robot systems (localization, navigation and perception) using ROS, for wine making regions and maize fields

Thermography to assess grapevine status and traits opportunities and limitations in crop monitoring and phenotyping – a review

Mestrado em Engenharia de Viticultura e Enologia (Double degree) / Instituto Superior de Agronomia. Universidade de Lisboa / Faculdade de Ciências. Universidade do PortoClimate change and the increasing water shortage pose increasing challenges to agriculture and viticulture, especially in typically dry and hot areas such as the Mediterranean and demand for solutions to use water resources more effectively. For this reason, new tools are needed to precisely monitor water stress in crops such as grapevine in order to save irrigation water, while guaranteeing yield. Imaging technologies and remote sensing tools are becoming more common in agriculture and plant/crop science research namely to perform phenotyping/selection or for crop stress monitoring purposes. Thermography emerged as important tool for the industry and agriculture. It allows detection of the emitted infrared thermal radiation and conversion of infrared radiation into temperature distribution maps. Considering that leaf temperature is a feasible indicator of stress and/or stomatal behavior, thermography showed to be capable to support characterization of novel genotypes and/or monitor crop’s stress. However, there are still limitations in the use of the technique that need to be minimized such as the accuracy of thermal data due to variable weather conditions, limitations due to the high costs of the equipment/platforms and limitations related to image analysis and processing to extract meaningful thermal data. This work revises the role of remote sensing and imaging in modern viticulture as well as the advantages and disadvantages of thermography and future developments, focusing on viticultureN/

Localização para Smart Devices tirando partido de iBeacons

A massificacao dos dispositivos de localização por satélite popularizou este tema, que hoje em dia se encontra amplamente experimentado e testado. Por outro lado, a evolução dos dispositivos móveis traz novas tecnologias e funcionalidades. O Bluetooth v4.0 é uma delas. Os iBeacons contém esta tecnologia e com base nas vantagens que traz, como o baixo custo e o baixo consumo, foi desenvolvido um sistema de localização baseado no RSSI. Este sistema é composto por dispositivos móveis, como os telemóveis de última geraçao, com o Sistema Operativo Android e uma aplicação criada para o efeito da localizaçao e os iBeacons como pontos de referênciaThe massification of devices based on satellite localization popularized this theme, which is widely well-tried and tested nowadays. On the other hand, the evolution of mobile devices brings new technologies and features. Bluetooth v4.0 is one of them. The iBeacons contains this technology and based on the advantages it brings, such as low cost, low consumption, a tracking system was developed. This system will be composed of mobile devices such as latest generation mobile phones, with the Android operating system and an application created for the purpose of location and iBeacons as reference point

Robot navigation in vineyards based on the visual vanish point concept

One of the biggest challenges of autonomous navigation in robots for agriculture is the path following in a large dimension map and various terrains. An important ability is to follow corridors and or vine rows which are frequent situation and with some complexity given the outline of real vegetation. One method to locate and guide the robot in between vineyards is making use of vanishing point detection on vine rows in order to obtain a reference point and send the adequate velocity commands to the motors. This detection will be conceived utilizing convectional image processing algorithms and Deep Learning techniques. It will be necessary to adapt the image processing algorithms or Deep Learning for use in ROS 2 context.One of the biggest challenges of autonomous navigation in robots for agriculture is the path following in a large dimension map and various terrains. An important ability is to follow corridors and or vine rows which are frequent situation and with some complexity given the outline of real vegetation. One method to locate and guide the robot in between vineyards is making use of vanishing point detection on vine rows in order to obtain a reference point and send the adequate velocity commands to the motors. This detection will be conceived utilizing convectional image processing algorithms and Deep Learning techniques. It will be necessary to adapt the image processing algorithms or Deep Learning for use in ROS 2 context

Segmentation and detection of Woody Trunks using Deep Learning for Agricultural Robotics

This project aims to help the implementation of image processing algorithms in agriculture robots so that they are robust to different aspects like weather conditions, vineyard terrain irregularities and efficient to operate in small robots with low energy consumption. Along with this, Deep Learning models became more complex. Thus, not all processors can handle such models. So, to develop a system with real-time detection for low-power processors becomes demanding because there is a lack of real datasets annotated for vine trunks and expedite tools to support this work. To support the deployment of deep-learning technology in agricultural robots, this dissertation presents the first public dataset of vine trunk images, called VineSet, with respective annotations for each trunk. This dataset was built from scratch, having a total of 9481 images of 5 different Douro vineyards, resulting from the images initially collected by AgRob V16 and various augmentation operations. Then, this dataset was used to train different state-of-the-art Deep Learning object detection models, together with Google Tensor Processing Unit. In parallel with this, this work presents an assisted labelling procedure that uses our trained models to reduce the time spent on labelling in the creation of new datasets. Also, this dissertation proposes the segmentation of vine trunks, using object detection models and semantic segmentation models. In this way, all the work done will allow the integration of edge-AI algorithms in SLAM, like Vine-SLAM, which will serve for the localisation and mapping of the robot, through natural markers in the vineyards.Agricultural robots need image processing algorithms, which should be reliable under all weather conditions and be computationally efficient. Furthermore, several limitations may arise, such as the characteristic vineyard terrain irregularities or overfitting in the training of neural networks that may affect the performance. In parallel with this, the evolution of Deep Learning models became more complex, demanding an increased computational complexity. Thus, not all processors can handle such models efficiently. So, developing a system with a real-time performance for low-power processors becomes demanding and is nowadays a research and development challenge because there is a lack of real data sets annotated and expedite tools to support this work. To support the deployment of deep-learning technology in agricultural robots, this dissertation presents a public VineSet dataset, the first public large collection of vine trunk images. The dataset was built from scratch, having a total of 9481 real image frames and providing the vine trunks annotations in each one of them. VineSet is composed of RGB and thermal images of 5 different Douro vineyards, with 952 initially collected by AgRob V16 robot, and others 8529 image frames resulting from a vast number of augmentation operations. To check the validity and usefulness of this VineSet dataset, in this work is presented an experimental baseline study, using state-of-the-art Deep Learning models together with Google Tensor Processing Unit. To simplify the task of augmentation in the creation of future datasets, we propose an assisted labelling procedure - by using our trained models - to reduce the labelling time, in some cases ten times faster per frame. This dissertation presents preliminary results to support future research in this topic, for example with VineSet leads possible to train (by transfer learning procedure) existing deep neural networks with Average Precision (AP) higher than 80% for vineyards trunks detection. For example, an AP of 84.16% was achieved for SSD MobileNet-V1. Also, the models trained with VineSet present good results in other environments such as orchards or forests. Our automatic labelling tool proves this, reducing annotation time by more than 30% in various areas of agriculture and more than 70% on vineyards. In this dissertation, we also propose the segmentation of the vine trunks. Firstly, object detection models were used together with VineSet to perform the trunk segmentation. To evaluate the performance of the different models, a script that implements some metrics of semantic segmentation was built. The results showed that the object detection models trained with VineSet were not only suitable for trunk detection but also trunk segmentation. For example, a DICE Similarity Index (DSI) of 70.78% was achieved for SSD MobileNet-V1. Finally, semantic segmentation was also briefly approached. A subset of the images of VineSet was used to train several models. Results show that semantic segmentation can substitute DL-based object detection models for pixel-based classification if a proper training set is provided. In this way, all the work done will allow the integration of edge-AI algorithms in SLAM, like Vine-SLAM, which will serve for the localisation and mapping of the robot, through natural markers in the vineyards

UAV and Machine Learning Based Refinement of a Satellite-Driven Vegetation Index for Precision Agriculture

Precision agriculture is considered to be a fundamental approach in pursuing a low-input, high-efficiency, and sustainable kind of agriculture when performing site-specific management practices. To achieve this objective, a reliable and updated description of the local status of crops is required. Remote sensing, and in particular satellite-based imagery, proved to be a valuable tool in crop mapping, monitoring, and diseases assessment. However, freely available satellite imagery with low or moderate resolutions showed some limits in specific agricultural applications, e.g., where crops are grown by rows. Indeed, in this framework, the satellite's output could be biased by intra-row covering, giving inaccurate information about crop status. This paper presents a novel satellite imagery refinement framework, based on a deep learning technique which exploits information properly derived from high resolution images acquired by unmanned aerial vehicle (UAV) airborne multispectral sensors. To train the convolutional neural network, only a single UAV-driven dataset is required, making the proposed approach simple and cost-effective. A vineyard in Serralunga d'Alba (Northern Italy) was chosen as a case study for validation purposes. Refined satellite-driven normalized difference vegetation index (NDVI) maps, acquired in four different periods during the vine growing season, were shown to better describe crop status with respect to raw datasets by correlation analysis and ANOVA. In addition, using a K-means based classifier, 3-class vineyard vigor maps were profitably derived from the NDVI maps, which are a valuable tool for growers