Local Motion Planner for Autonomous Navigation in Vineyards with a RGB-D Camera-Based Algorithm and Deep Learning Synergy

With the advent of agriculture 3.0 and 4.0, researchers are increasingly focusing on the development of innovative smart farming and precision agriculture technologies by introducing automation and robotics into the agricultural processes. Autonomous agricultural field machines have been gaining significant attention from farmers and industries to reduce costs, human workload, and required resources. Nevertheless, achieving sufficient autonomous navigation capabilities requires the simultaneous cooperation of different processes; localization, mapping, and path planning are just some of the steps that aim at providing to the machine the right set of skills to operate in semi-structured and unstructured environments. In this context, this study presents a low-cost local motion planner for autonomous navigation in vineyards based only on an RGB-D camera, low range hardware, and a dual layer control algorithm. The first algorithm exploits the disparity map and its depth representation to generate a proportional control for the robotic platform. Concurrently, a second back-up algorithm, based on representations learning and resilient to illumination variations, can take control of the machine in case of a momentaneous failure of the first block. Moreover, due to the double nature of the system, after initial training of the deep learning model with an initial dataset, the strict synergy between the two algorithms opens the possibility of exploiting new automatically labeled data, coming from the field, to extend the existing model knowledge. The machine learning algorithm has been trained and tested, using transfer learning, with acquired images during different field surveys in the North region of Italy and then optimized for on-device inference with model pruning and quantization. Finally, the overall system has been validated with a customized robot platform in the relevant environment

Automatic plant pest detection and recognition using k-means clustering algorithm and correspondence filters

Plant pest recognition and detection is vital for food security, quality of life and a stable agricultural economy. This research demonstrates the combination of the k-means clustering algorithm and the correspondence filter to achieve pest detection and recognition. The detection of the dataset is achieved by partitioning the data space into Voronoi cells, which tends to find clusters of comparable spatial extents, thereby separating the objects (pests) from the background (pest habitat). The detection is established by extracting the variant distinctive attributes between the pest and its habitat (leaf, stem) and using the correspondence filter to identify the plant pests to obtain correlation peak values for different datasets. This work further establishes that the recognition probability from the pest image is directly proportional to the height of the output signal and inversely proportional to the viewing angles, which further confirmed that the recognition of plant pests is a function of their position and viewing angle. It is encouraging to note that the correspondence filter can achieve rotational invariance of pests up to angles of 360 degrees, which proves the effectiveness of the algorithm for the detection and recognition of plant pests

Concept study of a Digital Twin of a Precision Agricultural Robot

When designing a digital twin, different properties are needed to be implemented so that the physical twin can be able to interact with the environment and fulfil the tasks that the physical asset was developed for. The methodology proposed in this thesis is of highly relevance when designing a digital twin solution, being simple to adapt to different necessities and with a clear architecture to utilize or to adjust to digital assets in different applications. The digital twin developed in the case study, on which this thesis is based, is the foundation of the development and creation of an innovative table grape harvesting robot. The main objective of this research is to review and identify potential methodologies that can be used in the design stage of a digital twin and to validate how the processes in the methodologies can support the system to fulfill the objectives of the project. The system involves the interactions between the robot, the environment, and the agronomical tasks that the robot needs to perform. This thesis creates the methodologies that will assist different stakeholders in easily identifying the processes that streamline the testing procedure of different algorithms in the digital twin, saving time and resources by doing the development in the digital twin and not in the physical object. The thesis assessed the challenges of limited testing time and transporting equipment and personnel difficulties to a fixed location, in this case, a vineyard located in Italy, defined later as the physical asset. It is of highly importance to incorporate the research structure to the digital twin development team early in the project's timeline. Based on the literature and discussion between stakeholders, the basic architecture was created, and from there, the cases defined in this thesis will allow the users and clients to test in a seamless way their products in the digital twin. The process gave the option to the users to select and use from a basic environment to a more complex and challenging one. The purpose of the thesis was to present and document certain architecture and methodologies used in the research and present them as a base for future developments in the area. This method can be used for projects when physical assets need to be created and tested, when time periods for testing are part of the challenges of the project, and the availability to allocate and integrate resources is complex. The main results and conclusion of this thesis is the methodology proposed, on how a simple processes and methodologies can be easily adapted to the necessities of any digital twin solution, and how the architecture proposed can have the ability to modify different cases for specific objectives. And finally, how it is possible to use, prepper and export the information needed to train the Machine learning (ML) algorithms, and to add noise specific to allow the evolution of the algorithms. The methodology proposed in this thesis can increase the quality and usability of any digital twin by proving how it can be successfully implemented during the planning developing process of a project. Furthermore, the methodology demonstrate that it can be easily adapted to the necessities of any digital twin solution and streamlined the progress in the future use of digital twins in any area. In the case study, the methodology helped all different stakeholders to utilize the digital twin to develop, test, and improve different algorithms from different locations through Europe without the need to build the physical robot, or being in one particular place, and without the restrictions of seasonal harvesting periods

Vineyard yeld estimation by VINBOT robot - preliminary results with the white variety Viosinho

Nowadays it is recognized that vineyard yield estimation can bring several benefits to all the vine and wine industry and, consequently, there is a strong demand for fast and reliable yield estimation methods. Recently a strong effort has been made on developing machine vision tools to automatically estimate vineyard yields evolving several research teams worldwide. In this paper we aim to present preliminary results obtained in the frame of an European research project (VINBOT: “Autonomous cloud-computing vineyard robot to optimise yield management and wine quality”) focus on yield estimation. A ground truth evaluation trial was set up in an experimental vineyard with the white variety Viosinho, trained on a vertical shoot positioning system and spur pruned. A sample of contiguous vines was labeled and submitted to a detailed assessment of vegetative and reproductive data to feed a viticulture data library. The vines were scanned during the ripening period of the 2015 season by the VINBOT sensor head composed with a set of sensors capable of capturing vineyard images and 3D data. Ground truth data was used to relate with images taken by the sensors and to test algorithms of image analysis. In this paper we present and discuss the relationships between actual and estimated yield computed using the surface occupied by the grape clusters in the images. Our preliminary results showed that, despite of a slight underestimation of the ground truth, caused mainly by cluster occlusion, when the canopy density allows visualization of most part of the clusters, the yield can be estimated by machine vision with a high fidelity. Further research is ongoing to test those devices and methodologies in other varieties and to improve the estimation accuracyinfo:eu-repo/semantics/publishedVersio

The Use of Agricultural Robots in Orchard Management

Book chapter that summarizes recent research on agricultural robotics in orchard management, including Robotic pruning, Robotic thinning, Robotic spraying, Robotic harvesting, Robotic fruit transportation, and future trends.Comment: 22 page

Analysis of the scientifc knowledge structure on automation in the wine industry: a bibliometric and systematic review

The objective of this research is to analyze the knowledge structure of the academic literature indexed in the Core Collection of the Web of Science on automation in the wine industry, from the frst registered article in 1996 to 2022, in order to identify the latest trends in the study of this subject. A bibliometric and systematic analysis of the literature was carried out. First, for the quantitative analysis of the scientifc production, the bibliometric study was conducted, using the WoS database for data collection and the VosViewer and Bibliometrix applications to create the network maps. Second, once the literature had been examined quantitatively, content analysis was undertaken using the PRISMA methodology. The results show, among other aspects, the uneven distribution of the examined scientifc production from 1996 to 2022, that computer vision, data aggregation, life cycle assessment, precision viticulture, extreme learning machine and collaborative platforms are the major current keywords and the predominance of Spain and Italy in terms of scientifc production in the feld. There are various justifcations which support the originality of this study. First, it contributes to the nderstanding of academic literature and the identifcation of the most recent trends in the study of automation in the wine industry. Second, to the best of our knowledge, no prior bibliometric studies have considered this topic. Third, this research evaluates the literature from the frst record to the year 2022, thereby providing a comprehensive analysis of the scientifc production.Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature

Thermography to assess grapevine status and traits opportunities and limitations in crop monitoring and phenotyping – a review

Mestrado em Engenharia de Viticultura e Enologia (Double degree) / Instituto Superior de Agronomia. Universidade de Lisboa / Faculdade de Ciências. Universidade do PortoClimate change and the increasing water shortage pose increasing challenges to agriculture and viticulture, especially in typically dry and hot areas such as the Mediterranean and demand for solutions to use water resources more effectively. For this reason, new tools are needed to precisely monitor water stress in crops such as grapevine in order to save irrigation water, while guaranteeing yield. Imaging technologies and remote sensing tools are becoming more common in agriculture and plant/crop science research namely to perform phenotyping/selection or for crop stress monitoring purposes. Thermography emerged as important tool for the industry and agriculture. It allows detection of the emitted infrared thermal radiation and conversion of infrared radiation into temperature distribution maps. Considering that leaf temperature is a feasible indicator of stress and/or stomatal behavior, thermography showed to be capable to support characterization of novel genotypes and/or monitor crop’s stress. However, there are still limitations in the use of the technique that need to be minimized such as the accuracy of thermal data due to variable weather conditions, limitations due to the high costs of the equipment/platforms and limitations related to image analysis and processing to extract meaningful thermal data. This work revises the role of remote sensing and imaging in modern viticulture as well as the advantages and disadvantages of thermography and future developments, focusing on viticultureN/