Grapevine bunch weight estimation using image-based features: comparing the predictive performance of number of visible berries and bunch area

Recent advances in machine vision technologies have provided a multitude of automatic tools for recognition and quantitative estimation of grapevine bunch features in 2D images. However, converting them into bunch weight (BuW) is still a big challenge. This paper aims to compare the explanatory power of the number of visible berries (#vBe) and the bunch area (BuA) in 2D images, in order to predict BuW. A set of 300 bunches from four grapevine cultivars were picked at harvest and imaged using a digital RGB camera. Then each bunch was manually assessed for several morphological attributes and, from each image, the #vBe was visually assessed while BuA was segmented using manual labelling combined with an image processing software. Single and multiple regression analysis between BuW and the image-based variables were performed and the obtained regression models were subsequently validated with two independent datasets. The high goodness of fit obtained for all the linear regression models indicates that either one of the imagebased variables can be used as an accurate proxy of actual bunch weight and that a general model is also suitable. The comparison of the explanatory power of the two image-based attributes for predicting bunch weight showed that the models based on the predictor #vBe had a slightly lower coefficient of determination (R2) than the models based on BuA. The combination of the two image-based explanatory variables in a multiple regression model produced predictor models with similar or noticeably higher R2 than those obtained for single-predictor models. However, adding a second variable produced a higher and more generalised gain in accuracy for the simple regression models based on the predictor #vBe than for the models based on BuA. Our results recommend the use of the models based on the two image-based variables, as they were generally more accurate and robust than the single variable models. When the gains in accuracy produced by adding a second image-based feature are small, the option of using only a single predictor can be chosen; in such a case, our results indicate that BuA would be a more accurate and less cultivardependent option than the #vBeinfo:eu-repo/semantics/publishedVersio

Development of a new non-invasive vineyard yield estimation method based on image analysis

Doutoramento em Engenharia Agronómica / Instituto Superior de Agronomia. Universidade de LisboaPredicting vineyard yield with accuracy can provide several advantages to the whole vine and wine industry. Today this is majorly done using manual and sometimes destructive methods, based on bunch samples. Yield estimation using computer vision and image analysis can potentially perform this task extensively, automatically, and non-invasively. In the present work this approach is explored in three main steps: image collection, occluded fruit estimation and image traits conversion to mass. On the first step, grapevine images were collected in field conditions along some of the main grapevine phenological stages. Visible yield components were identified in the image and compared to ground truth. When analyzing inflorescences and bunches, more than 50% were occluded by leaves or other plant organs, on three cultivars. No significant differences were observed on bunch visibility after fruit set. Visible bunch projected area explained an average of 49% of vine yield variation, between veraison and harvest. On the second step, vine images were collected, in field conditions, with different levels of defoliation intensity at bunch zone. A regression model was computed combining canopy porosity and visible bunch area, obtained via image analysis, which explained 70-84% of bunch exposure variation. This approach allowed for an estimation of the occluded fraction of bunches with average errors below |10|%. No significant differences were found between the model’s output at veraison and harvest. On the last step, the conversion of bunch image traits into mass was explored in laboratory and field conditions. In both cases, cultivar differences related to bunch architecture were found to affect weight estimation. A combination of derived variables which included visible bunch area, estimated total bunch area, visible bunch perimeter, visible berry number and bunch compactness was used to estimate yield on undisturbed grapevines. The final model achieved a R2 = 0.86 between actual and estimated yield (n = 213). If performed automatically, the final approach suggested in this work has the potential to provide a non-invasive method that can be performed accurately across whole vineyards.N/

A Multicultivar Approach for Grape Bunch Weight Estimation Using Image Analysis

The determination of bunch features that are relevant for bunch weight estimation is an important step in automatic vineyard yield estimation using image analysis. The conversion of 2D image features into mass can be highly dependent on grapevine cultivar, as the bunch morphology varies greatly. This paper aims to explore the relationships between bunch weight and bunch features obtained from image analysis considering a multicultivar approach. A set of 192 bunches from four cultivars, collected at sites located in Portugal and South Africa, were imaged using a conventional digital RGB camera, followed by image analysis, where several bunch features were extracted, along with physical measurements performed in laboratory conditions. Image data features were explored as predictors of bunch weight, individually and in a multiple stepwise regression analysis, which were then tested on 37% of the data. The results show that the variables bunch area and visible berries are good predictors of bunch weight (R2 ranging from 0.72 to 0.90); however, the simple regression lines fitted between these predictors and the response variable presented significantly different slopes among cultivars, indicating cultivar dependency. The elected multiple regression model used a combination of four variables: bunch area, bunch perimeter, visible berry number, and average berry area. The regression analysis between the actual and estimated bunch weight yielded a R2 = 0.91 on the test set. Our results are an important step towards automatic yield estimation in the vineyard, as they increase the possibility of applying image-based approaches using a generalized model, independent of the cultivarinfo:eu-repo/semantics/publishedVersio

GeoAI approach to Vineyard Yield Estimation

A thesis submitted in partial fulfillment of the requirements for the degree of Doctor in Information Management, specialization in Geographic Information SystemsKnowing in advance vineyard yield is a key issue for growers, winemakers, policy makers, and regulators being fundamental to achieve the best balance between vegetative and reproductive growth, and to allow more informed decisions like thinning, irrigation and nutrient management, schedule harvest, optimize winemaking operations, program crop insurance, fraud detection and grape picking workforce demand. In a long-term scenario of perceived climate change, it is also essential for planning and regulatory purposes at the regional level. Estimating yield is complex and requires knowing driving factors related to climate, plant, and crop management that directly influence the number of clusters per vine, berries per cluster, and berry weight. These three yield components explain 60%, 30%, and 10% of the yield. The traditional methods are destructive, labor-demanding, and time-consuming, with low accuracy primarily due to operator errors and sparse sampling (compared to the inherent spatial variability in a production vineyard). Those are supported by manual sampling, where yield is estimated by sampling clusters weight and the number of clusters per vine, historical data, and extrapolation considering the number of vines in a plot. As the extensive research in the area clearly shows, improved applied methodologies are needed at different spatial scales. The methodological approaches for yield estimation based on indirect methods are primarily applicable at small scale and can provide better estimates than the traditional manual sampling. They mainly depend on computer vision and image processing algorithms, data-driven models based on vegetation indices and pollen data, and on relating climate, soil, vegetation, and crop management variables that can support dynamic crop simulation models. Despite surpassing the limitations assigned to traditional manual sampling methods with the same or better results on accuracy, they still lack a fundamental key aspect: the real application in commercial vineyards. Another gap is the lack of solutions for estimating yield at broader scales (e.g., regional level). The perception is that decisions are more likely to take place on a smaller scale, which in some cases is inaccurate. It might be the case in regulated areas and areas where support for small viticulturists is needed and made by institutions with proper resources and a large area of influence. This is corroborated by the fact that data-driven models based on Trellis Tension and Pollen traps are being used for yield estimation at regional scales in real environments in different regions of the world. The current dissertation consists of the first study to identify through a systematic literature review the research approaches for predicting yield in vineyards for wine production that can serve as an alternative to traditional estimation methods, to characterize the different new approaches identifying and comparing their applicability under field conditions, scalability concerning the objective, accuracy, advantages, and shortcomings. In the second study following the identified research gap, a yield estimation model based on Geospatial Artificial Intelligence (GeoAI) with remote sensing and climate data and a machine-learning approach was developed. Using a satellite-based time-series of Normalized Difference Vegetation Index (NDVI) calculated from Sentinel 2 images and climate data acquired by local automatic weather stations, a system for yield prediction based on a Long Short-Term Memory (LSTM) neural network was implemented. The results show that this approach makes it possible to estimate wine grape yield accurately in advance at different scales

Assessing berry number for grapevine yield estimation by image analysis: case study with the white variety “Encruzado”

Mestrado em Engenharia de Viticultura e Enologia (Double degree) / Instituto Superior de Agronomia. Universidade de Lisboa / Faculdade de Ciências. Universidade do PortoNowadays, yield estimation represents one of the most important topics in viticulture. It can lead to a better vineyard management and to a better organization of harvesting operations in the vineyard and in the cellar. In recent years, image analysis has become an important tool to improve yield forecast, with the advantages of saving time and being non-invasive. This research aims to estimate the yield of the white cultivar ‘Encruzado’ using visible berry number counted in the images aquired at veraison and near harvest, using a manual RGB camera and the robot VINBOT. Images were collected in laboratory and in the field at the experimental vineyard of the Instituto Superior de Agronomia (ISA) in Lisbon. In the field images the number of visible berries per canopy meter was higher at maturation than at veraison, respectively 72.6 and 66.3. Regarding the percentage of visible berries, 30.2% where visible at veraison and 24.1% at maturation. Concerning percentage of berries occluded by other berries it was observed 28.7% at veraison and 24.3% at maturation. Regression analysis showed that the number of berries in the image explained a very high proportion of bunch weight variability, R2=0.64 at veraison and 0.91 at maturation. Regression analysis also showed that the canopy porosity explained a very high proportion of visible berries variability, R2=0.81 at veraison and 0.88 at maturation. The obtained regression models underestimated the yield with an higher error at veraison than at maturation. This underestimation indicates that the use of visible berry number on the images to estimate yield still needs further research to improve the algorithms accuracyN/

Fotogrametría de rango cercano aplicada a la Ingeniería Agroforestal

Tesis por compendio de publicaciones[EN]Since the late twentieth century, Geotechnologies are being applied in different research lines in Agroforestry Engineering aimed at advancing in the modeling of biophysical parameters in order to improve the productivity. In this study, low-cost and close range photogrammetry has been used in different agroforestry scenarios to solve identified gaps in the results and improve procedures and technology hitherto practiced in this field. Photogrammetry offers the advantage of being a non-destructive and non-invasive technique, never changing physical properties of the studied element, providing rigor and completeness to the captured information. In this PhD dissertation, the following contributions are presented divided into three research papers: • A methodological proposal to acquire georeferenced multispectral data of high spatial resolution using a low-cost manned aerial platform, to monitor and sustainably manage extensive áreas of crops. The vicarious calibration is exposed as radiometric calibration method of the multispectral sensor embarked on a paraglider. Low-cost surfaces are performed as control coverages. • The development of a method able to determine crop productivity under field conditions, from the combination of close range photogrammetry and computer vision, providing a constant operational improvement and a proactive management in the crop monitoring. An innovate methodology in the sector is proposed, ensuring flexibility and simplicity in the data collection by non-invasive technologies, automation in processing and quality results with low associated cost. • A low cost, efficient and accurate methodology to obtain Digital Height Models of vegatal cover intended for forestry inventories by integrating public data from LiDAR into photogrammetric point clouds coming from low cost flights. This methodology includes the potentiality of LiDAR to register ground points in areas with high density of vegetation and the better spatial, radiometric and temporal resolution from photogrammetry for the top of vegetal covers.[ES]Desde finales del siglo XX se están aplicando Geotecnologías en diferentes líneas de investigación en Ingeniería Agroforestal orientadas a avanzar en la modelización de parámetros biofísicos con el propósito de mejorar la productividad. En este estudio se ha empleado fotogrametría de bajo coste y rango cercano en distintos escenarios agroforestales para solventar carencias detectadas en los resultados obtenidos y mejorar los procedimientos y la tecnología hasta ahora usados en este campo. La fotogrametría ofrece como ventaja el ser una técnica no invasiva y no destructiva, por lo que no altera en ningún momento las propiedades físicas del elemento estudiado, dotando de rigor y exhaustividad a la información capturada. En esta Tesis Doctoral se presentan las siguientes contribuciones, divididas en tres artículos de investigación: • Una propuesta metodológica de adquisición de datos multiespectrales georreferenciados de alta resolución espacial mediante una plataforma aérea tripulada de bajo coste, para monitorizar y gestionar sosteniblemente amplias extensiones de cultivos. Se expone la calibración vicaria como método de calibración radiométrico del sensor multiespectral embarcado en un paramotor empleando como coberturas de control superficies de bajo coste. • El desarrollo de un método capaz de determinar la productividad del cultivo en condiciones de campo, a partir de la combinación de fotogrametría de rango cercano y visión computacional, facilitando una mejora operativa constante así como una gestión proactiva en la monitorización del cultivo. Se propone una metodología totalmente novedosa en el sector, garantizando flexibilidad y sencillez en la toma de datos mediante tecnologías no invasivas, automatismo en el procesado, calidad en los resultados y un bajo coste asociado. • Una metodología de bajo coste, eficiente y precisa para la obtención de Modelos Digitales de Altura de Cubierta Vegetal destinados al inventario forestal mediante la integración de datos públicos procedentes del LiDAR en las nubes de puntos fotogramétricas obtenidas con un vuelo de bajo coste. Esta metodología engloba la potencialidad del LiDAR para registrar el terreno en zonas con alta densidad de vegetación y una mejor resolución espacial, radiométrica y temporal procedente de la fotogrametría para la parte superior de las cubiertas vegetales

Overcoming the challenge of bunch occlusion by leaves for vineyard yield estimation using image analysis

Accurate yield estimation is of utmost importance for the entire grape and wine production chain, yet it remains an extremely challenging process due to high spatial and temporal variability in vineyards. Recent research has focused on using image analysis for vineyard yield estimation, with one of the major obstacles being the high degree of occlusion of bunches by leaves. This work uses canopy features obtained from 2D images (canopy porosity and visible bunch area) as proxies for estimating the proportion of occluded bunches by leaves to enable automatic yield estimation on non-disturbed canopies. Data was collected from three grapevine varieties, and images were captured from 1 m segments at two phenological stages (veraison and full maturation) in non-defoliated and partially defoliated vines. Visible bunches (bunch exposure; BE) varied between 16 and 64 %. This percentage was estimated using a multiple regression model that includes canopy porosity and visible bunch area as predictors, yielding a R2 between 0.70 and 0.84 on a training set composed of 70 % of all data, showing an explanatory power 10 to 43 % higher than when using the predictors individually. A model based on the combined data set (all varieties and phenological stages) was selected for BE estimation, achieving a R2 = 0.80 on the validation set. This model did not show validation metrics differences when applied on data collected at veraison or full maturation, suggesting that BE can be accurately estimated at any stage. Bunch exposure was then used to estimate total bunch area (tBA), showing low errors (< 10 %) except for the variety Arinto, which presents specific morphological traits such as large leaves and bunches. Finally, yield estimation computed from estimated tBA presented a very low error (0.2 %) on the validation data set with pooled data. However, when performed on every single variety, the simplified approach of area-to-mass conversion was less accurate for the variety Syrah. The method demonstrated in this work is an important step towards a fully automated non-invasive yield estimation approach, as it offers a solution to estimate bunches that are not visible to imaging sensorsinfo:eu-repo/semantics/publishedVersio

Assessing berry number for grapevine yield estimation by image analysis: case study with the white variety “Arinto”

Mestrado em Engenharia de Viticultura e Enologia (Double degree) / Instituto Superior de Agronomia. Universidade de Lisboa / Faculdade de Ciências. Universidade do PortoYield estimation in recent years is identified as one of more important topics in viticulture because it can lead to more efficiently managed vineyards producing wines of highly quality. Recently, to improve the efficiency of yield estimation, image analysis is becoming an important tool to collect detailed information from the vines regarding the yield. New technologies were developed for yield estimation using a new ground platform, such as VINBOT, using image analysis. This work was done in a vineyard of the “Instituto Superior de Agronomia”, with the aim to estimate the final yield, during the growing cycle 2019 of the variety “Arinto”, using images collected in three different modality: laboratory condition (1), field condition (2) and VINBOT robot. In the every condition, the images were captured with the RGB-D camera. For (1) and (2) the photos were acquired manually through the use of a digital camera placed on a tripod but in the (3) the RGB-D camera was fixed on the VINBOT robot. In this work, the correlation of yield components between field data and images data was evaluated. In addition, throught MATLAB, it was evaluate the number of visible berries in the images and the percentage of visible berries not occluded by leaves and by other berries. Througt the laboratory results was calculate a growth factor of bunches on the periods pea-size and veraison. On the VINBOT analysis the efficacy to estimate the total yield from the number of berries was higher at maturation with a 10% error ratio. The relationship between canopy porosity and exposed berries showed for all the stages high and significant R2 indicating that we can use it to estimate berries occlusion through image analysis. This accuracy makes the proposed methodology ideal for early yield prediction as a very helpful tool for the grape and wine industryN/

Assessing Berry Number for Grapevine Yield Estimation by Image Analysis: Case Study with the Red Variety “Syrah”

Mestrado em Engenharia de Viticultura e Enologia (Double degree) / Instituto Superior de Agronomia. Universidade de Lisboa / Faculdade de Ciências. Universidade do PortoThe yield estimation provides information that help growers to make decisions in order to optimize crop growth and to organize the harvest operations in field and in the cellar. In most vineyard estates yield is forecasted using manual methods. However, image analysis methods, which are less invasive low cost and more representative are now being developed. The main objective of this work was to estimate yield through data obtained in the frame of Vinbot project during the 2019 season. In this thesis, images of the grapevine variety Syrah taken in the laboratory and in the vineyards of the “Instituto Superior de Agronomia” in Lisbon were analyzed. In the laboratory the images were taken manually with an RGB camera, while in the field vines were imaged either manually and by the Vinbot robot. From these images, the number of visible berries were counted with MATLAB. From the laboratory values, the relationships between the number of visible berries and actual bunch weight and berry number were studied. From the data obtained in the field, it was analyzed the visibility of the berries at different levels of defoliation and the relationship between the area of visible bunches and the visible berries. Berry-by-berry occlusion showed a value of 6.4% at pea-size, 14.5% at veraison and 25% at maturation. In addition, high and significant determination coefficient were obtained between actual yield and visible berries. The comparison of estimated yield, obtained using the regression models with actual yield, showed an underestimation at all the three phonological stages. This low accuracy of the developed models show that the use of algorithms based on visible berry number on the images to estimate yield still needs further researchN/

Utilisation d'un robot terrestre pour estimer les caracteristiques de la canopee et le rendement au vignoble

This paper aims to present some results of the EU VINBOT (Autonomous cloud-computing vineyard robot to optimize yield management and wine quality) project focused on vineyard yield estimation. A ground truth evaluation trial was set up in an experimental vineyard with two plots of the white varieties ‘Alvarinho’ and ‘Arinto’, trained on a vertical shoot positioning system and spur pruned. For each varietal plot, six smart points were selected with 10 contiguous vines each. During the ripening period of the 2016 season the vines were manually assessed for canopy dimensions and yield and then scanned by the VINBOT sensor head composed with a 2D laser rangefinder, a Kinect v2 camera and a set of robot navigation sensors. Ground truth data was used to compare with the canopy data estimated by the rangefinder and with the output of the image analysis algorithms. Regarding canopy features (height, volume and exposed leaf area), in general an acceptable fit between actual and estimated values was observed with canopy height showing the best agreement. The regression analysis between actual and estimated values of canopy features showed a significant linear relationship for all the features however the lower values of the R2 indicate a weak relationship. Regarding the yield, despite the significant R2 (0.31) showed by the regression analysis between actual and estimated values, the equation of the fitted line indicate that the VINBOT algorithms underestimated the yield by an additive factor. Our results showed that canopy features can be estimated by the VINBOT platform with an acceptable accuracy. However, the underestimation of actual yield, caused mainly by bunch occlusion, deserves further research to improve the algorithms accuracyN/