PHENOquad: A new multi sensor platform for field phenotyping and screening of yield relevant characteristics within grapevine breeding research

Balanced and stable yield is a major trait in grapevine breeding and breeding research. Grapevine yield hereby is a complex quantitative trait, as it is influenced by multiple plant parameters, like berry size, number of berries per bunch, number of bunches per shoot, management, and environmental factors. In the current breeding process, the complexity of this trait has shown that a classification according to descriptive factors for marker development is only possible to a limited extent. Precise field phenotyping of yield-related traits is the basic prerequisite to be able to measure such quantitative traits. This, however, is the major bottleneck due to labor, time and constrains of plant material in the breeding process. For this reason, one of our main goals with the newly developed phenotyping platform PHENOquad with its multisensor system PHENOboxx is to improve phenotyping efficiency of grapevine yield to overcome the phenotyping bottleneck. The newly developed embedded vision system PHENOboxx is mounted on an "all-terrain vehicle (ATV)". This allows a fast data acquisition on a large number of individual vines. In order to evaluate the yield potential of breeding material in comparison to established grapevine cultivars, various yield-related parameters of the vines are quantified directly in the field with high spatial and temporal resolution. As key parameters for yield-related phenotyping, the number of shoots, bunches, berries and the weight of dormant pruning wood was identified. The image data acquired are annotated to train the artificial intelligence (AI). Within the process, the image analysis results are compared to annotated ground truth data and correlated with the field reference data. We expect to increase the precision, target specificity and throughput of screening grapevine material without reducing its accuracy over time by using the PHENOquad. In addition, a weighting of yield-relevant parameters would be possible. This opens up new possibilities for efficient plant evaluation in the scope of grapevine breeding. Also new application possibilities for precision viticulture are conceivable

Development of a new non-invasive vineyard yield estimation method based on image analysis

Doutoramento em Engenharia Agronómica / Instituto Superior de Agronomia. Universidade de LisboaPredicting vineyard yield with accuracy can provide several advantages to the whole vine and wine industry. Today this is majorly done using manual and sometimes destructive methods, based on bunch samples. Yield estimation using computer vision and image analysis can potentially perform this task extensively, automatically, and non-invasively. In the present work this approach is explored in three main steps: image collection, occluded fruit estimation and image traits conversion to mass. On the first step, grapevine images were collected in field conditions along some of the main grapevine phenological stages. Visible yield components were identified in the image and compared to ground truth. When analyzing inflorescences and bunches, more than 50% were occluded by leaves or other plant organs, on three cultivars. No significant differences were observed on bunch visibility after fruit set. Visible bunch projected area explained an average of 49% of vine yield variation, between veraison and harvest. On the second step, vine images were collected, in field conditions, with different levels of defoliation intensity at bunch zone. A regression model was computed combining canopy porosity and visible bunch area, obtained via image analysis, which explained 70-84% of bunch exposure variation. This approach allowed for an estimation of the occluded fraction of bunches with average errors below |10|%. No significant differences were found between the model’s output at veraison and harvest. On the last step, the conversion of bunch image traits into mass was explored in laboratory and field conditions. In both cases, cultivar differences related to bunch architecture were found to affect weight estimation. A combination of derived variables which included visible bunch area, estimated total bunch area, visible bunch perimeter, visible berry number and bunch compactness was used to estimate yield on undisturbed grapevines. The final model achieved a R2 = 0.86 between actual and estimated yield (n = 213). If performed automatically, the final approach suggested in this work has the potential to provide a non-invasive method that can be performed accurately across whole vineyards.N/

Overcoming the challenge of bunch occlusion by leaves for vineyard yield estimation using image analysis

Accurate yield estimation is of utmost importance for the entire grape and wine production chain, yet it remains an extremely challenging process due to high spatial and temporal variability in vineyards. Recent research has focused on using image analysis for vineyard yield estimation, with one of the major obstacles being the high degree of occlusion of bunches by leaves. This work uses canopy features obtained from 2D images (canopy porosity and visible bunch area) as proxies for estimating the proportion of occluded bunches by leaves to enable automatic yield estimation on non-disturbed canopies. Data was collected from three grapevine varieties, and images were captured from 1 m segments at two phenological stages (veraison and full maturation) in non-defoliated and partially defoliated vines. Visible bunches (bunch exposure; BE) varied between 16 and 64 %. This percentage was estimated using a multiple regression model that includes canopy porosity and visible bunch area as predictors, yielding a R2 between 0.70 and 0.84 on a training set composed of 70 % of all data, showing an explanatory power 10 to 43 % higher than when using the predictors individually. A model based on the combined data set (all varieties and phenological stages) was selected for BE estimation, achieving a R2 = 0.80 on the validation set. This model did not show validation metrics differences when applied on data collected at veraison or full maturation, suggesting that BE can be accurately estimated at any stage. Bunch exposure was then used to estimate total bunch area (tBA), showing low errors (< 10 %) except for the variety Arinto, which presents specific morphological traits such as large leaves and bunches. Finally, yield estimation computed from estimated tBA presented a very low error (0.2 %) on the validation data set with pooled data. However, when performed on every single variety, the simplified approach of area-to-mass conversion was less accurate for the variety Syrah. The method demonstrated in this work is an important step towards a fully automated non-invasive yield estimation approach, as it offers a solution to estimate bunches that are not visible to imaging sensorsinfo:eu-repo/semantics/publishedVersio

Vineyard pruning weight assessment by machine vision: towards an on-the-go measurement system

Aim: Pruning weight is an indicator of vegetative growth and vigour in grapevine. Traditionally, it is manually determined, which is time-consuming and labour-demanding. This study aims at providing a new, non-invasive and low-cost method for pruning weight estimation in commercial vineyards based on computer vision. Methods and results: The methodology relies on computer-based analysis of RGB images captured manually and on-the-go in a VSP Tempranillo vineyard. Firstly, the pruning weight estimation was evaluated using manually taken photographs using a controlled background. These images were analysed to generate a model of wood pruning weight estimation, resulting in a coefficient of determination (R2) of 0.91 (p<0.001) and a root-mean-square error (RMSE) of 87.7 g. After this, a mobile sensor platform (modified ATV) was used to take vine images automatically and on-the-go without background. These RGB images were analysed using a fully automated computer vision algorithm, resulting in R2 = 0.75 (p<0.001) and RMSE = 147.9 g. Finally, the mobile sensor platform was also used to sample a commercial VSP vineyard to map the spatial variability of wood pruning weight, and hereafter vine vigour. Conclusions: The results showed that the developed computer vision methodology was able to estimate the vine pruning weight in commercial vineyards and to map the spatial variation of the pruning weight across a vineyard. Significance and impact of the study: The presented methodology may become a valuable tool for the wine industry for rapid assessment and mapping of vine vigour. This information can be used to support decision making on pruning, fertilization and canopy management

Vineyard yield estimation using image analysis – a review

Mestrado em Engenharia de Viticultura e Enologia (Double degree) / Instituto Superior de Agronomia. Universidade de Lisboa / Faculdade de Ciências. Universidade do PortoYield estimation is one of the main goals of the wine industry, this because with an accurate yield estimation it is possible to have a significant reduction in production costs and a better management of the wine industry. Traditional methods for yield estimation are laborious and time consuming, for these reasons in the last years we are witnessing to the development of new methodologies, most of which are based on image analysis. Thanks to the continuous updating and improvement of the computer vision techniques and of the robotic platforms, image analysis applied to the yield estimation is becoming more and more efficient. In fact the results shown by the different studies are very satisfying, at least as regards the estimation of what is possible to see, while are under development several procedures which have the objective to estimate what is not possible to see, due to bunch occlusion by leaves and by others clusters. I this work the different methodologies and the different approaches used for yield estimation are described, including both traditional methods and new approaches based on image analysis, in order to present the advantages and disadvantages of each of themN/

Assessing berry number for grapevine yield estimation by image analysis: case study with the white variety “Arinto”

Mestrado em Engenharia de Viticultura e Enologia (Double degree) / Instituto Superior de Agronomia. Universidade de Lisboa / Faculdade de Ciências. Universidade do PortoYield estimation in recent years is identified as one of more important topics in viticulture because it can lead to more efficiently managed vineyards producing wines of highly quality. Recently, to improve the efficiency of yield estimation, image analysis is becoming an important tool to collect detailed information from the vines regarding the yield. New technologies were developed for yield estimation using a new ground platform, such as VINBOT, using image analysis. This work was done in a vineyard of the “Instituto Superior de Agronomia”, with the aim to estimate the final yield, during the growing cycle 2019 of the variety “Arinto”, using images collected in three different modality: laboratory condition (1), field condition (2) and VINBOT robot. In the every condition, the images were captured with the RGB-D camera. For (1) and (2) the photos were acquired manually through the use of a digital camera placed on a tripod but in the (3) the RGB-D camera was fixed on the VINBOT robot. In this work, the correlation of yield components between field data and images data was evaluated. In addition, throught MATLAB, it was evaluate the number of visible berries in the images and the percentage of visible berries not occluded by leaves and by other berries. Througt the laboratory results was calculate a growth factor of bunches on the periods pea-size and veraison. On the VINBOT analysis the efficacy to estimate the total yield from the number of berries was higher at maturation with a 10% error ratio. The relationship between canopy porosity and exposed berries showed for all the stages high and significant R2 indicating that we can use it to estimate berries occlusion through image analysis. This accuracy makes the proposed methodology ideal for early yield prediction as a very helpful tool for the grape and wine industryN/

Assessing berry number for grapevine yield estimation by image analysis: case study with the white variety “Encruzado”

Mestrado em Engenharia de Viticultura e Enologia (Double degree) / Instituto Superior de Agronomia. Universidade de Lisboa / Faculdade de Ciências. Universidade do PortoNowadays, yield estimation represents one of the most important topics in viticulture. It can lead to a better vineyard management and to a better organization of harvesting operations in the vineyard and in the cellar. In recent years, image analysis has become an important tool to improve yield forecast, with the advantages of saving time and being non-invasive. This research aims to estimate the yield of the white cultivar ‘Encruzado’ using visible berry number counted in the images aquired at veraison and near harvest, using a manual RGB camera and the robot VINBOT. Images were collected in laboratory and in the field at the experimental vineyard of the Instituto Superior de Agronomia (ISA) in Lisbon. In the field images the number of visible berries per canopy meter was higher at maturation than at veraison, respectively 72.6 and 66.3. Regarding the percentage of visible berries, 30.2% where visible at veraison and 24.1% at maturation. Concerning percentage of berries occluded by other berries it was observed 28.7% at veraison and 24.3% at maturation. Regression analysis showed that the number of berries in the image explained a very high proportion of bunch weight variability, R2=0.64 at veraison and 0.91 at maturation. Regression analysis also showed that the canopy porosity explained a very high proportion of visible berries variability, R2=0.81 at veraison and 0.88 at maturation. The obtained regression models underestimated the yield with an higher error at veraison than at maturation. This underestimation indicates that the use of visible berry number on the images to estimate yield still needs further research to improve the algorithms accuracyN/

Grapevine yield estimation using image analysis for the variety Syrah

Mestrado em Engenharia de Viticultura e Enologia (Double Degree) / Instituto Superior de Agronomia. Universidade de Lisboa / Faculdade de Ciências. Universidade do PortoYield estimation in recent years is identified as one of more important topics in viticulture because it can lead to more efficiently managed vineyards producing wines of highly quality. Recently, to improve the efficiency of yield estimation, image analysis is becoming an important tool to collect detailed information from the vines regarding the yield. New technologies were developed for yield estimation using a new ground platform, such as VINBOT, using image analysis. This work was done in a vineyard of the “Instituto Superior de Agronomia”, with the aim to estimate the final yield, during the growing cycle 2019 of the variety “Syrah”, using images collected by the VINBOT robot. The images were captured with the RGB-D camera placed on the VINBOT robot in the vineyard and in addition, we obtained laboratory images using an RGB-D manual camera. In this work, the correlation of yield components between ground truth data and images data was evaluated. In addition, it was evaluate the projected bunches area in the images and the percentage of visible bunches not occluded by leaves and by other bunches. It was found a growth factor of bunches on the periods from pea-size to harvest. The efficacy to estimate bunch weight from the projected area was higher at maturation. The relationship between canopy porosity and exposed bunches showed for all the stages high and significant R2 indicating that we can use it to estimate bunches covered by leaves through image analysis. The percentage of visible bunches without the leaves occlusion and bunch occlusion was 29% at pea-size, 21% at veraison and 45% at maturation. It was estimated the final yield at pea-size, with an MA%E of 54%, at veraison and maturation were observed values of MA%E of 7% and 5%, respectively. Our results enable to conclude that the image analysis is an alternative to the traditional way to estimate the yieldN/

Assessing Berry Number for Grapevine Yield Estimation by Image Analysis: Case Study with the Red Variety “Syrah”

Mestrado em Engenharia de Viticultura e Enologia (Double degree) / Instituto Superior de Agronomia. Universidade de Lisboa / Faculdade de Ciências. Universidade do PortoThe yield estimation provides information that help growers to make decisions in order to optimize crop growth and to organize the harvest operations in field and in the cellar. In most vineyard estates yield is forecasted using manual methods. However, image analysis methods, which are less invasive low cost and more representative are now being developed. The main objective of this work was to estimate yield through data obtained in the frame of Vinbot project during the 2019 season. In this thesis, images of the grapevine variety Syrah taken in the laboratory and in the vineyards of the “Instituto Superior de Agronomia” in Lisbon were analyzed. In the laboratory the images were taken manually with an RGB camera, while in the field vines were imaged either manually and by the Vinbot robot. From these images, the number of visible berries were counted with MATLAB. From the laboratory values, the relationships between the number of visible berries and actual bunch weight and berry number were studied. From the data obtained in the field, it was analyzed the visibility of the berries at different levels of defoliation and the relationship between the area of visible bunches and the visible berries. Berry-by-berry occlusion showed a value of 6.4% at pea-size, 14.5% at veraison and 25% at maturation. In addition, high and significant determination coefficient were obtained between actual yield and visible berries. The comparison of estimated yield, obtained using the regression models with actual yield, showed an underestimation at all the three phonological stages. This low accuracy of the developed models show that the use of algorithms based on visible berry number on the images to estimate yield still needs further researchN/

Grapevine yield estimation using image analysis for the variety Arinto

Mestrado em Engenharia de Viticultura e Enologia (Double Degree) / Instituto Superior de Agronomia. Universidade de Lisboa / Faculdade de Ciências. Universidade do PortoYield estimation can lead to difficulties in the vineyard and winery, if it is done inaccurately following wrong procedures, doing a non-representative sampling or for the human error. Moreover, the traditional yield estimation methods are time consuming and destructive because they need someone that goes into the vineyard to count the yield components and that take out from the vineyard inflorescence or bunches to count and weight the flowers and the berries. To avoid these problems and the errors that can occur on this way, the development and application of new and innovative techniques to estimate the yield through the analysis of RGB images taken under field conditions are under study from different groups of research. In our research work we’ve studied the application of counting the yield components in the images throughout all the growing season. Furthermore, we’ve studied two different algorithms that starting from the survey of canopy porosity and/or visible bunches area, can help to do an estimation of the yield. The most promising yield estimation, based on the counting of the yield components done through image analysis, was found to be at the phenological stage of four leaves out, which shown a mean absolute percent error (MA%E) of 32 ± 2% and an correlaion coefficient (r Obs,Est) between observed and estimated shoots of 0.62. The two algorithms used different models: for estimating the area of the bunches covered by leaves and to estimate the weight of the bunches per linear canopy meter. When the area of the bunches without leaf occlusion was estimated, an average percentage of occlusion generated by the bunches on the other bunches of 8%, 6% and 12% respectively at pea size, veraison and maturation, was used to estimate the total area of the bunches. When the total area of the bunches per linear canopy meter was estimated the two models to estimate the grape weight were used. Finally, to estimate the weight at harvest, the growth factors of 6.6 and 1.7 respectively, at pea size and veraison were used. The first algorithm shown a MA%E, between the estimated and observed values of yield, of - 33.59%, -9.24% and -11.25%, instead the second algorithm shown a MA%E of -6.81%, -1.35% and 0.01% respectively at pea-size, veraison and maturationN/