Fruit detection system and an end effector for robotic harvesting of Fuji apples

 The challenges in developing a fruit harvesting robot are recognizing the fruit in the foliage and detaching the fruit from the tree without damaging either the fruit or the tree.  The objectives of this study were to develop a real-time fruit detection system using machine vision and a laser ranging sensor and to develop an end effector capable of detaching the fruit in a way similar to manual pick.  The Fuji apple variety was used in this study. In the detection of the fruit, machine vision was combined with a laser ranging sensor.  The machine vision recognized the fruit and the laser ranging sensor determined the distance.  The system detected a single fruit with 100% accuracy in both front and back lighted scenes with ±3 mm accuracy in distance measurement.  To detach the fruit from the tree, an end effector was developed with a peduncle holder and a wrist; the peduncle holder pinches the peduncle of the fruit and the wrist rotates the peduncle holder to detach the fruit.  Field test results of the end effector showed more than 90% success rate in detaching the fruit with average time use of 7.1 seconds.Keywords: apple, end effector, image processing, machine vision, robotic harvesting, Japan Citation: Bulanon D. M., and T. Kataoka.  Fruit detection system and an end effector for robotic harvesting of Fuji apples.  Agric Eng Int: CIGR Journal, 2010, 12(1): 203-210.&nbsp

A Machine Vision System for the Apple Harvesting Robot

Rosana G. Moreira, Editor-in-Chief; Texas A&M UniversityThis is a Technical article from International Commission of Agricultural Engineering (CIGR, Commission Internationale du Genie Rural) E-Journal Volume 3 (2001): D.M. Bulanon, T. Kataoka, Y. Ota, and T. Hiroma. A Machine Vision System for the Apple Harvesting Robot. Vol. III, December 2001

A Separating Method of Adjacent Apples Based on Machine Vision and Chain Code Information

Part 1: Decision Support Systems, Intelligent Systems and Artificial Intelligence ApplicationsInternational audienceFruit location is an important parameter for apple harvesting robot to conduct picking task. However, it is difficult to obtain coordinates of each apple under natural conditions. One of the major challenges is detecting adjacent fruits accurately. Previous studies for adjacent detection have shortcomings such as vast computation, difficulty in implementation and over-segmentation. In this paper, we propose a novel and effective separating method for adjacent apples recognition based on chain code information and obtain the centroid coordinates of each fruit. Firstly, those valid regions of fruit are extracted by pre-processing the initial image. Secondly, chain code information is obtained by following the contour of extracted regions. Thirdly, through observing the changing law of chain code difference and adopting local optimum principle, concave points are found. Finally, the best point pairs are determined with different matching principles, and those adjacent apples are separated exactly. The experimental results show that the average rate of successful separation is greater than 91.2% with the proposed method, which can meet the requirements of applications in harvesting robots

Performance improvements of a sweet pepper harvesting robot in protected cropping environments

Using robots to harvest sweet peppers in protected cropping environments has remained unsolved despite considerable effort by the research community over several decades. In this paper, we present the robotic harvester, Harvey, designed for sweet peppers in protected cropping environments that achieved a 76.5% success rate on 68 fruit (within a modified scenario) which improves upon our prior work which achieved 58% on 24 fruit and related sweet pepper harvesting work which achieved 33% on 39 fruit (for their best tool in a modified scenario). This improvement was primarily achieved through the introduction of a novel peduncle segmentation system using an efficient deep convolutional neural network, in conjunction with three-dimensional postfiltering to detect the critical cutting location. We benchmark the peduncle segmentation against prior art demonstrating an improvement in performance with a (Formula presented.) score of 0.564 compared to 0.302. The robotic harvester uses a perception pipeline to detect a target sweet pepper and an appropriate grasp and cutting pose used to determine the trajectory of a multimodal harvesting tool to grasp the sweet pepper and cut it from the plant. A novel decoupling mechanism enables the gripping and cutting operations to be performed independently. We perform an in-depth analysis of the full robotic harvesting system to highlight bottlenecks and failure points that future work could address.</p