NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image

This paper reviews the second challenge on spectral reconstruction from RGB images, i.e., the recovery of whole- scene hyperspectral (HS) information from a 3-channel RGB image. As in the previous challenge, two tracks were provided: (i) a "Clean" track where HS images are estimated from noise-free RGBs, the RGB images are themselves calculated numerically using the ground-truth HS images and supplied spectral sensitivity functions (ii) a "Real World" track, simulating capture by an uncalibrated and unknown camera, where the HS images are recovered from noisy JPEG-compressed RGB images. A new, larger-than-ever, natural hyperspectral image data set is presented, containing a total of 510 HS images. The Clean and Real World tracks had 103 and 78 registered participants respectively, with 14 teams competing in the final testing phase. A description of the proposed methods, alongside their challenge scores and an extensive evaluation of top performing methods is also provided. They gauge the state-of-the-art in spectral reconstruction from an RGB image

Controlled Lighting and Illumination-Independent Target Detection for Real-Time Cost-Efficient Applications. The Case Study of Sweet Pepper Robotic Harvesting

Current harvesting robots are limited by low detection rates due to the unstructured and dynamic nature of both the objects and the environment. State-of-the-art algorithms include color- and texture-based detection, which are highly sensitive to the illumination conditions. Deep learning algorithms promise robustness at the cost of significant computational resources and the requirement for intensive databases. In this paper we present a Flash-No-Flash (FNF) controlled illumination acquisition protocol that frees the system from most ambient illumination effects and facilitates robust target detection while using only modest computational resources and no supervised training. The approach relies on the simultaneous acquisition of two images—with/without strong artificial lighting (“Flash„/“no-Flash„). The difference between these images represents the appearance of the target scene as if only the artificial light was present, allowing a tight control over ambient light for color-based detection. A performance evaluation database was acquired in greenhouse conditions using an eye-in-hand RGB camera mounted on a robotic manipulator. The database includes 156 scenes with 468 images containing a total of 344 yellow sweet peppers. Performance of both color blob and deep-learning detection algorithms are compared on Flash-only and FNF images. The collected database is made public

SWEEPER Sweet Pepper Harvesting Robot : Report on test scenarios and definition performance measures

Deliverable 7.1 reports the lay-out and the objectives of the experimental setups for individual module testing and integrated system testing of the sweet-pepper harvesting robot. Testing environments and conditions for laboratory test as well as for greenhouse experiments are defined and described. The laboratory environment and conditions are created to mimic the end-user greenhouse. This includes imitation sweet-pepper plant parts and an indoor rail system. The laboratory setup is created to be more controlled and has less variance than a real greenhouse. Further on performance indicators are specified for all hard- and software modules. Modules to be tested can be categorized in hardware (trolley, manipulator, end-effector, cameras, illumination, depth sensor) and software (GUI, state machine, motion planning, fruit detection, ripeness determination, obstacle detection). For the integrated system performance indicators were selected based on measures known from the predecessor project Crops. These measures include next to others the overall harvest success rate, fruit damage rate and cycle time. Different test scenarios for both laboratory and greenhouse conditions are defined to be able to obtain quantitative data of the performance of individual modules and also of the integrated system. For the modules of the advanced system more detailed information will become available later in the project and after the evaluation of the results of the basic system. Upcoming deliverables of work package 7 will therefore contain a special section describing the reviewed and updated test plan and the performance measures for these modules

Robotic data acquisition of sweet pepper images for research and development

A main problem limiting the development of robotic harvesters is robust fruit detection [5]. Despite intensive research conducted in identifying the fruits and their location [2,3], current fruit detection algorithms have a limited detection rate of 0.87 which is unfeasible from an economic perspective [5]. The complexity of the fruit detection task is due to the unstructured and dynamic nature of both the objects and the environment [4-6]: the fruit have inherent high variability in size, shape, texture, and location; occlusion and variable illumination conditions significantly influence the detection performance[3]. A common practice for image processing R&D for complicated problems is the acquisition of a large database (e.g., Labelme open source labeling database [1], Oxford building dataset [2]). These datasets enable to advance vision algorithms development [7] and provide a benchmark for evaluating new algorithms. To the best of our knowledge, to date there is no open dataset available for R&D in image processing of agricultural objects. Evaluation of previously reported algorithms was based on limited data [5]. Previous research indicated the importance of evaluating algorithms for a wide range of sensory, crop, and environmental conditions [5]. A robotic acquisition system and procedure was developed using a 6 degree of freedom manipulator, equipped with 3 different sensors to automatically acquire images from several viewpoints with different sensors and illumination conditions. Measurements were conducted along the day and at night in a commercial greenhouse and resulted in a total of 1764 images from 14 viewpoints for each scene. Additionally, drawbacks and advantages of the proposed approach as compared to other approaches previously utilized will be discussed along with recommendations for future acquisitions

Development of a sweet pepper harvesting robot

This paper presents the development, testing and validation of SWEEPER, a robot for harvesting sweet pepper fruit in greenhouses. The robotic system includes a six degrees of freedom industrial arm equipped with a specially designed end effector, RGB-D camera, high-end computer with graphics processing unit, programmable logic controllers, other electronic equipment, and a small container to store harvested fruit. All is mounted on a cart that autonomously drives on pipe rails and concrete floor in the end-user environment. The overall operation of the harvesting robot is described along with details of the algorithms for fruit detection and localization, grasp pose estimation, and motion control. The main contributions of this paper are the integrated system design and its validation and extensive field testing in a commercial greenhouse for different varieties and growing conditions. A total of 262 fruits were involved in a 4-week long testing period. The average cycle time to harvest a fruit was 24â\u80\u89s. Logistics took approximately 50% of this time (7.8â\u80\u89s for discharge of fruit and 4.7â\u80\u89s for platform movements). Laboratory experiments have proven that the cycle time can be reduced to 15â\u80\u89s by running the robot manipulator at a higher speed. The harvest success rates were 61% for the best fit crop conditions and 18% in current crop conditions. This reveals the importance of finding the best fit crop conditions and crop varieties for successful robotic harvesting. The SWEEPER robot is the first sweet pepper harvesting robot to demonstrate this kind of performance in a commercial greenhouse