Robotic Crop Interaction in Agriculture for Soft Fruit Harvesting

Autonomous tree crop harvesting has been a seemingly attainable, but elusive, robotics goal for the past several decades. Limiting grower reliance on uncertain seasonal labour is an economic driver of this, but the ability of robotic systems to treat each plant individually also has environmental benefits, such as reduced emissions and fertiliser use. Over the same time period, effective grasping and manipulation (G&M) solutions to warehouse product handling, and more general robotic interaction, have been demonstrated. Despite research progress in general robotic interaction and harvesting of some specific crop types, a commercially successful robotic harvester has yet to be demonstrated. Most crop varieties, including soft-skinned fruit, have not yet been addressed. Soft fruit, such as plums, present problems for many of the techniques employed for their more robust relatives and require special focus when developing autonomous harvesters. Adapting existing robotics tools and techniques to new fruit types, including soft skinned varieties, is not well explored. This thesis aims to bridge that gap by examining the challenges of autonomous crop interaction for the harvesting of soft fruit. Aspects which are known to be challenging include mixed obstacle planning with both hard and soft obstacles present, poor outdoor sensing conditions, and the lack of proven picking motion strategies. Positioning an actuator for harvesting requires solving these problems and others specific to soft skinned fruit. Doing so effectively means addressing these in the sensing, planning and actuation areas of a robotic system. Such areas are also highly interdependent for grasping and manipulation tasks, so solutions need to be developed at the system level. In this thesis, soft robotics actuators, with simplifying assumptions about hard obstacle planes, are used to solve mixed obstacle planning. Persistent target tracking and filtering is used to overcome challenging object detection conditions, while multiple stages of object detection are applied to refine these initial position estimates. Several picking motions are developed and tested for plums, with varying degrees of effectiveness. These various techniques are integrated into a prototype system which is validated in lab testing and extensive field trials on a commercial plum crop. Key contributions of this thesis include I. The examination of grasping & manipulation tools, algorithms, techniques and challenges for harvesting soft skinned fruit II. Design, development and field-trial evaluation of a harvester prototype to validate these concepts in practice, with specific design studies of the gripper type, object detector architecture and picking motion for this III. Investigation of specific G&M module improvements including: o Application of the autocovariance least squares (ALS) method to noise covariance matrix estimation for visual servoing tasks, where both simulated and real experiments demonstrated a 30% improvement in state estimation error using this technique. o Theory and experimentation showing that a single range measurement is sufficient for disambiguating scene scale in monocular depth estimation for some datasets. o Preliminary investigations of stochastic object completion and sampling for grasping, active perception for visual servoing based harvesting, and multi-stage fruit localisation from RGB-Depth data. Several field trials were carried out with the plum harvesting prototype. Testing on an unmodified commercial plum crop, in all weather conditions, showed promising results with a harvest success rate of 42%. While a significant gap between prototype performance and commercial viability remains, the use of soft robotics with carefully chosen sensing and planning approaches allows for robust grasping & manipulation under challenging conditions, with both hard and soft obstacles

Design and implementation of a compliant robot with force feedback and strategy planning software

Force-feedback robotics techniques are being developed for automated precision assembly and servicing of NASA space flight equipment. Design and implementation of a prototype robot which provides compliance and monitors forces is in progress. Computer software to specify assembly steps and makes force feedback adjustments during assembly are coded and tested for three generically different precision mating problems. A model program demonstrates that a suitably autonomous robot can plan its own strategy

3D Printed Soft Robotic Hand

Soft robotics is an emerging industry, largely dominated by companies which hand mold their actuators. Our team set out to design an entirely 3D printed soft robotic hand, powered by a pneumatic control system which will prove both the capabilities of soft robots and those of 3D printing. Through research, computer aided design, finite element analysis, and experimental testing, a functioning actuator was created capable of a deflection of 2.17” at a maximum pressure input of 15 psi. The single actuator was expanded into a 4 finger gripper and the design was printed and assembled. The created prototype was ultimately able to lift both a 100-gram apple and a 4-gram pill, proving its functionality in two prominent industries: pharmaceutical and food packing

Ground Robotic Hand Applications for the Space Program study (GRASP)

This document reports on a NASA-STDP effort to address research interests of the NASA Kennedy Space Center (KSC) through a study entitled, Ground Robotic-Hand Applications for the Space Program (GRASP). The primary objective of the GRASP study was to identify beneficial applications of specialized end-effectors and robotic hand devices for automating any ground operations which are performed at the Kennedy Space Center. Thus, operations for expendable vehicles, the Space Shuttle and its components, and all payloads were included in the study. Typical benefits of automating operations, or augmenting human operators performing physical tasks, include: reduced costs; enhanced safety and reliability; and reduced processing turnaround time

Autonomous Grasping Using Novel Distance Estimator

This paper introduces a novel distance estimator using monocular vision for autonomous underwater grasping. The presented method is also applicable to topside grasping operations. The estimator is developed for robot manipulators with a monocular camera placed near the gripper. The fact that the camera is attached near the gripper makes it possible to design a method for capturing images from different positions, as the relative position change can be measured. The presented system can estimate relative distance to an object of unknown size with good precision. The manipulator applied in the presented work is the SeaArm-2, a fully electric underwater small modular manipulator. The manipulator is unique in its integrated monocular camera in the end-effector module, and its design facilitates the use of different end-effector tools. The camera is used for supervision, object detection, and tracking. The distance estimator was validated in a laboratory setting through autonomous grasping experiments. The manipulator was able to search for and find, estimate the relative distance of, grasp, and retrieve the relevant object in 12 out of 12 trials.publishedVersio

Integrating sensors and actuators for robotic assembly

This thesis addresses the problem of integrating sensors and actuators for closed-loop control of a robotic assembly cell. In addition to the problems of interfacing the physical components of the work-cell, the difficulties of representing sensory feedback at a high level within the robot control program are investigated. A new level of robot programming, called sensor-level programming, is introduced. In this, the movements of the actuators are not given explicitly, but rather are inferred by the programming system to achieve new sensor conditions given by the programmer.Control of each sensor and actuator is distributed through a master-slave hierarchy, with each sensor and actuator having its own slave controller. A protocol for information interchange between each controller and the master is defined. If possible, the control of the kinematics of a robot arm is achieved through the manufacturer's existing control system. Under these circumstances, the actuator slave would be acting as aninterface between the generic command codes issued from the central controller, and the syntax of the corresponding control instructions required by the commercial system.Sensor information is preprocessed in the sensor slaves and a set of high-level descriptors, called attributes, are sent to the central controller. Closed-loop control is achieved on the basis of these attributes.The processing of sensor information which is corrupted by noise is investigated. Sources of sensor noise are identified and new algorithms are developed to quantify the noise based on information obtained from the closed-loop servoing. Once the relative magnitudes of the system andmeasurement noise have been estimated, a Kalman filter is used to weight the sensor information and hence reduce the credibility given to noisy sensors; in the limit ignoring the information completely. The improvements in system performance by processing the sensor information in this way are demonstrated.The sensor-level representation and automatic error processing are embedded in a software control system, which can be used to interface commercial systems as well as purpose-built devices. An'industrial research project associated with the lay-up of carbon-fibre provides anexample of its operation.A list of publications resulting from the work in this thesis is given in Appendix E