Automatic plant features recognition using stereo vision for crop monitoring

Machine vision and robotic technologies have potential to accurately monitor plant parameters which reflect plant stress and water requirements, for use in farm management decisions. However, autonomous identification of individual plant leaves on a growing plant under natural conditions is a challenging task for vision-guided agricultural robots, due to the complexity of data relating to various stage of growth and ambient environmental conditions. There are numerous machine vision studies that are concerned with describing the shape of leaves that are individually-presented to a camera. The purpose of these studies is to identify plant species, or for the autonomous detection of multiple leaves from small seedlings under greenhouse conditions. Machine vision-based detection of individual leaves and challenges presented by overlapping leaves on a developed plant canopy using depth perception properties under natural outdoor conditions is yet to be reported. Stereo vision has recently emerged for use in a variety of agricultural applications and is expected to provide an accurate method for plant segmentation and identification which can benefit from depth properties and robustness. This thesis presents a plant leaf extraction algorithm using a stereo vision sensor. This algorithm is used on multiple leaf segmentation and overlapping leaves separation using a combination of image features, specifically colour, shape and depth. The separation between the connected and the overlapping leaves relies on the measurement of the discontinuity in depth gradient for the disparity maps. Two techniques have been developed to implement this task based on global and local measurement. A geometrical plane from each segmented leaf can be extracted and used to parameterise a 3D model of the plant image and to measure the inclination angle of each individual leaf. The stem and branch segmentation and counting method was developed based on the vesselness measure and Hough transform technique. Furthermore, a method for reconstructing the segmented parts of hibiscus plants is presented and a 2.5D model is generated for the plant. Experimental tests were conducted with two different selected plants: cotton of different sizes, and hibiscus, in an outdoor environment under varying light conditions. The proposed algorithm was evaluated using 272 cotton and hibiscus plant images. The results show an observed enhancement in leaf detection when utilising depth features, where many leaves in various positions and shapes (single, touching and overlapping) were detected successfully. Depth properties were more effective in separating between occluded and overlapping leaves with a high separation rate of 84% and these can be detected automatically without adding any artificial tags on the leaf boundaries. The results exhibit an acceptable segmentation rate of 78% for individual plant leaves thereby differentiating the leaves from their complex backgrounds and from each other. The results present almost identical performance for both species under various lighting and environmental conditions. For the stem and branch detection algorithm, experimental tests were conducted on 64 colour images of both species under different environmental conditions. The results show higher stem and branch segmentation rates for hibiscus indoor images (82%) compared to hibiscus outdoor images (49.5%) and cotton images (21%). The segmentation and counting of plant features could provide accurate estimation about plant growth parameters which can be beneficial for many agricultural tasks and applications

Automated Find Fix and Track with a Medium Altitude Long Endurance Remotely Piloted Aircraft

A limitation in RPA ISR operations is loss of target track if the command link is severed. For an RPA to effectively execute the ISR mission without a command link, it needs the capability to F2T targets autonomously. Automated Find Fix and Track (AFFTRAC) was developed to help solve this problem by demonstrating a proof of concept tactical autopilot. Monocular stereo vision was used to process sequential images acquired during orbit to produce a partial structural point cloud of the original structure. This partial structural point cloud was then exploited to create a holding area density for the aircraft to stay within. A simple greedy algorithm exploited this holding area density to produce aircraft turn commands to approximate tactical ISR holding. The result was that imagery from existing MQ-9 sensors was used to provide command guidance to autonomously to maintain line of sight to a target. Overall, AFFTRAC is a promising initial framework for a tactical autopilot, but additional development is needed to mature component algorithms

Computational intelligence approaches to robotics, automation, and control [Volume guest editors]

No abstract available

Local Features to a Global View: Recognition of Occluded Objects by Spectral Matching Using Pairwise Feature Relationships

Ph.DDOCTOR OF PHILOSOPH

View synthesis for kinetic depth X-ray imaging

This thesis reports the development and analysis of feature based synthesis of transmission X-ray images. The synthetic imagery is formed through matching and morphing or warping line-scan format images produced by a novel multi-view X-ray machine. In this way video type sequences, which periodically alternate between synthetic and detector based views, may be formed. The purpose of these sequences is to provide depth from motion or kinetic depth effect (KDE) in a visual display; while the role of the synthesis is to reduce the total number of detector arrays, associated collimators and X-ray flux per inspection. A specific challenge is to explore the bounds for producing synthetic imagery that can be seamlessly introduced into the resultant sequences. This work is distinct from the image collection and display technique, termed KDEX, previously undertaken by the Imaging Science Group at NTU. The ultimate aim of the research programme in collaboration with The UK Home Office and The US Dept. of Homeland Security is to enhance the detection and identification of threats in X-ray scans of luggage. A multi-view „KDEX scanner‟ was employed to collect greyscale and colour coded image sequences of 30 different bags; each sequence comprised of 7 perspective views separated from one another by 10. This imagery was organised and stored in a database to enable a coherent series of experiments to be conducted. Corresponding features in sequential pairs of images, at various different angular separations, were identified by applying a scale invariant feature transform (SIFT)

Feature-based hybrid inspection planning for complex mechanical parts

Globalization and emerging new powers in the manufacturing world are among many challenges, major manufacturing enterprises are facing. This resulted in increased alternatives to satisfy customers\u27 growing needs regarding products\u27 aesthetic and functional requirements. Complexity of part design and engineering specifications to satisfy such needs often require a better use of advanced and more accurate tools to achieve good quality. Inspection is a crucial manufacturing function that should be further improved to cope with such challenges. Intelligent planning for inspection of parts with complex geometric shapes and free form surfaces using contact or non-contact devices is still a major challenge. Research in segmentation and localization techniques should also enable inspection systems to utilize modern measurement technologies capable of collecting huge number of measured points. Advanced digitization tools can be classified as contact or non-contact sensors. The purpose of this thesis is to develop a hybrid inspection planning system that benefits from the advantages of both techniques. Moreover, the minimization of deviation of measured part from the original CAD model is not the only characteristic that should be considered when implementing the localization process in order to accept or reject the part; geometric tolerances must also be considered. A segmentation technique that deals directly with the individual points is a necessary step in the developed inspection system, where the output is the actual measured points, not a tessellated model as commonly implemented by current segmentation tools. The contribution of this work is three folds. First, a knowledge-based system was developed for selecting the most suitable sensor using an inspection-specific features taxonomy in form of a 3D Matrix where each cell includes the corresponding knowledge rules and generate inspection tasks. A Travel Salesperson Problem (TSP) has been applied for sequencing these hybrid inspection tasks. A novel region-based segmentation algorithm was developed which deals directly with the measured point cloud and generates sub-point clouds, each of which represents a feature to be inspected and includes the original measured points. Finally, a new tolerance-based localization algorithm was developed to verify the functional requirements and was applied and tested using form tolerance specifications. This research enhances the existing inspection planning systems for complex mechanical parts with a hybrid inspection planning model. The main benefits of the developed segmentation and tolerance-based localization algorithms are the improvement of inspection decisions in order not to reject good parts that would have otherwise been rejected due to misleading results from currently available localization techniques. The better and more accurate inspection decisions achieved will lead to less scrap, which, in turn, will reduce the product cost and improve the company potential in the market