Segmentation and 3D reconstruction of rose plants from stereoscopic images

The method proposed in this paper is part of the vision module of a garden robot capable of navigating towards rose bushes and clip them according to a set of pruning rules. The method is responsible for performing the segmentation of the branches and recovering their morphology in 3D. The obtained reconstruction allows the manipulator of the robot to select the candidate branches to be pruned. This method first obtains a stereo pair of images and calculates the disparity image using block matching and the segmentation of the branches using a Fully Convolutional Neuronal Network modified to return a map with the probability at the pixel level of the presence of a branch. A post-processing step combines the segmentation and the disparity in order to improve the results. Then, the skeleton of the plant and the branching structure are calculated, and finally, the 3D reconstruction is obtained. The proposed approach is evaluated with five different datasets, three of them compiled by the authors and two from the state of the art, including indoor and outdoor scenes with uncontrolled environments. The different steps of the proposed pipeline are evaluated and compared with other state-of-the-art methods, showing that the accuracy of the segmentation improves other methods for this task, even with variable lighting, and also that the skeletonization and the reconstruction processes obtain robust results.This work was funded by the European Horizon 2020 program, under the project TrimBot2020 (Grant No. 688007)

Inter- and Intra-species Variation in Three Crown Condition Indicators for Seven Tree Species in the Southeastern United States

The USDA Forest Service utilizes assessments of tree crowns, specifically crown density, crown dieback, and foliage transparency, to accomplish in part its mission of reporting the long-term status, changes, and trends in forest ecosystem health in the United States. To aid interpretation and provide general guidelines of health across all species the crown condition assessments are classed into categories ranging from “good” to “poor.” The purpose of this research was to evaluate and describe the variation in crown density, crown dieback, and foliage transparency between and within species, and to critique the appropriateness of the current threshold levels. In addition, inter-observer deviation between two assessment crews was evaluated for crown density; however, the attempts to effectively predict between-crew variation were unsuccessful. The seven species included in the analyses were slash pine (Pinus elliottii), loblolly pine (Pinus taeda), Virginia pine (Pinus virginiana), red maple (Acer rubrum), sweetgum (Liquidambar styraciflua), yellow-poplar (Liriodendron tulipifera), and white oak (Quercus alba). Between- and within-species differences were determined via pair-wise comparisons at the 10h, 25th, 50th, 75th, and 90th percentiles of the empirical distribution function of each crown condition indicator. Random “error” drawn from uniform distributions on the intervals (-2.5, +2.5) and (-7.5, +7.5) was added to the percentile estimates in order to capture the possible within-crew variation in the crown assessments. Bootstrapping was used to compute two-sided 90 percent confidence intervals (CIs) for each percentile with the percentile CI method. A clear gradient of expected crown conditions was found among the species, but uncertainty in the data made it difficult to confidently pinpoint species-specific differences for the three crown condition indicators. Assuming limited measurement error in the data, the greatest disparity among species was found in crown density. Dissimilarity was apparent between hardwood and softwood crown densities in general, but only scattered differences were found among the species in each group. In terms of foliage transparency, Virginia pine was the most dissimilar overall. No major differences were found among the species in terms of crown dieback. In addition, relatively little variation was found within the two species (loblolly pine and sweetgum) examined for intraspecies variation. Modifications to the current threshold levels were recommended for all three crown condition indicators. The suggested changes resulted in only small adjustments to the percentage of observations in each category and better reflect the distribution of observations across the range of the crown conditions. The proposed thresholds are: • crown density: exceptional, 51-100 percent; good, 41-50 percent;moderate, 31-40 percent; and poor, 0-30 percent; • crown dieback: none, 0-5 percent; light, 6-19 percent; moderate, 20-35 percent; and severe, 36-100 percent; and • foliage transparency: normal, 0-20 percent; moderate, 21-40 percent; and severe, 41-100 percent

3D segmentation and localization using visual cues in uncontrolled environments

3D scene understanding is an important area in robotics, autonomous vehicles, and virtual reality. The goal of scene understanding is to recognize and localize all the objects around the agent. This is done through semantic segmentation and depth estimation. Current approaches focus on improving the robustness to solve each task but fail in making them efficient for real-time usage. This thesis presents four efficient methods for scene understanding that work in real environments. The methods also aim to provide a solution for 2D and 3D data. The first approach presents a pipeline that combines the block matching algorithm for disparity estimation, an encoder-decoder neural network for semantic segmentation, and a refinement step that uses both outputs to complete the regions that were not labelled or did not have any disparity assigned to them. This method provides accurate results in 3D reconstruction and morphology estimation of complex structures like rose bushes. Due to the lack of datasets of rose bushes and their segmentation, we also made three large datasets. Two of them have real roses that were manually labelled, and the third one was created using a scene modeler and 3D rendering software. The last dataset aims to capture diversity, realism and obtain different types of labelling. The second contribution provides a strategy for real-time rose pruning using visual servoing of a robotic arm and our previous approach. Current methods obtain the structure of the plant and plan the cutting trajectory using only a global planner and assume a constant background. Our method works in real environments and uses visual feedback to refine the location of the cutting targets and modify the planned trajectory. The proposed visual servoing allows the robot to reach the cutting points 94% of the time. This is an improvement compared to only using a global planner without visual feedback, which reaches the targets 50% of the time. To the best of our knowledge, this is the first robot able to prune a complete rose bush in a natural environment. Recent deep learning image segmentation and disparity estimation networks provide accurate results. However, most of these methods are computationally expensive, which makes them impractical for real-time tasks. Our third contribution uses multi-task learning to learn the image segmentation and disparity estimation together end-to-end. The experiments show that our network has at most 1/3 of the parameters of the state-of-the-art of each individual task and still provides competitive results. The last contribution explores the area of scene understanding using 3D data. Recent approaches use point-based networks to do point cloud segmentation and find local relations between points using only the latent features provided by the network, omitting the geometric information from the point clouds. Our approach aggregates the geometric information into the network. Given that the geometric and latent features are different, our network also uses a two-headed attention mechanism to do local aggregation at the latent and geometric level. This additional information helps the network to obtain a more accurate semantic segmentation, in real point cloud data, using fewer parameters than current methods. Overall, the method obtains the state-of-the-art segmentation in the real datasets S3DIS with 69.2% and competitive results in the ModelNet40 and ShapeNetPart datasets

Satisfying giant appetites : mechanisms of small scale foraging by large African herbivores

Variation in body mass allows for resource partitioning and co-existence of different species. Body mass is also seen as the main factor governing nutrient requirements in herbivores as metabolic rate and requirements have often been found to scale to ¾ power of body mass. Although the consequences of body mass on foraging behaviour of herbivores has been extensively studied, the mechanism behind how body mass differences determines the small scale foraging patterns of especially larger herbivores, has up to now been unclear. In this study, I looked at how body mass and small scale vegetation characteristics shaped the mouth morphology of herbivores and how body mass of a herbivore affects the scale at which intake is maximized. The results indicate that the dilution of plant mass and more specifically leaf mass in space requires that mega-herbivores such as elephant have enlarged soft mouth parts to compensate for this dilution. Finally, I demonstrate, using linear programming techniques with multiple nutrients as constraints, how a mega-herbivore’s daily diet choice is determined by forage abundance whereas a small herbivore is more constrained by fibre

Aerospace medicine and biology: A continuing bibliography with indexes, supplement 128, May 1974

This special bibliography lists 282 reports, articles, and other documents introduced into the NASA scientific and technical information system in April 1974

Clinical Decision Support Systems with Game-based Environments, Monitoring Symptoms of Parkinson’s Disease with Exergames

Parkinson’s Disease (PD) is a malady caused by progressive neuronal degeneration, deriving in several physical and cognitive symptoms that worsen with time. Like many other chronic diseases, it requires constant monitoring to perform medication and therapeutic adjustments. This is due to the significant variability in PD symptomatology and progress between patients. At the moment, this monitoring requires substantial participation from caregivers and numerous clinic visits. Personal diaries and questionnaires are used as data sources for medication and therapeutic adjustments. The subjectivity in these data sources leads to suboptimal clinical decisions. Therefore, more objective data sources are required to better monitor the progress of individual PD patients. A potential contribution towards more objective monitoring of PD is clinical decision support systems. These systems employ sensors and classification techniques to provide caregivers with objective information for their decision-making. This leads to more objective assessments of patient improvement or deterioration, resulting in better adjusted medication and therapeutic plans. Hereby, the need to encourage patients to actively and regularly provide data for remote monitoring remains a significant challenge. To address this challenge, the goal of this thesis is to combine clinical decision support systems with game-based environments. More specifically, serious games in the form of exergames, active video games that involve physical exercise, shall be used to deliver objective data for PD monitoring and therapy. Exergames increase engagement while combining physical and cognitive tasks. This combination, known as dual-tasking, has been proven to improve rehabilitation outcomes in PD: recent randomized clinical trials on exergame-based rehabilitation in PD show improvements in clinical outcomes that are equal or superior to those of traditional rehabilitation. In this thesis, we present an exergame-based clinical decision support system model to monitor symptoms of PD. This model provides both objective information on PD symptoms and an engaging environment for the patients. The model is elaborated, prototypically implemented and validated in the context of two of the most prominent symptoms of PD: (1) balance and gait, as well as (2) hand tremor and slowness of movement (bradykinesia). While balance and gait affections increase the risk of falling, hand tremors and bradykinesia affect hand dexterity. We employ Wii Balance Boards and Leap Motion sensors, and digitalize aspects of current clinical standards used to assess PD symptoms. In addition, we present two dual-tasking exergames: PDDanceCity for balance and gait, and PDPuzzleTable for tremor and bradykinesia. We evaluate the capability of our system for assessing the risk of falling and the severity of tremor in comparison with clinical standards. We also explore the statistical significance and effect size of the data we collect from PD patients and healthy controls. We demonstrate that the presented approach can predict an increased risk of falling and estimate tremor severity. Also, the target population shows a good acceptance of PDDanceCity and PDPuzzleTable. In summary, our results indicate a clear feasibility to implement this system for PD. Nevertheless, long-term randomized clinical trials are required to evaluate the potential of PDDanceCity and PDPuzzleTable for physical and cognitive rehabilitation effects

Distribution automation applications of fiber optics

Motivations for interest and research in distribution automation are discussed. The communication requirements of distribution automation are examined and shown to exceed the capabilities of power line carrier, radio, and telephone systems. A fiber optic based communication system is described that is co-located with the distribution system and that could satisfy the data rate and reliability requirements. A cost comparison shows that it could be constructed at a cost that is similar to that of a power line carrier system. The requirements for fiber optic sensors for distribution automation are discussed. The design of a data link suitable for optically-powered electronic sensing is presented. Empirical results are given. A modeling technique that was used to understand the reflections of guided light from a variety of surfaces is described. An optical position-indicator design is discussed. Systems aspects of distribution automation are discussed, in particular, the lack of interface, communications, and data standards. The economics of distribution automation are examined

Instrumented shoes for daily activity monitoring in healthy and at risk populations

Daily activity reflects the health status of an individual. Ageing and disease drastically affect all dimensions of mobility, from the number of active bouts to their duration and intensity. Performing less activity leads to muscle deterioration and further weakness that could lead to increased fall risk. Gait performance is also affected by ageing and could be detrimental for daily mobility. Therefore, activity monitoring in older adults and at risk persons is crucial to obtain relevant quantitative information about daily life performance. Activity evaluation has mainly been established through questionnaires or daily logs. These methods are simple but not sufficiently accurate and are prone to errors. With the advent of microelectromechanical systems (MEMS), the availability of wearable sensors has shifted activity analysis towards ambulatory monitoring. In particular, inertial measurement units consisting of accelerometers and gyroscopes have shown to be extremely relevant for characterizing human movement. However, monitoring daily activity requires comfortable and easy to use systems that are strategically placed on the body or integrated in clothing to avoid movement hindrance. Several research based systems have employed multiple sensors placed at different locations, capable of recognizing activity types with high accuracy, but not comfortable for daily use. Single sensor systems have also been used but revealed inaccuracies in activity recognition. To this end, we propose an instrumented shoe system consisting of an inertial measurement unit and a pressure sensing insole with all the sensors placed at the shoe/foot level. By measuring the foot movement and loading, the recognition of locomotion and load bearing activities would be appropriate for activity classification. Furthermore, inertial measurement units placed on the foot can perform detailed gait analysis, providing the possibility of characterizing locomotion. The system and dedicated activity classification algorithms were first designed, tested and validated during the first part of the thesis. Their application to clinical rehabilitation of at risk persons was demonstrated over the second part. In the first part of the thesis, the designed instrumented shoes system was tested in standardized conditions with healthy elderly subjects performing a sequence of structured activities. An algorithm based on movement biomechanics was built to identify each activity, namely sitting, standing, level walking, stairs, ramps, and elevators. The rich array of sensors present in the system included a 3D accelerometer, 3D gyroscope, 8 force sensors, and a barometer allowing the algorithm to reach a high accuracy in classifying different activity types. The tuning parameters of the algorithm were shown to be robust to small changes, demonstrating the suitability of the algorithm to activity classification in older adults. Next, the system was tested in daily life conditions on the same elderly participants. Using a wearable reference system, the concurrent validity of the instrumented shoes in classifying daily activity was shown. Additionally, daily gait metrics were obtained and compared to the literature. Further insight into the relationship between some gait parameters as well as a global activity metric, the activity âcomplexityâ, was discussed. Participants positively rated their comfort while using the system... (Please refer to thesis for full abstract