A Robotic System for Learning Visually-Driven Grasp Planning (Dissertation Proposal)

We use findings in machine learning, developmental psychology, and neurophysiology to guide a robotic learning system\u27s level of representation both for actions and for percepts. Visually-driven grasping is chosen as the experimental task since it has general applicability and it has been extensively researched from several perspectives. An implementation of a robotic system with a gripper, compliant instrumented wrist, arm and vision is used to test these ideas. Several sensorimotor primitives (vision segmentation and manipulatory reflexes) are implemented in this system and may be thought of as the innate perceptual and motor abilities of the system. Applying empirical learning techniques to real situations brings up such important issues as observation sparsity in high-dimensional spaces, arbitrary underlying functional forms of the reinforcement distribution and robustness to noise in exemplars. The well-established technique of non-parametric projection pursuit regression (PPR) is used to accomplish reinforcement learning by searching for projections of high-dimensional data sets that capture task invariants. We also pursue the following problem: how can we use human expertise and insight into grasping to train a system to select both appropriate hand preshapes and approaches for a wide variety of objects, and then have it verify and refine its skills through trial and error. To accomplish this learning we propose a new class of Density Adaptive reinforcement learning algorithms. These algorithms use statistical tests to identify possibly interesting regions of the attribute space in which the dynamics of the task change. They automatically concentrate the building of high resolution descriptions of the reinforcement in those areas, and build low resolution representations in regions that are either not populated in the given task or are highly uniform in outcome. Additionally, the use of any learning process generally implies failures along the way. Therefore, the mechanics of the untrained robotic system must be able to tolerate mistakes during learning and not damage itself. We address this by the use of an instrumented, compliant robot wrist that controls impact forces

Integrated 2-D Optical Flow Sensor

I present a new focal-plane analog VLSI sensor that estimates optical flow in two visual dimensions. The chip significantly improves previous approaches both with respect to the applied model of optical flow estimation as well as the actual hardware implementation. Its distributed computational architecture consists of an array of locally connected motion units that collectively solve for the unique optimal optical flow estimate. The novel gradient-based motion model assumes visual motion to be translational, smooth and biased. The model guarantees that the estimation problem is computationally well-posed regardless of the visual input. Model parameters can be globally adjusted, leading to a rich output behavior. Varying the smoothness strength, for example, can provide a continuous spectrum of motion estimates, ranging from normal to global optical flow. Unlike approaches that rely on the explicit matching of brightness edges in space or time, the applied gradient-based model assures spatiotemporal continuity on visual information. The non-linear coupling of the individual motion units improves the resulting optical flow estimate because it reduces spatial smoothing across large velocity differences. Extended measurements of a 30x30 array prototype sensor under real-world conditions demonstrate the validity of the model and the robustness and functionality of the implementation

Classification of Marine Vessels in a Littoral Environment Using a Novel Training Database

Research into object classification has led to the creation of hundreds of databases for use as training sets in object classification algorithms. Datasets made up of thousands of cars, people, boats, faces and everyday objects exist for general classification techniques. However, no commercially available database exists for use with detailed classification and categorization of marine vessels commonly found in littoral environments. This research seeks to fill this void and is the combination of a multi-stage research endeavor designed to provide the missing marine vessel ontology. The first of the two stages performed to date introduces a novel training database called the Lister Littoral Database 900 (LLD-900) made up of over 900 high-quality images. These images consist of high-resolution color photos of marine vessels in working, active conditions taken directly from the field and edited for best possible use. Segmentation masks of each boat have been developed to separate the image into foreground and background sections. Segmentation masks that include boat wakes as part of the foreground section are the final image type included. These are included to allow for wake affordance detection algorithms rely on the small changes found in wakes made by different moving vessels. Each of these three types of images are split into their respective general classification folders, which consist of a differing number of boat categories dependent on the research stage. In the first stage of research, the initial database is tested using a simple, readily available classification algorithm known as the Nearest Neighbor Classifier. The accuracy of the database as a training set is tested and recorded and potential improvements are documented. The second stage incorporates these identified improvements and reconfigures the database before retesting the modifications using the same Nearest Neighbor Classifier along with two new methods known as the K-Nearest Neighbor Classifier and the Min-Mean Distance Classifier. These additional algorithms are also readily available and offer basic classification testing using different classification techniques. Improvements in accuracy are calculated and recorded. Finally, further improvements for a possible third iteration are discussed. The goal of this research is to establish the basis for a training database to be used with classification algorithms to increase the security of ports, harbors, shipping channels and bays. The purpose of the database is to train existing and newly created algorithms to properly identify and classify all boats found in littoral areas so that anomalous behavior detection techniques can be applied to determine when a threat is present. This research represents the completion of the initial steps in accomplishing this goal delivering a novel framework for use with littoral area marine vessel classification. The completed work is divided and presented in two separate papers written specifically for submission to and publication at appropriate conferences. When fully integrated with computer vision techniques, the database methodology and ideas presented in this thesis research will help to provide a vital new level of security in the littoral areas around the world

Automatic Change-based Diagnosis of Structures Using Spatiotemporal Data and As- Designed Model

abstract: Civil infrastructures undergo frequent spatial changes such as deviations between as-designed model and as-is condition, rigid body motions of the structure, and deformations of individual elements of the structure, etc. These spatial changes can occur during the design phase, the construction phase, or during the service life of a structure. Inability to accurately detect and analyze the impact of such changes may miss opportunities for early detections of pending structural integrity and stability issues. Commercial Building Information Modeling (BIM) tools could hardly track differences between as-designed and as-built conditions as they mainly focus on design changes and rely on project managers to manually update and analyze the impact of field changes on the project performance. Structural engineers collect detailed onsite data of a civil infrastructure to perform manual updates of the model for structural analysis, but such approach tends to become tedious and complicated while handling large civil infrastructures. Previous studies started collecting detailed geometric data generated by 3D laser scanners for defect detection and geometric change analysis of structures. However, previous studies have not yet systematically examined methods for exploring the correlation between the detected geometric changes and their relation to the behaviors of the structural system. Manually checking every possible loading combination leading to the observed geometric change is tedious and sometimes error-prone. The work presented in this dissertation develops a spatial change analysis framework that utilizes spatiotemporal data collected using 3D laser scanning technology and the as-designed models of the structures to automatically detect, classify, and correlate the spatial changes of a structure. The change detection part of the developed framework is computationally efficient and can automatically detect spatial changes between as-designed model and as-built data or between two sets of as-built data collected using 3D laser scanning technology. Then a spatial change classification algorithm automatically classifies the detected spatial changes as global (rigid body motion) and local deformations (tension, compression). Finally, a change correlation technique utilizes a qualitative shape-based reasoning approach for identifying correlated deformations of structure elements connected at joints that contradicts the joint equilibrium. Those contradicting deformations can help to eliminate improbable loading combinations therefore guiding the loading path analysis of the structure.Dissertation/ThesisDoctoral Dissertation Civil and Environmental Engineering 201