Manipulating Highly Deformable Materials Using a Visual Feedback Dictionary

The complex physical properties of highly deformable materials such as clothes pose significant challenges fanipulation systems. We present a novel visual feedback dictionary-based method for manipulating defoor autonomous robotic mrmable objects towards a desired configuration. Our approach is based on visual servoing and we use an efficient technique to extract key features from the RGB sensor stream in the form of a histogram of deformable model features. These histogram features serve as high-level representations of the state of the deformable material. Next, we collect manipulation data and use a visual feedback dictionary that maps the velocity in the high-dimensional feature space to the velocity of the robotic end-effectors for manipulation. We have evaluated our approach on a set of complex manipulation tasks and human-robot manipulation tasks on different cloth pieces with varying material characteristics.Comment: The video is available at goo.gl/mDSC4

Clustering and Community Detection in Directed Networks: A Survey

Networks (or graphs) appear as dominant structures in diverse domains, including sociology, biology, neuroscience and computer science. In most of the aforementioned cases graphs are directed - in the sense that there is directionality on the edges, making the semantics of the edges non symmetric. An interesting feature that real networks present is the clustering or community structure property, under which the graph topology is organized into modules commonly called communities or clusters. The essence here is that nodes of the same community are highly similar while on the contrary, nodes across communities present low similarity. Revealing the underlying community structure of directed complex networks has become a crucial and interdisciplinary topic with a plethora of applications. Therefore, naturally there is a recent wealth of research production in the area of mining directed graphs - with clustering being the primary method and tool for community detection and evaluation. The goal of this paper is to offer an in-depth review of the methods presented so far for clustering directed networks along with the relevant necessary methodological background and also related applications. The survey commences by offering a concise review of the fundamental concepts and methodological base on which graph clustering algorithms capitalize on. Then we present the relevant work along two orthogonal classifications. The first one is mostly concerned with the methodological principles of the clustering algorithms, while the second one approaches the methods from the viewpoint regarding the properties of a good cluster in a directed network. Further, we present methods and metrics for evaluating graph clustering results, demonstrate interesting application domains and provide promising future research directions.Comment: 86 pages, 17 figures. Physics Reports Journal (To Appear

Illumination Invariant Deep Learning for Hyperspectral Data

Motivated by the variability in hyperspectral images due to illumination and the difficulty in acquiring labelled data, this thesis proposes different approaches for learning illumination invariant feature representations and classification models for hyperspectral data captured outdoors, under natural sunlight. The approaches integrate domain knowledge into learning algorithms and hence does not rely on a priori knowledge of atmospheric parameters, additional sensors or large amounts of labelled training data. Hyperspectral sensors record rich semantic information from a scene, making them useful for robotics or remote sensing applications where perception systems are used to gain an understanding of the scene. Images recorded by hyperspectral sensors can, however, be affected to varying degrees by intrinsic factors relating to the sensor itself (keystone, smile, noise, particularly at the limits of the sensed spectral range) but also by extrinsic factors such as the way the scene is illuminated. The appearance of the scene in the image is tied to the incident illumination which is dependent on variables such as the position of the sun, geometry of the surface and the prevailing atmospheric conditions. Effects like shadows can make the appearance and spectral characteristics of identical materials to be significantly different. This degrades the performance of high-level algorithms that use hyperspectral data, such as those that do classification and clustering. If sufficient training data is available, learning algorithms such as neural networks can capture variability in the scene appearance and be trained to compensate for it. Learning algorithms are advantageous for this task because they do not require a priori knowledge of the prevailing atmospheric conditions or data from additional sensors. Labelling of hyperspectral data is, however, difficult and time-consuming, so acquiring enough labelled samples for the learning algorithm to adequately capture the scene appearance is challenging. Hence, there is a need for the development of techniques that are invariant to the effects of illumination that do not require large amounts of labelled data. In this thesis, an approach to learning a representation of hyperspectral data that is invariant to the effects of illumination is proposed. This approach combines a physics-based model of the illumination process with an unsupervised deep learning algorithm, and thus requires no labelled data. Datasets that vary both temporally and spatially are used to compare the proposed approach to other similar state-of-the-art techniques. The results show that the learnt representation is more invariant to shadows in the image and to variations in brightness due to changes in the scene topography or position of the sun in the sky. The results also show that a supervised classifier can predict class labels more accurately and more consistently across time when images are represented using the proposed method. Additionally, this thesis proposes methods to train supervised classification models to be more robust to variations in illumination where only limited amounts of labelled data are available. The transfer of knowledge from well-labelled datasets to poorly labelled datasets for classification is investigated. A method is also proposed for enabling small amounts of labelled samples to capture the variability in spectra across the scene. These samples are then used to train a classifier to be robust to the variability in the data caused by variations in illumination. The results show that these approaches make convolutional neural network classifiers more robust and achieve better performance when there is limited labelled training data. A case study is presented where a pipeline is proposed that incorporates the methods proposed in this thesis for learning robust feature representations and classification models. A scene is clustered using no labelled data. The results show that the pipeline groups the data into clusters that are consistent with the spatial distribution of the classes in the scene as determined from ground truth

Novel system of pavement cracking detection algorithms using 1mm 3D surface data

Pavement cracking is one of the major concerns for pavement design and management. There have been rapid developments of automated pavement cracking detection in recent years. However, none of them has been widely accepted so far due to lack of capability of maintaining consistently high detection accuracy for various pavement surfaces. Using 1mm 3D data collected by WayLink Digital Highway Data Vehicle (DHDV), an entire system of algorithms, which consists of Fully Automated Cracking Detection Subsystem, Interactive Cracking Detection Subsystem and Noisy Pattern Detection Subsystem, is proposed in this study for improvements in adaptability, reliability and interactivity of pavement cracking detection.The Fully Automated Cracking Detection Subsystem utilizes 3D Shadow Simulation to find lower areas in local neighborhood, and then eliminates noises by subsequent noise suppressing procedures. The assumption behind 3D Shadow Simulation is that local lower areas will be shadowed under light with a certain projection angle. According to the Precision-Recall Analysis on two real pavement segments, the fully automated subsystem can achieve a high level of Precision and Recall on both pavement segments.The Interactive Cracking Detection Subsystem implements an interactive algorithm proposed in this study, which is capable of improving its detection accuracy by adjustments based on the operator's feedback, to provide a slower but more flexible as well as confident approach to pavement cracking detection. It is demonstrated in the case study that the interactive subsystem can retrieve almost 100 percent of cracks with nearly no noises.The Noisy Pattern Detection Subsystem is proposed to exclude pavement joints and grooves from cracking detection so that false-positive errors on rigid pavements can be reduced significantly. This subsystem applies Support Vector Machines (SVM) to train the classifiers for the recognition of transverse groove, transverse joint, longitudinal groove and longitudinal joint respectively. Based on the trained classifiers, pattern extraction procedures are developed to find the exact locations of pavement joints and grooves.Non-dominated Sorting Genetic Algorithm II (NSGA-II), which is one of multi objective genetic algorithms, is employed in this study to optimize parameters of the fully automated subsystem for the pursuing of high Precision and high Recall simultaneously. In addition to NSGA-II, an Auxiliary Prediction Model (APM) is proposed in this study to assist NSGA-II for faster convergence and better diversity.Finally, CPU-based and GPU-based Parallel Computing Techniques, including MultiGPU, GPU streaming, Multi-Core and Multi-Threading are combined in this study to increase the processing speed for all computational tasks that can be synchronous

A new machine-learning framework to generate star cluster models

openThe birthplaces of stars are complex places, where turbulent interstellar gas collapses and fragments into star-forming cores, giving rise to non-trivial substructure. While the formation process can be modelled with hydrodynamical simulations, these are quite expensive in terms of computational resources. Moreover, primordial star clusters that are still embedded in their parent gas cloud are hard to constrain observationally. In this context, most efforts aimed at simulating the dynamical evolution of star clusters assume simplified initial conditions, such as truncated Maxwellian models. We aim to improve on this state-of-the-art by introducing a set of tools to generate realistic initial conditions for star clusters by training an appropriate class of machine learning models on a limited set of hydrodynamical simulations. In particular, we will exploit a new approach based on Gaussian process (GP) models, which have the advantage of differentiability and of being more tractable, allowing for seamless inclusion in a downstream machine learning pipeline for e.g. inference purposes. The proposed learning framework is a two-step process including the model training and the sampling of new stellar clusters based on the inference results. We investigate different sampling approaches in order to find samplers that are able to generate realistic realizations.The birthplaces of stars are complex places, where turbulent interstellar gas collapses and fragments into star-forming cores, giving rise to non-trivial substructure. While the formation process can be modelled with hydrodynamical simulations, these are quite expensive in terms of computational resources. Moreover, primordial star clusters that are still embedded in their parent gas cloud are hard to constrain observationally. In this context, most efforts aimed at simulating the dynamical evolution of star clusters assume simplified initial conditions, such as truncated Maxwellian models. We aim to improve on this state-of-the-art by introducing a set of tools to generate realistic initial conditions for star clusters by training an appropriate class of machine learning models on a limited set of hydrodynamical simulations. In particular, we will exploit a new approach based on Gaussian process (GP) models, which have the advantage of differentiability and of being more tractable, allowing for seamless inclusion in a downstream machine learning pipeline for e.g. inference purposes. The proposed learning framework is a two-step process including the model training and the sampling of new stellar clusters based on the inference results. We investigate different sampling approaches in order to find samplers that are able to generate realistic realizations

Harnessing Teamwork in Networks: Prediction, Optimization, and Explanation

abstract: Teams are increasingly indispensable to achievements in any organizations. Despite the organizations' substantial dependency on teams, fundamental knowledge about the conduct of team-enabled operations is lacking, especially at the {\it social, cognitive} and {\it information} level in relation to team performance and network dynamics. The goal of this dissertation is to create new instruments to {\it predict}, {\it optimize} and {\it explain} teams' performance in the context of composite networks (i.e., social-cognitive-information networks). Understanding the dynamic mechanisms that drive the success of high-performing teams can provide the key insights into building the best teams and hence lift the productivity and profitability of the organizations. For this purpose, novel predictive models to forecast the long-term performance of teams ({\it point prediction}) as well as the pathway to impact ({\it trajectory prediction}) have been developed. A joint predictive model by exploring the relationship between team level and individual level performances has also been proposed. For an existing team, it is often desirable to optimize its performance through expanding the team by bringing a new team member with certain expertise, or finding a new candidate to replace an existing under-performing member. I have developed graph kernel based performance optimization algorithms by considering both the structural matching and skill matching to solve the above enhancement scenarios. I have also worked towards real time team optimization by leveraging reinforcement learning techniques. With the increased complexity of the machine learning models for predicting and optimizing teams, it is critical to acquire a deeper understanding of model behavior. For this purpose, I have investigated {\em explainable prediction} -- to provide explanation behind a performance prediction and {\em explainable optimization} -- to give reasons why the model recommendations are good candidates for certain enhancement scenarios.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Computer Aided Multi-Data Fusion Dismount Modeling

Recent research efforts strive to address the growing need for dismount surveillance, dismount tracking and characterization. Current work in this area utilizes hyperspectral and multispectral imaging systems to exploit spectral properties in order to detect areas of exposed skin and clothing characteristics. Because of the large bandwidth and high resolution, hyperspectral imaging systems pose great ability to characterize and detect dismounts. A multi-data dismount modeling system where the development and manipulation of dismount models is a necessity. This thesis demonstrates a computer aided multi-data fused dismount model, which facilitates studies of dismount detection, characterization and identification. The system is created by fusing: pixel mapping, signature attachment, and pixel mixing algorithms. The developed multi-data dismount model produces simulated hyperspectral images that closely represent an image collected by a hyperspectral imager. The dismount model can be modified to fit the researcher\u27s needs. The multi-data model structure allows the employment of a database of signatures acquired from several sources. The model is flexible enough to allow further exploitation, enhancement and manipulation. The multi-data dismount model developed in this effort fulfills the need for a dismount modeling tool in a hyperspectral imaging environment

Fast hierarchical low-rank view factor matrices for thermal irradiance on planetary surfaces

We present an algorithm for compressing the radiosity view factor model commonly used in radiation heat transfer and computer graphics. We use a format inspired by the hierarchical off-diagonal low rank format, where elements are recursively partitioned using a quadtree or octree and blocks are compressed using a sparse singular value decomposition -- the hierarchical matrix is assembled using dynamic programming. The motivating application is time-dependent thermal modeling on vast planetary surfaces, with a focus on permanently shadowed craters which receive energy through indirect irradiance. In this setting, shape models are comprised of a large number of triangular facets which conform to a rough surface. At each time step, a quadratic number of triangle-to-triangle scattered fluxes must be summed; that is, as the sun moves through the sky, we must solve the same view factor system of equations for a potentially unlimited number of time-varying righthand sides. We first conduct numerical experiments with a synthetic spherical cap-shaped crater, where the equilibrium temperature is analytically available. We also test our implementation with triangle meshes of planetary surfaces derived from digital elevation models recovered by orbiting spacecrafts. Our results indicate that the compressed view factor matrix can be assembled in quadratic time, which is comparable to the time it takes to assemble the full view matrix itself. Memory requirements during assembly are reduced by a large factor. Finally, for a range of compression tolerances, the size of the compressed view factor matrix and the speed of the resulting matrix vector product both scale linearly (as opposed to quadratically for the full matrix), resulting in orders of magnitude savings in processing time and memory space.Comment: 21 pages, 10 figure

Integrating spatial and spectral information for automatic feature identification in high -resolution remotely sensed images

This research used image objects, instead of pixels, as the basic unit of analysis in high-resolution imagery. Thus, not only spectral radiance and texture were used in the analysis, but also spatial context. Furthermore, the automated identification of attributed objects is potentially useful for integrating remote sensing with a vector-based GIS.;A study area in Morgantown, WV was chosen as a site for the development and testing of automated feature extraction methods with high-resolution data. In the first stage of the analysis, edges were identified using texture. Experiments with simulated data indicated that a linear operator identified curved and sharp edges more accurately than square shaped operators. Areas with edges that formed a closed boundary were used to delineate sub-patches. In the region growing step, the similarities of all adjacent subpatches were examined using a multivariate Hotelling T2 test that draws on the classes\u27 covariance matrices. Sub-patches that were not sufficiently dissimilar were merged to form image patches.;Patches were then classified into seven classes: Building, Road, Forest, Lawn, Shadowed Vegetation, Water, and Shadow. Six classification methods were compared: the pixel-based ISODATA and maximum likelihood approaches, field-based ECHO, and region based maximum likelihood using patch means, a divergence index, and patch probability density functions (pdfs). Classification with the divergence index showed the lowest accuracy, a kappa index of 0.254. The highest accuracy, 0.783, was obtained from classification using the patch pdf. This classification also produced a visually pleasing product, with well-delineated objects and without the distracting salt-and-pepper effect of isolated misclassified pixels. The accuracies of classification with patch mean, pixel based maximum likelihood, ISODATA and ECHO were 0.735, 0.687, 0.610, and 0.605, respectively.;Spatial context was used to generate aggregate land cover information. An Urbanized Rate Index, defined based on the percentage of Building and Road area within a local window, was used to segment the image. Five summary landcover classes were identified from the Urbanized Rate segmentation and the image object classification: High Urbanized Rate and large building sizes, Intermediate Urbanized Rate and intermediate building sizes, Low urbanized rate and small building sizes, Forest, and Water