Learning To Grasp

Providing robots with the ability to grasp objects has, despite decades of research, remained a challenging problem. The problem is approachable in constrained environments where there is ample prior knowledge of the scene and objects that will be manipulated. The challenge is in building systems that scale beyond specific situational instances and gracefully operate in novel conditions. In the past, heuristic and simple rule based strategies were used to accomplish tasks such as scene segmentation or reasoning about occlusion. These heuristic strategies work in constrained environments where a roboticist can make simplifying assumptions about everything from the geometries of the objects to be interacted with, level of clutter, camera position, lighting, and a myriad of other relevant variables. With these assumptions in place, it becomes tractable for a roboticist to hardcode desired behaviour and build a robotic system capable of completing repetitive tasks. These hardcoded behaviours will quickly fail if the assumptions about the environment are invalidated. In this thesis we will demonstrate how a robust grasping system can be built that is capable of operating under a more variable set of conditions without requiring significant engineering of behavior by a roboticist. This robustness is enabled by a new found ability to empower novel machine learning techniques with massive amounts of synthetic training data. The ability of simulators to create realistic sensory data enables the generation of massive corpora of labeled training data for various grasping related tasks. The use of simulation allows for the creation of a wide variety of environments and experiences exposing the robotic system to a large number of scenarios before ever operating in the real world. This thesis demonstrates that it is now possible to build systems that work in the real world trained using deep learning on synthetic data. The sheer volume of data that can be produced via simulation enables the use of powerful deep learning techniques whose performance scales with the amount of data available. This thesis will explore how deep learning and other techniques can be used to encode these massive datasets for efficient runtime use. The ability to train and test on synthetic data allows for quick iterative development of new perception, planning and grasp execution algorithms that work in a large number of environments. Creative applications of machine learning and massive synthetic datasets are allowing robotic systems to learn skills, and move beyond repetitive hardcoded tasks

Multi-Modal Geometric Learning for Grasping and Manipulation

This work provides an architecture that incorporates depth and tactile information to create rich and accurate 3D models useful for robotic manipulation tasks. This is accomplished through the use of a 3D convolutional neural network (CNN). Offline, the network is provided with both depth and tactile information and trained to predict the object's geometry, thus filling in regions of occlusion. At runtime, the network is provided a partial view of an object. Tactile information is acquired to augment the captured depth information. The network can then reason about the object's geometry by utilizing both the collected tactile and depth information. We demonstrate that even small amounts of additional tactile information can be incredibly helpful in reasoning about object geometry. This is particularly true when information from depth alone fails to produce an accurate geometric prediction. Our method is benchmarked against and outperforms other visual-tactile approaches to general geometric reasoning. We also provide experimental results comparing grasping success with our method

Human Robot Interface for Assistive Grasping

This work describes a new human-in-the-loop (HitL) assistive grasping system for individuals with varying levels of physical capabilities. We investigated the feasibility of using four potential input devices with our assistive grasping system interface, using able-bodied individuals to define a set of quantitative metrics that could be used to assess an assistive grasping system. We then took these measurements and created a generalized benchmark for evaluating the effectiveness of any arbitrary input device into a HitL grasping system. The four input devices were a mouse, a speech recognition device, an assistive switch, and a novel sEMG device developed by our group that was connected either to the forearm or behind the ear of the subject. These preliminary results provide insight into how different interface devices perform for generalized assistive grasping tasks and also highlight the potential of sEMG based control for severely disabled individuals.Comment: 8 pages, 21 figure

Pin1-dependent signaling negatively affects GABAergic transmission by modulating neuroligin2/gephyrin interaction

The cell adhesion molecule Neuroligin2 (NL2) is localized selectively at GABAergic synapses, where it interacts with the scaffolding protein gephyrin in the post-synaptic density. However, the role of this interaction for formation and plasticity of GABAergic synapses is unclear. Here, we demonstrate that endogenous NL2 undergoes proline-directed phosphorylation at its unique S714-P consensus site, leading to the recruitment of the peptidyl-prolyl cis-trans isomerase Pin1. This signalling cascade negatively regulates NL2' s ability to interact with gephyrin at GABAergic post-synaptic sites. As a consequence, enhanced accumulation of NL2, gephyrin and GABA A receptors was detected at GABAergic synapses in the hippocampus of Pin1-knockout mice (Pin1\ufffd/\ufffd) associated with an increase in amplitude of spontaneous GABA A -mediated post-synaptic currents. Our results suggest that Pin1-dependent signalling represents a mechanism to modulate GABAergic transmission by regulating NL2/gephyrin interaction. \ufffd 2014 Macmillan Publishers Limited. All rights reserved

Loss of the Urothelial Differentiation Marker FOXA1 Is Associated with High Grade, Late Stage Bladder Cancer and Increased Tumor Proliferation

Approximately 50% of patients with muscle-invasive bladder cancer (MIBC) develop metastatic disease, which is almost invariably lethal. However, our understanding of pathways that drive aggressive behavior of MIBC is incomplete. Members of the FOXA subfamily of transcription factors are implicated in normal urogenital development and urologic malignancies. FOXA proteins are implicated in normal urothelial differentiation, but their role in bladder cancer is unknown. We examined FOXA expression in commonly used in vitro models of bladder cancer and in human bladder cancer specimens, and used a novel in vivo tissue recombination system to determine the functional significance of FOXA1 expression in bladder cancer. Logistic regression analysis showed decreased FOXA1 expression is associated with increasing tumor stage (p<0.001), and loss of FOXA1 is associated with high histologic grade (p<0.001). Also, we found that bladder urothelium that has undergone keratinizing squamous metaplasia, a precursor to the development of squamous cell carcinoma (SCC) exhibited loss of FOXA1 expression. Furthermore, 81% of cases of SCC of the bladder were negative for FOXA1 staining compared to only 40% of urothelial cell carcinomas. In addition, we showed that a subpopulation of FOXA1 negative urothelial tumor cells are highly proliferative. Knockdown of FOXA1 in RT4 bladder cancer cells resulted in increased expression of UPK1B, UPK2, UPK3A, and UPK3B, decreased E-cadherin expression and significantly increased cell proliferation, while overexpression of FOXA1 in T24 cells increased E-cadherin expression and significantly decreased cell growth and invasion. In vivo recombination of bladder cancer cells engineered to exhibit reduced FOXA1 expression with embryonic rat bladder mesenchyme and subsequent renal capsule engraftment resulted in enhanced tumor proliferation. These findings provide the first evidence linking loss of FOXA1 expression with histological subtypes of MIBC and urothelial cell proliferation, and suggest an important role for FOXA1 in the malignant phenotype of MIBC