Duodepth: Static Gesture Recognition Via Dual Depth Sensors

Static gesture recognition is an effective non-verbal communication channel between a user and their devices; however many modern methods are sensitive to the relative pose of the user's hands with respect to the capture device, as parts of the gesture can become occluded. We present two methodologies for gesture recognition via synchronized recording from two depth cameras to alleviate this occlusion problem. One is a more classic approach using iterative closest point registration to accurately fuse point clouds and a single PointNet architecture for classification, and the other is a dual Point-Net architecture for classification without registration. On a manually collected data-set of 20,100 point clouds we show a 39.2% reduction in misclassification for the fused point cloud method, and 53.4% for the dual PointNet, when compared to a standard single camera pipeline.Comment: 26th International Conference on Image Processin

Sensor fusion for semantic segmentation of urban scenes

Abstract—Semantic understanding of environments is an important problem in robotics in general and intelligent au-tonomous systems in particular. In this paper, we propose a semantic segmentation algorithm which effectively fuses infor-mation from images and 3D point clouds. The proposed method incorporates information from multiple scales in an intuitive and effective manner. A late-fusion architecture is proposed to maximally leverage the training data in each modality. Finally, a pairwise Conditional Random Field (CRF) is used as a post-processing step to enforce spatial consistency in the structured prediction. The proposed algorithm is evaluated on the publicly available KITTI dataset [1] [2], augmented with additional pixel and point-wise semantic labels for building, sky, road, vegetation, sidewalk, car, pedestrian, cyclist, sign/pole, and fence regions. A per-pixel accuracy of 89.3 % and average class accuracy of 65.4 % is achieved, well above current state-of-the-art [3]. I

Digital Signal Processing

Contains an introduction and reports on fourteen research projects.National Science Foundation FellowshipNational Science Foundation (Grant ECS84-07285)U.S. Navy - Office of Naval Research (Contract N00014-81-K-0742)Sanders Associates, Inc.U.S. Air Force - Office of Scientific Research (Contract F19628-85-K-0028)Advanced Television Research ProgramAmoco Foundation FellowshipHertz Foundation Fellowshi

Digital Signal Processing

Contains an introduction and reports on fifteen research projects.National Science Foundation FellowshipU.S. Navy - Office of Naval Research (Contract N00014-81-K-0742)National Science Foundation (Grant ECS 84-07285)Sanders Associates, Inc.U.S. Air Force - Office of Scientific Research (Contract F19628-85-K-0028)AT&T Bell Laboratories Doctoral Support ProgramCanada, Bell Northern Research ScholarshipCanada, Fonds pour la Formation de Chercheurs et /'Aide a la Recherche Postgraduate FellowshipCanada, Natural Science and Engineering Research Council Postgraduate FellowshipAmoco Foundation FellowshipFannie and John Hertz Foundation Fellowshi

Digital Signal Processing

Contains an introduction and reports on fifteen research projects.U.S. Navy - Office of Naval Research (Contract N00O14-81-K-0742)U.S. Navy - Office of Naval Research (Contract N00014-77-C-0266)National Science Foundation (Grant ECS80-07102)National Science Foundation (Grant ECS84-07285)Amoco Foundation FellowshipSanders Associates, Inc.Advanced Television Research ProgramM.I.T. Vinton Hayes FellowshipHertz Foundation Fellowshi

Digital Signal Processing

Contains an introduction and reports on twenty research projects.National Science Foundation (Grant ECS 84-07285)U.S. Navy - Office of Naval Research (Contract N00014-81-K-0742)National Science Foundation FellowshipSanders Associates, Inc.U.S. Air Force - Office of Scientific Research (Contract F19628-85-K-0028)Canada, Bell Northern Research ScholarshipCanada, Fonds pour la Formation de Chercheurs et l'Aide a la Recherche Postgraduate FellowshipCanada, Natural Science and Engineering Research Council Postgraduate FellowshipU.S. Navy - Office of Naval Research (Contract N00014-81-K-0472)Fanny and John Hertz Foundation FellowshipCenter for Advanced Television StudiesAmoco Foundation FellowshipU.S. Air Force - Office of Scientific Research (Contract F19628-85-K-0028