LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop

While there has been remarkable progress in the performance of visual recognition algorithms, the state-of-the-art models tend to be exceptionally data-hungry. Large labeled training datasets, expensive and tedious to produce, are required to optimize millions of parameters in deep network models. Lagging behind the growth in model capacity, the available datasets are quickly becoming outdated in terms of size and density. To circumvent this bottleneck, we propose to amplify human effort through a partially automated labeling scheme, leveraging deep learning with humans in the loop. Starting from a large set of candidate images for each category, we iteratively sample a subset, ask people to label them, classify the others with a trained model, split the set into positives, negatives, and unlabeled based on the classification confidence, and then iterate with the unlabeled set. To assess the effectiveness of this cascading procedure and enable further progress in visual recognition research, we construct a new image dataset, LSUN. It contains around one million labeled images for each of 10 scene categories and 20 object categories. We experiment with training popular convolutional networks and find that they achieve substantial performance gains when trained on this dataset

3D ShapeNets: A Deep Representation for Volumetric Shapes

3D shape is a crucial but heavily underutilized cue in today's computer vision systems, mostly due to the lack of a good generic shape representation. With the recent availability of inexpensive 2.5D depth sensors (e.g. Microsoft Kinect), it is becoming increasingly important to have a powerful 3D shape representation in the loop. Apart from category recognition, recovering full 3D shapes from view-based 2.5D depth maps is also a critical part of visual understanding. To this end, we propose to represent a geometric 3D shape as a probability distribution of binary variables on a 3D voxel grid, using a Convolutional Deep Belief Network. Our model, 3D ShapeNets, learns the distribution of complex 3D shapes across different object categories and arbitrary poses from raw CAD data, and discovers hierarchical compositional part representations automatically. It naturally supports joint object recognition and shape completion from 2.5D depth maps, and it enables active object recognition through view planning. To train our 3D deep learning model, we construct ModelNet -- a large-scale 3D CAD model dataset. Extensive experiments show that our 3D deep representation enables significant performance improvement over the-state-of-the-arts in a variety of tasks.Comment: to be appeared in CVPR 201

Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge

Robot warehouse automation has attracted significant interest in recent years, perhaps most visibly in the Amazon Picking Challenge (APC) [1]. A fully autonomous warehouse pick-and-place system requires robust vision that reliably recognizes and locates objects amid cluttered environments, self-occlusions, sensor noise, and a large variety of objects. In this paper we present an approach that leverages multiview RGB-D data and self-supervised, data-driven learning to overcome those difficulties. The approach was part of the MIT-Princeton Team system that took 3rd- and 4th-place in the stowing and picking tasks, respectively at APC 2016. In the proposed approach, we segment and label multiple views of a scene with a fully convolutional neural network, and then fit pre-scanned 3D object models to the resulting segmentation to get the 6D object pose. Training a deep neural network for segmentation typically requires a large amount of training data. We propose a self-supervised method to generate a large labeled dataset without tedious manual segmentation. We demonstrate that our system can reliably estimate the 6D pose of objects under a variety of scenarios. All code, data, and benchmarks are available at http://apc.cs.princeton.edu

Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Picking Challenge

Robot warehouse automation has attracted significant interest in recent years, perhaps most visibly in the Amazon Picking Challenge (APC) [1]. A fully autonomous warehouse pick-and-place system requires robust vision that reliably recognizes and locates objects amid cluttered environments, self-occlusions, sensor noise, and a large variety of objects. In this paper we present an approach that leverages multiview RGB-D data and self-supervised, data-driven learning to overcome those difficulties. The approach was part of the MIT-Princeton Team system that took 3rd- and 4th-place in the stowing and picking tasks, respectively at APC 2016. In the proposed approach, we segment and label multiple views of a scene with a fully convolutional neural network, and then fit pre-scanned 3D object models to the resulting segmentation to get the 6D object pose. Training a deep neural network for segmentation typically requires a large amount of training data. We propose a self-supervised method to generate a large labeled dataset without tedious manual segmentation. We demonstrate that our system can reliably estimate the 6D pose of objects under a variety of scenarios. All code, data, and benchmarks are available at http://apc.cs.princeton.edu

Case Report: Prenatal Diagnosis and Treatment of Fetal Autoimmune-Associated First-Degree Atrioventricular Block: First Report From China

Background: The rapid progression from fetal first-degree atrioventricular block (AVB) to third-degree AVB had been reported. However, how to define fetal first-degree AVB with proper technique and the necessity of the treatment in utero for fetal autoimmune-associated first-degree AVB are still controversial.Purpose: To explore the diagnosis and the effect of treatment for fetal first-degree AVB.Cases Presentation: Four pregnant women with positive autoantibodies anti-SSA/Ro were admitted into our hospital with complaints of rapid prolonged atrioventricular (AV) intervals of their fetuses. Fetal AV intervals were re-measured by tissue Doppler imaging (TDI) from the onset of atrial contraction to ventricular systole (Aa-Sa), which were 170 ms (case 1-twin A), 160 ms (case 1-twin B), 163 ms (case 2) and 172 ms (case 3) and 170 ms (case 4), respectively. The histories of medication usage or infection during gestation were denied. Amniotic fluid genetic screenings and virological tests were negative in all cases. No structural cardiac disorders were found and the cardiovascular profile scores were 10 for each fetus. Oral dexamethasone (initial dose of 4.5 mg daily) and hydroxychloroquine (200 mg bid) plus weekly follow-up surveillance were suggested. The dosage of dexamethasone was adjusted according to the changes of the AV intervals and fetal development of biparietal diameters (BPD) and femur lengths (FL). All fetal AV intervals were controlled well. Maternal and fetal adverse effects were noted as diabetes in 1 mother and growth retardation in all fetuses. All fetuses were delivered via cesarean section at 35+4, 37, 38, and 37+1 gestational weeks, with 10 scores of Apgar score. Postnatally, positive anti-SSA/Ro was found in all neonates. However, there were no clinical or laboratory evidence of neonatal lupus syndrome. No abnormal signs were found on postnatal electrocardiogram and echocardiography for all neonates. With a follow-up of 8–53 months, there was no progression of disease and all infants demonstrated normal physical, mental, and motor development.Conclusion: Prenatal treatment for fetal autoimmune-associated first-degree AVB could be an alternative. Strict surveillance and timely adjustment of the treatment according to the conditions of the mother and the fetus are indicated. Further studies are necessary to prove our concept

Research on a UAV spray system combined with grid atomized droplets

BackgroundsUAVs for crop protection hold significant potential for application in mountainous orchard areas in China. However, certain issues pertaining to UAV spraying need to be addressed for further technological advancement, aimed at enhancing crop protection efficiency and reducing pesticide usage. These challenges include the potential for droplet drift, limited capacity for pesticide solution. Consequently, efforts are required to overcome these limitations and optimize UAV spraying technology.MethodsIn order to balance high deposition and low drift in plant protection UAV spraying, this study proposes a plant protection UAV spraying method. In order to study the operational effects of this spraying method, this study conducted a UAV spray and grid impact test to investigate the effects of different operational parameters on droplet deposition and drift. Meanwhile, a spray model was constructed using machine learning techniques to predict the spraying effect of this method.Results and discussionThis study investigated the droplet deposition rate and downwind drift rate on three types of citrus trees: traditional densely planted trees, dwarf trees, and hedged trees, considering different particle sizes and UAV flight altitudes. Analyzing the effect of increasing the grid on droplet coverage and deposition density for different tree forms. The findings demonstrated a significantly improved droplet deposition rate on dwarf and hedged citrus trees compared to traditional densely planted trees and adopting a fixed-height grid increased droplet coverage and deposition density for both the densely planted and trellised citrus trees, but had the opposite effect on dwarfed citrus trees. When using the grid system. Among the factors examined, the height of the sampling point exhibited the greatest influence on the droplet deposition rate, whereas UAV flight height and droplet particle size had no significant impact. The distance in relation to wind direction had the most substantial effect on droplet drift rate. In terms of predicting droplet drift rate, the BP neural network performed inadequately with a coefficient of determination of 0.88. Conversely, REGRESS, ELM, and RBFNN yielded similar and notably superior results with a coefficient of determination greater than 0.95. Notably, ELM demonstrated the smallest root mean square error

Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching

This paper presents a robotic pick-and-place system that is capable of grasping and recognizing both known and novel objects in cluttered environments. The key new feature of the system is that it handles a wide range of object categories without needing any task-specific training data for novel objects. To achieve this, it first uses a category-agnostic affordance prediction algorithm to select and execute among four different grasping primitive behaviors. It then recognizes picked objects with a cross-domain image classification framework that matches observed images to product images. Since product images are readily available for a wide range of objects (e.g., from the web), the system works out-of-the-box for novel objects without requiring any additional training data. Exhaustive experimental results demonstrate that our multi-affordance grasping achieves high success rates for a wide variety of objects in clutter, and our recognition algorithm achieves high accuracy for both known and novel grasped objects. The approach was part of the MIT-Princeton Team system that took 1st place in the stowing task at the 2017 Amazon Robotics Challenge. All code, datasets, and pre-trained models are available online at http://arc.cs.princeton.eduComment: Project webpage: http://arc.cs.princeton.edu Summary video: https://youtu.be/6fG7zwGfIk

Case Report: Hypothyroidism Misdiagnosed as Fulminant Myocarditis in a Child

Background: Hypothyroidism can lead to bradycardia, reduced cardiac output, cardiac enlargement, and abnormal electrocardiogram. However, hemodynamic instability and malignant arrhythmias due to hypothyroidism is rarely reported in children.Patient Findings: We report the case of a child with third-degree atrioventricular block, cardiogenic shock, and Adams Stokes Syndrome, who was initially misdiagnosed with fulminant myocarditis and was later found to have hypothyroidism during treatment.Summary: The child's condition did not improve after the administration of gamma globulin, methylprednisolone, and isoproterenol. Even after the placement of temporary pacemakers, the therapeutic effect was still not ideal. Upon reviewing the medical history, the child's condition improved rapidly after levothyroxine supplementation.Conclusions: Hypothyroidism is a common disease, but secondary severe cardiovascular lesions are particularly rare in children. Therefore, the delay in diagnosis can lead to serious cardiovascular manifestations. When pediatric patients develop severe AVB and bradycardia, hypothyroidism should be considered as a possible cause