Combining Local Appearance and Holistic View: Dual-Source Deep Neural Networks for Human Pose Estimation

We propose a new learning-based method for estimating 2D human pose from a single image, using Dual-Source Deep Convolutional Neural Networks (DS-CNN). Recently, many methods have been developed to estimate human pose by using pose priors that are estimated from physiologically inspired graphical models or learned from a holistic perspective. In this paper, we propose to integrate both the local (body) part appearance and the holistic view of each local part for more accurate human pose estimation. Specifically, the proposed DS-CNN takes a set of image patches (category-independent object proposals for training and multi-scale sliding windows for testing) as the input and then learns the appearance of each local part by considering their holistic views in the full body. Using DS-CNN, we achieve both joint detection, which determines whether an image patch contains a body joint, and joint localization, which finds the exact location of the joint in the image patch. Finally, we develop an algorithm to combine these joint detection/localization results from all the image patches for estimating the human pose. The experimental results show the effectiveness of the proposed method by comparing to the state-of-the-art human-pose estimation methods based on pose priors that are estimated from physiologically inspired graphical models or learned from a holistic perspective.Comment: CVPR 201

Co-interest Person Detection from Multiple Wearable Camera Videos

Wearable cameras, such as Google Glass and Go Pro, enable video data collection over larger areas and from different views. In this paper, we tackle a new problem of locating the co-interest person (CIP), i.e., the one who draws attention from most camera wearers, from temporally synchronized videos taken by multiple wearable cameras. Our basic idea is to exploit the motion patterns of people and use them to correlate the persons across different videos, instead of performing appearance-based matching as in traditional video co-segmentation/localization. This way, we can identify CIP even if a group of people with similar appearance are present in the view. More specifically, we detect a set of persons on each frame as the candidates of the CIP and then build a Conditional Random Field (CRF) model to select the one with consistent motion patterns in different videos and high spacial-temporal consistency in each video. We collect three sets of wearable-camera videos for testing the proposed algorithm. All the involved people have similar appearances in the collected videos and the experiments demonstrate the effectiveness of the proposed algorithm.Comment: ICCV 201

Visual Attention Consistency under Image Transforms for Multi-Label Image Classification

Human visual perception shows good consistency for many multi-label image classification tasks under certain spatial transforms, such as scaling, rotation, flipping and translation. This has motivated the data augmentation strategy widely used in CNN classifier training -- transformed images are included for training by assuming the same class labels as their original images. In this paper, we further propose the assumption of perceptual consistency of visual attention regions for classification under such transforms, i.e., the attention region for a classification follows the same transform if the input image is spatially transformed. While the attention regions of CNN classifiers can be derived as an attention heatmap in middle layers of the network, we find that their consistency under many transforms are not preserved. To address this problem, we propose a two-branch network with an original image and its transformed image as inputs and introduce a new attention consistency loss that measures the attention heatmap consistency between two branches. This new loss is then combined with multi-label image classification loss for network training. Experiments on three datasets verify the superiority of the proposed network by achieving new state-of-the-art classification performance

Question Directed Graph Attention Network for Numerical Reasoning over Text

Numerical reasoning over texts, such as addition, subtraction, sorting and counting, is a challenging machine reading comprehension task, since it requires both natural language understanding and arithmetic computation. To address this challenge, we propose a heterogeneous graph representation for the context of the passage and question needed for such reasoning, and design a question directed graph attention network to drive multi-step numerical reasoning over this context graph.Comment: Accepted at EMNLP 202

A constrained, total-variation minimization algorithm for low-intensity X-ray CT

Purpose: We develop an iterative image-reconstruction algorithm for application to low-intensity computed tomography (CT) projection data, which is based on constrained, total-variation (TV) minimization. The algorithm design focuses on recovering structure on length scales comparable to a detector-bin width. Method: Recovering the resolution on the scale of a detector bin, requires that pixel size be much smaller than the bin width. The resulting image array contains many more pixels than data, and this undersampling is overcome with a combination of Fourier upsampling of each projection and the use of constrained, TV-minimization, as suggested by compressive sensing. The presented pseudo-code for solving constrained, TV-minimization is designed to yield an accurate solution to this optimization problem within 100 iterations. Results: The proposed image-reconstruction algorithm is applied to a low-intensity scan of a rabbit with a thin wire, to test resolution. The proposed algorithm is compared with filtered back-projection (FBP). Conclusion: The algorithm may have some advantage over FBP in that the resulting noise-level is lowered at equivalent contrast levels of the wire.Comment: This article has been submitted to "Medical Physics" on 9/13/201

Associations between Interleukin-31 Gene Polymorphisms and Dilated Cardiomyopathy in a Chinese Population

To explore the role of Interkeulin-31 (IL-31) in dilated cardiomyopathy (DCM), in our study, two SNPs of IL-31, rs4758680 (C/A) and rs7977932 (C/G), were analyzed in 331 DCM patients and 493 controls in a Chinese Han population. The frequencies of C allele and CC genotype of rs4758680 were significantly increased in DCM patients (P = 0 005, P = 0 001, resp.). Compared to CC genotype of rs4758680, the A carriers (CA/AA genotypes) were the protect factors in DCM susceptibility while the frequencies of CA/AA genotypes were decreased in the dominant model for DCM group (P < 0 001, OR = 0.56, 95%CI = 0.39-0.79). Moreover, IL-31 mRNA expression level of white blood cells was increased in DCM patients (0.072 (0.044-0.144) versus 0.036 (0.020-0.052), P < 0 001). In survival analysis of 159 DCM patients, Kaplan-Meier curve revealed the correlation between CC homozygote of rs4758680 and worse prognosis for DCM group (P = 0 005). Compared to CC genotype, the CA/AA genotypes were the independent factors in both univariate (HR = 0.530, 95%CI = 0.337-0.834, P = 0 006) and multivariate analyses after age, gender, left ventricular end-diastolic diameter, and left ventricular ejection fraction adjusted (HR = 0.548, 95%CI = 0.345-0.869, P = 0 011). Thus, we concluded that IL-31 gene polymorphisms were tightly associated with DCM susceptibility and contributed to worse prognosis in DCM patients

Effect of night shift work on metabolic syndrome in adults who suffered from earthquake stress in early life

ObjectiveTo examine the role of night shift work on the risk of metabolic syndrome (MetS) in adults suffered from earthquakes prenatally or as infants and to analyse the effect of stress on factors that influence MetS in this population.MethodsWe included 870 subjects from 2014 to 2015. All subjects work as miners for the Kailuan Mining Group and were born were living in Tangshan. Participants were classified into two groups on basis of their work schedules: day shift and night shift. They were further classified into the prenatal exposure group, the infancy exposure group, and the control group based on their age during the Tangshan earthquake. This study was conducted 38 years after the earthquake. Participants’ general demographic data, smoking and drinking habits, as well as work schedules were collected. All participants’ sleep status was assessed with the Pittsburgh Sleep Quality Index. The measurement of all subjects’ waist circumference and blood pressure was made, and triglycerides, fasting blood glucose, high-density lipoproteins, and low-density lipoproteins were measured by collecting blood samples. The definition of MetS was made after the guidelines for preventing and controlling type 2 diabetes in China (2017 Edition).ResultsA total of 187 (21.5%) workers were determined to have MetS. The incidence of MetS was greatly higher in night shift workers who were exposed to an earthquake during infancy than in day shift workers (χ2 = 8.053, p = 0.005). A multivariate logistic regression analysis displayed male participants had a higher risk develop MetS than female participants (p = 0.042, OR = 0.368, 95% CI = 0.140, 0.965). Current smokers (p = 0.030, OR = 1.520, 95%CI = 1.042, 2.218) and participants who sleep fewer than 7 h per night (p = 0.015, OR = 1.638, 95%CI = 1.101, 2.437) had a higher risk of MetS. Prenatal earthquake stress was also a risk element for MetS (p = 0.012, OR = 1.644, 95%CI = 1.115, 2.423).ConclusionThe risk of MetS is significantly higher in night shift workers exposed to earthquake stress during infancy than day shift workers. Earthquake exposure during pregnancy is an independent risk factor for MetS. Smoking and sleeping less than 7 h have a higher risk of MetS than the control group