Articulated Pose Estimation by a Graphical Model with Image Dependent Pairwise Relations

We present a method for estimating articulated human pose from a single static image based on a graphical model with novel pairwise relations that make adaptive use of local image measurements. More precisely, we specify a graphical model for human pose which exploits the fact the local image measurements can be used both to detect parts (or joints) and also to predict the spatial relationships between them (Image Dependent Pairwise Relations). These spatial relationships are represented by a mixture model. We use Deep Convolutional Neural Networks (DCNNs) to learn conditional probabilities for the presence of parts and their spatial relationships within image patches. Hence our model combines the representational flexibility of graphical models with the efficiency and statistical power of DCNNs. Our method significantly outperforms the state of the art methods on the LSP and FLIC datasets and also performs very well on the Buffy dataset without any training.Comment: NIPS 2014 Camera Read

Parsing Occluded People by Flexible Compositions

This paper presents an approach to parsing humans when there is significant occlusion. We model humans using a graphical model which has a tree structure building on recent work [32, 6] and exploit the connectivity prior that, even in presence of occlusion, the visible nodes form a connected subtree of the graphical model. We call each connected subtree a flexible composition of object parts. This involves a novel method for learning occlusion cues. During inference we need to search over a mixture of different flexible models. By exploiting part sharing, we show that this inference can be done extremely efficiently requiring only twice as many computations as searching for the entire object (i.e., not modeling occlusion). We evaluate our model on the standard benchmarked "We Are Family" Stickmen dataset and obtain significant performance improvements over the best alternative algorithms.Comment: CVPR 15 Camera Read

Joint Multi-Person Pose Estimation and Semantic Part Segmentation

Human pose estimation and semantic part segmentation are two complementary tasks in computer vision. In this paper, we propose to solve the two tasks jointly for natural multi-person images, in which the estimated pose provides object-level shape prior to regularize part segments while the part-level segments constrain the variation of pose locations. Specifically, we first train two fully convolutional neural networks (FCNs), namely Pose FCN and Part FCN, to provide initial estimation of pose joint potential and semantic part potential. Then, to refine pose joint location, the two types of potentials are fused with a fully-connected conditional random field (FCRF), where a novel segment-joint smoothness term is used to encourage semantic and spatial consistency between parts and joints. To refine part segments, the refined pose and the original part potential are integrated through a Part FCN, where the skeleton feature from pose serves as additional regularization cues for part segments. Finally, to reduce the complexity of the FCRF, we induce human detection boxes and infer the graph inside each box, making the inference forty times faster. Since there's no dataset that contains both part segments and pose labels, we extend the PASCAL VOC part dataset with human pose joints and perform extensive experiments to compare our method against several most recent strategies. We show that on this dataset our algorithm surpasses competing methods by a large margin in both tasks.Comment: This paper has been accepted by CVPR 201

Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts

Detecting objects becomes difficult when we need to deal with large shape deformation, occlusion and low resolution. We propose a novel approach to i) handle large deformations and partial occlusions in animals (as examples of highly deformable objects), ii) describe them in terms of body parts, and iii) detect them when their body parts are hard to detect (e.g., animals depicted at low resolution). We represent the holistic object and body parts separately and use a fully connected model to arrange templates for the holistic object and body parts. Our model automatically decouples the holistic object or body parts from the model when they are hard to detect. This enables us to represent a large number of holistic object and body part combinations to better deal with different “detectability” patterns caused by deformations, occlusion and/or low resolution. We apply our method to the six animal categories in the PASCAL VOC dataset and show that our method significantly improves state-of-the-art (by 4.1% AP) and provides a richer representation for objects. During training we use annotations for body parts (e.g., head, torso, etc), making use of a new dataset of fully annotated object parts for PASCAL VOC 2010, which provides a mask for each part.This material is based upon work supported by the Center for Minds, Brains and Machines (CBMM), funded by NSF STC award CCF-1231216

Mutual Information as Intrinsic Reward of Reinforcement Learning Agents for On-demand Ride Pooling

The emergence of on-demand ride pooling services allows each vehicle to serve multiple passengers at a time, thus increasing drivers' income and enabling passengers to travel at lower prices than taxi/car on-demand services (only one passenger can be assigned to a car at a time like UberX and Lyft). Although on-demand ride pooling services can bring so many benefits, ride pooling services need a well-defined matching strategy to maximize the benefits for all parties (passengers, drivers, aggregation companies and environment), in which the regional dispatching of vehicles has a significant impact on the matching and revenue. Existing algorithms often only consider revenue maximization, which makes it difficult for requests with unusual distribution to get a ride. How to increase revenue while ensuring a reasonable assignment of requests brings a challenge to ride pooling service companies (aggregation companies). In this paper, we propose a framework for vehicle dispatching for ride pooling tasks, which splits the city into discrete dispatching regions and uses the reinforcement learning (RL) algorithm to dispatch vehicles in these regions. We also consider the mutual information (MI) between vehicle and order distribution as the intrinsic reward of the RL algorithm to improve the correlation between their distributions, thus ensuring the possibility of getting a ride for unusually distributed requests. In experimental results on a real-world taxi dataset, we demonstrate that our framework can significantly increase revenue up to an average of 3\% over the existing best on-demand ride pooling method.Comment: Accepted by AAMAS 202

Uncertainty-informed Mutual Learning for Joint Medical Image Classification and Segmentation

Classification and segmentation are crucial in medical image analysis as they enable accurate diagnosis and disease monitoring. However, current methods often prioritize the mutual learning features and shared model parameters, while neglecting the reliability of features and performances. In this paper, we propose a novel Uncertainty-informed Mutual Learning (UML) framework for reliable and interpretable medical image analysis. Our UML introduces reliability to joint classification and segmentation tasks, leveraging mutual learning with uncertainty to improve performance. To achieve this, we first use evidential deep learning to provide image-level and pixel-wise confidences. Then, an Uncertainty Navigator Decoder is constructed for better using mutual features and generating segmentation results. Besides, an Uncertainty Instructor is proposed to screen reliable masks for classification. Overall, UML could produce confidence estimation in features and performance for each link (classification and segmentation). The experiments on the public datasets demonstrate that our UML outperforms existing methods in terms of both accuracy and robustness. Our UML has the potential to explore the development of more reliable and explainable medical image analysis models. We will release the codes for reproduction after acceptance.Comment: 13 page

Efferocytosis-related gene IL33 predicts prognosis and immune response and mediates proliferation and migration in vitro and in vivo of breast cancer

BackgroundBreast cancer (BRCA) has a high incidence among women, with poor prognosis and high mortality, which is increasing year by year. Efferocytosis is a process of phagocytosis of abnormal cells and is of great value in tumor research. Our study seeks to create a predictive model for BRCA using efferocytosis-related genes (ERGs) to explore the significance of efferocytosis in this disease.MethodsIn this research, Differential analysis, and univariate Cox regression were employed to identify genes linked to prognosis in BRCA patients. Then the BRCA patients were categorized into distinct groups using consensus clustering based on prognosis genes. Survival analysis, PCA, and t-SNE were performed to verify these groups. The enrichment of metabolic pathways within the detected clusters was evaluated using gene set variation analysis (GSVA) and gene set enrichment analysis (GSEA). Additionally, single-sample GSEA (ssGSEA) was used to examine changes in immune infiltration and enrichment. A risk prognostic model was constructed utilizing multivariable Cox regression and Least Absolute Shrinkage and Selection Operator (LASSO) analyses, and subsequently validated its predictive accuracy by stratifying patients according to the median risk score. Ultimately, some crucial independent prognostic genes were pinpointed and their expression, roles, and immune characteristics were explored in both laboratory and live models.ResultsFindings revealed 52 differentially expressed genes (DEGs), of which 21 were significantly linked to BRCA outcomes. These 21 genes were utilized for consensus clustering to categorize BRCA patients into two subtypes. Subtype B was linked to a worse prognosis compared to Subtype A, though both subtypes were distinguishable. The enriched pathways were mainly concentrated in Subtype A and were actively expressed in this group. Following this, a prognostic risk model was constructed using five risk genes, which was proven to possess significant predictive value. A significant link was identified between the immune microenvironment and the risk-associated genes and scores. IL33 was identified as an independent prognostic gene with important research value. Its in vivo expression results aligned with the data analysis findings, showing low expression in BRCA. Furthermore, overexpression of IL33 significantly inhibited BRCA growth and motility in vitro and in vivo, while also enhancing their vulnerability to destruction by activated CD8+ T cells.ConclusionThe ERG-based risk model effectively predicts the prognosis of BRCA patients and shows a strong link with the immune microenvironment. IL33 stands out as a significant prognostic marker, crucial in the onset and advancement of BRCA. This highlights the necessity for additional studies and indicates that IL33 might be a potential target for BRCA treatment