Monocular Real-time Full Body Capture with Inter-part Correlations

We present the first method for real-time full body capture that estimates shape and motion of body and hands together with a dynamic 3D face model from a single color image. Our approach uses a new neural network architecture that exploits correlations between body and hands at high computational efficiency. Unlike previous works, our approach is jointly trained on multiple datasets focusing on hand, body or face separately, without requiring data where all the parts are annotated at the same time, which is much more difficult to create at sufficient variety. The possibility of such multi-dataset training enables superior generalization ability. In contrast to earlier monocular full body methods, our approach captures more expressive 3D face geometry and color by estimating the shape, expression, albedo and illumination parameters of a statistical face model. Our method achieves competitive accuracy on public benchmarks, while being significantly faster and providing more complete face reconstructions

Language-Guided Traffic Simulation via Scene-Level Diffusion

Realistic and controllable traffic simulation is a core capability that is necessary to accelerate autonomous vehicle (AV) development. However, current approaches for controlling learning-based traffic models require significant domain expertise and are difficult for practitioners to use. To remedy this, we present CTG++, a scene-level conditional diffusion model that can be guided by language instructions. Developing this requires tackling two challenges: the need for a realistic and controllable traffic model backbone, and an effective method to interface with a traffic model using language. To address these challenges, we first propose a scene-level diffusion model equipped with a spatio-temporal transformer backbone, which generates realistic and controllable traffic. We then harness a large language model (LLM) to convert a user's query into a loss function, guiding the diffusion model towards query-compliant generation. Through comprehensive evaluation, we demonstrate the effectiveness of our proposed method in generating realistic, query-compliant traffic simulations

Learning to Generate Image Embeddings with User-level Differential Privacy

Small on-device models have been successfully trained with user-level differential privacy (DP) for next word prediction and image classification tasks in the past. However, existing methods can fail when directly applied to learn embedding models using supervised training data with a large class space. To achieve user-level DP for large image-to-embedding feature extractors, we propose DP-FedEmb, a variant of federated learning algorithms with per-user sensitivity control and noise addition, to train from user-partitioned data centralized in the datacenter. DP-FedEmb combines virtual clients, partial aggregation, private local fine-tuning, and public pretraining to achieve strong privacy utility trade-offs. We apply DP-FedEmb to train image embedding models for faces, landmarks and natural species, and demonstrate its superior utility under same privacy budget on benchmark datasets DigiFace, EMNIST, GLD and iNaturalist. We further illustrate it is possible to achieve strong user-level DP guarantees of $\epsilon<2$ while controlling the utility drop within 5%, when millions of users can participate in training

Comprehensive FGFR3 alteration-related transcriptomic characterization is involved in immune infiltration and correlated with prognosis and immunotherapy response of bladder cancer

BackgroundBladder cancer (BC) threatens the health of human beings worldwide because of its high recurrence rate and mortality. As an actionable biomarker, fibroblast growth factor receptor 3 (FGFR3) alterations have been revealed as a vital biomarker and associated with favorable outcomes in BC. However, the comprehensive relationship between the FGFR3 alteration associated gene expression profile and the prognosis of BC remains ambiguous.Materials and MethodsGenomic alteration profile, gene expression data, and related clinical information of BC patients were downloaded from The Cancer Genomics database (TCGA), as a training cohort. Subsequently, the Weighted Gene Co-expression Network Analysis (WGCNA) was conducted to identify the hub modules correlated with FGFR3 alteration. The univariate, multivariate, and least absolute shrinkage and selection operator (LASSO) Cox regression analyses were used to obtain an FGFR3 alteration-related gene (FARG) prognostic signature and FARG-based nomogram. The receiver operating characteristic (ROC) curve analysis was used for evaluation of the ability of prognosis prediction. The FARG signature was validated in four independent datasets, namely, GSE13507, GSE31684, GSE32548, and GSE48075, from Gene Expression Omnibus (GEO). Then, clinical feature association analysis, functional enrichment, genomic alteration enrichment, and tumor environment analysis were conducted to reveal differential clinical and molecular characterizations in different risk groups. Lastly, the treatment response was evaluated in the immunotherapy-related dataset of the IMvigor210 cohort and the frontline chemotherapy dataset of GSE48276, and the chemo-drug sensitivity was estimated via Genomics of Drug Sensitivity in Cancer (GDSC).ResultsThere were a total of eleven genes (CERCAM, TPST1, OSBPL10, EMP1, CYTH3, NCRNA00201, PCDH10, GAP43, COLQ, DGKB, and SETBP1) identified in the FARG signature, which divided BC patients from the TCGA cohort into high- and low-risk groups. The Kaplan–Meier curve analysis demonstrated that BC patients in the low-risk group have superior overall survival (OS) than those in the high-risk group (median OS: 27.06 months vs. 104.65 months, p &lt; 0.0001). Moreover, the FARG signature not only showed a good performance in prognosis prediction, but also could distinguish patients with different neoplasm disease stages, notably whether patients presented with muscle invasive phenotype. Compared to clinicopathological features, the FARG signature was found to be the only independent prognostic factor, and subsequently, a FARG-based prognostic nomogram was constructed with better ability of prognosis prediction, indicated by area under ROC curve (AUC) values for 1-, 3-, and 5-year OS of 0.69, 0.71, and 0.79, respectively. Underlying the FARG signature, multiple kinds of metabolism- and immune-related signaling pathways were enriched. Genomic alteration enrichment further identified that FGFR3 alterations, especially c.746C&gt;G (p.Ser249Cys), were more prevalent in the low-risk group. Additionally, FARG score was positively correlated with ESTIMATE and TIDE scores, and the low-risk group had abundant enrichment of plasma B cells, CD8+ T cells, CD4+ naive T cells, and helper follicular T cells, implying that patients in the low-risk group were likely to make significant responses to immunotherapy, which was further supported by the analysis in the IMvigor210 cohort as there was a significantly higher response rate among patients with lower FARG scores. The analysis of the GDSC database finally demonstrated that low-risk samples were more sensitive to methotrexate and tipifarnib, whereas those in the high-risk group had higher sensitivities in cisplatin, docetaxel, and paclitaxel, instead.ConclusionThe novel established FARG signature based on a comprehensive FGFR3 alteration-related transcriptomic profile performed well in prognosis prediction and was also correlated with immunotherapy and chemotherapy treatment responses, which had great potential in future clinical applications