MMFL-Net: Multi-scale and Multi-granularity Feature Learning for Cross-domain Fashion Retrieval

Instance-level image retrieval in fashion is a challenging issue owing to its increasing importance in real-scenario visual fashion search. Cross-domain fashion retrieval aims to match the unconstrained customer images as queries for photographs provided by retailers; however, it is a difficult task due to a wide range of consumer-to-shop (C2S) domain discrepancies and also considering that clothing image is vulnerable to various non-rigid deformations. To this end, we propose a novel multi-scale and multi-granularity feature learning network (MMFL-Net), which can jointly learn global-local aggregation feature representations of clothing images in a unified framework, aiming to train a cross-domain model for C2S fashion visual similarity. First, a new semantic-spatial feature fusion part is designed to bridge the semantic-spatial gap by applying top-down and bottom-up bidirectional multi-scale feature fusion. Next, a multi-branch deep network architecture is introduced to capture global salient, part-informed, and local detailed information, and extracting robust and discrimination feature embedding by integrating the similarity learning of coarse-to-fine embedding with the multiple granularities. Finally, the improved trihard loss, center loss, and multi-task classification loss are adopted for our MMFL-Net, which can jointly optimize intra-class and inter-class distance and thus explicitly improve intra-class compactness and inter-class discriminability between its visual representations for feature learning. Furthermore, our proposed model also combines the multi-task attribute recognition and classification module with multi-label semantic attributes and product ID labels. Experimental results demonstrate that our proposed MMFL-Net achieves significant improvement over the state-of-the-art methods on the two datasets, DeepFashion-C2S and Street2Shop.Comment: 27 pages, 12 figures, Published by <Multimedia Tools and Applications

Expressive Compositing for Augmented Reality

International audienceIn this presentation, we introduce the work on Augmented Reality in the MANAO research group. We focus on our Focus and Context Rendering approach for Augmented Reality. We demonstrate that the use of expressive stylization of videos and 3D expressive rendering techniques, once combined, ensures that cues from real videos and virtual 3D objects are clearly noticeable

Simultaneous Interrogation of Cancer Omics to Identify Subtypes With Significant Clinical Differences

Recent advances in high-throughput sequencing have accelerated the accumulation of omics data on the same tumor tissue from multiple sources. Intensive study of multi-omics integration on tumor samples can stimulate progress in precision medicine and is promising in detecting potential biomarkers. However, current methods are restricted owing to highly unbalanced dimensions of omics data or difficulty in assigning weights between different data sources. Therefore, the appropriate approximation and constraints of integrated targets remain a major challenge. In this paper, we proposed an omics data integration method, named high-order path elucidated similarity (HOPES). HOPES fuses the similarities derived from various omics data sources to solve the dimensional discrepancy, and progressively elucidate the similarities from each type of omics data into an integrated similarity with various high-order connected paths. Through a series of incremental constraints for commonality, HOPES can take both specificity of single data and consistency between different data types into consideration. The fused similarity matrix gives global insight into patients' correlation and efficiently distinguishes subgroups. We tested the performance of HOPES on both a simulated dataset and several empirical tumor datasets. The test datasets contain three omics types including gene expression, DNA methylation, and microRNA data for five different TCGA cancer projects. Our method was shown to achieve superior accuracy and high robustness compared with several benchmark methods on simulated data. Further experiments on five cancer datasets demonstrated that HOPES achieved superior performances in cancer classification. The stratified subgroups were shown to have statistically significant differences in survival. We further located and identified the key genes, methylation sites, and microRNAs within each subgroup. They were shown to achieve high potential prognostic value and were enriched in many cancer-related biological processes or pathways

Importance-Driven Composition of Multiple Rendering Styles

International audienceWe introduce a non-uniform composition that integrates multiple rendering styles in a picture driven by an importance map. This map, either issued from saliency estimation or designed by a user, is introduced both in the creation of the multiple styles and in the final composition. Our approach accommodates a variety of stylization techniques, such as color desaturation, line drawing, blurring, edge-preserving smoothing and enhancement. We illustrate the versatility of the proposed approach and the variety of rendering styles on different applications such as images, videos, 3D scenes and even mixed reality. We also demonstrate that such an approach may help in directing user attention

On-Line Visualization of Underground Structures using Context Features

International audienceWe introduce an on-line framework for the visualizing of underground structures that improves X-Ray vision and Focus and Context Rendering for Augmented Reality. Our approach does not require an accurate reconstruction of the 3D environment and runs on-line on modern hardwares. For these purposes, we extract characteristic features from video frames and create visual cues to reveal occlusion relationships. To enhance the perception of occluding order, the extracted features are either directly rendered, or used to create hybrid blending masks: we thus ensures that the resulting cues are clearly noticeable

A clinically relevant online patient QA solution with daily CT scans and EPID-based in vivo dosimetry: A feasible study on rectal cancer

Adaptive radiation therapy (ART) could protect organs at risk (OARs) while maintain high dose coverage to targets. However, there still lack efficient online patient QA methods. We aim to develop a clinically relevant online patient quality assurance (QA) solution for ART using daily CT scans and electronic portal imaging device (EPID)-based in vivo dosimetry. Ten patients with rectal cancer at our center were included. Patients' daily CT scans and portal images were collected to generate reconstructed 3D dose distributions. Contours of targets and OARs were recontoured on these daily CT scans by a clinician or an auto-segmentation algorithm, then dose-volume indices were calculated, and the percent deviation of these indices to their original plans were determined. This deviation was regarded as the metric for clinically relevant patient QA. The tolerance level was obtained using a 95% interval of the QA metric distribution. These deviations could be further divided into anatomically relevant or delivery relevant indicators for error source analysis. Finally, our QA solution was validated on an additional six clinical patients. In rectal cancer, the lower and upper tolerance of the QA metric for PTV {\Delta}D95 (%) were [-3.11%, 2.35%], and for PTV {\Delta}D2 (%) were [-0.78%, 3.23%]. In validation, the 68% for PTV {\Delta}D95 (%) and the 79% for PTV {\Delta}D2 ({%)of the 28 fractions are within tolerances of the QA metrics. By using four or more out-of-tolerance QA metrics as an action level, there were 5 fractions (18%) have four or more out-of-tolerance QA metrics in validation patient dataset. The online patient QA solution using daily CT scans and EPID-based in vivo dosimetry is clinically feasible. Source of error analysis has the potential for distinguishing sources of error and guiding ART for future treatments