End-to-End Cross-Modality Retrieval with CCA Projections and Pairwise Ranking Loss

Cross-modality retrieval encompasses retrieval tasks where the fetched items are of a different type than the search query, e.g., retrieving pictures relevant to a given text query. The state-of-the-art approach to cross-modality retrieval relies on learning a joint embedding space of the two modalities, where items from either modality are retrieved using nearest-neighbor search. In this work, we introduce a neural network layer based on Canonical Correlation Analysis (CCA) that learns better embedding spaces by analytically computing projections that maximize correlation. In contrast to previous approaches, the CCA Layer (CCAL) allows us to combine existing objectives for embedding space learning, such as pairwise ranking losses, with the optimal projections of CCA. We show the effectiveness of our approach for cross-modality retrieval on three different scenarios (text-to-image, audio-sheet-music and zero-shot retrieval), surpassing both Deep CCA and a multi-view network using freely learned projections optimized by a pairwise ranking loss, especially when little training data is available (the code for all three methods is released at: https://github.com/CPJKU/cca_layer).Comment: Preliminary version of a paper published in the International Journal of Multimedia Information Retrieva

Multivariate Copula Models at Work: Outperforming the desert island copula?

Since the pioneering work of Embrechts and co-authors in 1999, copula models enjoy steadily increasing popularity in finance. Whereas copulas are well-studied in the bivariate case, the higher-dimensional case still offers several open issues and it is by far not clear how to construct copulas which sufficiently capture the characteristics of financial returns. For this reason, elliptical copulas (i.e. Gaussian and Student-t copula) still dominate both empirical and practical applications. On the other hand, several attractive construction schemes appeared in the recent literature prom sing flexible but still manageable dependence models. The aim of this work is to empirically investigate whether these models are really capable to outperform its benchmark, i.e. the Student-t copula (which is termed by Paul Embrechts as "desert island copula" on account of its excellent fit to financial returns) and, in addition, to compare the fit of these different copula classes among themselves. --KS-copula,Hierarchical Archimedian,Product copulas,Pair-copula decomposition

In-Vitro MPI-Guided IVOCT Catheter Tracking in Real Time for Motion Artifact Compensation

Purpose: Using 4D magnetic particle imaging (MPI), intravascular optical coherence tomography (IVOCT) catheters are tracked in real time in order to compensate for image artifacts related to relative motion. Our approach demonstrates the feasibility for bimodal IVOCT and MPI in-vitro experiments. Material and Methods: During IVOCT imaging of a stenosis phantom the catheter is tracked using MPI. A 4D trajectory of the catheter tip is determined from the MPI data using center of mass sub-voxel strategies. A custom built IVOCT imaging adapter is used to perform different catheter motion profiles: no motion artifacts, motion artifacts due to catheter bending, and heart beat motion artifacts. Two IVOCT volume reconstruction methods are compared qualitatively and quantitatively using the DICE metric and the known stenosis length. Results: The MPI-tracked trajectory of the IVOCT catheter is validated in multiple repeated measurements calculating the absolute mean error and standard deviation. Both volume reconstruction methods are compared and analyzed whether they are capable of compensating the motion artifacts. The novel approach of MPI-guided catheter tracking corrects motion artifacts leading to a DICE coefficient with a minimum of 86% in comparison to 58% for a standard reconstruction approach. Conclusions: IVOCT catheter tracking with MPI in real time is an auspicious method for radiation free MPI-guided IVOCT interventions. The combination of MPI and IVOCT can help to reduce motion artifacts due to catheter bending and heart beat for optimized IVOCT volume reconstructions.Comment: 19 pages, 11 figure