End-to-End Cross-Modality Retrieval with CCA Projections and Pairwise Ranking Loss

Cross-modality retrieval encompasses retrieval tasks where the fetched items are of a different type than the search query, e.g., retrieving pictures relevant to a given text query. The state-of-the-art approach to cross-modality retrieval relies on learning a joint embedding space of the two modalities, where items from either modality are retrieved using nearest-neighbor search. In this work, we introduce a neural network layer based on Canonical Correlation Analysis (CCA) that learns better embedding spaces by analytically computing projections that maximize correlation. In contrast to previous approaches, the CCA Layer (CCAL) allows us to combine existing objectives for embedding space learning, such as pairwise ranking losses, with the optimal projections of CCA. We show the effectiveness of our approach for cross-modality retrieval on three different scenarios (text-to-image, audio-sheet-music and zero-shot retrieval), surpassing both Deep CCA and a multi-view network using freely learned projections optimized by a pairwise ranking loss, especially when little training data is available (the code for all three methods is released at: https://github.com/CPJKU/cca_layer).Comment: Preliminary version of a paper published in the International Journal of Multimedia Information Retrieva

On non-normality and classification of amplification mechanisms in stability and resolvent analysis

We seek to quantify non-normality of the most amplified resolvent modes and predict their features based on the characteristics of the base or mean velocity profile. A 2-by-2 model linear Navier-Stokes (LNS) operator illustrates how non-normality from mean shear distributes perturbation energy in different velocity components of the forcing and response modes. The inverse of their inner product, which is unity for a purely normal mechanism, is proposed as a measure to quantify non-normality. In flows where there is downstream spatial dependence of the base/mean, mean flow advection separates the spatial support of forcing and response modes which impacts the inner product. Success of mean stability analysis depends on the normality of amplification. If the amplification is normal, the resolvent operator written in its dyadic representation reveals that the adjoint and forward stability modes are proportional to the forcing and response resolvent modes. If the amplification is non-normal, then resolvent analysis is required to understand the origin of observed flow structures. Eigenspectra and pseudospectra are used to characterize these phenomena. Two test cases are studied: low Reynolds number cylinder flow and turbulent channel flow. The first deals mainly with normal mechanisms and quantification of non-normality using the inverse inner product of the leading forcing and response modes agrees well with the product of the resolvent norm and distance between the imaginary axis and least stable eigenvalue. In turbulent channel flow, structures result from both normal and non-normal mechanisms. Mean shear is exploited most efficiently by stationary disturbances while bounds on the pseudospectra illustrate how non-normality is responsible for the most amplified disturbances at spatial wavenumbers and temporal frequencies corresponding to well-known turbulent structures

Accurate and Efficient Expression Evaluation and Linear Algebra

We survey and unify recent results on the existence of accurate algorithms for evaluating multivariate polynomials, and more generally for accurate numerical linear algebra with structured matrices. By "accurate" we mean that the computed answer has relative error less than 1, i.e., has some correct leading digits. We also address efficiency, by which we mean algorithms that run in polynomial time in the size of the input. Our results will depend strongly on the model of arithmetic: Most of our results will use the so-called Traditional Model (TM). We give a set of necessary and sufficient conditions to decide whether a high accuracy algorithm exists in the TM, and describe progress toward a decision procedure that will take any problem and provide either a high accuracy algorithm or a proof that none exists. When no accurate algorithm exists in the TM, it is natural to extend the set of available accurate operations by a library of additional operations, such as $x+y+z$, dot products, or indeed any enumerable set which could then be used to build further accurate algorithms. We show how our accurate algorithms and decision procedure for finding them extend to this case. Finally, we address other models of arithmetic, and the relationship between (im)possibility in the TM and (in)efficient algorithms operating on numbers represented as bit strings.Comment: 49 pages, 6 figures, 1 tabl

An Overview of Polynomially Computable Characteristics of Special Interval Matrices

It is well known that many problems in interval computation are intractable, which restricts our attempts to solve large problems in reasonable time. This does not mean, however, that all problems are computationally hard. Identifying polynomially solvable classes thus belongs to important current trends. The purpose of this paper is to review some of such classes. In particular, we focus on several special interval matrices and investigate their convenient properties. We consider tridiagonal matrices, {M,H,P,B}-matrices, inverse M-matrices, inverse nonnegative matrices, nonnegative matrices, totally positive matrices and some others. We focus in particular on computing the range of the determinant, eigenvalues, singular values, and selected norms. Whenever possible, we state also formulae for determining the inverse matrix and the hull of the solution set of an interval system of linear equations. We survey not only the known facts, but we present some new views as well

A Singular Value Thresholding Algorithm for Matrix Completion

This paper introduces a novel algorithm to approximate the matrix with minimum nuclear norm among all matrices obeying a set of convex constraints. This problem may be understood as the convex relaxation of a rank minimization problem and arises in many important applications as in the task of recovering a large matrix from a small subset of its entries (the famous Netflix problem). Off-the-shelf algorithms such as interior point methods are not directly amenable to large problems of this kind with over a million unknown entries. This paper develops a simple first-order and easy-to-implement algorithm that is extremely efficient at addressing problems in which the optimal solution has low rank. The algorithm is iterative, produces a sequence of matrices {X^k,Y^k}, and at each step mainly performs a soft-thresholding operation on the singular values of the matrix Y^k. There are two remarkable features making this attractive for low-rank matrix completion problems. The first is that the soft-thresholding operation is applied to a sparse matrix; the second is that the rank of the iterates {X^k} is empirically nondecreasing. Both these facts allow the algorithm to make use of very minimal storage space and keep the computational cost of each iteration low. On the theoretical side, we provide a convergence analysis showing that the sequence of iterates converges. On the practical side, we provide numerical examples in which 1,000 × 1,000 matrices are recovered in less than a minute on a modest desktop computer. We also demonstrate that our approach is amenable to very large scale problems by recovering matrices of rank about 10 with nearly a billion unknowns from just about 0.4% of their sampled entries. Our methods are connected with the recent literature on linearized Bregman iterations for ℓ_1 minimization, and we develop a framework in which one can understand these algorithms in terms of well-known Lagrange multiplier algorithms

The structured distance to singularity of a symmetric tridiagonal Toeplitz matrix

This paper is concerned with the distance of a symmetric tridiagonal Toeplitz matrix $T$ to the variety of similarly structured singular matrices, and with determining the closest matrix to $T$ in this variety. Explicit formulas are presented, that exploit the analysis of the sensitivity of the spectrum of $T$ with respect to structure-preserving perturbations of its entries.Comment: 16 pages, 5 Figure

A framework for joint design of pilot sequence and linear precoder

Most performance measures of pilot-assisted multiple-input multiple-output systems are functions of the linear precoder and the pilot sequence. A framework for the optimization of these two parameters is proposed, based on a matrix-valued generalization of the concept of effective signal-to-noise ratio (SNR) introduced in the famous work by Hassibi and Hochwald. Our framework aims to extend the work of Hassibi and Hochwald by allowing for transmit-side fading correlations, and by considering a class of utility functions of said effective SNR matrix, most notably including the well-known capacity lower bound used by Hassibi and Hochwald. We tackle the joint optimization problem by recasting the optimization of the precoder (resp. pilot sequence) subject to a fixed pilot sequence (resp. precoder) into a convex problem. Furthermore, we prove that joint optimality requires that the eigenbases of the precoder and pilot sequence be both aligned along the eigenbasis of the channel correlation matrix. We finally describe how to wrap all studied subproblems into an iteration that converges to a local optimum of the joint optimization.Peer ReviewedPostprint (author's final draft