Combining Language and Vision with a Multimodal Skip-gram Model

We extend the SKIP-GRAM model of Mikolov et al. (2013a) by taking visual information into account. Like SKIP-GRAM, our multimodal models (MMSKIP-GRAM) build vector-based word representations by learning to predict linguistic contexts in text corpora. However, for a restricted set of words, the models are also exposed to visual representations of the objects they denote (extracted from natural images), and must predict linguistic and visual features jointly. The MMSKIP-GRAM models achieve good performance on a variety of semantic benchmarks. Moreover, since they propagate visual information to all words, we use them to improve image labeling and retrieval in the zero-shot setup, where the test concepts are never seen during model training. Finally, the MMSKIP-GRAM models discover intriguing visual properties of abstract words, paving the way to realistic implementations of embodied theories of meaning.Comment: accepted at NAACL 2015, camera ready version, 11 page

Local strong maximal monotonicity and full stability for parametric variational systems

The paper introduces and characterizes new notions of Lipschitzian and H\"olderian full stability of solutions to general parametric variational systems described via partial subdifferential and normal cone mappings acting in Hilbert spaces. These notions, postulated certain quantitative properties of single-valued localizations of solution maps, are closely related to local strong maximal monotonicity of associated set-valued mappings. Based on advanced tools of variational analysis and generalized differentiation, we derive verifiable characterizations of the local strong maximal monotonicity and full stability notions under consideration via some positive-definiteness conditions involving second-order constructions of variational analysis. The general results obtained are specified for important classes of variational inequalities and variational conditions in both finite and infinite dimensions

Constraint Qualifications and Optimality Conditions for Nonconvex Semi-Infinite and Infinite Programs

The paper concerns the study of new classes of nonlinear and nonconvex optimization problems of the so-called infinite programming that are generally defined on infinite-dimensional spaces of decision variables and contain infinitely many of equality and inequality constraints with arbitrary (may not be compact) index sets. These problems reduce to semi-infinite programs in the case of finite-dimensional spaces of decision variables. We extend the classical Mangasarian-Fromovitz and Farkas-Minkowski constraint qualifications to such infinite and semi-infinite programs. The new qualification conditions are used for efficient computing the appropriate normal cones to sets of feasible solutions for these programs by employing advanced tools of variational analysis and generalized differentiation. In the further development we derive first-order necessary optimality conditions for infinite and semi-infinite programs, which are new in both finite-dimensional and infinite-dimensional settings.Comment: 28 page