End-to-End Cross-Modality Retrieval with CCA Projections and Pairwise Ranking Loss

Cross-modality retrieval encompasses retrieval tasks where the fetched items are of a different type than the search query, e.g., retrieving pictures relevant to a given text query. The state-of-the-art approach to cross-modality retrieval relies on learning a joint embedding space of the two modalities, where items from either modality are retrieved using nearest-neighbor search. In this work, we introduce a neural network layer based on Canonical Correlation Analysis (CCA) that learns better embedding spaces by analytically computing projections that maximize correlation. In contrast to previous approaches, the CCA Layer (CCAL) allows us to combine existing objectives for embedding space learning, such as pairwise ranking losses, with the optimal projections of CCA. We show the effectiveness of our approach for cross-modality retrieval on three different scenarios (text-to-image, audio-sheet-music and zero-shot retrieval), surpassing both Deep CCA and a multi-view network using freely learned projections optimized by a pairwise ranking loss, especially when little training data is available (the code for all three methods is released at: https://github.com/CPJKU/cca_layer).Comment: Preliminary version of a paper published in the International Journal of Multimedia Information Retrieva

Multilingual websites evaluation: methodology and heuristic tool

A methodology for evaluating the suitability of websites to multilingual and multicultural audiences is proposed. Standards and validated recommendations have been checked in order to create a (heuristic) guideline. The result is a checklist for evaluating websites with international audiences, both monolingual (with or without globalization) and multilingual (with or without localization)

Impact of Listening Behavior on Music Recommendation.

The next generation of music recommendation systems willbe increasingly intelligent and likely take into account userbehavior for more personalized recommendations. In thiswork we consider user behavior when making recommendationswith features extracted from a user’s history of listeningevents. We investigate the impact of listener’s behaviorby considering features such as play counts, “mainstreaminess”,and diversity in music taste on the performanceof various music recommendation approaches. Theunderlying dataset has been collected by crawling socialmedia (specifically Twitter) for listening events. Each user’slistening behavior is characterized into a three dimensionalfeature space consisting of play count, “mainstreaminess”(i.e. the degree to which the observed user listens to currentlypopular artists), and diversity (i.e. the diversity ofgenres the observed user listens to). Drawing subsets ofthe 28,000 users in our dataset, according to these threedimensions, we evaluate whether these dimensions influencefigures of merit of various music recommendation approaches,in particular, collaborative filtering (CF) and CFenhanced by cultural information such as users located inthe same city or country

Feature-combination hybrid recommender systems for automated music playlist continuation

Music recommender systems have become a key technology to support the interaction of users with the increasingly larger music catalogs of on-line music streaming services, on-line music shops, and personal devices. An important task in music recommender systems is the automated continuation of music playlists, that enables the recommendation of music streams adapting to given (possibly short) listening sessions. Previous works have shown that applying collaborative filtering to collections of curated music playlists reveals underlying playlist-song co-occurrence patterns that are useful to predict playlist continuations. However, most music collections exhibit a pronounced long-tailed distribution. The majority of songs occur only in few playlists and, as a consequence, they are poorly represented by collaborative filtering. We introduce two feature-combination hybrid recommender systems that extend collaborative filtering by integrating the collaborative information encoded in curated music playlists with any type of song feature vector representation. We conduct off-line experiments to assess the performance of the proposed systems to recover withheld playlist continuations, and we compare them to competitive pure and hybrid collaborative filtering baselines. The results of the experiments indicate that the introduced feature-combination hybrid recommender systems can more accurately predict fitting playlist continuations as a result of their improved representation of songs occurring in few playlists(VLID)328909

A Hybrid Approach to Music Playlist Continuation Based on Playlist-Song Membership

Automated music playlist continuation is a common task of music recommender systems, that generally consists in providing a fitting extension to a given playlist. Collaborative filtering models, that extract abstract patterns from curated music playlists, tend to provide better playlist continuations than content-based approaches. However, pure collaborative filtering models have at least one of the following limitations: (1) they can only extend playlists profiled at training time; (2) they misrepresent songs that occur in very few playlists. We introduce a novel hybrid playlist continuation model based on what we name "playlist-song membership", that is, whether a given playlist and a given song fit together. The proposed model regards any playlist-song pair exclusively in terms of feature vectors. In light of this information, and after having been trained on a collection of labeled playlist-song pairs, the proposed model decides whether a playlist-song pair fits together or not. Experimental results on two datasets of curated music playlists show that the proposed playlist continuation model compares to a state-of-the-art collaborative filtering model in the ideal situation of extending playlists profiled at training time and where songs occurred frequently in training playlists. In contrast to the collaborative filtering model, and as a result of its general understanding of the playlist-song pairs in terms of feature vectors, the proposed model is additionally able to (1) extend non-profiled playlists and (2) recommend songs that occurred seldom or never in training~playlists

Evaluación de sitios web multilingües: metodología y herramienta heurística

A methodology for evaluating the suitability of websites to multilingual and multicultural audiences is proposed. Standards and validated recommendations have been checked in order to create a (heuristic) guideline. The result is a checklist for evaluating websites with international audiences, both monolingual (with or without globalization) and multilingual (with or without localization)

Machine learning approaches to hybrid music recommender systems

Music catalogs in music streaming services, on-line music shops and private collections become increasingly larger and consequently difficult to navigate. Music recommender systems are technologies devised to support users accessing such large catalogs by automatically identifying and suggesting music that may interest them. This thesis focuses on the machine learning aspects of music recommendation with contributions at the intersection of recommender systems and music information retrieval: I investigate and propose recommender systems that observe and exploit the particularities of the music domain. The thesis specializes in "hybrid" music recommender systems, so called because they combine two fundamentally different types of data: (1) user-music interaction histories (e.g., the music that users recently listened to, or "liked"), with (2) descriptions of the musical content (e.g., the genre, or acoustical properties of a song). The proposed hybrid music recommender systems integrate the strengths of these two types of data into enhanced standalone systems. This is in contrast to most previous approaches in the literature, where hybridization was achieved through the heuristic combination of music recommendations issued by independent systems. The proposed hybrid music recommender systems are thoroughly evaluated against competitive recommender system baselines, for different music recommendation tasks, and on different datasets. According to the conducted experiments, the proposed systems predict music recommendations comparably or more accurately than the considered baselines, with the improvements being largely explained by their superior ability to handle infrequent music items. In this way, the proposed hybrid music recommender systems provide means to alleviate the so-called "cold-start" problem for new releases and infrequent music and enable the discovery of music beyond the charts of popular music. Special attention is paid to the particularities of the music domain. I focus on two important music recommendation tasks: music artist recommendation, focusing on general, stable user music preferences, and music playlist continuation, focusing on local relationships in short listening sessions. I exploit data sources abundant in the context of on-line music consumption: user listening histories, hand-curated music playlists, music audio signal, and social tags. I investigate challenges specific to modeling music playlists: the choice and the arrangement of songs within playlists, and the effectiveness of different types of music descriptions to identify songs that fit well together.submitted by Andreu Vall PortabellaUniversität Linz, Dissertation, 2018(VLID)336676