Creating Capsule Wardrobes from Fashion Images

We propose to automatically create capsule wardrobes. Given an inventory of candidate garments and accessories, the algorithm must assemble a minimal set of items that provides maximal mix-and-match outfits. We pose the task as a subset selection problem. To permit efficient subset selection over the space of all outfit combinations, we develop submodular objective functions capturing the key ingredients of visual compatibility, versatility, and user-specific preference. Since adding garments to a capsule only expands its possible outfits, we devise an iterative approach to allow near-optimal submodular function maximization. Finally, we present an unsupervised approach to learn visual compatibility from "in the wild" full body outfit photos; the compatibility metric translates well to cleaner catalog photos and improves over existing methods. Our results on thousands of pieces from popular fashion websites show that automatic capsule creation has potential to mimic skilled fashionistas in assembling flexible wardrobes, while being significantly more scalable.Comment: Accepted to CVPR 201

Towards Unsupervised Graph Completion Learning on Graphs with Features and Structure Missing

In recent years, graph neural networks (GNN) have achieved significant developments in a variety of graph analytical tasks. Nevertheless, GNN's superior performance will suffer from serious damage when the collected node features or structure relationships are partially missing owning to numerous unpredictable factors. Recently emerged graph completion learning (GCL) has received increasing attention, which aims to reconstruct the missing node features or structure relationships under the guidance of a specifically supervised task. Although these proposed GCL methods have made great success, they still exist the following problems: the reliance on labels, the bias of the reconstructed node features and structure relationships. Besides, the generalization ability of the existing GCL still faces a huge challenge when both collected node features and structure relationships are partially missing at the same time. To solve the above issues, we propose a more general GCL framework with the aid of self-supervised learning for improving the task performance of the existing GNN variants on graphs with features and structure missing, termed unsupervised GCL (UGCL). Specifically, to avoid the mismatch between missing node features and structure during the message-passing process of GNN, we separate the feature reconstruction and structure reconstruction and design its personalized model in turn. Then, a dual contrastive loss on the structure level and feature level is introduced to maximize the mutual information of node representations from feature reconstructing and structure reconstructing paths for providing more supervision signals. Finally, the reconstructed node features and structure can be applied to the downstream node classification task. Extensive experiments on eight datasets, three GNN variants and five missing rates demonstrate the effectiveness of our proposed method.Comment: Accepted by 23rd IEEE International Conference on Data Mining (ICDM 2023

Modélisation des comportements de recherche basé sur les interactions des utilisateurs

Les utilisateurs de systèmes d'information divisent normalement les tâches en une séquence de plusieurs étapes pour les résoudre. En particulier, les utilisateurs divisent les tâches de recherche en séquences de requêtes, en interagissant avec les systèmes de recherche pour mener à bien le processus de recherche d'informations. Les interactions des utilisateurs sont enregistrées dans des journaux de requêtes, ce qui permet de développer des modèles pour apprendre automatiquement les comportements de recherche à partir des interactions des utilisateurs avec les systèmes de recherche. Ces modèles sont à la base de multiples applications d'assistance aux utilisateurs qui aident les systèmes de recherche à être plus interactifs, faciles à utiliser, et cohérents. Par conséquent, nous proposons les contributions suivantes : un modèle neuronale pour apprendre à détecter les limites des tâches de recherche dans les journaux de requête ; une architecture de regroupement profond récurrent qui apprend simultanément les représentations de requête et regroupe les requêtes en tâches de recherche ; un modèle non supervisé et indépendant d'utilisateur pour l'identification des tâches de recherche prenant en charge les requêtes dans seize langues ; et un modèle de tâche de recherche multilingue, une approche non supervisée qui modélise simultanément l'intention de recherche de l'utilisateur et les tâches de recherche. Les modèles proposés améliorent les méthodes existantes de modélisation, en tenant compte de la confidentialité des utilisateurs, des réponses en temps réel et de l'accessibilité linguistique. Le respect de la vie privée de l'utilisateur est une préoccupation majeure, tandis que des réponses rapides sont essentielles pour les systèmes de recherche qui interagissent avec les utilisateurs en temps réel, en particulier dans la recherche par conversation. Dans le même temps, l'accessibilité linguistique est essentielle pour aider les utilisateurs du monde entier, qui interagissent avec les systèmes de recherche dans de nombreuses langues. Les contributions proposées peuvent bénéficier à de nombreuses applications d'assistance aux utilisateurs, en aidant ces derniers à mieux résoudre leurs tâches de recherche lorsqu'ils accèdent aux systèmes de recherche pour répondre à leurs besoins d'information.Users of information systems normally divide tasks in a sequence of multiple steps to solve them. In particular, users divide search tasks into sequences of queries, interacting with search systems to carry out the information seeking process. User interactions are registered on search query logs, enabling the development of models to automatically learn search patterns from the users' interactions with search systems. These models underpin multiple user assisting applications that help search systems to be more interactive, user-friendly, and coherent. User assisting applications include query suggestion, the ranking of search results based on tasks, query reformulation analysis, e-commerce applications, retrieval of advertisement, query-term prediction, mapping of queries to search tasks, and so on. Consequently, we propose the following contributions: a neural model for learning to detect search task boundaries in query logs; a recurrent deep clustering architecture that simultaneously learns query representations through self-training, and cluster queries into groups of search tasks; Multilingual Graph-Based Clustering, an unsupervised, user-agnostic model for search task identification supporting queries in sixteen languages; and Language-agnostic Search Task Model, an unsupervised approach that simultaneously models user search intent and search tasks. Proposed models improve on existing methods for modeling user interactions, taking into account user privacy, realtime response times, and language accessibility. User privacy is a major concern in Ethics for intelligent systems, while fast responses are critical for search systems interacting with users in realtime, particularly in conversational search. At the same time, language accessibility is essential to assist users worldwide, who interact with search systems in many languages. The proposed contributions can benefit many user assisting applications, helping users to better solve their search tasks when accessing search systems to fulfill their information needs

Improving personalized elderly care: an approach using cognitive agents to better assist elderly people

Tesis por compendio de publicaciones[ES]El envejecimiento de la población a nivel global es una constante cada vez más presente en el día a día y las consecuencias derivadas de este problema son cada vez más impactantes para el correcto funcionamiento y estructuración de la sociedad. En este contexto, hablamos de consecuencias a nivel de crecimiento económico, estilos de vida (y jubilación), relaciones familiares, recursos disponibles por el gobierno a la franja etaria más anciana e inevitablemente la prevalencia de enfermedades crónicas. Es ante esta realidad que surge la necesidad de desarrollo y promoción de estrategias eficaces en el acompañamiento, prevención y estímulo al envejecimiento activo y saludable de la población para garantizar que las personas ancianas continúen teniendo un papel relevante en la sociedad en lugar de someterse al aislamiento y fácil deterioro de las capacidades físicas, cognitivas, emocionales y sociales. De esta forma, tiene todo el sentido aprovechar todos los desarrollos tecnológicos verificados en los últimos años, principalmente en lo que se refiere a avances en las áreas de dispositivos móviles, inteligencia artificial y sistemas de monitoreo y crear soluciones capaces de brindar apoyo diariamente al recopilar datos e indicadores del estado de salud y, en respuesta, proporcionar diversas acciones personalizadas que motiven la adopción de mejores hábitos de salud y medios para lograr este envejecimiento activo y saludable. El desafío consiste en motivar a esta población a conciliar su día a día con el interés y la voluntad de utilizar aplicaciones y sistemas que brinden este apoyo personalizado. Algunas de las abordajes recientemente explorados en la literatura con este objetivo y que han alcanzado resultados prometedores se basan en la utilización de técnicas de gamificación e incentivo al cumplimiento de desafíos a nivel de salud (como si la persona estuviera jugando un juego) y la utilización de interacciones personalizadas con objetos (ya sean físicos como robots o virtuales como avatares) capaces de brindar feedback más personal, creando así una conexión más cercana entre ambas entidades. El trabajo aquí presentado combina estas ideas y resulta en un enfoque inteligente para la promoción del bienestar de la población anciana a través de un sistema de cuidados de salud personalizado. Este sistema incorpora diversas técnicas de gamificación para la promoción de mejores hábitos y comportamientos, y la utilización de un asistente virtual cognitivo capaz de entender las necesidades e intereses del usuario para posibilitar un feedback e interacción personalizados con el fin de ayudar y motivar al cumplimiento de los diferentes desafíos y objetivos que se identifiquen. El enfoque propuesto fue validado a través de un estudio con 12 usuarios ancianos y se lograron resultados significativos en términos de usabilidad, aceptación y efectos de salud. Específicamente, los resultados obtenidos permiten respaldar la importancia y el efecto positivo de combinar técnicas de gamificación e interacción con un asistente virtual cognitivo que traduzca el progreso del estado de salud del usuario, ya que se lograron mejoras significativas en los resultados de salud después de la intervención. Además, los resultados de usabilidad obtenidos mediante la cumplimentación de un cuestionario de usabilidad confirmaron la buena adhesión a el enfoque presentado. Estos resultados validan la hipótesis de la investigación estudiada en el desarrollo de esta disertación

Recommender Systems with Generative Retrieval

Modern recommender systems perform large-scale retrieval by first embedding queries and item candidates in the same unified space, followed by approximate nearest neighbor search to select top candidates given a query embedding. In this paper, we propose a novel generative retrieval approach, where the retrieval model autoregressively decodes the identifiers of the target candidates. To that end, we create semantically meaningful tuple of codewords to serve as a Semantic ID for each item. Given Semantic IDs for items in a user session, a Transformer-based sequence-to-sequence model is trained to predict the Semantic ID of the next item that the user will interact with. To the best of our knowledge, this is the first Semantic ID-based generative model for recommendation tasks. We show that recommender systems trained with the proposed paradigm significantly outperform the current SOTA models on various datasets. In addition, we show that incorporating Semantic IDs into the sequence-to-sequence model enhances its ability to generalize, as evidenced by the improved retrieval performance observed for items with no prior interaction history.Comment: Preprint versio

Evolutionary Clustering of Apprentices' Self- Regulated Learning Behavior in Learning Journals

Learning journals are increasingly used in vocational education to foster self-regulated learning and reflective learning practices. However, for many apprentices, documenting working experiences is a difficult task. In this article, we profile apprentices' learning behavior in an online learning journal. Based on a pedagogical framework, we propose a novel multistep clustering pipeline that integrates different learning dimensions into a combined profile. Specifically, the profiles are described in terms of effort, consistency, regularity, help-seeking behavior, and quality of the written entries. Our results on two populations of chef apprentices (183 apprentices) interacting with an online learning journal (over 121K entries) show that our pipeline captures changes in learning patterns over time and yields interpretable profiles that can be related to academic performance. The obtained profiles can be used as a basis for personalized interventions, with the ultimate goal of improving the apprentices' learning experience