Quaternion-Based Graph Convolution Network for Recommendation

Graph Convolution Network (GCN) has been widely applied in recommender systems for its representation learning capability on user and item embeddings. However, GCN is vulnerable to noisy and incomplete graphs, which are common in real world, due to its recursive message propagation mechanism. In the literature, some work propose to remove the feature transformation during message propagation, but making it unable to effectively capture the graph structural features. Moreover, they model users and items in the Euclidean space, which has been demonstrated to have high distortion when modeling complex graphs, further degrading the capability to capture the graph structural features and leading to sub-optimal performance. To this end, in this paper, we propose a simple yet effective Quaternion-based Graph Convolution Network (QGCN) recommendation model. In the proposed model, we utilize the hyper-complex Quaternion space to learn user and item representations and feature transformation to improve both performance and robustness. Specifically, we first embed all users and items into the Quaternion space. Then, we introduce the quaternion embedding propagation layers with quaternion feature transformation to perform message propagation. Finally, we combine the embeddings generated at each layer with the mean pooling strategy to obtain the final embeddings for recommendation. Extensive experiments on three public benchmark datasets demonstrate that our proposed QGCN model outperforms baseline methods by a large margin.Comment: 13 pages, 7 figures, 6 tables. Submitted to ICDE 202

Quaternion-Based Self-Attentive Long Short-Term User Preference Encoding for Recommendation

Quaternion space has brought several benefits over the traditional Euclidean space: Quaternions (i) consist of a real and three imaginary components, encouraging richer representations; (ii) utilize Hamilton product which better encodes the inter-latent interactions across multiple Quaternion components; and (iii) result in a model with smaller degrees of freedom and less prone to overfitting. Unfortunately, most of the current recommender systems rely on real-valued representations in Euclidean space to model either user's long-term or short-term interests. In this paper, we fully utilize Quaternion space to model both user's long-term and short-term preferences. We first propose a QUaternion-based self-Attentive Long term user Encoding (QUALE) to study the user's long-term intents. Then, we propose a QUaternion-based self-Attentive Short term user Encoding (QUASE) to learn the user's short-term interests. To enhance our models' capability, we propose to fuse QUALE and QUASE into one model, namely QUALSE, by using a Quaternion-based gating mechanism. We further develop Quaternion-based Adversarial learning along with the Bayesian Personalized Ranking (QABPR) to improve our model's robustness. Extensive experiments on six real-world datasets show that our fused QUALSE model outperformed 11 state-of-the-art baselines, improving 8.43% at HIT@1 and 10.27% at NDCG@1 on average compared with the best baseline

Continuous Input Embedding Size Search For Recommender Systems

Latent factor models are the most popular backbones for today's recommender systems owing to their prominent performance. Latent factor models represent users and items as real-valued embedding vectors for pairwise similarity computation, and all embeddings are traditionally restricted to a uniform size that is relatively large (e.g., 256-dimensional). With the exponentially expanding user base and item catalog in contemporary e-commerce, this design is admittedly becoming memory-inefficient. To facilitate lightweight recommendation, reinforcement learning (RL) has recently opened up opportunities for identifying varying embedding sizes for different users/items. However, challenged by search efficiency and learning an optimal RL policy, existing RL-based methods are restricted to highly discrete, predefined embedding size choices. This leads to a largely overlooked potential of introducing finer granularity into embedding sizes to obtain better recommendation effectiveness under a given memory budget. In this paper, we propose continuous input embedding size search (CIESS), a novel RL-based method that operates on a continuous search space with arbitrary embedding sizes to choose from. In CIESS, we further present an innovative random walk-based exploration strategy to allow the RL policy to efficiently explore more candidate embedding sizes and converge to a better decision. CIESS is also model-agnostic and hence generalizable to a variety of latent factor RSs, whilst experiments on two real-world datasets have shown state-of-the-art performance of CIESS under different memory budgets when paired with three popular recommendation models.Comment: To appear in SIGIR'2

Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks

Many state-of-the-art neural models for NLP are heavily parameterized and thus memory inefficient. This paper proposes a series of lightweight and memory efficient neural architectures for a potpourri of natural language processing (NLP) tasks. To this end, our models exploit computation using Quaternion algebra and hypercomplex spaces, enabling not only expressive inter-component interactions but also significantly ($75\%$) reduced parameter size due to lesser degrees of freedom in the Hamilton product. We propose Quaternion variants of models, giving rise to new architectures such as the Quaternion attention Model and Quaternion Transformer. Extensive experiments on a battery of NLP tasks demonstrates the utility of proposed Quaternion-inspired models, enabling up to $75\%$ reduction in parameter size without significant loss in performance.Comment: ACL 201

Robust GNSS Carrier Phase-based Position and Attitude Estimation Theory and Applications

Mención Internacional en el título de doctorNavigation information is an essential element for the functioning of robotic platforms and intelligent transportation systems. Among the existing technologies, Global Navigation Satellite Systems (GNSS) have established as the cornerstone for outdoor navigation, allowing for all-weather, all-time positioning and timing at a worldwide scale. GNSS is the generic term for referring to a constellation of satellites which transmit radio signals used primarily for ranging information. Therefore, the successful operation and deployment of prospective autonomous systems is subject to our capabilities to support GNSS in the provision of robust and precise navigational estimates. GNSS signals enable two types of ranging observations: –code pseudorange, which is a measure of the time difference between the signal’s emission and reception at the satellite and receiver, respectively, scaled by the speed of light; –carrier phase pseudorange, which measures the beat of the carrier signal and the number of accumulated full carrier cycles. While code pseudoranges provides an unambiguous measure of the distance between satellites and receiver, with a dm-level precision when disregarding atmospheric delays and clock offsets, carrier phase measurements present a much higher precision, at the cost of being ambiguous by an unknown number of integer cycles, commonly denoted as ambiguities. Thus, the maximum potential of GNSS, in terms of navigational precision, can be reach by the use of carrier phase observations which, in turn, lead to complicated estimation problems. This thesis deals with the estimation theory behind the provision of carrier phase-based precise navigation for vehicles traversing scenarios with harsh signal propagation conditions. Contributions to such a broad topic are made in three directions. First, the ultimate positioning performance is addressed, by proposing lower bounds on the signal processing realized at the receiver level and for the mixed real- and integer-valued problem related to carrier phase-based positioning. Second, multi-antenna configurations are considered for the computation of a vehicle’s orientation, introducing a new model for the joint position and attitude estimation problems and proposing new deterministic and recursive estimators based on Lie Theory. Finally, the framework of robust statistics is explored to propose new solutions to code- and carrier phase-based navigation, able to deal with outlying impulsive noises.La información de navegación es un elemental fundamental para el funcionamiento de sistemas de transporte inteligentes y plataformas robóticas. Entre las tecnologías existentes, los Sistemas Globales de Navegación por Satélite (GNSS) se han consolidado como la piedra angular para la navegación en exteriores, dando acceso a localización y sincronización temporal a una escala global, irrespectivamente de la condición meteorológica. GNSS es el término genérico que define una constelación de satélites que transmiten señales de radio, usadas primordinalmente para proporcionar información de distancia. Por lo tanto, la operatibilidad y funcionamiento de los futuros sistemas autónomos pende de nuestra capacidad para explotar GNSS y estimar soluciones de navegación robustas y precisas. Las señales GNSS permiten dos tipos de observaciones de alcance: –pseudorangos de código, que miden el tiempo transcurrido entre la emisión de las señales en los satélites y su acquisición en la tierra por parte de un receptor; –pseudorangos de fase de portadora, que miden la fase de la onda sinusoide que portan dichas señales y el número acumulado de ciclos completos. Los pseudorangos de código proporcionan una medida inequívoca de la distancia entre los satélites y el receptor, con una precisión de decímetros cuando no se tienen en cuenta los retrasos atmosféricos y los desfases del reloj. En contraposición, las observaciones de la portadora son super precisas, alcanzando el milímetro de exactidud, a expensas de ser ambiguas por un número entero y desconocido de ciclos. Por ende, el alcanzar la máxima precisión con GNSS queda condicionado al uso de las medidas de fase de la portadora, lo cual implica unos problemas de estimación de elevada complejidad. Esta tesis versa sobre la teoría de estimación relacionada con la provisión de navegación precisa basada en la fase de la portadora, especialmente para vehículos que transitan escenarios donde las señales no se propagan fácilmente, como es el caso de las ciudades. Para ello, primero se aborda la máxima efectividad del problema de localización, proponiendo cotas inferiores para el procesamiento de la señal en el receptor y para el problema de estimación mixto (es decir, cuando las incógnitas pertenecen al espacio de números reales y enteros). En segundo lugar, se consideran las configuraciones multiantena para el cálculo de la orientación de un vehículo, presentando un nuevo modelo para la estimación conjunta de posición y rumbo, y proponiendo estimadores deterministas y recursivos basados en la teoría de Lie. Por último, se explora el marco de la estadística robusta para proporcionar nuevas soluciones de navegación precisa, capaces de hacer frente a los ruidos atípicos.Programa de Doctorado en Ciencia y Tecnología Informática por la Universidad Carlos III de MadridPresidente: José Manuel Molina López.- Secretario: Giorgi Gabriele.- Vocal: Fabio Dovi