Bayesian Optimisation for Planning And Reinforcement Learning

This thesis addresses the problem of achieving efficient non-myopic decision making by explicitly balancing exploration and exploitation. Decision making, both in planning and reinforcement learning (RL), enables agents or robots to complete tasks by acting on their environments. Complexity arises when completing objectives requires sacrificing short-term performance in order to achieve better long-term performance. Decision making algorithms with this characteristic are known as non-myopic, and require long sequences of actions to be evaluated, thereby greatly increasing the search space size. Optimal behaviours need balance two key quantities: exploration and exploitation. Exploitation takes advantage of previously acquired information or high performing solutions, whereas exploration focuses on acquiring more informative data. The balance between these quantities is crucial in both RL and planning. This thesis brings the following contributions: Firstly, a reward function trading off exploration and exploitation of gradients for sequential planning is proposed. It is based on Bayesian optimisation (BO) and is combined to a non-myopic planner to achieve efficient spatial monitoring. Secondly, the algorithm is extended to continuous actions spaces, called continuous belief tree search (CBTS), and uses BO to dynamically sample actions within a tree search, balancing high-performing actions and novelty. Finally, the framework is extended to RL, for which a multi-objective methodology for explicit exploration and exploitation balance is proposed. The two objectives are modelled explicitly and balanced at a policy level, as in BO. This allows for online exploration strategies, as well as a data-efficient model-free RL algorithm achieving exploration by minimising the uncertainty of Q-values (EMU-Q). The proposed algorithms are evaluated on different simulated and real-world robotics problems, displaying superior performance in terms of sample efficiency and exploration

LA ARQUITECTURA COMO IDENTIDAD DE MARCA EN LAS CASAS MUSEO: UN ESTUDIO SOBRE LAS CASAS MUSEO DE AUTOR EN GALICIA

House museums are spaces whose brand works as a promise to the visitor, showing the most intimate-affective part of the daily life of a relevant character. For this reason, it is necessary to manage the brand identity in such a way that the institution disseminates its mission through the most relevant aspects, such as architecture. This study focuses on the author's house museums in Galicia, using a qualitative methodology based on in-depth interviews, with the aim of analysing how these spaces manage their brand identity through architecture. A total of sixteen house museums were interviewed, which represents 100% of the author’s house museums open at that moment. The results show that architecture is seen as part of the core product, forming a fundamental axis within the brand identity of the institution. However, it seems that there is still no widespread trend in establishing branding strategies that emphasise the architecture-artist and architecture-territory link. This study aims to invite reflection on the importance of brand identity management in museums, specifically in-house museums, in order to serve as a guide for museum managers.Las Casas museo son espacios cuya marca funciona como una promesa hacia el visitante en la que muestra la parte más íntimo-afectiva de la cotidianidad de un personaje relevante. Por este motivo, resulta necesario una buena gestión de la identidad de marca gracias a la cual la institución difunda su misión por medio de aquellos aspectos más relevantes, como la arquitectura. Este estudio se centra en las Casas museo de autor de Galicia, a través de una metodología cualitativa basada en entrevistas en profundidad, con el objetivo de analizar cómo estos espacios gestionan su identidad de marca mediante la arquitectura. Se entrevistaron un total de dieciséis Casa museo de autor, lo que representa un 100% de las Casas museo de autor abiertas en el momento de la investigación. Los resultados reflejan que la arquitectura se vislumbra como parte del producto principal conformando un eje fundamental dentro de la identidad de marca de la institución. Sin embargo, parece que todavía no existe una corriente generalizada a la hora de establecer estrategias de branding per se que enfaticen el vínculo arquitectura-artista y arquitectura-territorio. Este estudio pretende invitar a la reflexión sobre la importancia de la gestión de la identidad de marca en los museos, concretamente en las Casas museo, con el fin de servir como guía para los gestores de los mismos

Compression of Deep Neural Networks for Image Instance Retrieval

Image instance retrieval is the problem of retrieving images from a database which contain the same object. Convolutional Neural Network (CNN) based descriptors are becoming the dominant approach for generating global image descriptors for the instance retrieval problem. One major drawback of CNN-based global descriptors is that uncompressed deep neural network models require hundreds of megabytes of storage making them inconvenient to deploy in mobile applications or in custom hardware. In this work, we study the problem of neural network model compression focusing on the image instance retrieval task. We study quantization, coding, pruning and weight sharing techniques for reducing model size for the instance retrieval problem. We provide extensive experimental results on the trade-off between retrieval performance and model size for different types of networks on several data sets providing the most comprehensive study on this topic. We compress models to the order of a few MBs: Two orders of magnitude smaller than the uncompressed models while achieving negligible loss in retrieval performance1

Empirical Comparison of Graph-based Recommendation Engines for an Apps Ecosystem

Recommendation engines (RE) are becoming highly popular, e.g., in the area of e-commerce. A RE offers new items (products or content) to users based on their profile and historical data. The most popular algorithms used in RE are based on collaborative filtering. This technique makes recommendations based on the past behavior of other users and the similarity between users and items. In this paper we have evaluated the performance of several RE based on the properties of the networks formed by users and items. The RE use in a novel way graph theoretic concepts like edges weights or network flow. The evaluation has been conducted in a real environment (ecosystem) for recommending apps to smartphone users. The analysis of the results allows concluding that the effectiveness of a RE can be improved if the age of the data, and if a global view of the data is considered. It also shows that graph-based RE are effective, but more experiments are required for a more accurate characterization of their properties

Lower-limb amputees can reduce the energy cost of walking when assisted by an Active Pelvis Orthosis

Exoskeletons could compete with active prostheses as effective aids to reduce the increased metabolic demands faced by lower-limb amputees during locomotion. However, little evidence of their efficacy with amputees has been provided so far. In this paper, a portable hip exoskeleton has been tested with seven healthy subjects and two transfemoral amputees, with the final goal to verify whether a hip flexion-extension assistance could be effective in reducing the metabolic cost of walking. The metabolic power of the participants was estimated through indirect calorimetry during alternated repetitions of three treadmill-based walking conditions: without the exoskeleton (NoExo), with the exoskeleton in zero-torque mode (ExoTM) and with the exoskeleton providing hip flexion-extension assistance (ExoAM). The results showed that the exoskeleton reduced the net metabolic power of the two amputees in ExoAM with respect to NoExo, by 5.0% and 3.4%. With healthy subjects, a 5.5±3.1% average reduction in the metabolic power was observed during ExoAM compared to ExoTM (differences were not statistically significant), whereas ExoAM required 3.9±3.0% higher metabolic power than NoExo (differences were not statistically significant). These results provide initial evidence of the potential of exoskeletal technologies for assisting lower-limb amputees, thereby paving the way for further experimentations