Visual Transfer Learning in the Absence of the Source Data

Image recognition has become one of the most popular topics in machine learning. With the development of Deep Convolutional Neural Networks (CNN) and the help of the large scale labeled image database such as ImageNet, modern image recognition models can achieve competitive performance compared to human annotation in some general image recognition tasks. Many IT companies have adopted it to improve their visual related tasks. However, training these large scale deep neural networks requires thousands or even millions of labeled images, which is an obstacle when applying it to a specific visual task with limited training data. Visual transfer learning is proposed to solve this problem. Visual transfer learning aims at transferring the knowledge from a source visual task to a target visual task. Typically, the target task is related to the source task, and the training data in the target task is relatively small. In visual transfer learning, the majority of existing methods assume that the source data is freely available and use the source data to measure the discrepancy between the source and target task to help the transfer process. However, in many real applications, source data are often a subject of legal, technical and contractual constraints between data owners and data customers. Beyond privacy and disclosure obligations, customers are often reluctant to share their data. When operating customer care, collected data may include information on recent technical problems which is a highly sensitive topic that companies are not willing to share. This scenario is often called Hypothesis Transfer Learning (HTL) where the source data is absent. Therefore, these previous methods cannot be applied to many real visual transfer learning problems. In this thesis, we investigate the visual transfer learning problem under HTL setting. Instead of using the source data to measure the discrepancy, we use the source model as the proxy to transfer the knowledge from the source task to the target task. Compared to the source data, the well-trained source model is usually freely accessible in many tasks and contains equivalent source knowledge as well. Specifically, in this thesis, we investigate the visual transfer learning in two scenarios: domain adaptation and learning new categories. In contrast to the previous methods in HTL, our methods can both leverage knowledge from more types of source models and achieve better transfer performance. In chapter 3, we investigate the visual domain adaptation problem under the setting of Hypothesis Transfer Learning. We propose Effective Multiclass Transfer Learning (EMTLe) that can effectively transfer the knowledge when the size of the target set is small. Specifically, EMTLe can effectively transfer the knowledge using the outputs of the source models as the auxiliary bias to adjust the prediction in the target task. Experiment results show that EMTLe can outperform other baselines under the setting of HTL. In chapter 4, we investigate the semi-supervised domain adaptation scenario under the setting of HTL and propose our framework Generalized Distillation Semi-supervised Domain Adaptation (GDSDA). Specifically, we show that GDSDA can effectively transfer the knowledge using the unlabeled data. We also demonstrate that the imitation parameter, the hyperparameter in GDSDA that balances the knowledge from source and target task, is important to the transfer performance. Then we propose GDSDA-SVM which uses SVMs as the base classifier in GDSDA. We show that GDSDA-SVM can determine the imitation parameter in GDSDA autonomously. Compared to previous methods, whose imitation parameter can only be determined by either brutal force search or background knowledge, GDSDA-SVM is more effective in real applications. In chapter 5, we investigate the problem of fine-tuning the deep CNN to learn new food categories using the large ImageNet database as our source. Without accessing to the source data, i.e. the ImageNet dataset, we show that by fine-tuning the parameters of the source model with our target food dataset, we can achieve better performance compared to those previous methods. To conclude, the main contribution of is that we investigate the visual transfer learning problem under the HTL setting. We propose several methods to transfer the knowledge from the source task in supervised and semi-supervised learning scenarios. Extensive experiments results show that without accessing to any source data, our methods can outperform previous work

Learn, don't forget: constructive methods for effective continual learning

L'objectiu distintiu de la intel·ligència artificial és aconseguir agents amb capacitat per adaptar-se a fluxos continus d'informació. L'aprenentatge continu pretén donar resposta a aquest repte. No obstant això, els models d'aprenentatge automàtic acumulen el coneixement d'una manera diferent de la dels humans, i l'aprenentatge de noves tasques condueix a la degradació de les passades, fenomen anomenat "oblit catastròfic". La majoria dels mètodes d'aprenentatge continu o penalitzen el canvi dels paràmetres considerats importants per a les tasques passades (mètodes basats en la regularització) o bé emprenen una petita memòria intermèdia de repetició (mètodes basats en la repetició) que alimenta el model amb exemples de tasques passades per preservar el rendiment. Tot i això, el paper exacte que juga la regularització i els altres possibles factors que fan que el procés d'aprenentatge continu sigui eficaç no es coneixen bé. El projecte dóna llum sobre aquestes qüestions i suggereix maneres de millorar el rendiment de l'aprenentatge continu en tasques de visió com la classificació.El objetivo distintivo de la inteligencia artificial reside en conseguir agentes con capacidad para adaptarse a flujos continuos de información. El aprendizaje continuo pretende dar respuesta a este reto. Sin embargo, los modelos de aprendizaje automático acumulan el conocimiento de una manera diferente a la de los humanos, y el aprendizaje de nuevas tareas conduce a la degradación de las pasadas, fenómeno denominado "olvido catastrófico". La mayoría de los métodos de aprendizaje continuo o bien penalizan el cambio de los parámetros considerados importantes para las tareas pasadas (métodos basados en la regularización) o bien emplean un pequeño búfer de repetición (métodos basados en la repetición) que alimenta el modelo con ejemplos de tareas pasadas para preservar el rendimiento. Sin embargo, el papel exacto que juega la regularización y los demás posibles factores que hacen que el proceso de aprendizaje continuo sea eficaz no se conocen bien. El proyecto arroja luz sobre estas cuestiones y sugiere formas de mejorar el rendimiento del aprendizaje continuo en tareas de visión como la clasificación.The hallmark of artificial intelligence lies in agents with capabilities to adapt to continuous streams of information and tasks. Continual Learning aims to address this challenge. However, machine learning models accumulate knowledge in a manner different from humans, and learning new tasks leads to degradation in past ones, a phenomenon aptly named "catastrophic forgetting". Most continual learning methods either penalize the change of parameters deemed important for past tasks (regularization-based methods) or employ a small replay buffer (replay-based methods) that feeds the model examples from past tasks in order to preserve performance. However, the role and nature of the regularization and the other possible factors that make the continual learning process effective are not well understood. The project sheds light on these questions and suggests ways to improve the performance of continual learning in vision tasks such as classification.Outgoin

Deep Emotion Recognition in Textual Conversations: A Survey

While Emotion Recognition in Conversations (ERC) has seen a tremendous advancement in the last few years, new applications and implementation scenarios present novel challenges and opportunities. These range from leveraging the conversational context, speaker and emotion dynamics modelling, to interpreting common sense expressions, informal language and sarcasm, addressing challenges of real time ERC, recognizing emotion causes, different taxonomies across datasets, multilingual ERC to interpretability. This survey starts by introducing ERC, elaborating on the challenges and opportunities pertaining to this task. It proceeds with a description of the emotion taxonomies and a variety of ERC benchmark datasets employing such taxonomies. This is followed by descriptions of the most prominent works in ERC with explanations of the Deep Learning architectures employed. Then, it provides advisable ERC practices towards better frameworks, elaborating on methods to deal with subjectivity in annotations and modelling and methods to deal with the typically unbalanced ERC datasets. Finally, it presents systematic review tables comparing several works regarding the methods used and their performance. The survey highlights the advantage of leveraging techniques to address unbalanced data, the exploration of mixed emotions and the benefits of incorporating annotation subjectivity in the learning phase

Chemoinformatics-Driven Approaches for Kinase Drug Discovery

Given their importance for the majority of cell physiology processes, protein kinases are among the most extensively studied protein targets in drug discovery. Inappropriate regulation of their basal levels results in pathophysiological disorders. In this regard, small-molecule inhibitors of human kinome have been developed to treat these conditions effectively and improve the survival rates and life quality of patients. In recent years, kinase-related data has become increasingly available in the public domain. These large amounts of data provide a rich knowledge source for the computational studies of kinase drug discovery concepts. This thesis aims to systematically explore properties of kinase inhibitors on the basis of publicly available data. Hence, an established "selectivity versus promiscuity" conundrum of kinase inhibitors is evaluated, close structural analogs with diverging promiscuity levels are analyzed, and machine learning is employed to classify different kinase inhibitor binding modes. In the first study, kinase inhibitor selectivity trends are explored on the kinase pair level where kinase structural features and phylogenetic relationships are used to explain the obtained selectivity information. Next, selectivity of clinical kinase inhibitors is inspected on the basis of cell-based profiling campaign results to consolidate the previous findings. Further, clinical candidates are mapped to medicinal chemistry sources and promiscuity levels of different inhibitor subsets are estimated, including designated chemical probes. Additionally, chemical probe analysis is extended to expert-curated representatives to correlate the views established by scientific community and evaluate their potential for chemical biology applications. Then, large-scale promiscuity analysis of kinase inhibitor data combining several public repositories is performed to subsequently explore promiscuity cliffs (PCs) and PC pathways and study structure-promiscuity relationships. Furthermore, an automated extraction protocol prioritizing the most informative pathways is proposed with focus on those containing promiscuity hubs. In addition, the generated promiscuity data structures including cliffs, pathways, and hubs are discussed for their potential in experimental and computational follow-ups and subsequently made publicly available. Finally, machine learning methods are used to develop classification models of kinase inhibitors with distinct experimental binding modes and their potential for the development of novel therapeutics is assessed

Efficient Learning Machines

Computer scienc