Discriminative Topic Modeling with Logistic LDA

Despite many years of research into latent Dirichlet allocation (LDA), applying LDA to collections of non-categorical items is still challenging. Yet many problems with much richer data share a similar structure and could benefit from the vast literature on LDA. We propose logistic LDA, a novel discriminative variant of latent Dirichlet allocation which is easy to apply to arbitrary inputs. In particular, our model can easily be applied to groups of images, arbitrary text embeddings, and integrates well with deep neural networks. Although it is a discriminative model, we show that logistic LDA can learn from unlabeled data in an unsupervised manner by exploiting the group structure present in the data. In contrast to other recent topic models designed to handle arbitrary inputs, our model does not sacrifice the interpretability and principled motivation of LDA

Structural Bootstrapping - A Novel, Generative Mechanism for Faster and More Efficient Acquisition of Action-Knowledge

eISSN: 1943-0612Humans, but also robots, learn to improve their behavior. Without existing knowledge, learning either needs to be explorative and, thus, slow or-to be more efficient-it needs to rely on supervision, which may not always be available. However, once some knowledge base exists an agent can make use of it to improve learning efficiency and speed. This happens for our children at the age of around three when they very quickly begin to assimilate new information by making guided guesses how this fits to their prior knowledge. This is a very efficient generative learning mechanism in the sense that the existing knowledge is generalized into as-yet unexplored, novel domains. So far generative learning has not been employed for robots and robot learning remains to be a slow and tedious process. The goal of the current study is to devise for the first time a general framework for a generative process that will improve learning and which can be applied at all different levels of the robot's cognitive architecture. To this end, we introduce the concept of structural bootstrapping-borrowed and modified from child language acquisition-to define a probabilistic process that uses existing knowledge together with new observations to supplement our robot's data-base with missing information about planning-, object-, as well as, action-relevant entities. In a kitchen scenario, we use the example of making batter by pouring and mixing two components and show that the agent can efficiently acquire new knowledge about planning operators, objects as well as required motor pattern for stirring by structural bootstrapping. Some benchmarks are shown, too, that demonstrate how structural bootstrapping improves performanceTaikomosios informatikos katedraVytauto Didžiojo universiteta

Inference, learning and optimization on structured domains : methods and applications

With the development of modern digitization, increasingly more data emerge in almost all areas. It is worth emphasizing that not only does the quantity of the data increase, but also the number of data types, or the sources where data are collected, are boosted. Undoubtedly, more information can be exploited with the presence of more comprehensive data. Nevertheless, merging different data together also makes the analysis of them more challenging. There exist various forms of dependencies or interactions among multiple data. Therefore, working with these data goes much beyond traditional machine leaning tasks: e.g. classification or regression, where the output is a single scalar. In this dissertation, multiple data sets together are considered as structures, in which different dependencies can hence be modeled. In particular, structures are encoded within three forms by using: graphs, kernels and manifolds respectively, which can match different application domains. This dissertation goes through inference, learning and optimiza- tion of structured data which are represented with different forms. Some existing work is reviewed while several new methods are put forward. In particular, to make the dissertation more practical, different methods were applied and evaluated on real-world application domains, including image segmentation, image annotation, protein function prediction, object-action relation modeling and 3D transformation estimation. Of course the applicabilities of these methods go far beyond those presented in the dissertation. Meanwhile, this dissertation attempts to, with practical case studies, provide some main principles or methodologies when confronting structured data, and empirical experience in above-mentioned domains should be easily transferred to other ones. Above all, the main contributions of this dissertation are several novel models and learning algorithms for structured outputs, including joint SVM and kernel generalized homogeneity analysis for multi-label learning, persistent sequential Monte Carlo for learning undirected graphical models. The study in this dissertation is expected to widen and/or deepen the understanding of relevant research.With the development of modern digitization, increasingly more data emerge in almost all areas. It is worth emphasizing that not only does the quantity of the data increase, but also the number of data types, or the sources where data are collected, are boosted. Undoubtedly, more information can be exploited with the presence of more comprehensive data. Nevertheless, merging different data together also makes the analysis of them more challenging. There exist various forms of dependencies or interactions among multiple data. Therefore, working with these data goes much beyond traditional machine leaning tasks: e.g. classification or regression, where the output is a single scalar. In this dissertation, multiple data sets together are considered as structures, in which different dependencies can hence be modeled. In particular, structures are encoded within three forms by using: graphs, kernels and manifolds respectively, which can match different application domains. This dissertation goes through inference, learning and optimiza- tion of structured data which are represented with different forms. Some existing work is reviewed while several new methods are put forward. In particular, to make the dissertation more practical, different methods were applied and evaluated on real-world application domains, including image segmentation, image annotation, protein function prediction, object-action relation modeling and 3D transformation estimation. Of course the applicabilities of these methods go far beyond those presented in the dissertation. Meanwhile, this dissertation attempts to, with practical case studies, provide some main principles or methodologies when confronting structured data, and empirical experience in above-mentioned domains should be easily transferred to other ones. Above all, theHanchen XiongEnth. u.a. 10 Veröff. d. Verf. aus den Jahren 2013 - 2015Innsbruck, Univ., Diss., 2015OeBB(VLID)43870

Discriminative topic modeling with logistic LDA

Despite many years of research into latent Dirichlet allocation (LDA), applying LDA to collections of non-categorical items is still challenging for practitioners. Yet many problems with much richer data share a similar structure and could benefit from the vast literature on LDA. We propose logistic LDA, a novel discriminative variant of latent Dirichlet allocation which is easy to apply to arbitrary inputs. In particular, our model can easily be applied to groups of images, arbitrary text embeddings, or integrate deep neural networks. Although it is a discriminative model, we show that logistic LDA can learn from unlabeled data in an unsupervised manner by exploiting the group structure present in the data. In contrast to other recent topic models designed to handle arbitrary inputs, our model does not sacrifice the interpretability and principled motivation of LDA