Bayes in the age of intelligent machines

The success of methods based on artificial neural networks in creating intelligent machines seems like it might pose a challenge to explanations of human cognition in terms of Bayesian inference. We argue that this is not the case, and that in fact these systems offer new opportunities for Bayesian modeling. Specifically, we argue that Bayesian models of cognition and artificial neural networks lie at different levels of analysis and are complementary modeling approaches, together offering a way to understand human cognition that spans these levels. We also argue that the same perspective can be applied to intelligent machines, where a Bayesian approach may be uniquely valuable in understanding the behavior of large, opaque artificial neural networks that are trained on proprietary data

Dirichlet belief networks for topic structure learning

Recently, considerable research effort has been devoted to developing deep architectures for topic models to learn topic structures. Although several deep models have been proposed to learn better topic proportions of documents, how to leverage the benefits of deep structures for learning word distributions of topics has not yet been rigorously studied. Here we propose a new multi-layer generative process on word distributions of topics, where each layer consists of a set of topics and each topic is drawn from a mixture of the topics of the layer above. As the topics in all layers can be directly interpreted by words, the proposed model is able to discover interpretable topic hierarchies. As a self-contained module, our model can be flexibly adapted to different kinds of topic models to improve their modelling accuracy and interpretability. Extensive experiments on text corpora demonstrate the advantages of the proposed model.Comment: accepted in NIPS 201

Domain adaptation in Natural Language Processing

Domain adaptation has received much attention in the past decade. It has been shown that domain knowledge is paramount for building successful Natural Language Processing (NLP) applications. To investigate the domain adaptation problem, we conduct several experiments from different perspectives. First, we automatically adapt sentiment dictionaries for predicting the financial outcomes “excess return” and “volatility”. In these experiments, we compare manual adaptation of the domain-general dictionary with automatic adaptation, and manual adaptation with a combination consisting of first manual, then automatic adaptation. We demonstrate that automatic adaptation performs better than manual adaptation, namely the automatically adapted sentiment dictionary outperforms the previous state of the art in predicting excess return and volatility. Furthermore, we perform qualitative and quantitative analyses finding that annotation based on an expert’s a priori belief about a word’s meaning is error-prone – the meaning of a word can only be recognized in the context that it appears in. Second, we develop the temporal transfer learning approach to account for the language change in social media. The language of social media is changing rapidly – new words appear in the vocabulary, and new trends are constantly emerging. Temporal transfer-learning allows us to model these temporal dynamics in the document collection. We show that this method significantly improves the prediction of movie sales from discussions on social media forums. In particular, we illustrate the success of parameter transfer, the importance of textual information for financial prediction, and show that temporal transfer learning can capture temporal trends in the data by focusing on those features that are relevant in a particular time step, i.e., we obtain more robust models preventing overfitting. Third, we compare the performance of various domain adaptation models in low-resource settings, i.e., when there is a lack of large amounts of high-quality training data. This is an important issue in computational linguistics since the success of NLP applications primarily depends on the availability of training data. In real-world scenarios, the data is often too restricted and specialized. In our experiments, we evaluate different domain adaptation methods under these assumptions and find the most appropriate techniques for such a low-data problem. Furthermore, we discuss the conditions under which one approach substantially outperforms the other. Finally, we summarize our work on domain adaptation in NLP and discuss possible future work topics.Die Domänenanpassung hat in den letzten zehn Jahren viel Aufmerksamkeit erhalten. Es hat sich gezeigt, dass das Domänenwissen für die Erstellung erfolgreicher NLP-Anwendungen (Natural Language Processing) von größter Bedeutung ist. Um das Problem der Domänenanpassung zu untersuchen, führen wir mehrere Experimente aus verschiedenen Perspektiven durch. Erstens passen wir Sentimentlexika automatisch an, um die Überschussrendite und die Volatilität der Finanzergebnisse besser vorherzusagen. In diesen Experimenten vergleichen wir die manuelle Anpassung des allgemeinen Lexikons mit der automatischen Anpassung und die manuelle Anpassung mit einer Kombination aus erst manueller und dann automatischer Anpassung. Wir zeigen, dass die automatische Anpassung eine bessere Leistung erbringt als die manuelle Anpassung: das automatisch angepasste Sentimentlexikon übertrifft den bisherigen Stand der Technik bei der Vorhersage der Überschussrendite und der Volatilität. Darüber hinaus führen wir eine qualitative und quantitative Analyse durch und stellen fest, dass Annotationen, die auf der a priori Überzeugung eines Experten über die Bedeutung eines Wortes basieren, fehlerhaft sein können. Die Bedeutung eines Wortes kann nur in dem Kontext erkannt werden, in dem es erscheint. Zweitens entwickeln wir den Ansatz, den wir Temporal Transfer Learning benennen, um den Sprachwechsel in sozialen Medien zu berücksichtigen. Die Sprache der sozialen Medien ändert sich rasant – neue Wörter erscheinen im Vokabular und es entstehen ständig neue Trends. Temporal Transfer Learning ermöglicht es, diese zeitliche Dynamik in der Dokumentensammlung zu modellieren. Wir zeigen, dass diese Methode die Vorhersage von Filmverkäufen aus Diskussionen in Social-Media-Foren erheblich verbessert. In unseren Experimenten zeigen wir (i) den Erfolg der Parameterübertragung, (ii) die Bedeutung von Textinformationen für die finanzielle Vorhersage und (iii) dass Temporal Transfer Learning zeitliche Trends in den Daten erfassen kann, indem es sich auf die Merkmale konzentriert, die in einem bestimmten Zeitschritt relevant sind, d. h. wir erhalten robustere Modelle, die eine Überanpassung verhindern. Drittens vergleichen wir die Leistung verschiedener Domänenanpassungsmodelle in ressourcenarmen Umgebungen, d. h. wenn große Mengen an hochwertigen Trainingsdaten fehlen. Das ist ein wichtiges Thema in der Computerlinguistik, da der Erfolg der NLP-Anwendungen stark von der Verfügbarkeit von Trainingsdaten abhängt. In realen Szenarien sind die Daten oft zu eingeschränkt und spezialisiert. In unseren Experimenten evaluieren wir verschiedene Domänenanpassungsmethoden unter diesen Annahmen und finden die am besten geeigneten Techniken dafür. Darüber hinaus diskutieren wir die Bedingungen, unter denen ein Ansatz den anderen deutlich übertrifft. Abschließend fassen wir unsere Arbeit zur Domänenanpassung in NLP zusammen und diskutieren mögliche zukünftige Arbeitsthemen

Movies, TV programs and Youtube channels

학위논문(박사) -- 서울대학교대학원 : 공과대학 산업공학과, 2021.8. 조성준.The content market, including video content market, is a high-risk, high-return industry. Because the cost of copying and distributing the created video content is very low, large profit can be generated upon success. However, as content is an experience good, its quality cannot be judged before purchase. Hence, marketing has an important role in the content market because of the asymmetry of information between suppliers and consumers. Additionally, it has the characteristics of One Source Multi Use; if it is successful, additional profits can be created through various channels. Therefore, it is important for the content industry to correctly distinguish content with a high probability of success from the one without it and to conduct effective marketing activities to familiarize consumers with the product. Herein, we propose a methodology to assist in data-based decision-making using machine learning models and help in identifying problematic issues in video content markets such as movies, TV programs, and over-the-top (OTT) market. In the film market, although marketing is very important, decisions are still made based on the sense of practitioners. We used the market research data collected through online and offline surveys to learn a model that can predict the number of audiences on the opening-week Saturday, and then use the learned model to propose a method for effective marketing activities. In the TV program market, programming is performed to improve the overall viewership by matching TV programs and viewer groups well. We learn a model that predicts the audience rating of a program using the characteristics of the program and the audience-rating information of the programs before, after, and at the same time, and use the resulting data to assist in decision-making to find the optimal programming scenario. The OTT market is facing a new problem of user's perception bias caused by the “recent recommendation” system. In the fields of politics and news particularly, if the user does not have access to different viewpoints because of the recommendation service, it may create and/or deepen a bias toward a specific political view without the user being aware of it. In order to compensate for this, it is important to use the recommended channel while the user is well aware of what kind of channel it is. We built a channel network in the news/political field using the data extracted from the comments left by users on the videos of each channel. In addition, we propose a method to compensate for the bias by classifying networks into conservative and progressive channel clusters and presenting the topography of the political tendencies of YouTube channels.1 Introduction 1 2 Prediction of Movie Audience on First Saturday with Decision Trees 5 2.1 Background 5 2.2 Related work 9 2.3 Predictive model construction 15 2.3.1 Data 15 2.3.2 Target variable 17 2.3.3 Predictor variable 19 2.3.4 Decision Tree and ensemble prediction models 28 2.4 Prediction model evaluation 29 2.5 Summary 37 3 Prediction of TV program ratings with Decision Trees 40 3.1 Background 40 3.2 Related work 42 3.2.1 Research on the ratings themselves 42 3.2.2 Research on broadcasting programming 44 3.3 Predictive model construction 45 3.3.1 Target variable 45 3.3.2 Predictor variable 46 3.3.3 Prediction Model 48 3.4 Prediction model evaluation 50 3.4.1 Data 50 3.4.2 Experimental results 51 3.5 Optimization strategy using the predictive model 54 3.5.1 Broadcasting programming change process 56 3.5.2 Case Study 57 3.6 Summary 60 4 Relation detection of YouTube channels 62 4.1 Background 62 4.2 Related work 65 4.3 Method 67 4.3.1 Channel representation 68 4.3.2 Channel clustering with large k and merging clusters by keywords 71 4.3.3 Relabeling with RWR 73 4.3.4 Isolation score 74 4.4 Result 74 4.4.1 Channel representation 74 4.4.2 Channel clustering with large k and merging clusters by keywords 76 4.4.3 Relabeling with RWR 77 4.4.4 Isolation score 79 4.5 Discussion 80 4.5.1 On the Representativeness of the Channel Preferences of the Users from Their Comments 80 4.5.2 On Relabeling with RWR 82 4.6 Summary 83 5 Conclusion 85 5.1 Contribution 85 5.2 Future Direction 87 Bibliography 91 국문초록 110박

Article Search Tool and Topic Classifier

This thesis focuses on 3 main tasks related to Document Recommendations. The first approach deals with applying existing techniques on Document Recommendations using Doc2Vec. A robust representation of the same is presented to understand how noise induced in the embedding space affects predictions of the recommendations. The next phase focuses on improving the above recommendations using a Topic Classifier. A Hierarchical Attention Network is employed for this purpose. In order to increase the accuracy of prediction, this work establishes a relation to embedding size of the words in the article. In the last phase, model-agnostic Explainable AI (XAI) techniques are implemented to prove the findings in this thesis. XAI techniques are also employed to show how we can fine tune model hyper-parameters for a black-box model