A Survey on Awesome Korean NLP Datasets

English based datasets are commonly available from Kaggle, GitHub, or recently published papers. Although benchmark tests with English datasets are sufficient to show off the performances of new models and methods, still a researcher need to train and validate the models on Korean based datasets to produce a technology or product, suitable for Korean processing. This paper introduces 15 popular Korean based NLP datasets with summarized details such as volume, license, repositories, and other research results inspired by the datasets. Also, I provide high-resolution instructions with sample or statistics of datasets. The main characteristics of datasets are presented on a single table to provide a rapid summarization of datasets for researchers.Comment: 11 pages, 1 horizontal page for large tabl

국제정치적 갈등이 남북한 경제에 미치는 영향: 주식 시장과 무역에 대한 분석

학위논문(박사)--서울대학교 대학원 :사회과학대학 경제학부,2020. 2. 김병연.본 논문은 국제정치적 갈등의 경제적 영향에 대해 남북한의 사례를 중심으로 살펴본다. 특히 북한 관련 리스크가 남한의 주식 시장에 미치는 영향과 경제 제재가 북한의 무역에 미치는 영향에 주목하였다. 전체 논문은 개별적인 소주제를 다루는 세 편의 실증 연구로 구성된다. 첫 번째 장에서는 남한의 기업 주가 수익률이 북한 리스크에 어떻게 반응하는지 분석한다. 이를 위해 남한 언론의 북한 관련 보도 자료를 사용하여 월별 북한 리스크 지수를 작성하였다. 이 지수는 남북 관계의 긴장이 확대되거나 완화되는 경우 언론보도에 등장할 것으로 기대되는 키워드를 포함한 기사의 빈도수를 바탕으로 산출된다. 1999~2018년의 언론 보도자료를 분석한 결과, 북한 발 리스크는 핵/미사일 실험, 군사도발 등 이벤트 시점에 급증하며, 반대로 정상회담, 6자 회담 등 대화의 시기에는 감소하는 것으로 나타났다. 기업 주가 수익률을 종속변수로 한 회귀 분석에서는 국내 투자자의 주식 보유 비율이 높은 기업일수록, 자산 규모가 크고 고정자산의 비중이 높은 기업일수록, 남북경협에 관여한 경험이 있는 기업일수록 북한 관련 리스크에 민감하게 반응하는 것으로 나타났다. 두 번째 장에서는 북한에 부과된 주요 경제 제재가 무역에 준 영향을 무역의 질적 측면을 중심으로 분석한다. 우선 1998~2018년의 북한-중국 간 무역을 외연적 확장 수준(extensive margin), 상대 가격(relative unit price), 물량(quantity)으로 분해하고, 이 중 무역의 질적 측면으로 볼 수 있는 외연적 확장 수준과 상대 가격 지수의 변화에 주목하였다. 이를 통해, 북한의 대중 수출이 지난 20년간 양적으로 성장하였을 뿐 질적으로는 정체되어 있거나 오히려 후퇴했다는 사실을 확인하였다. 회귀분석에서는 북한의 무역을 직접적으로 타격하고자 한 한국과 일본의 독자 제재 및 2017년 UN안보리에서 결의된 다자 제재를 핵심 설명 변수로 설정하였고, 분석 방법으로는 전기 종속 변수가 포함된 동적 패널 모형(dynamic panel model)을 사용하였다. 추정 결과, 2017년 UN의 제재가 북한의 대중 수출에서 품목의 외연적 확장 수준을 축소 시킨 것으로 나타났다. 또한 2003년 일본의 제재는 중국 수입 시장에서 북한 생산품의 상대 가격을 유의하게 하락시킨 것으로 추정되었다. 추가적인 회귀분석에 따르면, 이러한 상대 가격 하락은 북-중 간 가격 협상력의 차이에서 기인한다. 이 연구 결과는 북한에 대한 무역 제재가 다른 주요 교역국과의 거래 관계를 차단하고 중국에 대한 의존도를 지나치게 높이면서 암묵적 비용을 발생시키고 있음을 시사한다. 마지막 장에서는 남한의 5.24 조치를 회피하기 위한 북-중 간의 우회무역 규모를 추정하였다. 이 연구는 중국의 기업-품목 단위의 자료를 사용하여 현재까지 북-중 무역에 대한 연구 중 가장 미시적 수준의 실증분석 결과를 제시한다. 분석 방법은 이중 차분법(difference-in-difference estimation)을 사용하였으며, 대 남한 수출과 대 북한 수입이 동시에 발생한 기업-품목들을 처치그룹으로 설정하여 2010년 전후의 변화를 추정하였다. 분석 결과, 북한의 중국을 경유한 남한으로의 간접 수출은 2010년 5.24 조치 이후 유의하게 증가한 것으로 나타났다. 산업별로 나누어 보면, 이러한 우회 무역은 주로 의류 임가공 부문에 집중되어 있으며 그 규모는 제재 이후 북한의 대남 직접 수출 감소분의 25%에 달하는 것으로 추정되었다.This dissertation investigates the economic impacts of international conflict, focusing on the cases of the two Koreas. Specifically, it examines the effects of North Korea-related risks on the South Korean stock market and economic sanctions on North Korea's foreign trade. It consists of three empirical studies covering subtopics. The first chapter analyzes how South Korean stock returns respond to North Korea-related risk. To do this, a monthly index for geopolitical risk from North Korea is constructed using South Korean media coverage database. The index is based on the frequency of articles containing keywords that are likely to appear in the media when inter-Korean tensions escalate or ease. Analysis of the media coverage from 1999 to 2018 show that the geopolitical risk index sharply increases in the occurrences of nuclear tests, missile launches, and military confrontations, while decreases significantly at around the times of summit meetings and multilateral talks. In the regression analysis, it is found that geopolitical risk related to North Korea has more negative effects on stock returns of firms with a higher share of domestic investors, larger assets and a higher proportion of fixed assets. It is also found that stock prices of companies involved in inter-Korean economic cooperation exhibit a more sensitive response to the North Korea risk. Chapter Ⅱ explores the impact of economic sanctions on North Koreas foreign trade, focusing on the quality of trade. It decomposes the trade between North Korea and China into the extensive margin, relative unit price and quantity, over the periods 1998-2018. Then it estimates sanction-induced changes in the former two elements of North Koreas export to China. The decomposition results show that the growth of North Koreas export to China is mostly attributed to the growth in quantity rather than quality. In the regression analysis, sanctions imposed by South Korea, Japan and the United Nations Security Council (UNSC) are used as key treatments. It is found that the UN sanctions in 2017 reduce the extensive margin in North Korean exports, and Japanese sanctions in 2003 have lowered the relative prices of North Korean products in the Chinese import market. The price impacts of sanctions are found to be associated with the bargaining power of China over North Korea. The findings suggest that trade sanctions against North Korea have created implicit costs by preventing North Korea from trading with alternative partners and increasing reliance on China. The last chapter estimates the size of the transit trade between North Korea and China to circumvent the sanctions imposed by South Korea. It exploits firm-product level variations in Chinese trade data to present micro evidence of the sanction-bypassing trade. Specifically, the transit trade is identified only when a firm import a product from North Korea and export the same product to South Korea in the same period. The difference-in-difference estimation results show that indirect exports from North Korea to South Korea via China are increased significantly by the 5.24 measures in 2010. The increase in North Koreas indirect export of apparels, in particular, accounts for a 25% of the decrease in North Korea's direct exports to South Korea.Introduction 1 Chapter Ⅰ. Geopolitical Risk from North Korea and Stock Market Reaction 4 1. Introduction 4 2. Related Literature 6 2.1. News-based Uncertainty Index 6 2.2. The Effects of Geopolitical Risk from North Korea 7 3. Measuring Geopolitical Risk from North Korea 7 3.1. Definition and Scope of Geopolitical Risk 7 3.2. Data and Methodology 9 3.3. Evaluating the GPRNK Index 12 4. Geopolitical Risk and Firm-level Stock Returns 20 4.1. Empirical Framework 20 4.2. Data and Descriptive Statistics 23 4.3. Baseline Results 26 4.4. Robustness Check 32 5. Conclusion 40 Chapter Ⅱ. Decomposing North Koreas Trade with China and Revisiting Sanction Effects 41 1. Introduction 41 2. Decomposing North Koreas Trade with China 46 2.1. Data 47 2.2. Methodology 50 2.3. Decomposition Results 52 3. Panel Regression Analysis 59 3.1. Empirical Framework 59 3.2. Baseline Results 63 3.3. Possible Channels 66 3.4. Robustness Check 71 4. Conclusion 73 Chapter Ⅲ. The Role of Chinese firms in Bypassing Sanctions on North Korea 75 1. Introduction 75 2. Data 78 2.1. Chinese Custom Trade Data 78 2.2. Stylized Facts 81 3. Empirical Strategy 83 4. Regression Results 86 4.1. North Koreas Indirect Exports via China 86 4.2. The Effects of Sanctions on the Indirect Exports 90 5. Conclusion 93 Concluding Remarks 95 References 98 Appendices 107 A1. Supplementary Materials for Chapter 1 107 A2. Supplementary Materials for Chapter Ⅱ 117 A3. Supplementary Materials for Chapter Ⅲ 121Docto

Understanding Social Media through Large Volume Measurements

The amount of user-generated web content has grown drastically in the past 15 years and many social media services are exceedingly popular nowadays. In this thesis we study social media content creation and consumption through large volume measurements of three prominent social media services, namely Twitter, YouTube, and Wikipedia. Common to the services is that they have millions of users, they are free to use, and the users of the services can both create and consume content. The motivation behind this thesis is to examine how users create and consume social media content, investigate why social media services are as popular as they are, what drives people to contribute on them, and see if it is possible to model the conduct of the users. We study how various aspects of social media content be that for example its creation and consumption or its popularity can be measured, characterized, and linked to real world occurrences. We have gathered more than 20 million tweets, metadata of more than 10 million YouTube videos and a complete six-year page view history of 19 different Wikipedia language editions. We show, for example, daily and hourly patterns for the content creation and consumption, content popularity distributions, characteristics of popular content, and user statistics. We will also compare social media with traditional news services and show the interaction with social media, news, and stock prices. In addition, we combine natural language processing with social media analysis, and discover interesting correlations between news and social media content. Moreover, we discuss the importance of correct measurement methods and show the effects of different sampling methods using YouTube measurements as an example.Sosiaalisen median suosio ja sen käyttäjien luoman sisällön määrä on kasvanut valtavasti viimeisen 15 vuoden aikana ja palvelut kuten Facebook, Instagram, Twitter, YouTube ja Wikipedia ovat erittäin suosittuja. Tässä väitöskirjassa tarkastellaan sosiaalisen median sisällön luonti- ja kulutusmalleja laajavoluumisen mittausdatan kautta. Väitöskirja sisältää mittausdataa Twitter-, YouTube- ja Wikipedia -palveluista. Yhteistä näille kolmelle palvelulle on muuan muassa se, että niillä on miljoonia käyttäjiä, niitä voi käyttää maksutta ja käyttäjät voivat luoda sekä kuluttaa sisältöä. Mittausdata sisältää yli 20 miljoona Twitter -viestiä, metadatatietoja yli kymmenestä miljoonasta YouTube -videosta ja täydellisen artikkelien katselukertojen tiedot kuudelta vuodelta 19 eri Wikipedian kieliversiosta. Tutkimuksen tarkoituksena on tarkastella kuinka käyttäjät luovat ja kuluttavat sisältöä sekä löytää niihin liittyviä malleja, joita voi hyödyntää tiedon jaossa, replikoinnissa ja tallentamisessa. Tutkimuksessa pyritään siis selvittämään miksi miksi sosiaalisen median palvelut ovat niin suosittuja kuin ne nyt ovat, mikä saa käyttäjät tuottamaan sisältöä niihin ja onko palveluiden käyttöä mahdollista mallintaa ja ennakoida. Väitöskirjassa verrataan myös sosiaalisen median ja tavallisten uutispalveluiden luonti- ja kulutusmalleja. Lisäksi näytetään kuinka sosiaalisen median sisältö, uutiset ja pörssikurssi hinnat ovat vuorovaikutuksessa toisiinsa. Väitöskirja sisältää myös pohdintaa oikean mittausmenetelmän valinnasta ja käyttämisestä sekä näytetään eri mittausmenetelmien vaikutuksista tuloksiin YouTube -mittausdatan avulla

Dirichlet belief networks for topic structure learning

Recently, considerable research effort has been devoted to developing deep architectures for topic models to learn topic structures. Although several deep models have been proposed to learn better topic proportions of documents, how to leverage the benefits of deep structures for learning word distributions of topics has not yet been rigorously studied. Here we propose a new multi-layer generative process on word distributions of topics, where each layer consists of a set of topics and each topic is drawn from a mixture of the topics of the layer above. As the topics in all layers can be directly interpreted by words, the proposed model is able to discover interpretable topic hierarchies. As a self-contained module, our model can be flexibly adapted to different kinds of topic models to improve their modelling accuracy and interpretability. Extensive experiments on text corpora demonstrate the advantages of the proposed model.Comment: accepted in NIPS 201

Impact of online public opinion regarding the Japanese nuclear wastewater incident on stock market based on the SOR model

The exposure of the Japanese nuclear wastewater incident has shaped online public opinion and has also caused a certain impact on stocks in aquaculture and feed industries. In order to explore the impact of network public opinion caused by public emergencies on relevant stocks, this paper uses the stimulus organism response(SOR) model to construct a framework model of the impact path of network public opinion on the financial stock market, and it uses emotional analysis, LDA and grounded theory methods to conduct empirical analysis. The study draws a new conclusion about the impact of online public opinion on the performance of relevant stocks in the context of the nuclear waste water incident in Japan. The positive change of media sentiment will lead to the decline of stock returns and the increase of volatility. The positive change of public sentiment will lead to the decline of stock returns in the current period and the increase of stock returns in the lag period. At the same time, we have proved that media attention, public opinion theme and prospect theory value have certain influences on stock performance in the context of the Japanese nuclear wastewater incident. The conclusion shows that after the public emergency, the government and investors need to pay attention to the changes of network public opinion caused by the event, so as to avoid the possible stock market risks

Semantic Knowledge Graphs for the News: A Review

ICT platforms for news production, distribution, and consumption must exploit the ever-growing availability of digital data. These data originate from different sources and in different formats; they arrive at different velocities and in different volumes. Semantic knowledge graphs (KGs) is an established technique for integrating such heterogeneous information. It is therefore well-aligned with the needs of news producers and distributors, and it is likely to become increasingly important for the news industry. This article reviews the research on using semantic knowledge graphs for production, distribution, and consumption of news. The purpose is to present an overview of the field; to investigate what it means; and to suggest opportunities and needs for further research and development.publishedVersio

Deep Learning-based Information Fusion Frameworks for Stock Price Movement Prediction

The challenges of modeling behaviour of financial markets, such as its high volatility, poor predictive behaviour, and its non-stationary nature, have continuously attracted attention of the researchers to employ advanced engineering methods. Within the context of financial econometrics, stock market movement prediction is a key and challenging problem. The research works reported in this thesis are motivated by the potentials of Artificial Intelligence (AI) and Machine Learning (ML)-based models, especially Deep Neural Network (DNN) architectures, for stock movement prediction. Considering recent progress in design and implementation of advanced DNN-based models, there has been a surge of interest in their application for predicting stock trends. In particular, the focus of the thesis is on utilization of information fusion to combine Twitter data with extended horizon market historical data for the task of price movement prediction. In this regard, the thesis made a number of contributions, first, the Noisy Deep Stock Movement Prediction Fusion (ND-SMPF) framework is proposed to extract news level temporal information; identify relevant words with highest correlation and effects on the stock trends, and; perform information fusion with historical price data. A real dataset is incorporated to evaluate performance of the proposed ND-SMPF framework. In addition, given that recent COVID-19 pandemic has negatively affected financial econometrics and stock markets across the globe, a unique COVID-19 related PRIce MOvement prediction (\CDATA) dataset is constructed. The objective is to incorporate effects of social media trends related to COVID-19 on stock market price movements. A novel hybrid and parallel DNN-based framework is then designed that integrates different and diversified learning architectures. Referred to as the COVID-19 adopted Hybrid and Parallel deep fusion framework for Stock price Movement Prediction (\SMP), innovative fusion strategies are used to combine scattered social media news related to COVID-19 with historical market data and perform accurate price movement prediction during a pandemic crisis