The Detection of Emerging Trends Using Wikipedia Traffic Data and Context Networks.

Can online media predict new and emerging trends, since there is a relationship between trends in society and their representation in online systems? While several recent studies have used Google Trends as the leading online information source to answer corresponding research questions, we focus on the online encyclopedia Wikipedia often used for deeper topical reading. Wikipedia grants open access to all traffic data and provides lots of additional (semantic) information in a context network besides single keywords. Specifically, we suggest and study context-normalized and time-dependent measures for a topic's importance based on page-view time series of Wikipedia articles in different languages and articles related to them by internal links. As an example, we present a study of the recently emerging Big Data market with a focus on the Hadoop ecosystem, and compare the capabilities of Wikipedia versus Google in predicting its popularity and life cycles. To support further applications, we have developed an open web platform to share results of Wikipedia analytics, providing context-rich and language-independent relevance measures for emerging trends

Version number versus release date (in weeks since 2004-01-01) for Apache Hadoop (gray) and Apache SOLR (green).

<p>The version numbers for different parallel development branches of both projects indicate ongoing improvements up to the ends of the active projects. The dates of the jump in interest derived from the plot of L.TRRI_<i>a</i>(<i>t</i>) in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0141892#pone.0141892.g004" target="_blank">Fig 4</a> are shown here as vertical black lines for SOLR (a) and Hadoop (b).</p

Project life cycle phases derived from Wikipedia usage data based on L.TRRI_<i>a</i>(<i>t</i>) (straight lines, Eq (5)) and G.TRRI_<i>a</i>(<i>t</i>) (dashed lines, Eq (6)) for Apache Hadoop (black) and Apache SOLR (green) versus time in weeks since 2009-01-01.

<p>For Hadoop L.TRRI_<i>a</i>(<i>t</i>) we find a strong linear increasing trend with some short-term fluctuations fading out after three months in the years 2009 and 2010. These short-term peaks reflect conference seasons. In 2011, finally, L.TRRI_<i>a</i>(<i>t</i>) gets greater than one: a sporadic jump in the user interest is followed by a saturation. G.TRRI_<i>a</i>(<i>t</i>) shows a significantly weaker trend during the same time and remains below one. This means that public interest in Hadoop related information is bound to the English language. The ‘break through’ of Apache Hadoop as a relevant topic is thus in 2011, about 18 months after Apache SOLR became a relevant topic. However, finally Apache SOLR is less relevant than Apache Hadoop. The gray areas indicate times for which no data was available for technical reasons. While the thin lines are the original L.TRRI_<i>a</i>(<i>t</i>) data, the thick lines have been obtained by applying a running average filter with a window length of 12 weeks.</p