Coping with noise in a real-world weblog crawler and retrieval system

In this paper we examine the effects of noise when creating a real-world weblog corpus for information retrieval. We focus on the DiffPost (Lee et al. 2008) approach to noise removal from blog pages, examining the difficulties encountered when crawling the blogosphere during the creation of a real-world corpus of blog pages. We introduce and evaluate a number of enhancements to the original DiffPost approach in order to increase the robustness of the algorithm. We then extend DiffPost by looking at the anchor-text to text ratio, and dis- cover that the time-interval between crawls is more impor- tant to the successful application of noise-removal algorithms within the blog context, than any additional improvements to the removal algorithm itself

Data Mining Algorithms for Internet Data: from Transport to Application Layer

Nowadays we live in a data-driven world. Advances in data generation, collection and storage technology have enabled organizations to gather data sets of massive size. Data mining is a discipline that blends traditional data analysis methods with sophisticated algorithms to handle the challenges posed by these new types of data sets. The Internet is a complex and dynamic system with new protocols and applications that arise at a constant pace. All these characteristics designate the Internet a valuable and challenging data source and application domain for a research activity, both looking at Transport layer, analyzing network tra c flows, and going up to Application layer, focusing on the ever-growing next generation web services: blogs, micro-blogs, on-line social networks, photo sharing services and many other applications (e.g., Twitter, Facebook, Flickr, etc.). In this thesis work we focus on the study, design and development of novel algorithms and frameworks to support large scale data mining activities over huge and heterogeneous data volumes, with a particular focus on Internet data as data source and targeting network tra c classification, on-line social network analysis, recommendation systems and cloud services and Big data

Using the Web Infrastructure for Real Time Recovery of Missing Web Pages

Given the dynamic nature of the World Wide Web, missing web pages, or 404 Page not Found responses, are part of our web browsing experience. It is our intuition that information on the web is rarely completely lost, it is just missing. In whole or in part, content often moves from one URI to another and hence it just needs to be (re-)discovered. We evaluate several methods for a \justin- time approach to web page preservation. We investigate the suitability of lexical signatures and web page titles to rediscover missing content. It is understood that web pages change over time which implies that the performance of these two methods depends on the age of the content. We therefore conduct a temporal study of the decay of lexical signatures and titles and estimate their half-life. We further propose the use of tags that users have created to annotate pages as well as the most salient terms derived from a page\u27s link neighborhood. We utilize the Memento framework to discover previous versions of web pages and to execute the above methods. We provide a work ow including a set of parameters that is most promising for the (re-)discovery of missing web pages. We introduce Synchronicity, a web browser add-on that implements this work ow. It works while the user is browsing and detects the occurrence of 404 errors automatically. When activated by the user Synchronicity offers a total of six methods to either rediscover the missing page at its new URI or discover an alternative page that satisfies the user\u27s information need. Synchronicity depends on user interaction which enables it to provide results in real time

Blogs as Infrastructure for Scholarly Communication.

This project systematically analyzes digital humanities blogs as an infrastructure for scholarly communication. This exploratory research maps the discourses of a scholarly community to understand the infrastructural dynamics of blogs and the Open Web. The text contents of 106,804 individual blog posts from a corpus of 396 blogs were analyzed using a mix of computational and qualitative methods. Analysis uses an experimental methodology (trace ethnography) combined with unsupervised machine learning (topic modeling), to perform an interpretive analysis at scale. Methodological findings show topic modeling can be integrated with qualitative and interpretive analysis. Special attention must be paid to data fitness, or the shape and re-shaping practices involved with preparing data for machine learning algorithms. Quantitative analysis of computationally generated topics indicates that while the community writes about diverse subject matter, individual scholars focus their attention on only a couple of topics. Four categories of informal scholarly communication emerged from the qualitative analysis: quasi-academic, para-academic, meta-academic, and extra-academic. The quasi and para-academic categories represent discourse with scholarly value within the digital humanities community, but do not necessarily have an obvious path into formal publication and preservation. A conceptual model, the (in)visible college, is introduced for situating scholarly communication on blogs and the Open Web. An (in)visible college is a kind of scholarly communication that is informal, yet visible at scale. This combination of factors opens up a new space for the study of scholarly communities and communication. While (in)invisible colleges are programmatically observable, care must be taken with any effort to count and measure knowledge work in these spaces. This is the first systematic, data driven analysis of the digital humanities and lays the groundwork for subsequent social studies of digital humanities.PhDInformationUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/111592/1/mcburton_1.pd

Invisible Search and Online Search Engines

" Invisible Search and Online Search Engines considers the use of search engines in contemporary everyday life and the challenges this poses for media and information literacy. Looking for mediated information is mostly done online and arbitrated by the various tools and devices that people carry with them on a daily basis. Because of this, search engines have a significant impact on the structure of our lives, and personal and public memories. Haider and Sundin consider what this means for society, whilst also uniting research on information retrieval with research on how people actually look for and encounter information. Search engines are now one of society’s key infrastructures for knowing and becoming informed. While their use is dispersed across myriads of social practices, where they have acquired close to naturalised positions, they are commercially and technically centralised. Arguing that search, searching, and search engines have become so widely used that we have stopped noticing them, Haider and Sundin consider what it means to be so reliant on this all-encompassing and increasingly invisible information infrastructure. Invisible Search and Online Search Engines is the first book to approach search and search engines from a perspective that combines insights from the technical expertise of information science research with a social science and humanities approach. As such, the book should be essential reading for academics, researchers, and students working on and studying information science, library and information science (LIS), media studies, journalism, digital cultures, and educational sciences.

Sentiment Analysis for Social Media

Sentiment analysis is a branch of natural language processing concerned with the study of the intensity of the emotions expressed in a piece of text. The automated analysis of the multitude of messages delivered through social media is one of the hottest research fields, both in academy and in industry, due to its extremely high potential applicability in many different domains. This Special Issue describes both technological contributions to the field, mostly based on deep learning techniques, and specific applications in areas like health insurance, gender classification, recommender systems, and cyber aggression detection

Semantic discovery and reuse of business process patterns

Patterns currently play an important role in modern information systems (IS) development and their use has mainly been restricted to the design and implementation phases of the development lifecycle. Given the increasing significance of business modelling in IS development, patterns have the potential of providing a viable solution for promoting reusability of recurrent generalized models in the very early stages of development. As a statement of research-in-progress this paper focuses on business process patterns and proposes an initial methodological framework for the discovery and reuse of business process patterns within the IS development lifecycle. The framework borrows ideas from the domain engineering literature and proposes the use of semantics to drive both the discovery of patterns as well as their reuse

A framework for adaptive personalised e-advertisements

The art of personalised e-advertising relies on attracting the user‟s attention to the recommended product, as it relates to their taste, interest and data. Whilst in practice, companies attempt various forms of personalisation; research of personalised e-advertising is rare, and seldom routed on solid theory. Adaptive hypermedia (AH) techniques have contributed to the development of personalised tools for adaptive content delivery, mostly in the educational domain. This study explores the use of these theories and techniques in a specific field – adaptive e-advertisements. This is accomplished firstly by structuring a theoretical framework that roots adaptive hypermedia into the domain of e-advertising and then uses this theoretical framework as the base for implementing and evaluating an adaptive e-advertisement system called “MyAds”. The novelty of this approach relies on a systematic design and evaluation based on adaptive hypermedia taxonomy. In particular, this thesis uses a user centric methodology to design and evaluate the proposed approach. It also reports on evaluations that investigated users‟ opinions on the appropriate design of MyAds. Another set of evaluations reported on users‟ perceptions of the implemented system, allowing for a reflection on the users‟ acceptance level of e-advertising. The results from both implicit and explicit feedback indicated that users found the MyAds system acceptable and agreed that the implemented user modelling and AH features within the system contributed to achieving acceptance, within their e-advertisement experience due to the different personalisation methods