Web Data Extraction, Applications and Techniques: A Survey

Web Data Extraction is an important problem that has been studied by means of different scientific tools and in a broad range of applications. Many approaches to extracting data from the Web have been designed to solve specific problems and operate in ad-hoc domains. Other approaches, instead, heavily reuse techniques and algorithms developed in the field of Information Extraction. This survey aims at providing a structured and comprehensive overview of the literature in the field of Web Data Extraction. We provided a simple classification framework in which existing Web Data Extraction applications are grouped into two main classes, namely applications at the Enterprise level and at the Social Web level. At the Enterprise level, Web Data Extraction techniques emerge as a key tool to perform data analysis in Business and Competitive Intelligence systems as well as for business process re-engineering. At the Social Web level, Web Data Extraction techniques allow to gather a large amount of structured data continuously generated and disseminated by Web 2.0, Social Media and Online Social Network users and this offers unprecedented opportunities to analyze human behavior at a very large scale. We discuss also the potential of cross-fertilization, i.e., on the possibility of re-using Web Data Extraction techniques originally designed to work in a given domain, in other domains.Comment: Knowledge-based System

Enabling Innovation across the Enterprise through Mashup-oriented Collaboration Environments

Nowadays enterprise collaboration is becoming essential for valuable innovation and competitive advantage. This collaboration must be brought a step forward from technical collaboration till collective smart exploitation of global intelligence. The Internet of Future is expected to be composed of a mesh of interoperable Web Services accessed from all over the Web. This approach has not yet caught on since a global user-service interaction is still an open issue. This paper states our vision with regard to the next generation front-end web technology that will enable integrated access to services, contents and things in the Future Internet. This approach will enable the massive deployment of services over Internet in a user-centric fashion. Having this in mind, the rationale behind EzWeb, a reference architecture and implementation of an open Enterprise 2.0 Collaboration Platform that empower its users to co-produce and share instant applications is presente

InnoJam: A Web 2.0 discussion platform featuring a recommender system

In this Master Thesis we have designed, implemented and evaluated a Web 2.0 platform for massive online-discussion, inspired by Innovation Jams. Innovation Jams, the original initiative from IBM, has proven to be successful at bringing together vast amounts of people, capturing their untapped knowledge and, while the participants are discussing, gather useful insights for a companyʼs innovation strategy [Spangler et al. 2006, Bjelland and Chapman Wood 2008]. Our approach, based in an open-source forum system, features visualization techniques and a recommender system in order to provide the participants in the Jam with useful insights and interesting discussion recommendations for an improved participation. A theoretical introduction and a state-of-the-art survey in recommender systems has been gathered in order to frame and support the design of the hybrid recommender system [Burke 2002], composed by a content-based and a collaborative filtering recommenders, developed for InnoJam

Prediction, Recommendation and Group Analytics Models in the domain of Mashup Services and Cyber-Argumentation Platform

Mashup application development is becoming a widespread software development practice due to its appeal for a shorter application development period. Application developers usually use web APIs from different sources to create a new streamlined service and provide various features to end-users. This kind of practice saves time, ensures reliability, accuracy, and security in the developed applications. Mashup application developers integrate these available APIs into their applications. Still, they have to go through thousands of available web APIs and chose only a few appropriate ones for their application. Recommending relevant web APIs might help application developers in this situation. However, very low API invocation from mashup applications creates a sparse mashup-web API dataset for the recommendation models to learn about the mashups and their web API invocation pattern. One research aims to analyze these mashup-specific critical issues, look for supplemental information in the mashup domain, and develop web API recommendation models for mashup applications. The developed recommendation model generates useful and accurate web APIs to reduce the impact of low API invocations in mashup application development. Cyber-Argumentation platform also faces a similarly challenging issue. In large-scale cyber argumentation platforms, participants express their opinions, engage with one another, and respond to feedback and criticism from others in discussing important issues online. Argumentation analysis tools capture the collective intelligence of the participants and reveal hidden insights from the underlying discussions. However, such analysis requires that the issues have been thoroughly discussed and participant’s opinions are clearly expressed and understood. Participants typically focus only on a few ideas and leave others unacknowledged and underdiscussed. This generates a limited dataset to work with, resulting in an incomplete analysis of issues in the discussion. One solution to this problem would be to develop an opinion prediction model for cyber-argumentation. This model would predict participant’s opinions on different ideas that they have not explicitly engaged. In cyber-argumentation, individuals interact with each other without any group coordination. However, the implicit group interaction can impact the participating user\u27s opinion, attitude, and discussion outcome. One of the objectives of this research work is to analyze different group analytics in the cyber-argumentation environment. The objective is to design an experiment to inspect whether the critical concepts of the Social Identity Model of Deindividuation Effects (SIDE) are valid in our argumentation platform. This experiment can help us understand whether anonymity and group sense impact user\u27s behavior in our platform. Another section is about developing group interaction models to help us understand different aspects of group interactions in the cyber-argumentation platform. These research works can help develop web API recommendation models tailored for mashup-specific domains and opinion prediction models for the cyber-argumentation specific area. Primarily these models utilize domain-specific knowledge and integrate them with traditional prediction and recommendation approaches. Our work on group analytic can be seen as the initial steps to understand these group interactions

Utilizing distributed web resources for enhanced knowledge representation

[no abstract

Privacy-preserving distributed service recommendation based on locality-sensitive hashing

With the advent of IoT (Internet of Things) age, considerable web services are emerging rapidly in service communities, which places a heavy burden on the target users’ service selection decisions. In this situation, various techniques, e.g., collaborative filtering (i.e., CF) is introduced in service recommendation to alleviate the service selection burden. However, traditional CF-based service recommendation approaches often assume that the historical user-service quality data is centralized, while neglect the distributed recommendation situation. Generally, distributed service recommendation involves inevitable message communication among different parties and hence, brings challenging efficiency and privacy concerns. In view of this challenge, a novel privacy-preserving distributed service recommendation approach based on Locality-Sensitive Hashing (LSH), i.e., DistSRLSH is put forward in this paper. Through LSH, DistSRLSH can achieve a good tradeoff among service recommendation accuracy, privacy-preservation and efficiency in distributed environment. Finally, through a set of experiments deployed on WS-DREAM dataset, we validate the feasibility of our proposal in handling distributed service recommendation problems

Semantic Collaborative Tagging for Web APIs Sharing and Reuse

Sharing and reuse of Web APIs for fast development of Web applications require advanced searching facilities to enable Web designers to find the Web APIs they need. In this paper we describe a Web API semantic collaborative tagging system to be implemented on top of the public ProgrammableWeb Web API repository. The system is designed to be used in a social context: the designers can take actively part in the semantic tagging of Web APIs, thus sharing their experience in developing their own Web applications. Moreover, they can exploit new searching facilities to find out relevant Web APIs according to different search scenarios and reuse them for fast deployment of new applications. To this aim, they rely in an hybrid fashion on the semantic tags and on the collective knowledge derived from past designers’ experiences. Proper matching and ranking metrics are defined and applied during Web API searching

On the Role of Context in the Design of Mobile Mashups

This paper presents a design methodology and an accompanying platform for the design and fast development of Context-Aware Mobile mashUpS (CAMUS). The approach is characterized by the role given to context as a first-class modeling dimension used to support i) the identification of the most adequate resources that can satisfy the users' situational needs and ii) the consequent tailoring at runtime of the provided data and functions. Context-based abstractions are exploited to generate models specifying how data returned by the selected services have to be merged and visualized by means of integrated views. Thanks to the adoption of Model-Driven Engineering (MDE) techniques, these models drive the flexible execution of the final mobile app on target mobile devices. A prototype of the platform, making use of novel and advanced Web and mobile technologies, is also illustrated

Recommendation and weaving of reusable mashup model patterns for assisted development

With this article, we give an answer to one of the open problems of mashup development that users may face when operating a model-driven mashup tool, namely the lack of modeling expertise. Although commonly considered simple applications, mashups can also be complex software artifacts depending on the number and types of Web resources (the components) they integrate. Mashup tools have undoubtedly simplified mashup development, yet the problem is still generally nontrivial and requires intimate knowledge of the components provided by the mashup tool, its underlying mashup paradigm, and of how to apply such to the integration of the components. This knowledge is generally neither intuitive nor standardized across different mashup tools and the consequent lack of modeling expertise affects both skilled programmers and end-user programmers alike. In this article, we show how to effectively assist the users of mashup tools with contextual, interactive recommendations of composition knowledge in the form of reusable mashup model patterns. We design and study three different recommendation algorithms and describe a pattern weaving approach for the one-click reuse of composition knowledge. We report on the implementation of three pattern recommender plugins for different mashup tools and demonstrate via user studies that recommending and weaving contextual mashup model patterns significantly reduces development times in all three cases

Design and development of a REST-based Web service platform for applications integration

Web services have attracted attention as a possible solution to share knowledge and application logic among different heterogeneous agents. A classic approach to this subject is using SOAP, a W3C protocol aimed to exchange structured information. The Web Services Interoperability organization (WS-I), defines a set of extensions, commonly called WS-*, which further enhance this knowledge exchange defining mechanisms and functionalities such as security, addressability or service composition. This thesis explores a relatively new alternative approach to the SOAP/WS-I stack: REST-based Web services. The acronym REST stands for Representational state transfer; this basically means that each unique URL is a representation of some object. You can get the contents of that object using an HTTP GET; you then might use a POST, PUT or DELETE to modify the object (in practice most of the services use a POST for this). All of Yahoo’s Web services use REST, including Flickr; del.icio.us API uses it; pubsub [http://www.pubsub.com/], Bloglines [http://www.bloglines.com/], Technorati [http://technorati.com/] and both, eBay and Amazon, have Web services for both REST and SOAP. Google seems to be consistent in implementing their Web services to use SOAP, with the exception of Blogger, which uses XML-RPC. The companies and organization that are using REST APIs have not been around for very long, and their APIs came out in the last seven years mostly. So REST is a new way to create and integrate Web services, whose main advantages are: being lightweight (not a lot of extra xml mark-up), human readable results, easy to build services (no toolkits required). Although REST is still generating discussion about possible implementations, and different proposals have been put forward, it provides enough mechanisms to allow knowledge-representations sharing among heterogeneous intelligent services. In this thesis, a novel way to integrate intelligent Web-services is designed and developed, and the resulting system is deployed in the domain of recommendation. Through a mashup, how different services are integrated and how a simple recommendation system consumes data coming from them to provide relevant information to users is presented. Part of this work has been carried out within the context of the Laboranova European project [http://www.laboranova.com/], and has been deployed to integrate a set of applications to create a virtual space to support innovation processes