Quality management in social business intelligence projects

Social networks have become a new source of useful information for companies. Increasing the value of social data requires, first, assessing and improving the quality of the relevant data and, subsequently, developing practical solutions that apply them in business intelligence tasks. This paper focuses on the Twitter social network and the processing of social data for business intelligence projects. With this purpose, the paper starts by defining the special requirements of the analysis cubes of a Social Business Intelligence (SoBI) project and by reviewing previous work to demonstrate the lack of valid approaches to this problem. Afterwards, we present a new data processing method for SoBI projects whose main contribution is a phase of data exploration and profiling that serves to build a quality data collection with respect to the analysis objectives of the project

Quality management architecture for social media data

Abstract Social media data has provided various insights into the behaviour of consumers and businesses. However, extracted data may be erroneous, or could have originated from a malicious source. Thus, quality of social media should be managed. Also, it should be understood how data quality can be managed across a big data pipeline, which may consist of several processing and analysis phases. The contribution of this paper is evaluation of data quality management architecture for social media data. The theoretical concepts based on previous work have been implemented for data quality evaluation of Twitter-based data sets. Particularly, reference architecture for quality management in social media data has been extended and evaluated based on the implementation architecture. Experiments indicate that 150–800 tweets/s can be evaluated with two cloud nodes depending on the configuration

Quality management architecture for social media data

Abstract Social media data has provided various insights into the behaviour of consumers and businesses. However, extracted data may be erroneous, or could have originated from a malicious source. Thus, quality of social media should be managed. Also, it should be understood how data quality can be managed across a big data pipeline, which may consist of several processing and analysis phases. The contribution of this paper is evaluation of data quality management architecture for social media data. The theoretical concepts based on previous work have been implemented for data quality evaluation of Twitter-based data sets. Particularly, reference architecture for quality management in social media data has been extended and evaluated based on the implementation architecture. Experiments indicate that 150–800 tweets/s can be evaluated with two cloud nodes depending on the configuration

MOESM2 of Quality management architecture for social media data

Additional file 1. The file contains modelling of metadata store, quality rules, and data store for Cassandra

MOESM1 of Quality management architecture for social media data

Additional file 2. The file contains quality metrics for evaluation of timeliness, relevancy, and popularity

MOESM3 of Quality management architecture for social media data

Additional file 3. An example for data quality management of tweets. Example code can be compared with the corresponding data views (Figs. 14, 15)