Big data reduction framework for value creation in sustainable enterprises

Value creation is a major sustainability factor for enterprises, in addition to profit maximization and revenue generation. Modern enterprises collect big data from various inbound and outbound data sources. The inbound data sources handle data generated from the results of business operations, such as manufacturing, supply chain management, marketing, and human resource management, among others. Outbound data sources handle customer-generated data which are acquired directly or indirectly from customers, market analysis, surveys, product reviews, and transactional histories. However, cloud service utilization costs increase because of big data analytics and value creation activities for enterprises and customers. This article presents a novel concept of big data reduction at the customer end in which early data reduction operations are performed to achieve multiple objectives, such as a) lowering the service utilization cost, b) enhancing the trust between customers and enterprises, c) preserving privacy of customers, d) enabling secure data sharing, and e) delegating data sharing control to customers. We also propose a framework for early data reduction at customer end and present a business model for end-to-end data reduction in enterprise applications. The article further presents a business model canvas and maps the future application areas with its nine components. Finally, the article discusses the technology adoption challenges for value creation through big data reduction in enterprise applications

Automating lead scoring with machine learning: An experimental study

Companies often gather a tremendous amount of data, such as browsing behavior, email activities and other contact data. This data can be the source of important competitive advantage by utilizing it in estimating a contact\u27s purchase probability using predictive analytics. The calculated purchase probability can then be used by companies to solve different business problems, such as optimizing their sales processes. The purpose of this article is to study how machine learning can be used to perform lead scoring as a special application case of making use of purchase probability. Historical behavioral data is used as training data for the classification algorithm, and purchase moments are used to limit the behavioral data for the contacts that have purchased a product in the past. Different ways of aggregating time-series data are tested to ensure that limiting the activities for buyers does not result in model bias. The results suggest that it is possible to estimate the purchase probability of leads using supervised learning algorithms, such as random forest, and that it is possible to obtain business insights from the results using visual analytic

New Approach for Market Intelligence Using Artificial and Computational Intelligence

Small and medium sized retailers are central to the private sector and a vital contributor to economic growth, but often they face enormous challenges in unleashing their full potential. Financial pitfalls, lack of adequate access to markets, and difficulties in exploiting technology have prevented them from achieving optimal productivity. Market Intelligence (MI) is the knowledge extracted from numerous internal and external data sources, aimed at providing a holistic view of the state of the market and influence marketing related decision-making processes in real-time. A related, burgeoning phenomenon and crucial topic in the field of marketing is Artificial Intelligence (AI) that entails fundamental changes to the skillssets marketers require. A vast amount of knowledge is stored in retailers’ point-of-sales databases. The format of this data often makes the knowledge they store hard to access and identify. As a powerful AI technique, Association Rules Mining helps to identify frequently associated patterns stored in large databases to predict customers’ shopping journeys. Consequently, the method has emerged as the key driver of cross-selling and upselling in the retail industry. At the core of this approach is the Market Basket Analysis that captures knowledge from heterogeneous customer shopping patterns and examines the effects of marketing initiatives. Apriori, that enumerates frequent itemsets purchased together (as market baskets), is the central algorithm in the analysis process. Problems occur, as Apriori lacks computational speed and has weaknesses in providing intelligent decision support. With the growth of simultaneous database scans, the computation cost increases and results in dramatically decreasing performance. Moreover, there are shortages in decision support, especially in the methods of finding rarely occurring events and identifying the brand trending popularity before it peaks. As the objective of this research is to find intelligent ways to assist small and medium sized retailers grow with MI strategy, we demonstrate the effects of AI, with algorithms in data preprocessing, market segmentation, and finding market trends. We show with a sales database of a small, local retailer how our Åbo algorithm increases mining performance and intelligence, as well as how it helps to extract valuable marketing insights to assess demand dynamics and product popularity trends. We also show how this results in commercial advantage and tangible return on investment. Additionally, an enhanced normal distribution method assists data pre-processing and helps to explore different types of potential anomalies.Små och medelstora detaljhandlare är centrala aktörer i den privata sektorn och bidrar starkt till den ekonomiska tillväxten, men de möter ofta enorma utmaningar i att uppnå sin fulla potential. Finansiella svårigheter, brist på marknadstillträde och svårigheter att utnyttja teknologi har ofta hindrat dem från att nå optimal produktivitet. Marknadsintelligens (MI) består av kunskap som samlats in från olika interna externa källor av data och som syftar till att erbjuda en helhetssyn av marknadsläget samt möjliggöra beslutsfattande i realtid. Ett relaterat och växande fenomen, samt ett viktigt tema inom marknadsföring är artificiell intelligens (AI) som ställer nya krav på marknadsförarnas färdigheter. Enorma mängder kunskap finns sparade i databaser av transaktioner samlade från detaljhandlarnas försäljningsplatser. Ändå är formatet på dessa data ofta sådant att det inte är lätt att tillgå och utnyttja kunskapen. Som AI-verktyg erbjuder affinitetsanalys en effektiv teknik för att identifiera upprepade mönster som statistiska associationer i data lagrade i stora försäljningsdatabaser. De hittade mönstren kan sedan utnyttjas som regler som förutser kundernas köpbeteende. I detaljhandel har affinitetsanalys blivit en nyckelfaktor bakom kors- och uppförsäljning. Som den centrala metoden i denna process fungerar marknadskorgsanalys som fångar upp kunskap från de heterogena köpbeteendena i data och hjälper till att utreda hur effektiva marknadsföringsplaner är. Apriori, som räknar upp de vanligt förekommande produktkombinationerna som köps tillsammans (marknadskorgen), är den centrala algoritmen i analysprocessen. Trots detta har Apriori brister som algoritm gällande låg beräkningshastighet och svag intelligens. När antalet parallella databassökningar stiger, ökar också beräkningskostnaden, vilket har negativa effekter på prestanda. Dessutom finns det brister i beslutstödet, speciellt gällande metoder att hitta sällan förekommande produktkombinationer, och i att identifiera ökande popularitet av varumärken från trenddata och utnyttja det innan det når sin höjdpunkt. Eftersom målet för denna forskning är att hjälpa små och medelstora detaljhandlare att växa med hjälp av MI-strategier, demonstreras effekter av AI med hjälp av algoritmer i förberedelsen av data, marknadssegmentering och trendanalys. Med hjälp av försäljningsdata från en liten, lokal detaljhandlare visar vi hur Åbo-algoritmen ökar prestanda och intelligens i datautvinningsprocessen och hjälper till att avslöja värdefulla insikter för marknadsföring, framför allt gällande dynamiken i efterfrågan och trender i populariteten av produkterna. Ytterligare visas hur detta resulterar i kommersiella fördelar och konkret avkastning på investering. Dessutom hjälper den utvidgade normalfördelningsmetoden i förberedelsen av data och med att hitta olika slags anomalier

Management Responses to Online Reviews: Big Data From Social Media Platforms

User-generated content from virtual communities helps businesses develop and sustain competitive advantages, which leads to asking how firms can strategically manage that content. This research, which consists of two studies, discusses management response strategies for hotel firms to gain a competitive advantage and improve customer relationship management by leveraging big data, social media analytics, and deep learning techniques. Since negative reviews' harmful effects are greater than positive comments' contribution, firms must strategise their responses to intervene in and minimise those damages. Although current literature includes a sheer amount of research that presents effective response strategies to negative reviews, they mostly overlook an extensive classification of response strategies. The first study consists of two phases and focuses on comprehensive response strategies to only negative reviews. The first phase is explorative and presents a correlation analysis between response strategies and overall ratings of hotels. It also reveals the differences in those strategies based on hotel class, average customer rating, and region. The second phase investigates effective response strategies for increasing the subsequent ratings of returning customers using logistic regression analysis. It presents that responses involving statements of admittance of mistake(s), specific action, and direct contact requests help increase following ratings of previously dissatisfied returning customers. In addition, personalising the response for better customer relationship management is particularly difficult due to the significant variability of textual reviews with various topics. The second study examines the impact of personalised management responses to positive and negative reviews on rating growth, integrating a novel method of multi-topic matching approach with a panel data analysis. It demonstrates that (a) personalised responses improve future ratings of hotels; (b) the effect of personalised responses is stronger for luxury hotels in increasing future ratings. Lastly, practical insights are provided

Analysing Transportation Data with Open Source Big Data Analytic Tools

Big data analytics allows a vast amount of structured and unstructured data to be effectively processed so that correlations, hidden patterns, and other useful information can be mined from the data. Several open source big data analytic tools that can perform tasks such as dimensionality reduction, feature extraction, transformation, optimization, are now available. One interesting area where such tools can provide effective solutions is transportation. Big data analytics can be used to efficiently manage transport infrastructure assets such as roads, airports, bus stations or ports. In this paper an overview of two open source big data analytic tools is first provided followed by a simple demonstration of application of these tools on transport dataset

Blending big data analytics : review on challenges and a recent study

With the collection of massive amounts of data every day, big data analytics has emerged as an important trend for many organizations. These collected data can contain important information that may be key to solving wide-ranging problems, such as cyber security, marketing, healthcare, and fraud. To analyze their large volumes of data for business analyses and decisions, large companies, such as Facebook and Google, adopt analytics. Such analyses and decisions impact existing and future technology. In this paper, we explore how big data analytics is utilized as a technique for solving problems of complex and unstructured data using such technologies as Hadoop, Spark, and MapReduce. We also discuss the data challenges introduced by big data according to the literature, including its six V's. Moreover, we investigate case studies of big data analytics on various techniques of such analytics, namely, text, voice, video, and network analytics. We conclude that big data analytics can bring positive changes in many fields, such as education, military, healthcare, politics, business, agriculture, banking, and marketing, in the future. © 2013 IEEE

Analyzing data streams using a dynamic compact stream pattern algorithm

A growing number of applications that generate massive streams of data need intelligent data processing and online analysis. Data & Knowledge Engineering (DKE) has been known to stimulate the exchange of ideas and interaction between these two related fields of interest. DKE makes it possible to understand, apply and assess knowledge and skills required for the development and application data mining systems. With present technology, companies are able to collect vast amounts of data with relative ease. With no hesitation, many companies now have more data than they can handle. A vital portion of this data entails large unstructured data sets which amount up to 90 percent of an organization’s data. With data quantities growing steadily, the explosion of data is putting a strain on infrastructures as diverse companies having to increase their data center capacity with more servers and storages. This study conceptualized handling enormous data as a stream mining problem that applies to continuous data stream and proposes an ensemble of unsupervised learning methods for efficiently detecting anomalies in stream data

Database marketing intelligence methodology supported by ontologies and knowlegde discovery in databases

Tese de doutoramento em Tecnologias e Sistemas de InformaçãoActualmente as organizações actuam em ambientes caracterizados pela inconstância, elevada competitividade e pressão no desenvolvimento de novas abordagens ao mercado e aos clientes. Nesse contexto, o acesso à informação, o suporte à tomada de decisão e a partilha de conhecimento tornam-se essenciais para o desempenho organizativo. No domínio do marketing têm surgido diversas abordagens para a exploração do conteúdo das suas bases de dados. Uma das abordagens, utilizadas com maior sucesso, tem sido o processo para a descoberta de conhecimento em bases de dados. Por outro lado, a necessidade de representação e partilha de conhecimento tem contribuído para um crescente desenvolvimento das ontologias em áreas diversas como sejam medicina, aviação ou segurança. O presente trabalho cruza diversas áreas: tecnologias e sistemas de informação (em particular a descoberta de conhecimento), o marketing (especificamente o database marketing) e as ontologias. O objectivo principal desta investigação foca o papel das ontologias em termos de suporte e assistência ao processo de descoberta de conhecimento em bases de dados num contexto de database marketing. Através de abordagens distintas foram formuladas duas ontologias: ontologia para o processo de descoberta de conhecimento em bases de dados e, a ontologia para o processo database marketing suportado na extracção de conhecimento em bases de dados (com reutilização da ontologia anterior). O processo para licitação e validação de conhecimento, baseou-se no método de Delphi (ontologia de database marketing) e no processo de investigação baseada na revisão de literatura (ontologia de descoberta de conhecimento). A concretização das ontologias suportou-se em duas metodologias: metodologia methontology, para a ontologia de descoberta de conhecimento e metodologia 101 para a ontologia de database marketing. A última, evidencia a reutilização de ontologias, viabilizando assim a reutilização da ontologia de descoberta de conhecimento na ontologia de database marketing. Ambas ontologias foram desenvolvidas sobre a ferramenta Protege-OWL permitindo não só a criação de toda a hierarquia de classes, propriedades e relações, como também, a realização de métodos de inferência através de linguagens baseadas em regras de Web semântica. Posteriormente, procedeu-se à experimentação da ontologia em casos práticos de extracção de conhecimento a partir de bases de dados de marketing. O emprego das ontologias neste contexto de investigação, representa uma abordagem pioneira e inovadora, uma vez que são propostas para assistirem em cada uma das fases do processo de extracção de conhecimento em bases de dados através de métodos de inferência. È assim possível assistir o utilizador em cada fase do processo de database marketing em acções tais como de selecção de actividades de marketing em função dos objectivos de marketing (e.g., perfil de cliente), em acções de selecção dados (e.g., tipos de dados a utilizar em função da actividade a desenvolver) ou mesmo no processo de selecção de algoritmos (e.g. inferir sobre o tipo de algoritmo a usar em função do objectivo definido). A integração das duas ontologias num contexto mais lato permite, propor uma metodologia com vista ao efectivo suporte do processo de database marketing baseado no processo de descoberta de conhecimento em bases de dados, denominado nesta dissertação como: Database Marketing Intelligence. Para a demonstração da viabilidade da metodologia proposta foi seguido o método action-research com o qual se observou e testou o papel das ontologias no suporte à descoberta de conhecimento em bases de dados (através de um caso prático) num contexto de database marketing. O trabalho de aplicação prática decorreu sobre uma base de dados real relativa a um cartão de fidelização de uma companhia petrolífera a operar em Portugal. Os resultados obtidos serviram para demonstrar em duas vertente o sucesso da abordagem proposta: por um lado foi possível formalizar e acompanhar todo o processo de descoberta de conhecimento em bases de dados; por outro lado, foi possível perspectivar uma metodologia para um domínio concreto suportado por ontologias (suporte á decisão na selecção de métodos e tarefas) e na descoberta de conhecimento em bases de dados.Nowadays, the environment in which companies work is turbulent, very competitive and pressure in the development of new approaches to the market and clients. In this context, the access to information, the decision support and knowledge sharing become essential for the organization performance. In the marketing domain several approaches for the exploration of database exploration have emerged. One of the most successfully used approaches has been the knowledge discovery process in databases. On the other hand, the necessity of knowledge representation and sharing and contributed to a growing development of ontologies in several areas such as in the medical, the aviation or safety areas. This work crosses several areas: technology and information systems (specifically knowledge discovery in databases), marketing (specifically database marketing) and ontologies in general. The main goal of this investigation is to focus on the role of ontologies in terms of support and aid to the knowledge discovery process in databases in a database marketing context. Through distinct approaches two ontologies were created: ontology for the knowledge discovery process in databases, and the ontology for the database marketing process supported on the knowledge extraction in databases (reusing the former ontology). The elicitation and validation of knowledge process was based on the Delphi method (database marketing ontology) and the investigation process was based on literature review (knowledge discovery ontology). The carrying out of both ontologies was based on two methodologies: methontology methodology, for the knowledge discovery process and 101 methodology for the database marketing ontology. The former methodology, stresses the reusing of ontologies, allowing the reusing of the knowledge discovery ontology in the database marketing ontology. Both ontologies were developed with the Protege-OWL tool. This tool allows not only the creation of all the hierarchic classes, properties and relationships, but also the carrying out of inference methods through web semantics based languages. Then, the ontology was tested in practical cases of knowledge extraction from marketing databases. The application of ontologies in this investigation represents a pioneer and innovative approach, once they are proposed to aid and execute an effective support in each phase of the knowledge extraction from databases in the database marketing context process. Through inference processes on the knowledge base created it was possible to assist the user in each phase of the database marketing process such as, in marketing activity selection actions according to the marketing objectives (e.g., client profile) or in data selection actions (e.g., type of data to use according to the activity to be preformed. In relation to aid in the knowledge discovery process in databases, it was also possible to infer on the type of algorithm to use according to the defined objective or even according to the type of data pre-processing activities to develop regarding the type of data and type of attribute information. The integration of both ontologies in a more general context allows proposing a methodology aiming to the effective support of the database marketing process based on the knowledge discovery process in databases, named in this dissertation as: Database Marketing Intelligence. To demonstrate the viability of the proposed methodology the action-research method was followed with which the role of ontologies in assisting knowledge discovery in databases (through a practical case) in the database marketing context was observed and tested. For the practical application work a real database about a customer loyalty card from a Portuguese oil company was used. The results achieved demonstrated the success of the proposed approach in two ways: on one hand, it was possible to formalize and follow the whole knowledge discovery in databases process; on the other hand, it was possible to perceive a methodology for a concrete domain supported by ontologies (support of the decision in the selection of methods and tasks) and in the knowledge discovery in databases.Fundação para a Ciência e a Tecnologia (FCT) - SFRH/BD/36541/200