New Approach for Market Intelligence Using Artificial and Computational Intelligence

Small and medium sized retailers are central to the private sector and a vital contributor to economic growth, but often they face enormous challenges in unleashing their full potential. Financial pitfalls, lack of adequate access to markets, and difficulties in exploiting technology have prevented them from achieving optimal productivity. Market Intelligence (MI) is the knowledge extracted from numerous internal and external data sources, aimed at providing a holistic view of the state of the market and influence marketing related decision-making processes in real-time. A related, burgeoning phenomenon and crucial topic in the field of marketing is Artificial Intelligence (AI) that entails fundamental changes to the skillssets marketers require. A vast amount of knowledge is stored in retailers’ point-of-sales databases. The format of this data often makes the knowledge they store hard to access and identify. As a powerful AI technique, Association Rules Mining helps to identify frequently associated patterns stored in large databases to predict customers’ shopping journeys. Consequently, the method has emerged as the key driver of cross-selling and upselling in the retail industry. At the core of this approach is the Market Basket Analysis that captures knowledge from heterogeneous customer shopping patterns and examines the effects of marketing initiatives. Apriori, that enumerates frequent itemsets purchased together (as market baskets), is the central algorithm in the analysis process. Problems occur, as Apriori lacks computational speed and has weaknesses in providing intelligent decision support. With the growth of simultaneous database scans, the computation cost increases and results in dramatically decreasing performance. Moreover, there are shortages in decision support, especially in the methods of finding rarely occurring events and identifying the brand trending popularity before it peaks. As the objective of this research is to find intelligent ways to assist small and medium sized retailers grow with MI strategy, we demonstrate the effects of AI, with algorithms in data preprocessing, market segmentation, and finding market trends. We show with a sales database of a small, local retailer how our Åbo algorithm increases mining performance and intelligence, as well as how it helps to extract valuable marketing insights to assess demand dynamics and product popularity trends. We also show how this results in commercial advantage and tangible return on investment. Additionally, an enhanced normal distribution method assists data pre-processing and helps to explore different types of potential anomalies.Små och medelstora detaljhandlare är centrala aktörer i den privata sektorn och bidrar starkt till den ekonomiska tillväxten, men de möter ofta enorma utmaningar i att uppnå sin fulla potential. Finansiella svårigheter, brist på marknadstillträde och svårigheter att utnyttja teknologi har ofta hindrat dem från att nå optimal produktivitet. Marknadsintelligens (MI) består av kunskap som samlats in från olika interna externa källor av data och som syftar till att erbjuda en helhetssyn av marknadsläget samt möjliggöra beslutsfattande i realtid. Ett relaterat och växande fenomen, samt ett viktigt tema inom marknadsföring är artificiell intelligens (AI) som ställer nya krav på marknadsförarnas färdigheter. Enorma mängder kunskap finns sparade i databaser av transaktioner samlade från detaljhandlarnas försäljningsplatser. Ändå är formatet på dessa data ofta sådant att det inte är lätt att tillgå och utnyttja kunskapen. Som AI-verktyg erbjuder affinitetsanalys en effektiv teknik för att identifiera upprepade mönster som statistiska associationer i data lagrade i stora försäljningsdatabaser. De hittade mönstren kan sedan utnyttjas som regler som förutser kundernas köpbeteende. I detaljhandel har affinitetsanalys blivit en nyckelfaktor bakom kors- och uppförsäljning. Som den centrala metoden i denna process fungerar marknadskorgsanalys som fångar upp kunskap från de heterogena köpbeteendena i data och hjälper till att utreda hur effektiva marknadsföringsplaner är. Apriori, som räknar upp de vanligt förekommande produktkombinationerna som köps tillsammans (marknadskorgen), är den centrala algoritmen i analysprocessen. Trots detta har Apriori brister som algoritm gällande låg beräkningshastighet och svag intelligens. När antalet parallella databassökningar stiger, ökar också beräkningskostnaden, vilket har negativa effekter på prestanda. Dessutom finns det brister i beslutstödet, speciellt gällande metoder att hitta sällan förekommande produktkombinationer, och i att identifiera ökande popularitet av varumärken från trenddata och utnyttja det innan det når sin höjdpunkt. Eftersom målet för denna forskning är att hjälpa små och medelstora detaljhandlare att växa med hjälp av MI-strategier, demonstreras effekter av AI med hjälp av algoritmer i förberedelsen av data, marknadssegmentering och trendanalys. Med hjälp av försäljningsdata från en liten, lokal detaljhandlare visar vi hur Åbo-algoritmen ökar prestanda och intelligens i datautvinningsprocessen och hjälper till att avslöja värdefulla insikter för marknadsföring, framför allt gällande dynamiken i efterfrågan och trender i populariteten av produkterna. Ytterligare visas hur detta resulterar i kommersiella fördelar och konkret avkastning på investering. Dessutom hjälper den utvidgade normalfördelningsmetoden i förberedelsen av data och med att hitta olika slags anomalier

Exploring fish purchasing behaviour using data analytics

Nas últimas décadas têm ocorrido mudanças significativas no setor do retalho resultantes da globalização, do aumento de competitividade e da transformação do comportamento de compra do consumidor. Esta mudança de paradigma também se aplica ao setor do peixe fresco, que tem sido alvo do interesse de investigadores internacionais por razões políticas e económicas. Tendo em conta este ambiente competitivo, que valoriza a qualidade e o serviço fornecido ao consumidor assente em custos aceitáveis, é necessário a adoção de estratégias focadas no cliente. Esta dissertação está integrada no projeto ValorMar, que nasceu do compromisso de um conjunto alargado de entidades, desde empresas até centros de investigação posicionados pela relevância da economia marítima na cadeia de valor do pescado. Assim, esta dissertação irá tentar compreender relações que se revelem críticas para a tomada de decisão dos consumidores no momento de compra de peixe fresco. Para tal, irão ser usados dados transacionais e técnicas de data mining adequadas ao problema.A metodologia proposta por esta dissertação tem como objetivo não só a identificação de clientes recorrendo a técnicas de segmentação, mas também uma análise ao carrinho de compras de um cliente de peixe fresco. Estas análises aos dados irão mostrar que a extração de conhecimento de grandes bases de dados permite melhorar as decisões estratégicas das empresas e a sua relação com os clientes.In the last decades there have been significant changes in the retail sector resulting from globalization, the increased competitiveness and transformation on consumer's purchasing behaviour. This paradigm shift also applies to the fish sector, that has been capturing the interest of researchers internationally for political and economic reasons. Taking this competitive environment into account, which values the quality and the service given to the customer based on acceptable costs, it is necessary to adopt customer focused strategies.This thesis is integrated in the ValorMar's project, which was born from the commitment of a broad spectrum of entities, from companies to research centers, positioned by the relevance of the sea economy in the fishery value chain. Thus, this dissertation will try to understand critical relations for the decision making of customers when buying fresh fish.For this, transactional data and data mining techniques appropriate to the problem will be used.The methodology proposed by this thesis aims not only to identify customers using clustering techniques, but also to analyze the market basket of a fresh fish customer. These data analyzis will show that the knowledge extraction from large databases allows to improve the companies strategic decisions and their relationship with customers

LC an effective classification based association rule mining algorithm

Classification using association rules is a research field in data mining that primarily uses association rule discovery techniques in classification benchmarks. It has been confirmed by many research studies in the literature that classification using association tends to generate more predictive classification systems than traditional classification data mining techniques like probabilistic, statistical and decision tree. In this thesis, we introduce a novel data mining algorithm based on classification using association called “Looking at the Class” (LC), which can be used in for mining a range of classification data sets. Unlike known algorithms in classification using the association approach such as Classification based on Association rule (CBA) system and Classification based on Predictive Association (CPAR) system, which merge disjoint items in the rule learning step without anticipating the class label similarity, the proposed algorithm merges only items with identical class labels. This saves too many unnecessary items combining during the rule learning step, and consequently results in large saving in computational time and memory. Furthermore, the LC algorithm uses a novel prediction procedure that employs multiple rules to make the prediction decision instead of a single rule. The proposed algorithm has been evaluated thoroughly on real world security data sets collected using an automated tool developed at Huddersfield University. The security application which we have considered in this thesis is about categorizing websites based on their features to legitimate or fake which is a typical binary classification problem. Also, experimental results on a number of UCI data sets have been conducted and the measures used for evaluation is the classification accuracy, memory usage, and others. The results show that LC algorithm outperformed traditional classification algorithms such as C4.5, PART and Naïve Bayes as well as known classification based association algorithms like CBA with respect to classification accuracy, memory usage, and execution time on most data sets we consider

Case Teknos Group Oy Paint Store Transaction Data

Companies operating in challenging business environments, characterized by the proliferation of disruptive technologies and intensifying competition, are obliged to re-evaluate their strategic approach. This has become the norm in the retail industry and traditional brick-and-mortar stores. Particularly local market players with scarce resources are looking into alternative solutions to delivering a unique customer experience with the intention to preserve their profitability. Customer experience has been an integral topic within academic research for decades, and has also substantiated its value in pragmatic contexts. Recent developments in this field have triggered the constitution of customer experience management functions, which aim to adopt a holistic approach to the customer experience. This enforces a quantitative perspective highlighting the role of customer transaction data. Association analysis is one of the most well-known methodology used to detect underlying patterns hidden in large transaction data sets. It uses machine learning techniques to firstly identify frequently purchased product combinations and secondly, to discover concealed associations among the products. The association rules derived and evaluated during the process can potentially reveal implicit, yet interesting customer insight, which may translate into actionable implications. The practical consequences in the framework of this study are referred to as sales increasing strategies, namely targeted marketing, cross-selling and space management. This thesis uses Python programming language in Anaconda’s Jupyter Notebook environment to perform association analysis on customer transaction data provided by the case company. The Apriori algorithm is applied to constitute the frequent itemsets and generate association rules between these itemsets. The interestingness and actionability of the rules will be evaluated based on various scoring measures computed for each rule. The outcomes of this study contribute to finding interesting customer insight and actionable recommendations for the case company to support their success in demanding market conditions. Furthermore, this research describes and discusses the relative success factors from the theoretical point of view and demonstrates the process of association rule mining when applied to customer transaction data

Building a Course Recommender System for The College of Wooster

The goal of this project is to investigate the approaches for building recommender systems and to apply them to implement a course recommender system for the College of Wooster. There are three main objectives of this project. The first is to understand the mathematics and computer science aspects behind it. The mathematic concepts built into this project include probability, statistics and linear algebra. The final product is consist of two components: a collection of Python scripts containing the implementation code of the course recommender system, and a simple user interface allowing people to use the recommender system without typing commands. The second goal is to analyze the pros and cons of different approaches by comparing their performance on the same training data set which have information about students and courses at the college in the last seven years. The final goal is to apply the best model to build the course recommender system that can provide helpful and personalized course recommendations to students

A Survey of Sequential Pattern Based E-Commerce Recommendation Systems

E-commerce recommendation systems usually deal with massive customer sequential databases, such as historical purchase or click stream sequences. Recommendation systems’ accuracy can be improved if complex sequential patterns of user purchase behavior are learned by integrating sequential patterns of customer clicks and/or purchases into the user–item rating matrix input of collaborative filtering. This review focuses on algorithms of existing E-commerce recommendation systems that are sequential pattern-based. It provides a comprehensive and comparative performance analysis of these systems, exposing their methodologies, achievements, limitations, and potential for solving more important problems in this domain. The review shows that integrating sequential pattern mining of historical purchase and/or click sequences into a user–item matrix for collaborative filtering can (i) improve recommendation accuracy, (ii) reduce user–item rating data sparsity, (iii) increase the novelty rate of recommendations, and (iv) improve the scalability of recommendation systems

A Taxonomy of Sequential Patterns Based Recommendation Systems

With remarkable expansion of information through the internet, users prefer to receive the exact information they need through some suggestions to save their time and money. Thus, recommendation systems have become the heart of business strategies of E-commerce as they can increase sales and revenue as well as customer loyalty. Recommendation systems techniques provide suggestions for items/products to be purchased, rented or used by a user. The most common type of recommendation system technique is Collaborative Filtering (CF), which takes user’s interest in an item (explicit rating) as input in a matrix known as the user-item rating matrix, and produces an output for unknown ratings of users for items from which top N recommended items for target users are defined. E-commerce recommendation systems usually deal with massive customer sequential databases such as historical purchase or click sequences. The time stamp of a click or purchase event is an important attribute of each dataset as the time interval between item purchases may be useful to learn the next items for purchase by users. Sequential Pattern Mining mines frequent or high utility sequential patterns from a sequential database. Recommendation systems accuracy will be improved if complex sequential patterns of user purchase behavior are learned by integrating sequential patterns of customer clicks and/or purchases into the user-item rating matrix input. Thus, integrating collaborative filtering (CF) and sequential pattern mining (SPM) of historical clicks and purchase data can improve recommendation accuracy, diversity and quality and this survey focuses on review of existing recommendation systems that are sequential pattern based exposing their methodologies, achievements, limitations, and potentials for solving more problems in this domain. This thesis provides a comprehensive and comparative study of the existing Sequential Pattern-based E-commerce recommendation systems (SP-based E-commerce RS) such as ChoRec05, ChenRec09, HuangRec09, LiuRec09, ChoiRec12, Hybrid Model RecSys16, Product RecSys16, SainiRec17, HPCRec18 and HSPCRec19. Thesis shows that integrating sequential patterns mining (SPM) of historical purchase and/or click sequences into user-item matrix for collaborative filtering (CF) (i) Improved recommendation accuracy (ii) Reduced limiting user-item rating data Sparsity (iii) Increased Novelty Rate of the recommendations and (iv) Improved Scalability of the recommendation system. Thus, the importance of sequential patterns of customer behavior in improving the quality of recommendation systems for the application domain of E-commerce is accentuated through this survey by having a comparative performance analysis of the surveyed systems