New Approach for Market Intelligence Using Artificial and Computational Intelligence

Small and medium sized retailers are central to the private sector and a vital contributor to economic growth, but often they face enormous challenges in unleashing their full potential. Financial pitfalls, lack of adequate access to markets, and difficulties in exploiting technology have prevented them from achieving optimal productivity. Market Intelligence (MI) is the knowledge extracted from numerous internal and external data sources, aimed at providing a holistic view of the state of the market and influence marketing related decision-making processes in real-time. A related, burgeoning phenomenon and crucial topic in the field of marketing is Artificial Intelligence (AI) that entails fundamental changes to the skillssets marketers require. A vast amount of knowledge is stored in retailers’ point-of-sales databases. The format of this data often makes the knowledge they store hard to access and identify. As a powerful AI technique, Association Rules Mining helps to identify frequently associated patterns stored in large databases to predict customers’ shopping journeys. Consequently, the method has emerged as the key driver of cross-selling and upselling in the retail industry. At the core of this approach is the Market Basket Analysis that captures knowledge from heterogeneous customer shopping patterns and examines the effects of marketing initiatives. Apriori, that enumerates frequent itemsets purchased together (as market baskets), is the central algorithm in the analysis process. Problems occur, as Apriori lacks computational speed and has weaknesses in providing intelligent decision support. With the growth of simultaneous database scans, the computation cost increases and results in dramatically decreasing performance. Moreover, there are shortages in decision support, especially in the methods of finding rarely occurring events and identifying the brand trending popularity before it peaks. As the objective of this research is to find intelligent ways to assist small and medium sized retailers grow with MI strategy, we demonstrate the effects of AI, with algorithms in data preprocessing, market segmentation, and finding market trends. We show with a sales database of a small, local retailer how our Åbo algorithm increases mining performance and intelligence, as well as how it helps to extract valuable marketing insights to assess demand dynamics and product popularity trends. We also show how this results in commercial advantage and tangible return on investment. Additionally, an enhanced normal distribution method assists data pre-processing and helps to explore different types of potential anomalies.Små och medelstora detaljhandlare är centrala aktörer i den privata sektorn och bidrar starkt till den ekonomiska tillväxten, men de möter ofta enorma utmaningar i att uppnå sin fulla potential. Finansiella svårigheter, brist på marknadstillträde och svårigheter att utnyttja teknologi har ofta hindrat dem från att nå optimal produktivitet. Marknadsintelligens (MI) består av kunskap som samlats in från olika interna externa källor av data och som syftar till att erbjuda en helhetssyn av marknadsläget samt möjliggöra beslutsfattande i realtid. Ett relaterat och växande fenomen, samt ett viktigt tema inom marknadsföring är artificiell intelligens (AI) som ställer nya krav på marknadsförarnas färdigheter. Enorma mängder kunskap finns sparade i databaser av transaktioner samlade från detaljhandlarnas försäljningsplatser. Ändå är formatet på dessa data ofta sådant att det inte är lätt att tillgå och utnyttja kunskapen. Som AI-verktyg erbjuder affinitetsanalys en effektiv teknik för att identifiera upprepade mönster som statistiska associationer i data lagrade i stora försäljningsdatabaser. De hittade mönstren kan sedan utnyttjas som regler som förutser kundernas köpbeteende. I detaljhandel har affinitetsanalys blivit en nyckelfaktor bakom kors- och uppförsäljning. Som den centrala metoden i denna process fungerar marknadskorgsanalys som fångar upp kunskap från de heterogena köpbeteendena i data och hjälper till att utreda hur effektiva marknadsföringsplaner är. Apriori, som räknar upp de vanligt förekommande produktkombinationerna som köps tillsammans (marknadskorgen), är den centrala algoritmen i analysprocessen. Trots detta har Apriori brister som algoritm gällande låg beräkningshastighet och svag intelligens. När antalet parallella databassökningar stiger, ökar också beräkningskostnaden, vilket har negativa effekter på prestanda. Dessutom finns det brister i beslutstödet, speciellt gällande metoder att hitta sällan förekommande produktkombinationer, och i att identifiera ökande popularitet av varumärken från trenddata och utnyttja det innan det når sin höjdpunkt. Eftersom målet för denna forskning är att hjälpa små och medelstora detaljhandlare att växa med hjälp av MI-strategier, demonstreras effekter av AI med hjälp av algoritmer i förberedelsen av data, marknadssegmentering och trendanalys. Med hjälp av försäljningsdata från en liten, lokal detaljhandlare visar vi hur Åbo-algoritmen ökar prestanda och intelligens i datautvinningsprocessen och hjälper till att avslöja värdefulla insikter för marknadsföring, framför allt gällande dynamiken i efterfrågan och trender i populariteten av produkterna. Ytterligare visas hur detta resulterar i kommersiella fördelar och konkret avkastning på investering. Dessutom hjälper den utvidgade normalfördelningsmetoden i förberedelsen av data och med att hitta olika slags anomalier

Enhancing Retail Strategies through Apriori, ECLAT& FP Growth Algorithms in Market Basket Analysis

"Market basket analysis" is a method employed in data mining to discover items that are commonly bought together by customers in a retail store. It is a crucial tool for retailers to understand consumer purchasing behavior and to improve their sales and marketing strategies. In this research paper, we present a comprehensive study on market basket analysis using three popular algorithms: Apriori, ECLAT, and FPGrowth. The paper begins with a brief synopsis of market basket analysis and the techniques adopted for itemset mining. We then introduce the dataset used in this study, which consists of real-life transaction data collected from a retail store. Next, we perform a thorough evaluation of the Apriori, ECLAT, and FPGrowth algorithms in terms of their computational time and the quality of the association rules generated. The results show that the FPGrowth algorithm is the fastest of the three algorithms, while the Apriori algorithm generates the most comprehensive and high-quality association rules.In addition, we also present a comparison of the performance of these algorithms that involve different assessment criteria like support, confidence, and lift. Our study highlights the importance of selecting the appropriate algorithm for market basket analysis depending on the specific requirements and constraints of the task. The paper concludes with an analysis on the limitations and future directions of research in this area. Overall, our study provides insights into the strengths and weaknesses of the Apriori, ECLAT, and FPGrowth algorithms and functions as a valuable resource for professionals and researchers in the field of market basket analysis

Application Areas of Data Mining in Indian Retail Banking Sector

Banking systems collect huge amounts of data on day to day basis be it customer information transaction details risk profiles credit card details credit limit and collateral details compliance and Anti Money Laundering AML related information trade finance data SWIFT and telex messages Thousands of decisions are taken in a bank daily These decisions include credit decisions default decisions relationship start up investment decisions AML and Illegal financing related One needs to depend on various reports and drill down tools provided by the banking systems to arrive at these critical decisions But this is a manual process and is error prone and time consuming due to large volume of transactional and historical data Interesting patterns and knowledge can be mined from this huge volume of data that in turn can be used for this decision making process This article explores and reviews various data mining techniques that can be applied in banking areas It provides an overview of data mining techniques and procedures It also provides an insight into how these techniques can be used in banking areas to make the decision making process easier and productiv

The Ubiquity of Large Graphs and Surprising Challenges of Graph Processing: Extended Survey

Graph processing is becoming increasingly prevalent across many application domains. In spite of this prevalence, there is little research about how graphs are actually used in practice. We performed an extensive study that consisted of an online survey of 89 users, a review of the mailing lists, source repositories, and whitepapers of a large suite of graph software products, and in-person interviews with 6 users and 2 developers of these products. Our online survey aimed at understanding: (i) the types of graphs users have; (ii) the graph computations users run; (iii) the types of graph software users use; and (iv) the major challenges users face when processing their graphs. We describe the participants' responses to our questions highlighting common patterns and challenges. Based on our interviews and survey of the rest of our sources, we were able to answer some new questions that were raised by participants' responses to our online survey and understand the specific applications that use graph data and software. Our study revealed surprising facts about graph processing in practice. In particular, real-world graphs represent a very diverse range of entities and are often very large, scalability and visualization are undeniably the most pressing challenges faced by participants, and data integration, recommendations, and fraud detection are very popular applications supported by existing graph software. We hope these findings can guide future research

RESEARCH ISSUES CONCERNING ALGORITHMS USED FOR OPTIMIZING THE DATA MINING PROCESS

In this paper, we depict some of the most widely used data mining algorithms that have an overwhelming utility and influence in the research community. A data mining algorithm can be regarded as a tool that creates a data mining model. After analyzing a set of data, an algorithm searches for specific trends and patterns, then defines the parameters of the mining model based on the results of this analysis. The above defined parameters play a significant role in identifying and extracting actionable patterns and detailed statistics. The most important algorithms within this research refer to topics like clustering, classification, association analysis, statistical learning, link mining. In the following, after a brief description of each algorithm, we analyze its application potential and research issues concerning the optimization of the data mining process. After the presentation of the data mining algorithms, we will depict the most important data mining algorithms included in Microsoft and Oracle software products, useful suggestions and criteria in choosing the most recommended algorithm for solving a mentioned task, advantages offered by these software products.data mining optimization, data mining algorithms, software solutions

Data Mining

Recently data mining has become more popular in the information industry. It is due to the availability of huge amounts of data. Industry needs turning such data into useful information and knowledge. This information and knowledge can be used in many applications ranging from business management, production control, and market analysis, to engineering design and science exploration. Database and information technology have been evolving systematically from primitive file processing systems to sophisticated and powerful databases systems. The research and development in database systems has led to the development of relational database systems, data modeling tools, and indexing and data organization techniques. In relational database systems data are stored in relational tables. In addition, users can get convenient and flexible access to data through query languages, optimized query processing, user interfaces and transaction management and optimized methods for On-Line Transaction Processing (OLTP). The abundant data, which needs powerful data analysis tools, has been described as a data rich but information poor situation. The fast-growing, tremendous amount of data, collected and stored in large and numerous databases. Humans can not analyze these large amounts of data. So we need powerful tools to analyze this large amount of data. As a result, data collected in large databases become data tombs. These are data archives that are seldom visited. So, important decisions are often not made based on the information-rich data stored in databases rather based on a decision maker's intuition. This is because the decision maker does not have the tools to extract the valuable knowledge embedded in the vast amounts of data. Data mining tools which perform data analysis may uncover important data patterns, contributing greatly to business strategies, knowledge bases, and scientific and medical research. So data mining tools will turn data tombs into golden nuggets of knowledge

Inter-organizational information sharing of customer data in retail

Thesis (S.M.)--Massachusetts Institute of Technology, Sloan School of Management, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 51-53).As massive online retailers are putting increasing pressure on the traditional brick-and-mortar retailers, new ways to compete for customers is needed. Identifying customers' behavior and understanding their needs could be a crucial way for firms to improve their competitive advantage. One way to better understand customers could be for retailers to collaborate and aggregate their customer data in order to gain more holistic understandings of customers. This thesis examines on a high-level how customer data is used in retail today and what new value can be generated by collaborating and aggregating customer data, both for businesses as well as for consumers. It then goes on to identify structures of customer data sharing that we observe today and what the underlying factors are for each structure. By projecting how these factors will evolve over time, different scenarios are drawn up of what we might expect to see in the retail industry in the future in terms of customer data sharing. The implications of those projections could possibly open up for new business opportunities, both for entrepreneurs as well as incumbent retailers.by John C.F. Tengberg.S.M

A Constraint Guided Progressive Sequential Mining Waterfall Model for CRM

CRM has been realized as a core for the growth of any enterprise. This requires both the customer satisfaction and fulfillment of customer requirement, which can only be achieved by analyzing consumer behaviors. The data mining has become an effective tool since often the organizations have large databases of information on customers. However, the traditional data mining techniques have no relevant mechanism to provide guidance for business understanding, model selection and dynamic changes made in the databases. This article helps in understanding and maintaining the requirement of continuous data mining process for CRM in dynamic environment. A novel integrative model, Constraint Guided Progressive SequentialMiningWaterfall (CGPSMW) for knowledge discovery process is proposed. The key performance factors that include management of marketing, sales, knowledge, technology among others those are required for the successful implementation of CRM. We have studied how the sequential pattern mining performed on progressive databases instead of static databases in conjunction with these CRM performance indicators can result in highly efficient and effective useful patterns. This would further help in classification of customers which any enterprise should focus on to achieve its growth and benefit. An organization has limited number of resources that it can only use for valuable customers to reap the fruits of CRM. The different steps of the proposed CGP-SMW model give a detailed elaboration how to keep focus on these customers in dynamic scenarios

Generic data warehousing for consumer electronics retail industry

The dynamic and highly competitive nature of the consumer electronics retail industry means that businesses in this industry are experiencing different decision making challenges in relation to pricing, inventory control, consumer satisfaction and product offerings. To overcome the challenges facing retailers and create opportunities, we propose a generic data warehousing solution which can be applied to a wide range of consumer electronics retailers with a minimum configuration. The solution includes a dimensional data model, a template SQL script, a high level architectural descriptions, ETL tool developed using C#, a set of APIs, and data access tools. It has been successfully applied by ASK Outlets Ltd UK resulting in improved productivity and enhanced sales growth