An academic review: applications of data mining techniques in finance industry

With the development of Internet techniques, data volumes are doubling every two years, faster than predicted by Moore’s Law. Big Data Analytics becomes particularly important for enterprise business. Modern computational technologies will provide effective tools to help understand hugely accumulated data and leverage this information to get insights into the finance industry. In order to get actionable insights into the business, data has become most valuable asset of financial organisations, as there are no physical products in finance industry to manufacture. This is where data mining techniques come to their rescue by allowing access to the right information at the right time. These techniques are used by the finance industry in various areas such as fraud detection, intelligent forecasting, credit rating, loan management, customer profiling, money laundering, marketing and prediction of price movements to name a few. This work aims to survey the research on data mining techniques applied to the finance industry from 2010 to 2015.The review finds that Stock prediction and Credit rating have received most attention of researchers, compared to Loan prediction, Money Laundering and Time Series prediction. Due to the dynamics, uncertainty and variety of data, nonlinear mapping techniques have been deeply studied than linear techniques. Also it has been proved that hybrid methods are more accurate in prediction, closely followed by Neural Network technique. This survey could provide a clue of applications of data mining techniques for finance industry, and a summary of methodologies for researchers in this area. Especially, it could provide a good vision of Data Mining Techniques in computational finance for beginners who want to work in the field of computational finance

Data-driven Technology Foresight: Text Analysis of Emerging Technologies

학위논문 (박사)-- 서울대학교 대학원 : 공과대학 산업·조선공학부, 2018. 2. 박용태.This dissertation argues for new directions in the field of technology foresight. Technology foresight was formulated on the basis of qualitative and participatory research. Initially, most foresight activities were triggered by the prospect of a handful number of experts, but recent studies highlight theoretical paradigm shifts toward a more comprehensive and data-driven approach to creating shared insights on the future of emerging technologies. Much of the research up to now, however, has been descriptive in nature, and a definite method of realizing the notion has not yet been addressed in the existing literature to a large extent. To this end, we have attempted to formalize the concept of data-driven technology foresight by incorporating unconventional data sources – future-oriented web data, Wikipedia data, and scientific publication data – and different analytical tools – Latent Semantic Analysis, IdeaGraph, and Morphological Analysis. Four distinct foresight frameworks were proposed for the proactive management process of emerging technologies: impact identification, impact analysis, plan development, and technology ideation. The study was guided by the following research questions: (1) what kinds of data sources are available on the web and which of those are considered useful in foresight studies? (2) Where could we incorporate these data sources and which techniques are most suitable for the given purposes? (3) Which foresight-related fields would particularly benefit from applying a data-driven approach and what are the positive effects? The proposals outlined should be considered exploratory and open-ended. It is designed to determine the nature of the problem, rather than to offer definitive and conclusive answers. Nevertheless, the proposed scheme may well provide not just a rationale but a theoretical grounding for this newly introduced notion. This dissertation is expected to yield a foothold for the readers to better comprehend and act on this new shift in the field of technology foresight.Chapter 1 Introduction 1 1.1 Emergence of Technology Foresight 1 1.2 Towards a Data-driven Technology Foresight 3 1.3 Problem Statement 6 1.4 Dissertation Overview 8 Chapter 2 Data Sources and Methodologies 15 2.1 Data Sources 15 2.1.1 Future-oriented Web Data 15 2.1.2 Wikipedia Data 17 2.1.3 Scientific Publication Data 19 2.2 Methodologies 21 2.2.1 Latent Semantic Analysis (LSA) 21 2.2.2 IdeaGraph 25 2.2.3 Morphological Analysis (MA) 29 Chapter 3 Foresight for Impact Identification 31 3.1 Introduction 32 3.2 Emerging Technology and its Social Impacts 36 3.2.1 Distinctive Nature of Emerging Technology 36 3.2.2 Technology Assessment 39 3.3 LSA for Constructing Scenarios 43 3.4 Research Framework 44 3.4.1 Step 1: Data Collection 46 3.4.2 Step 2: Scenario Development 49 3.4.2.1 Pre-LSA: Preprocessing Future-oriented Web Data 49 3.4.2.2 LSA: Applying Latent Semantic Analysis 52 3.4.2.3 Post-LSA: Constructing Scenarios 54 3.5 Illustrative Case Study: Drone Technology 55 3.6 Discussion 65 3.6.1 Categorization of Social Impacts 65 3.6.2 Comparative Analysis 72 3.6.3 Implication for Theory, Practice, and Policy 74 3.7 Conclusion 76 Chapter 4 Foresight for Impact Analysis 79 4.1 Introduction 80 4.2 Uncertainty and Complexity 82 4.3 Data-driven Foresight Process 84 4.4 Scenario Building Beyond the Obvious 86 4.4.1 Capturing Plausibility using LSA 90 4.4.2 Capturing Creativity using IdeaGraph 92 4.5 Research Framework 93 4.5.1 Step 1. Pre-Analysis: Data Preparation 94 4.5.1.1 Target Technology Selection 94 4.5.1.2 Data Acquisition 95 4.5.1.3 Data Preprocessing 95 4.5.2 Step 2. Text Analysis: Scenario Building 96 4.5.2.1 General Glimpse using Overt Structures 96 4.5.2.2 Hidden Details using Latent Structures 98 4.5.3 Step 3. Post-Analysis: Analytical Interpretation 101 4.5.3.1 Individual Impact Scenario 101 4.5.3.2 Overall Latent Impacts 101 4.6 Illustrative Case Study: 3D Printing Technology 102 4.7 Discussion 110 4.7.1 Scenarios Beyond the Obvious 110 4.7.2 Comparative Analysis 113 4.8 Conclusion 115 Chapter 5 Foresight for Plan Development 117 5.1 Introduction 118 5.2 Theoretical Paradigm Shift 120 5.2.1 Technology-focused vs. Society-focused 120 5.2.2 Co-evolution of Technology and Society 122 5.2.3 Responsible Development 125 5.3 Methodological Paradigm Shift 127 5.3.1 Participatory Approach 127 5.3.2 Data-driven Approach 129 5.4 Rationale for using LSA 131 5.5 Research Framework 132 5.5.1 Step 1. Envisioning Social Issues 133 5.5.1.1 Collection of Future-oriented Web Data 133 5.5.1.2 Construction of Impact Scenarios 135 5.5.1.3 Conceptualization of Impact Scenarios 137 5.5.2 Step 2. Deriving Technical Solutions 138 5.5.2.1 Collection of Scientific Publication Data 138 5.5.2.2 Construction of Solution Concepts 139 5.6 Illustrative Case Study: Autonomous Vehicle 140 5.7 Discussion 149 5.7.1 Comparative Analysis 149 5.7.2 Major Strengths in Envisioning Social Impacts 152 5.7.3 Major Strengths in Overviewing Solutions 154 5.8 Conclusion 156 Chapter 6 Foresight for Technology Ideation 158 6.1 Introduction 159 6.2 Related Studies 161 6.2.1 Generating Creative Ideas 161 6.2.2 Data-driven Morphological Analysis 163 6.3 Technology Foresight using Wikipedia 165 6.3.1 Wikipedia as a Good Remedy 165 6.3.2 Preliminaries: How to Apply Wikipedia 168 6.4 Research Framework 173 6.4.1 Basic Model 174 6.4.2 Extended Model 175 6.4.2.1 Phase 1: Preliminary Phase 177 6.4.2.2 Phase 2: Dimension Development Phase 177 6.4.2.3 Phase 3: Value Development Phase 179 6.4.2.4 Phase 4: Sub-dimension Development Phase 182 6.5 Illustrative Case Study: Drone Technology 183 6.5.1 Basic Model 183 6.5.2 Extended Model 185 6.6 Comparative Analysis 193 6.6.1 Experimental Setup 193 6.6.2 Comparison of Results 195 6.7 Intrinsic Limitations of Applying Wikipedia 199 6.8 Conclusion 201 Chapter 7 Concluding Remarks 203 Bibliography 211 Appendix 236 Appendix A Result of overt and latent structures of each impact scenario 236 Appendix B Result of Wikipedia-based morphological matrix (basic model) 240 Appendix C Result of Wikipedia-based morphological matrix using superordinate seed terms (extended model) 241 Appendix D Result of Wikipedia-based morphological matrix after applying subordinate value seed terms (extended model) 243 Appendix E Result of Wikipedia-based morphological matrix after developing sub-dimensions (extended model) 247Docto

Machine learning methods for sign language recognition: a critical review and analysis.

Sign language is an essential tool to bridge the communication gap between normal and hearing-impaired people. However, the diversity of over 7000 present-day sign languages with variability in motion position, hand shape, and position of body parts making automatic sign language recognition (ASLR) a complex system. In order to overcome such complexity, researchers are investigating better ways of developing ASLR systems to seek intelligent solutions and have demonstrated remarkable success. This paper aims to analyse the research published on intelligent systems in sign language recognition over the past two decades. A total of 649 publications related to decision support and intelligent systems on sign language recognition (SLR) are extracted from the Scopus database and analysed. The extracted publications are analysed using bibliometric VOSViewer software to (1) obtain the publications temporal and regional distributions, (2) create the cooperation networks between affiliations and authors and identify productive institutions in this context. Moreover, reviews of techniques for vision-based sign language recognition are presented. Various features extraction and classification techniques used in SLR to achieve good results are discussed. The literature review presented in this paper shows the importance of incorporating intelligent solutions into the sign language recognition systems and reveals that perfect intelligent systems for sign language recognition are still an open problem. Overall, it is expected that this study will facilitate knowledge accumulation and creation of intelligent-based SLR and provide readers, researchers, and practitioners a roadmap to guide future direction

Intelligent Systems

This book is dedicated to intelligent systems of broad-spectrum application, such as personal and social biosafety or use of intelligent sensory micro-nanosystems such as "e-nose", "e-tongue" and "e-eye". In addition to that, effective acquiring information, knowledge management and improved knowledge transfer in any media, as well as modeling its information content using meta-and hyper heuristics and semantic reasoning all benefit from the systems covered in this book. Intelligent systems can also be applied in education and generating the intelligent distributed eLearning architecture, as well as in a large number of technical fields, such as industrial design, manufacturing and utilization, e.g., in precision agriculture, cartography, electric power distribution systems, intelligent building management systems, drilling operations etc. Furthermore, decision making using fuzzy logic models, computational recognition of comprehension uncertainty and the joint synthesis of goals and means of intelligent behavior biosystems, as well as diagnostic and human support in the healthcare environment have also been made easier

A finder and representation system for knowledge carriers based on granular computing

In one of his publications Aristotle states ”All human beings by their nature desire to know” [Kraut 1991]. This desire is initiated the day we are born and accompanies us for the rest of our life. While at a young age our parents serve as one of the principle sources for knowledge, this changes over the course of time. Technological advances and particularly the introduction of the Internet, have given us new possibilities to share and access knowledge from almost anywhere at any given time. Being able to access and share large collections of written down knowledge is only one part of the equation. Just as important is the internalization of it, which in many cases can prove to be difficult to accomplish. Hence, being able to request assistance from someone who holds the necessary knowledge is of great importance, as it can positively stimulate the internalization procedure. However, digitalization does not only provide a larger pool of knowledge sources to choose from but also more people that can be potentially activated, in a bid to receive personalized assistance with a given problem statement or question. While this is beneficial, it imposes the issue that it is hard to keep track of who knows what. For this task so-called Expert Finder Systems have been introduced, which are designed to identify and suggest the most suited candidates to provide assistance. Throughout this Ph.D. thesis a novel type of Expert Finder System will be introduced that is capable of capturing the knowledge users within a community hold, from explicit and implicit data sources. This is accomplished with the use of granular computing, natural language processing and a set of metrics that have been introduced to measure and compare the suitability of candidates. Furthermore, are the knowledge requirements of a problem statement or question being assessed, in order to ensure that only the most suited candidates are being recommended to provide assistance

New Approach for Market Intelligence Using Artificial and Computational Intelligence

Small and medium sized retailers are central to the private sector and a vital contributor to economic growth, but often they face enormous challenges in unleashing their full potential. Financial pitfalls, lack of adequate access to markets, and difficulties in exploiting technology have prevented them from achieving optimal productivity. Market Intelligence (MI) is the knowledge extracted from numerous internal and external data sources, aimed at providing a holistic view of the state of the market and influence marketing related decision-making processes in real-time. A related, burgeoning phenomenon and crucial topic in the field of marketing is Artificial Intelligence (AI) that entails fundamental changes to the skillssets marketers require. A vast amount of knowledge is stored in retailers’ point-of-sales databases. The format of this data often makes the knowledge they store hard to access and identify. As a powerful AI technique, Association Rules Mining helps to identify frequently associated patterns stored in large databases to predict customers’ shopping journeys. Consequently, the method has emerged as the key driver of cross-selling and upselling in the retail industry. At the core of this approach is the Market Basket Analysis that captures knowledge from heterogeneous customer shopping patterns and examines the effects of marketing initiatives. Apriori, that enumerates frequent itemsets purchased together (as market baskets), is the central algorithm in the analysis process. Problems occur, as Apriori lacks computational speed and has weaknesses in providing intelligent decision support. With the growth of simultaneous database scans, the computation cost increases and results in dramatically decreasing performance. Moreover, there are shortages in decision support, especially in the methods of finding rarely occurring events and identifying the brand trending popularity before it peaks. As the objective of this research is to find intelligent ways to assist small and medium sized retailers grow with MI strategy, we demonstrate the effects of AI, with algorithms in data preprocessing, market segmentation, and finding market trends. We show with a sales database of a small, local retailer how our Åbo algorithm increases mining performance and intelligence, as well as how it helps to extract valuable marketing insights to assess demand dynamics and product popularity trends. We also show how this results in commercial advantage and tangible return on investment. Additionally, an enhanced normal distribution method assists data pre-processing and helps to explore different types of potential anomalies.Små och medelstora detaljhandlare är centrala aktörer i den privata sektorn och bidrar starkt till den ekonomiska tillväxten, men de möter ofta enorma utmaningar i att uppnå sin fulla potential. Finansiella svårigheter, brist på marknadstillträde och svårigheter att utnyttja teknologi har ofta hindrat dem från att nå optimal produktivitet. Marknadsintelligens (MI) består av kunskap som samlats in från olika interna externa källor av data och som syftar till att erbjuda en helhetssyn av marknadsläget samt möjliggöra beslutsfattande i realtid. Ett relaterat och växande fenomen, samt ett viktigt tema inom marknadsföring är artificiell intelligens (AI) som ställer nya krav på marknadsförarnas färdigheter. Enorma mängder kunskap finns sparade i databaser av transaktioner samlade från detaljhandlarnas försäljningsplatser. Ändå är formatet på dessa data ofta sådant att det inte är lätt att tillgå och utnyttja kunskapen. Som AI-verktyg erbjuder affinitetsanalys en effektiv teknik för att identifiera upprepade mönster som statistiska associationer i data lagrade i stora försäljningsdatabaser. De hittade mönstren kan sedan utnyttjas som regler som förutser kundernas köpbeteende. I detaljhandel har affinitetsanalys blivit en nyckelfaktor bakom kors- och uppförsäljning. Som den centrala metoden i denna process fungerar marknadskorgsanalys som fångar upp kunskap från de heterogena köpbeteendena i data och hjälper till att utreda hur effektiva marknadsföringsplaner är. Apriori, som räknar upp de vanligt förekommande produktkombinationerna som köps tillsammans (marknadskorgen), är den centrala algoritmen i analysprocessen. Trots detta har Apriori brister som algoritm gällande låg beräkningshastighet och svag intelligens. När antalet parallella databassökningar stiger, ökar också beräkningskostnaden, vilket har negativa effekter på prestanda. Dessutom finns det brister i beslutstödet, speciellt gällande metoder att hitta sällan förekommande produktkombinationer, och i att identifiera ökande popularitet av varumärken från trenddata och utnyttja det innan det når sin höjdpunkt. Eftersom målet för denna forskning är att hjälpa små och medelstora detaljhandlare att växa med hjälp av MI-strategier, demonstreras effekter av AI med hjälp av algoritmer i förberedelsen av data, marknadssegmentering och trendanalys. Med hjälp av försäljningsdata från en liten, lokal detaljhandlare visar vi hur Åbo-algoritmen ökar prestanda och intelligens i datautvinningsprocessen och hjälper till att avslöja värdefulla insikter för marknadsföring, framför allt gällande dynamiken i efterfrågan och trender i populariteten av produkterna. Ytterligare visas hur detta resulterar i kommersiella fördelar och konkret avkastning på investering. Dessutom hjälper den utvidgade normalfördelningsmetoden i förberedelsen av data och med att hitta olika slags anomalier