An Overview of the Use of Neural Networks for Data Mining Tasks

In the recent years the area of data mining has experienced a considerable demand for technologies that extract knowledge from large and complex data sources. There is a substantial commercial interest as well as research investigations in the area that aim to develop new and improved approaches for extracting information, relationships, and patterns from datasets. Artificial Neural Networks (NN) are popular biologically inspired intelligent methodologies, whose classification, prediction and pattern recognition capabilities have been utilised successfully in many areas, including science, engineering, medicine, business, banking, telecommunication, and many other fields. This paper highlights from a data mining perspective the implementation of NN, using supervised and unsupervised learning, for pattern recognition, classification, prediction and cluster analysis, and focuses the discussion on their usage in bioinformatics and financial data analysis tasks

Shareholder Voting and Directors’ Remuneration Report Legislation: Say on Pay in the U.K. (CRI 2009-004)

This paper investigates shareholder voting in the UK. The Directors’ Remuneration Report (DRR) Regulations of 2002 gave shareholders a mandatory non-binding vote on boardroom pay. First, using data on about 50,000 resolutions over the period 2002 to 2007 we find that less than 10% of shareholders abstain or vote against the mandated Directors’ Remuneration Report (DRR) resolution. Second, investors are more likely to vote against DRR resolutions compared to non-pay resolutions. Third, shareholders are more likely to vote against general executive pay resolutions, such as stock options, long term incentive plans and bonus resolutions compared to non-pay resolutions. Forth, firms with higher CEO pay attract greater voting dissent. Fifth, there is little evidence that CEO pay is lower in firms that previously experienced high levels of shareholder dissent. In addition, there is little evidence that the equity pay-mix, representing better owner-manager alignment, is greater in such firms. Currently, we find limited evidence that, on average, ‘say on pay’ materially alters the subsequent level and design of CEO compensation

A prediction scheme using perceptually important points and dynamic time warping

An algorithmic method for assessing statistically the efficient market hypothesis (EMH) is developed based on two data mining tools, perceptually important points (PIPs) used to dynamically segment price series into subsequences, and dynamic time warping (DTW) used to find similar historical subsequences. Then predictions are made from the mappings of the most similar subsequences, and the prediction error statistic is used for the EMH assessment. The predictions are assessed on simulated price paths composed of stochastic trend and chaotic deterministic time series, and real financial data of 18 world equity markets and the GBP/USD exchange rate. The main results establish that the proposed algorithm can capture the deterministic structure in simulated series, confirm the validity of EMH on the examined equity indices, and indicate that prediction of the exchange rates using PIPs and DTW could beat at cases the prediction of last available price

NEW METHODS FOR MINING SEQUENTIAL AND TIME SERIES DATA

Data mining is the process of extracting knowledge from large amounts of data. It covers a variety of techniques aimed at discovering diverse types of patterns on the basis of the requirements of the domain. These techniques include association rules mining, classification, cluster analysis and outlier detection. The availability of applications that produce massive amounts of spatial, spatio-temporal (ST) and time series data (TSD) is the rationale for developing specialized techniques to excavate such data. In spatial data mining, the spatial co-location rule problem is different from the association rule problem, since there is no natural notion of transactions in spatial datasets that are embedded in continuous geographic space. Therefore, we have proposed an efficient algorithm (GridClique) to mine interesting spatial co-location patterns (maximal cliques). These patterns are used as the raw transactions for an association rule mining technique to discover complex co-location rules. Our proposal includes certain types of complex relationships – especially negative relationships – in the patterns. The relationships can be obtained from only the maximal clique patterns, which have never been used until now. Our approach is applied on a well-known astronomy dataset obtained from the Sloan Digital Sky Survey (SDSS). ST data is continuously collected and made accessible in the public domain. We present an approach to mine and query large ST data with the aim of finding interesting patterns and understanding the underlying process of data generation. An important class of queries is based on the flock pattern. A flock is a large subset of objects moving along paths close to each other for a predefined time. One approach to processing a “flock query” is to map ST data into high-dimensional space and to reduce the query to a sequence of standard range queries that can be answered using a spatial indexing structure; however, the performance of spatial indexing structures rapidly deteriorates in high-dimensional space. This thesis sets out a preprocessing strategy that uses a random projection to reduce the dimensionality of the transformed space. We use probabilistic arguments to prove the accuracy of the projection and to present experimental results that show the possibility of managing the curse of dimensionality in a ST setting by combining random projections with traditional data structures. In time series data mining, we devised a new space-efficient algorithm (SparseDTW) to compute the dynamic time warping (DTW) distance between two time series, which always yields the optimal result. This is in contrast to other approaches which typically sacrifice optimality to attain space efficiency. The main idea behind our approach is to dynamically exploit the existence of similarity and/or correlation between the time series: the more the similarity between the time series, the less space required to compute the DTW between them. Other techniques for speeding up DTW, impose a priori constraints and do not exploit similarity characteristics that may be present in the data. Our experiments demonstrate that SparseDTW outperforms these approaches. We discover an interesting pattern by applying SparseDTW algorithm: “pairs trading” in a large stock-market dataset, of the index daily prices from the Australian stock exchange (ASX) from 1980 to 2002

Much ado about making money: The impact of disclosure, News and Rumors over the Formation of Security Market Prices over Time

This article develops an agent-based model of security market pricing process, capable to capture main stylised facts. It features collective market pricing mechanisms based upon evolving heterogenous expectations that incorporate signals of security issuer fundamental performance over time. Distinctive signaling sources on this performance correspond to institutional mechanisms of information diffusion. These sources differ by duration effect (temporary, persistent, and permanent), confidence, and diffusion degree among investors over space and time. Under full and immediate diffusion and balanced reaction by all the investors, the value content of these sources is expected to be consistently and timely integrated by the market price process, implying efficient pricing. By relaxing these quite heroic conditions, we assess the impact of distinctive information sources over market price dynamics, through financial systemic properties such as market price volatility, exuberance and errancy, as well as market liquidity. Our simulation analysis shows that transient information shocks can have permanent effects through mismatching reactions and self-reinforcing feedbacks, involving mispricing in both value and timing relative to the efficient market price series. This mispricing depends on both the information diffusion process and the ongoing information confidence mood among investors over space and time. We illustrate our results through paradigmatic cases of stochastic news, before generalising them to autocorrelated news. Our results are further corroborated by robustness checks over the parameter space and across several market trading mechanisms