A knowledge discovery approach to urban analysis

Enhancing our knowledge of the complexities of cities in order to empower ourselves to make more informed decisions has always been a challenge for urban research. Recent developments in large-scale computing, together with the new techniques and automated tools for data collection and analysis are opening up promising opportunities for addressing this problem. The main motivation that served as the driving force behind this research is how these developments may contribute to urban data analysis. On this basis, the thesis focuses on urban data analysis in order to search for findings that can enhance our knowledge of urban environments, using the generic process of knowledge discovery using data mining. A knowledge discovery process based on data mining is a fully automated or semi-automated process which involves the application of computational tools and techniques to explore the “previously unknown, and potentially useful information” (Witten & Frank, 2005) hidden in large and often complex and multi-dimensional databases. This information can be obtained in the form of correlations amongst variables, data groupings (classes and clusters) or more complex hypotheses (probabilistic rules of co-occurrence, performance vectors of prediction models etc.). This research targets researchers and practitioners working in the field of urban studies who are interested in quantitative/ computational approaches to urban data analysis and specifically aims to engage the interest of architects, urban designers and planners who do not have a background in statistics or in using data mining methods in their work. Accordingly, the overall aim of the thesis is the development of a knowledge discovery approach to urban analysis; a domain-specific adaptation of the generic process of knowledge discovery using data mining enabling the analyst to discover ‘relational urban knowledge’. ‘Relational urban knowledge’ is a term employed in this thesis to refer to the potentially ‘useful’ and/or ‘valuable’ information patterns and relationships that can be discovered in urban databases by applying data mining algorithms. A knowledge discovery approach to urban analysis through data mining can help us to understand site-specific characteristics of urban environments in a more profound and useful way. On a more specific level, the thesis aims towards ‘knowledge discovery’ in traditional thematic maps published in 2008 by the Istanbul Metropolitan Municipality as a basis of the Master Plan for the Beyoğlu Preservation Area. These thematic maps, which represent urban components, namely buildings, streets, neighbourhoods and their various attributes such as floor space use of the buildings, land price, population density or historical importance, do not really extend our knowledge of Beyoğlu Preservation Area beyond documenting its current state and do not contribute to the interventions presented in the master plan. However it is likely that ‘useful’ and ‘valuable’ information patterns discoverable using data mining algorithms are hidden in them. In accordance with the stated aims, three research questions of the thesis concerns (1) the development of a general process model to adapt the generic process of knowledge discovery using data mining for urban data analysis, (2) the investigation of information patterns and relationships that can be extracted from the traditional thematic maps of the Beyoğlu Preservation Area by further developing and implementing this model and (3) the investigation of how could this ‘relational urban knowledge’ support architects, urban designers or urban planners whilst developing intervention proposals for urban regeneration. A Knowledge Discovery Process Model (KDPM) for urban analysis was developed, as an answer to the the first research question. The KDPM for urban analysis is a domain-specific adaptation of the widely accepted process of knowledge discovery in databases defined by Fayyad, Piatetsky-Shapiro, and Smyth (1996b). The model describes a semi-automated process of database formulation, analysis and evaluation for extracting information patterns and relationships from raw data by combining both GIS and data mining functionalities in a complementary way. The KDPM for urban analysis suggests that GIS functionalities can be used to formulate a database, and GIS and data mining can complement each other in analyzing the database and evaluating the outcomes. The model illustrates that the output of a GIS platform can become the input for a data mining platform and vice versa, resulting in an interlinked analytical process which allows for a more sophisticated analysis of urban data. To investigate the second and third research questions, firstly the KDPM for urban analysis was further developed to construct a GIS database of the Beyoğlu Preservation Area from the thematic maps. Then, three implementations were performed using this GIS database; the Beyoğlu Preservation Area Building Features Database consisting of multiple features attributed to the buildings. In Implementation (1), the KDPM for urban analysis was used to investigate a variety of patterns and relationships that can be extracted from the database using three different data mining methods. In Implementations (2) and (3), the KDPM for urban analysis was implemented to test how the knowledge discovery approach through data mining proposed in this thesis can assist in developing draft plans for the regeneration of a run-down neighbourhood in the Beyoğlu Preservation Area (Tarlabaşı). In Implementation (2), the KDPM for urban analysis is implemented in combination with an evolutionary process to apply a regeneration approach developed by the author; a computational process which generates draft plans for ground floor use, user-profile and tenure-type allocation was developed. In Implementation (3), students applied the KDPM for urban analysis during the course of an international workshop. The model enabled them to explore site-specific particularities of Tarlabaşı that would support their urban intervention proposals. Among the outputs of the thesis three of them are considered as utilizable outputs that distinguish this thesis from previous studies: The KDPM for urban analysis. Although there have been other studies which make use of data mining methods and techniques combined with GIS technology, to the best of our knowledge no previous research has implemented a process model to depict this process and used the model to extract ‘knowledge’ from traditional thematic maps. Researchers and practitioners can re-use this process model to analyze other urban environments. The KDPM for urban analysis is, therefore, one of the main utilizable outputs of the thesis and an important scientific contribution of this study. The Beyoğlu Preservation Area Building Features Database. A large and quite comprehensive GIS database which consists of 45 spatial and non-spatial features attributed to the 11,984 buildings located in the Beyoğlu Preservation Area was constructed. This database is one of the original features of this study. To the best of our knowledge, there are no other examples of applications of data mining using such a comprehensive GIS database, constructed from a range of actual micro-scale data representing such a variety of features attributed to the buildings. This database can be re-used by analysts interested in studying the Beyoğlu Preservation Area. The Beyoğlu Preservation Area Building Features Database is therefore one of the main utilizable outputs of the thesis and represents a scientific contribution to the research material on the Beyoğlu Preservation Area. A computational process which generates draft plans for ground floor use, user-profile and tenure-type allocation, using GIS and data mining functionalities with evolutionary computation. This output of the thesis was generated by Implementation (2), which aimed to investigate Research Question (3). The overall process involved the successive application of Naïve Bayes Classification, Association Rule Analysis and an Evolutionary Algorithm to a subset of the Beyoğlu Preservation Area Building Features Database representing the Tarlabaşı neighbourhood. Briefly, the findings of the data mining analysis were used to formulate a set of rules for assigning ground floor use information to the buildings. These rules were then used for fitness measurements of an Evolutionary Algorithm, together with other fitness measurements for assigning user-profile and tenure-type information (defined by the author according to the regeneration approach developed by the author). As a result, the algorithm transformed the existing allocation of the ground floor use in the buildings located in Tarlabaşı in accordance with the given rules and assigned user-profile and tenure type information for each building. This computational process demonstrated one way to use the data mining analysis findings in developing intervention proposals for urban regeneration. A similar computational process can be implemented in other urban contexts by researchers and practitioners. To the best of our knowledge, no prior research has used data mining analysis findings for fitness measurements of an Evolutionary Algorithm in order to produce draft plans for ground floor use, user-profile and tenure-type allocation. This is, therefore, the most original scientific contribution and utilizable output of the thesis. As a result of the research, on the basis of the data that is available in the thematic maps of the Beyoğlu Preservation Area, the potential of a knowledge discovery approach to urban analysis in revealing the relationships between various components of urban environments and their various attributes is demonstrated. It is also demonstrated that these relationships can reveal site-specific characteristics of urban environments and if found ‘valuable’ by the the targeted researchers and practitioners, these can lead to the development of more informed intervention proposals. Thereby the knowledge discovery approach to urban analysis developed in this thesis may help to improve the quality of urban intervention proposals and consequently the quality of built environments. On the other hand, the implementations carried out in the thesis also exposed the major limitation of the knowledge discovery approach to urban analysis through data mining, which is the fact that the findings discoverable by this approach are limited by the relevant data that is collectable and accessible

Machine learning paradigms for modeling spatial and temporal information in multimedia data mining

Multimedia data mining and knowledge discovery is a fast emerging interdisciplinary applied research area. There is tremendous potential for effective use of multimedia data mining (MDM) through intelligent analysis. Diverse application areas are increasingly relying on multimedia under-standing systems. Advances in multimedia understanding are related directly to advances in signal processing, computer vision, machine learning, pattern recognition, multimedia databases, and smart sensors. The main mission of this special issue is to identify state-of-the-art machine learning paradigms that are particularly powerful and effective for modeling and combining temporal and spatial media cues such as audio, visual, and face information and for accomplishing tasks of multimedia data mining and knowledge discovery. These models should be able to bridge the gap between low-level audiovisual features which require signal processing and high-level semantics. A number of papers have been submitted to the special issue in the areas of imaging, artificial intelligence; and pattern recognition and five contributions have been selected covering state-of-the-art algorithms and advanced related topics. The first contribution by D. Xiang et al. “Evaluation of data quality and drought monitoring capability of FY-3A MERSI data” describes some basic parameters and major technical indicators of the FY-3A, and evaluates data quality and drought monitoring capability of the Medium-Resolution Imager (MERSI) onboard the FY-3A. The second contribution by A. Belatreche et al. “Computing with biologically inspired neural oscillators: application to color image segmentation” investigates the computing capabilities and potential applications of neural oscillators, a biologically inspired neural model, to gray scale and color image segmentation, an important task in image understanding and object recognition. The major contribution of this paper is the ability to use neural oscillators as a learning scheme for solving real world engineering problems. The third paper by A. Dargazany et al. entitled “Multibandwidth Kernel-based object tracking” explores new methods for object tracking using the mean shift (MS). A bandwidth-handling MS technique is deployed in which the tracker reach the global mode of the density function not requiring a specific staring point. It has been proven via experiments that the Gradual Multibandwidth Mean Shift tracking algorithm can converge faster than the conventional kernel-based object tracking (known as the mean shift). The fourth contribution by S. Alzu’bi et al. entitled “3D medical volume segmentation using hybrid multi-resolution statistical approaches” studies new 3D volume segmentation using multiresolution statistical approaches based on discrete wavelet transform and hidden Markov models. This system commonly reduced the percentage error achieved using the traditional 2D segmentation techniques by several percent. Furthermore, a contribution by G. Cabanes et al. entitled “Unsupervised topographic learning for spatiotemporal data mining” proposes a new unsupervised algorithm, suitable for the analysis of noisy spatiotemporal Radio Frequency Identification (RFID) data. The new unsupervised algorithm depicted in this article is an efficient data mining tool for behavioral studies based on RFID technology. It has the ability to discover and compare stable patterns in a RFID signal, and is appropriate for continuous learning. Finally, we would like to thank all those who helped to make this special issue possible, especially the authors and the reviewers of the articles. Our thanks go to the Hindawi staff and personnel, the journal Manager in bringing about the issue and giving us the opportunity to edit this special issue

Linking data mining, spatial analysis and algorithmic design: A review on a primer workshop based on Python

The field of data mining, the practical application of machine learning, has recently become a full flagged science known as Data Science. An interdisciplinary discipline in the intersection of A.I., computer science, statistics, data visualization and database management, its main objectives are pattern recognition/knowledge discovery in datasets and prediction/data modelling. The application of this latter objective has become recently the subject of intense debate following the case of its use outside the scientific research. From political campaigns to the first mortal accident involving a self-driven car, these events brought the field to the highlights and, although its tools are not new, the scale of their implementation raises important questions considering their application, the nature of personal digital data and free will. In research its application is most relevant in data rich fields and quantitative analysis. Here we can include spatial and urban analysis, which nowadays deal with huge datasets, e.g. combining Big Data from the internet, time series or unstructured data fluxes with urban form and structure, helping to assess or construct new investigation hypotheses. Using case-based reasoning and optimization data mining becomes a predictive tool able to assist the design process, producing scenarios or helping to explore constrained design solution spaces. The presentation will (i) briefly introduce the topic of data mining; (ii) its usage in urban analysis and design, and, mainly, (iii) report on a preliminary evaluation of the related workshop carried out in the context of the present seminar. The workshop introduces data mining to participants in a hands-on approach, focusing in simple tasks so concepts are internalized by playing with tools and scripting. The focus is on python scripting using Anaconda python data analysis package and Jupyter interactive Notebooks. In this way participants get a glimpse on one of the most flexible and widely used programming languages across a variety of fields, from algorithm design to data analysis, that is able of customize the tools that sometimes customize our own investigation or practice.info:eu-repo/semantics/publishedVersio

A Spatial Decision Support System for Property Valuation

May 6-

Data mining as a tool for environmental scientists

Over recent years a huge library of data mining algorithms has been developed to tackle a variety of problems in fields such as medical imaging and network traffic analysis. Many of these techniques are far more flexible than more classical modelling approaches and could be usefully applied to data-rich environmental problems. Certain techniques such as Artificial Neural Networks, Clustering, Case-Based Reasoning and more recently Bayesian Decision Networks have found application in environmental modelling while other methods, for example classification and association rule extraction, have not yet been taken up on any wide scale. We propose that these and other data mining techniques could be usefully applied to difficult problems in the field. This paper introduces several data mining concepts and briefly discusses their application to environmental modelling, where data may be sparse, incomplete, or heterogenous

Image mining: trends and developments

[Abstract]: Advances in image acquisition and storage technology have led to tremendous growth in very large and detailed image databases. These images, if analyzed, can reveal useful information to the human users. Image mining deals with the extraction of implicit knowledge, image data relationship, or other patterns not explicitly stored in the images. Image mining is more than just an extension of data mining to image domain. It is an interdisciplinary endeavor that draws upon expertise in computer vision, image processing, image retrieval, data mining, machine learning, database, and artificial intelligence. In this paper, we will examine the research issues in image mining, current developments in image mining, particularly, image mining frameworks, state-of-the-art techniques and systems. We will also identify some future research directions for image mining

Doctor of Philosophy

dissertationRecent advancements in mobile devices - such as Global Positioning System (GPS), cellular phones, car navigation system, and radio-frequency identification (RFID) - have greatly influenced the nature and volume of data about individual-based movement in space and time. Due to the prevalence of mobile devices, vast amounts of mobile objects data are being produced and stored in databases, overwhelming the capacity of traditional spatial analytical methods. There is a growing need for discovering unexpected patterns, trends, and relationships that are hidden in the massive mobile objects data. Geographic visualization (GVis) and knowledge discovery in databases (KDD) are two major research fields that are associated with knowledge discovery and construction. Their major research challenges are the integration of GVis and KDD, enhancing the ability to handle large volume mobile objects data, and high interactivity between the computer and users of GVis and KDD tools. This dissertation proposes a visualization toolkit to enable highly interactive visual data exploration for mobile objects datasets. Vector algebraic representation and online analytical processing (OLAP) are utilized for managing and querying the mobile object data to accomplish high interactivity of the visualization tool. In addition, reconstructing trajectories at user-defined levels of temporal granularity with time aggregation methods allows exploration of the individual objects at different levels of movement generality. At a given level of generality, individual paths can be combined into synthetic summary paths based on three similarity measures, namely, locational similarity, directional similarity, and geometric similarity functions. A visualization toolkit based on the space-time cube concept exploits these functionalities to create a user-interactive environment for exploring mobile objects data. Furthermore, the characteristics of visualized trajectories are exported to be utilized for data mining, which leads to the integration of GVis and KDD. Case studies using three movement datasets (personal travel data survey in Lexington, Kentucky, wild chicken movement data in Thailand, and self-tracking data in Utah) demonstrate the potential of the system to extract meaningful patterns from the otherwise difficult to comprehend collections of space-time trajectories