Design of Machine Learning Framework for Products Placement Strategy in Grocery Store

The well-known and most used support-confidence framework for Association rule mining has some drawbacks when employ to generate strong rules, this weakness has led to its poor predictive performances. This framework predict customers buying behavior based on the assumption of the confidence value, which limits its competent at making good business decision. This work presents a better Association Rule Mining conceptualized framework for mining previous customers transactions dataset of grocery store for the optimal prediction of products placement on the shelves, physical shelf arrangement and identification of products that needs promotion. Sampled transaction records were used to demonstrate the proposed framework. The proposed framework leverage on the ability of lift metric at improving the predictive performance of Association Rule Mining. The Lift discloses how much better an association rule is at predicting products to be placed together on the shelve rather than assuming. The proposed conceptualized framework will assist retailers and grocery stores owners to easily unlock the latent knowledge or patterns in their large day to day stored transaction dataset to make important business decision that will make them competitive and maximized their profit margin

Mining of uncertain Web log sequences with access history probabilities

An uncertain data sequence is a sequence of data that exist with some level of doubt or probability. Each data item in the uncertain sequence is represented with a label and probability values, referred to as existential probability, ranging from 0 to 1. Existing algorithms are either unsuitable or inefficient for discovering frequent sequences in uncertain data. This thesis presents mining of uncertain Web sequences with a method that combines access history probabilities from several Web log sessions with features of the PLWAP web sequential miner. The method is Uncertain Position Coded Pre-order Linked Web Access Pattern (U-PLWAP) algorithm for mining frequent sequential patterns in uncertain web logs. While PLWAP only considers a session of weblogs, U-PLWAP takes more sessions of weblogs from which existential probabilities are generated. Experiments show that U-PLWAP is at least 100% faster than U-apriori, and 33% faster than UF-growth. The UF-growth algorithm also fails to take into consideration the order of the items, thereby making U-PLWAP a richer algorithm in terms of the information its result contains

Penerapan Data Mining Dalam Mencari Pola Asosiasi Data Tracer Study Menggunakan Equivalence Class Transformation (ECLAT)

Tracer study adalah sebuah pendekatan yang diterapkan universitas untuk memperoleh informasi tentang kemungkinan kelemahan dalam proses pendidikan dan proses pembelajaran yang menjadi dasar perencanaan aktivitas untuk penyempurnaan di masa mendatang. Pada Universitas Islam Negeri Sultan Syarif Kasim Riau belum pernah ada tracer study yang komprehensif dan terstruktur dalam ruang lingkup universitas. Tracer study yang dilakukan hanya dalam lingkup program studi dan hanya dilaksanakan menjelang proses akreditasi prodi. Tidak ada tracer study yang rutin dilakukan di tingkat program studi dan universitas setiap tahunnya. Pada penelitian ini akan berfokus kepada penerapan data mining untuk mencari pola asosiasi pada data tracer study menggunakan Equivalence Class Transformation (ECLAT). Dari hasil penelitian terdapat 4 pola yang memenuhi support 13% dan confidence 80% dengan pengujian lift rasio > 1. Pola tersebut diantaranya Jika ipk antara 3 – 3,5 dan gaji pertama dibawah 3 juta dan laki-laki maka status kelulusan tidak tepat waktu dan “masa tunggu mendapatkan pekerjaan pertama kurang dari 6 bulan” dengan support 17% dan confidence 84%. Jika ipk antara 3 – 3,5 dan perempuan maka “masa tunggu mendapatkan pekerjaan pertama kurang dari 6 bulan” dan hubungan pekerjaan dengan jurusan sesuai dengan support 14 % dan confidence 100%

PENERAPAN DATA MINING DALAM MENCARI POLA ASOSIASI DATA TRACER STUDY MENGGUNAKAN EQUIVALENCE CLASS TRANSFORMATION (ECLAT)

Abstrak - Tracer study adalah sebuah pendekatan yang diterapkan universitas untuk memperoleh informasi tentang kemungkinan kelemahan dalam proses pendidikan dan proses pembelajaran yang menjadi dasar perencanaan aktivitas untuk penyempurnaan di masa mendatang. Pada Universitas Islam Negeri Sultan Syarif Kasim Riau belum pernah ada tracer study yang komprehensif dan terstruktur dalam ruang lingkup universitas. Tracer study yang dilakukan hanya dalam lingkup program studi dan hanya dilaksanakan menjelang proses akreditasi prodi. Tidak ada tracer study yang rutin dilakukan di tingkat program studi dan universitas setiap tahunnya. Pada penelitian ini akan berfokus kepada penerapan data mining untuk mencari pola asosiasi pada data tracer study menggunakan Equivalence Class Transformation (ECLAT). Dari hasil penelitian terdapat 4 pola yang memenuhi support 13% dan confidence 80% dengan pengujian lift rasio > 1. Pola tersebut diantaranya Jika ipk antara 3 – 3,5 dan gaji pertama dibawah 3 juta dan laki-laki maka status kelulusan tidak tepat waktu dan “masa tunggu mendapatkan pekerjaan pertama kurang dari 6 bulan” dengan support 17% dan confidence 84%. Jika ipk antara 3 – 3,5 dan perempuan maka “masa tunggu mendapatkan pekerjaan pertama kurang dari 6 bulan” dan hubungan pekerjaan dengan jurusan sesuai dengan support 14 % dan confidence 100%

Implementation of an interactive pattern mining framework on electronic health record datasets

Large collections of electronic patient records contain a broad range of clinical information highly relevant for data analysis. However, they are maintained primarily for patient administration, and automated methods are required to extract valuable knowledge for predictive, preventive, personalized and participatory medicine. Sequential pattern mining is a fundamental task in data mining which can be used to find statistically relevant, non-trivial temporal dependencies of events such as disease comorbidities. This works objective is to use this mining technique to identify disease associations based on ICD-9-CM codes data of the entire Taiwanese population obtained from Taiwan’s National Health Insurance Research Database. This thesis reports the development and implementation of the Disease Pattern Miner – a pattern mining framework in a medical domain. The framework was designed as a Web application which can be used to run several state-of-the-art sequence mining algorithms on electronic health records, collect and filter the results to reduce the number of patterns to a meaningful size, and visualize the disease associations as an interactive model in a specific population group. This may be crucial to discover new disease associations and offer novel insights to explain disease pathogenesis. A structured evaluation of the data and models are required before medical data-scientist may use this application as a tool for further research to get a better understanding of disease comorbidities

Breaking rules: taking Complex Ontology Alignment beyond rule­based approaches

Tese de mestrado, Ciência de Dados, Universidade de Lisboa, Faculdade de Ciências, 2021As ontologies are developed in an uncoordinated manner, differences in scope and design compromise interoperability. Ontology matching is critical to address this semantic heterogeneity problem, as it finds correspondences that enable integrating data across the Semantic Web. One of the biggest challenges in this field is that ontology schemas often differ conceptually, and therefore reconciling many real¬world ontology pairs (e.g., in geography or biomedicine) involves establishing complex mappings that contain multiple entities from each ontology. Yet, for the most part, ontology matching algorithms are restricted to finding simple equivalence mappings between ontology entities. This work presents novel algorithms for Complex Ontology Alignment based on Association Rule Mining over a set of shared instances between two ontologies. Its strategy relies on a targeted search for known complex patterns in instance and schema data, reducing the search space. This allows the application of semantic¬based filtering algorithms tailored to each kind of pattern, to select and refine the most relevant mappings. The algorithms were evaluated in OAEI Complex track datasets under two automated approaches: OAEI’s entity¬based approach and a novel element¬overlap–based approach which was developed in the context of this work. The algorithms were able to find mappings spanning eight distinct complex patterns, as well as combinations of patterns through disjunction and conjunction. They were able to efficiently reduce the search space and showed competitive performance results comparing to the State of the Art of complex alignment systems. As for the comparative analysis of evaluation methodologies, the proposed element¬overlap–based evaluation strategy was shown to be more accurate and interpretable than the reference-based automatic alternative, although none of the existing strategies fully address the challenges discussed in the literature. For future work, it would be interesting to extend the algorithms to cover more complex patterns and combine them with lexical approaches

Structural advances for pattern discovery in multi-relational databases

With ever-growing storage needs and drift towards very large relational storage settings, multi-relational data mining has become a prominent and pertinent field for discovering unique and interesting relational patterns. As a consequence, a whole suite of multi-relational data mining techniques is being developed. These techniques may either be extensions to the already existing single-table mining techniques or may be developed from scratch. For the traditionalists, single-table mining algorithms can be used to work on multi-relational settings by making inelegant and time consuming joins of all target relations. However, complex relational patterns cannot be expressed in a single-table format and thus, cannot be discovered. This work presents a new multi-relational frequent pattern mining algorithm termed Multi-Relational Frequent Pattern Growth (MRFP Growth). MRFP Growth is capable of mining multiple relations, linked with referential integrity, for frequent patterns that satisfy a user specified support threshold. Empirical results on MRFP Growth performance and its comparison with the state-of-the-art multirelational data mining algorithms like WARMR and Decentralized Apriori are discussed at length. MRFP Growth scores over the latter two techniques in number of patterns generated and speed. The realm of multi-relational clustering is also explored in this thesis. A multi-Relational Item Clustering approach based on Hypergraphs (RICH) is proposed. Experimentally RICH combined with MRFP Growth proves to be a competitive approach for clustering multi-relational data. The performance and iii quality of clusters generated by RICH are compared with other clustering algorithms. Finally, the thesis demonstrates the applied utility of the theoretical implications of the above mentioned algorithms in an application framework for auto-annotation of images in an image database. The system is called CoMMA which stands for Combining Multi-relational Multimedia for Associations

Similarity search and mining in uncertain spatial and spatio-temporal databases

Both the current trends in technology such as smart phones, general mobile devices, stationary sensors and satellites as well as a new user mentality of utilizing this technology to voluntarily share information produce a huge flood of geo-spatial and geo-spatio-temporal data. This data flood provides a tremendous potential of discovering new and possibly useful knowledge. In addition to the fact that measurements are imprecise, due to the physical limitation of the devices, some form of interpolation is needed in-between discrete time instances. From a complementary perspective - to reduce the communication and bandwidth utilization, along with the storage requirements, often the data is subjected to a reduction, thereby eliminating some of the known/recorded values. These issues introduce the notion of uncertainty in the context of spatio-temporal data management - an aspect raising an imminent need for scalable and flexible data management. The main scope of this thesis is to develop effective and efficient techniques for similarity search and data mining in uncertain spatial and spatio-temporal data. In a plethora of research fields and industrial applications, these techniques can substantially improve decision making, minimize risk and unearth valuable insights that would otherwise remain hidden. The challenge of effectiveness in uncertain data is to correctly determine the set of possible results, each associated with the correct probability of being a result, in order to give a user a confidence about the returned results. The contrary challenge of efficiency, is to compute these result and corresponding probabilities in an efficient manner, allowing for reasonable querying and mining times, even for large uncertain databases. The paradigm used to master both challenges, is to identify a small set of equivalent classes of possible worlds, such that members of the same class can be treated as equivalent in the context of a given query predicate or data mining task. In the scope of this work, this paradigm will be formally defined, and applied to the most prominent classes of spatial queries on uncertain data, including range queries, k-nearest neighbor queries, ranking queries and reverse k-nearest neighbor queries. For this purpose, new spatial and probabilistic pruning approaches are developed to further speed up query processing. Furthermore, the proposed paradigm allows to develop the first efficient solution for the problem of frequent co-location mining on uncertain data. Special emphasis is taken on the temporal aspect of applications using modern data collection technologies. While the aforementioned techniques work well for single points of time, the prediction of query results over time remains a challenge. This thesis fills this gap by modeling an uncertain spatio-temporal object as a stochastic process, and by applying the above paradigm to efficiently query, index and mine historical spatio-temporal data.Moderne Technologien, z.B. Sattelitentechnologie und Technologie in Smart Phones, erzeugen eine Flut räumlicher Geo-Daten. Zudem ist in der Gesellschaft ein Trend zu beobachten diese erzeugten Daten freiwillig auf öffentlich zugänglichen Plattformen zur Verfügung zu stellen. Diese Datenflut hat immenses Potential, um neues und nützliches Wissen zu entdecken. Diese Daten sind jedoch grundsätzlich unsichere räumliche Daten. Die Unsicherheit ergibt sich aus mehreren Aspekten. Zum einen kommt es bei Messungen grundsätzlich zu Messungenauigkeiten, zum anderen ist zwischen diskreten Messzeitpunkten eine Interpolation nötig, die zusätzliche Unsicherheit erzeugt. Auerdem werden die Daten oft absichtlich reduziert, um Speicherplatz und Transfervolumen einzusparen, wodurch weitere Information verloren geht. Diese Unsicherheit schafft einen sofortigen Bedarf für skalierbare und flexible Methoden zur Verwaltung und Auswertung solcher Daten. Im Rahmen dieser Arbeit sollen effektive und effiziente Techniken zur Ähnlichkeitssuche und zum Data Mining bei unsicheren räumlichen und unsicheren räumlich-zeitlichen Daten erarbeitet werden. Diese Techniken liefern wertvolles Wissen, das auf verschiedenen Forschungsgebieten, als auch bei industriellen Anwendungen zur Entscheidungsfindung genutzt werden kann. Bei der Entwicklung dieser Techniken gibt es zwei Herausforderungen. Einerseits müssen die entwickelten Techniken effektiv sein, um korrekte Ergebnisse und Wahrscheinlichkeiten dieser Ergebnisse zurückzugeben. Andererseits müssen die entwickelten Techniken effizient sein, um auch in sehr großen Datenbanken Ergebnisse in annehmbarer Zeit zu liefern. Die Dissertation stellt ein neues Paradigma vor, das beide Herausforderungen meistert. Dieses Paradigma identifiziert mögliche Datenbankwelten, die bezüglich des gegebenen Anfrageprädikats äquivalent sind. Es wird formal definiert und auf die relevantesten räumlichen Anfragetypen angewendet, um effiziente Lösungen zu entwickeln. Dazu gehören Bereichanfragen, k-Nächste-Nachbarnanfragen, Rankinganfragen und Reverse k-Nächste-Nachbarnanfragen. Räumliche und probabilistische Pruningkriterien werden entwickelt, um insignifikante Ergebnisse früh auszuschlieen. Zudem wird die erste effiziente Lösung für das Problem des "Spatial Co-location Minings" auf unsicheren Daten präsentiert. Ein besonderer Schwerpunkt dieser Arbeit liegt auf dem temporalen Aspekt moderner Geo-Daten. Während obig genannte Techniken dieser Arbeit für einzelne Zeitpunkt sehr gut funktionieren, ist die effektive und effiziente Verwaltung von unsicheren räumlich zeitlichen Daten immer noch ein weitestgehend ungelöstes Problem. Diese Dissertation löst dieses Problem, indem unsichere räumlich-zeitliche Daten durch stochastische Prozesse modeliert werden. Auf diese stochastischen Prozesse lässt sich das oben genannte Paradigma anwenden, um unsichere räumlich-zeitliche Daten effizient anzufragen, zu indexieren, und zu minen