Dissimilarity-based learning for complex data

Mokbel B. Dissimilarity-based learning for complex data. Bielefeld: Universität Bielefeld; 2016.Rapid advances of information technology have entailed an ever increasing amount of digital data, which raises the demand for powerful data mining and machine learning tools. Due to modern methods for gathering, preprocessing, and storing information, the collected data become more and more complex: a simple vectorial representation, and comparison in terms of the Euclidean distance is often no longer appropriate to capture relevant aspects in the data. Instead, problem-adapted similarity or dissimilarity measures refer directly to the given encoding scheme, allowing to treat information constituents in a relational manner. This thesis addresses several challenges of complex data sets and their representation in the context of machine learning. The goal is to investigate possible remedies, and propose corresponding improvements of established methods, accompanied by examples from various application domains. The main scientific contributions are the following: (I) Many well-established machine learning techniques are restricted to vectorial input data only. Therefore, we propose the extension of two popular prototype-based clustering and classification algorithms to non-negative symmetric dissimilarity matrices. (II) Some dissimilarity measures incorporate a fine-grained parameterization, which allows to configure the comparison scheme with respect to the given data and the problem at hand. However, finding adequate parameters can be hard or even impossible for human users, due to the intricate effects of parameter changes and the lack of detailed prior knowledge. Therefore, we propose to integrate a metric learning scheme into a dissimilarity-based classifier, which can automatically adapt the parameters of a sequence alignment measure according to the given classification task. (III) A valuable instrument to make complex data sets accessible are dimensionality reduction techniques, which can provide an approximate low-dimensional embedding of the given data set, and, as a special case, a planar map to visualize the data's neighborhood structure. To assess the reliability of such an embedding, we propose the extension of a well-known quality measure to enable a fine-grained, tractable quantitative analysis, which can be integrated into a visualization. This tool can also help to compare different dissimilarity measures (and parameter settings), if ground truth is not available. (IV) All techniques are demonstrated on real-world examples from a variety of application domains, including bioinformatics, motion capturing, music, and education

Adaptive structure metrics for automated feedback provision in intelligent tutoring systems

Paaßen B, Mokbel B, Hammer B. Adaptive structure metrics for automated feedback provision in intelligent tutoring systems. Neurocomputing. 2016;192(SI):3-13.Typical intelligent tutoring systems rely on detailed domain-knowledge which is hard to obtain and difficult to encode. As a data-driven alternative to explicit domain-knowledge, one can present learners with feedback based on similar existing solutions from a set of stored examples. At the heart of such a data-driven approach is the notion of similarity. We present a general-purpose framework to construct structure metrics on sequential data and to adapt those metrics using machine learning techniques. We demonstrate that metric adaptation improves the classification of wrong versus correct learner attempts in a simulated data set from sports training, and the classification of the underlying learner strategy in a real Java programming dataset

A Toolbox for Adaptive Sequence Dissimilarity Measures for Intelligent Tutoring Systems

ABSTRACT We present the TCS Alignment Toolbox, which offers a flexible framework to calculate and visualize (dis)similarities between sequences in the context of educational data mining and intelligent tutoring systems. The toolbox offers a variety of alignment algorithms, allows for complex input sequences comprised of multi-dimensional elements, and is adjustable via rich parameterization options, including mechanisms for an automatic adaptation based on given data. Our demo shows an example in which the alignment measure is adapted to distinguish students' Java programs w.r.t. different solution strategies, via a machine learning technique

Adaptive structure metrics for automated feedback provision in Java programming

Paaßen B, Mokbel B, Hammer B. Adaptive structure metrics for automated feedback provision in Java programming. In: Verleysen M, ed. Proceedings of the ESANN, 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. 2015: 307-312.Today's learning supporting systems for programming mostly rely on pre-coded feedback provision, such that their applicability is restricted to modelled tasks. In this contribution, we investigate the suitability of machine learning techniques to automate this process by means of a presentationm of similar solution strategies from a set of stored examples. To this end we apply structure metric learning methods in local and global alignment which can be used to compare Java programs. We demonstrate that automatically adapted metrics better identify the underlying programming strategy as compared to their default counterparts in a benchmark example from programming

Inferring Feature Relevances From Metric Learning

Schulz A, Mokbel B, Biehl M, Hammer B. Inferring Feature Relevances From Metric Learning. In: 2015 IEEE Symposium Series on Computational Intelligence. Piscataway, NJ: IEEE; 2015

Adaptive distance measures for sequential data

Mokbel B, Paaßen B, Hammer B. Adaptive distance measures for sequential data. In: Verleysen M, ed. ESANN, 22nd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges, Belgium: i6doc.com; 2014: 265-270.Recent extensions of learning vector quantization (LVQ) to general (dis-)similarity data have paved the way towards LVQ classifiers for possibly discrete, structured objects such as sequences addressed by classical alignment. In this contribution, we propose a metric learning scheme based on this framework which allows for autonomous learning of the underlying scoring matrix according to a given discriminative task. Besides facilitating the often crucial and problematic choice of the scoring matrix in applications, this extension offers an increased interpretability of the results by pointing out structural invariances for the given task

Learning Feedback in Intelligent Tutoring Systems

Gross S, Mokbel B, Hammer B, Pinkwart N. Learning Feedback in Intelligent Tutoring Systems. KI - Künstliche Intelligenz. 2015;29(4):1-6.Intelligent Tutoring Systems (ITSs) are adaptive learning systems that aim to support learners by providing one-on-one individualized instruction. Typically, instructing learners in ITSs is build on formalized domain knowledge and, thus, the applicability is restricted to well-defined domains where knowledge about the domain being taught can be explicitly modeled. For ill-defined domains, human tutors still by far outperform the performance of ITSs, or the latter are not applicable at all. As part of the DFG priority programme "Autonomous Learning", the FIT project has been conducted over a period of 3 years pursuing the goal to develop novel ITS methods, that are also applicable for ill-defined problems, based on implicit domain knowledge extracted from educational data sets. Here, machine learning techniques have been used to autonomously infer structures from given learning data (e.g., student solutions) and, based on these structures, to develop strategies for instructing learners

Towards Dimensionality Reduction for Smart Home Sensor Data

Mokbel B, Schulz A. Towards Dimensionality Reduction for Smart Home Sensor Data. In: Hammer B, Martinetz T, Villmann T, eds. Proceedings of the Workshop New Challenges in Neural Computation (NC² 2015). Machine Learning Reports. 2015: 41-48.In this paper, we investigate in how far nonlinear dimensionality reduction (DR) techniques can be utilized to tackle particular challenges of sensor data from smart home environments. Smart homes often contain a large number of sensors of various types, providing output in real time, which results in a sequence of high-dimensional, heterogeneous data vectors. We propose that DR techniques can provide a truthful low-dimensional representation (i.e. a compression) of this kind of data, together with a corresponding reconstruction (i.e. decompression). This yields an automatic fusion of uncoordinated raw sensor signals, as well as an economical storage format, with a certain robustness against sensor failure. In proof-of-concept experiments, we present first empirical results to test our approach based on real-world data

Autonomous Learning of Representations

Walter O, Häb-Umbach R, Mokbel B, Paaßen B, Hammer B. Autonomous Learning of Representations. KI - Künstliche Intelligenz. 2015;29(4):339–351.Besides the core learning algorithm itself, one major question in machine learning is how to best encode given training data such that the learning technology can efficiently learn based thereon and generalize to novel data. While classical approaches often rely on a hand coded data representation, the topic of autonomous representation or feature learning plays a major role in modern learning architectures. The goal of this contribution is to give an overview about different principles of autonomous feature learning, and to exemplify two principles based on two recent examples: autonomous metric learning for sequences, and autonomous learning of a deep representation for spoken language, respectively