Algorithm Selection Framework: A Holistic Approach to the Algorithm Selection Problem

A holistic approach to the algorithm selection problem is presented. The “algorithm selection framework uses a combination of user input and meta-data to streamline the algorithm selection for any data analysis task. The framework removes the conjecture of the common trial and error strategy and generates a preference ranked list of recommended analysis techniques. The framework is performed on nine analysis problems. Each of the recommended analysis techniques are implemented on the corresponding data sets. Algorithm performance is assessed using the primary metric of recall and the secondary metric of run time. In six of the problems, the recall of the top ranked recommendation is considered excellent with at least 95 percent of the best observed recall; the average of this metric is 79 percent due to two poorly performing recommendations. The top recommendation is Pareto efficient for three of the problems. The framework measures well against an a-priori set of criteria. The framework provides value by filtering the candidate of analytic techniques and, often, selecting a high performing technique as the top ranked recommendation. The user input and meta-data used by the framework contain information with high potential for effective algorithm selection. Future work should optimize the recommendation logic and expand the scope of techniques for other types of analysis problems. Further, the results of this proposed study should be leveraged in order to better understand the behavior of meta-learning models

A comparison of statistical machine learning methods in heartbeat detection and classification

In health care, patients with heart problems require quick responsiveness in a clinical setting or in the operating theatre. Towards that end, automated classification of heartbeats is vital as some heartbeat irregularities are time consuming to detect. Therefore, analysis of electro-cardiogram (ECG) signals is an active area of research. The methods proposed in the literature depend on the structure of a heartbeat cycle. In this paper, we use interval and amplitude based features together with a few samples from the ECG signal as a feature vector. We studied a variety of classification algorithms focused especially on a type of arrhythmia known as the ventricular ectopic fibrillation (VEB). We compare the performance of the classifiers against algorithms proposed in the literature and make recommendations regarding features, sampling rate, and choice of the classifier to apply in a real-time clinical setting. The extensive study is based on the MIT-BIH arrhythmia database. Our main contribution is the evaluation of existing classifiers over a range sampling rates, recommendation of a detection methodology to employ in a practical setting, and extend the notion of a mixture of experts to a larger class of algorithms

TOWARDS BUILDING AN INTELLIGENT INTEGRATED MULTI-MODE TIME DIARY SURVEY FRAMEWORK

Enabling true responses is an important characteristic in surveys; where the responses are free from bias and satisficing. In this thesis, we examine the current state of surveys, briefly touching upon questionnaire surveys, and then on time diary surveys (TDS). TDS are open-ended conversational surveys of a free-form nature with both, the interviewer and the respondent, playing a part in its progress and successful completion. With limited research available on how intelligent and assistive components can affect TDS respondents, we explore ways in which intelligent systems such as Computer Adaptive Testing, Intelligent Tutoring Systems, Recommender Systems, and Decision Support Systems can be leveraged for use in TDS. The motivation for this work is from realizing the opportunity that an enhanced web based instrument can offer the survey domain to unite the various facets of web based surveys to create an intelligent integrated multi-mode TDS framework. We envision the framework to provide all the advantages of web based surveys and interviewer assisted surveys. The two primary challenges are in determining what data is to be used by the system and how to interact with the user – specifically integrating the (1) Interviewer-assisted mode, and (2) Self-administered mode. Our proposed solution – the intelligent integrated multi-mode framework – is essentially the solution to a set of modeling problems and we propose two sets of overreaching mechanisms: (1) Knowledge Engineering Mechanisms (KEM), and (2) Interaction Mechanisms (IxM), where KEM serves the purpose of understanding what data can be created, used and stored while IxM deals with interacting with the user. We build and study a prototype instrument in the interviewer-assisted mode based on the framework. We are able to determine that the instrument improves the interview process as intended and increases the data quality of the response data and is able to assist the interviewer. We also observe that the framework’s mechanisms contribute towards reducing interviewers’ cognitive load, data entry times and interview time by predicting the next activity. Advisor: Leenkiat So

Advanced techniques for personalized, interactive question answering

Using a computer to answer questions has been a human dream since the beginning of the digital era. A first step towards the achievement of such an ambitious goal is to deal with naturallangilage to enable the computer to understand what its user asks. The discipline that studies the conD:ection between natural language and the represen~ tation of its meaning via computational models is computational linguistics. According to such discipline, Question Answering can be defined as the task that, given a question formulated in natural language, aims at finding one or more concise answers in the form of sentences or phrases. Question Answering can be interpreted as a sub-discipline of information retrieval with the added challenge of applying sophisticated techniques to identify the complex syntactic and semantic relationships present in text. Although it is widely accepted that Question Answering represents a step beyond standard infomiation retrieval, allowing a more sophisticated and satisfactory response to the user's information needs, it still shares a series of unsolved issues with the latter. First, in most state-of-the-art Question Answering systems, the results are created independently of the questioner's characteristics, goals and needs. This is a serious limitation in several cases: for instance, a primary school child and a History student may need different answers to the questlon: When did, the Middle Ages begin? Moreover, users often issue queries not as standalone but in the context of a wider information need, for instance when researching a specific topic. Although it has recently been proposed that providing Question Answering systems with dialogue interfaces would encourage and accommodate the submission of multiple related questions and handle the user's requests for clarification, interactive Question Answering is still at its early stages: Furthermore, an i~sue which still remains open in current Question Answering is that of efficiently answering complex questions, such as those invoking definitions and descriptions (e.g. What is a metaphor?). Indeed, it is difficult to design criteria to assess the correctness of answers to such complex questions. .. These are the central research problems addressed by this thesis, and are solved as follows. An in-depth study on complex Question Answering led to the development of classifiers for complex answers. These exploit a variety of lexical, syntactic and shallow semantic features to perform textual classification using tree-~ernel functions for Support Vector Machines. The issue of personalization is solved by the integration of a User Modelling corn': ponent within the the Question Answering model. The User Model is able to filter and fe-rank results based on the user's reading level and interests. The issue ofinteractivity is approached by the development of a dialogue model and a dialogue manager suitable for open-domain interactive Question Answering. The utility of such model is corroborated by the integration of an interactive interface to allow reference resolution and follow-up conversation into the core Question Answerin,g system and by its evaluation. Finally, the models of personalized and interactive Question Answering are integrated in a comprehensive framework forming a unified model for future Question Answering research