Interpretability of AI in Computer Systems and Public Policy

Advances in Artificial Intelligence (AI) have led to spectacular innovations and sophisticated systems for tasks that were thought to be capable only by humans. Examples include playing chess and Go, face and voice recognition, driving vehicles, and more. In recent years, the impact of AI has moved beyond offering mere predictive models into building interpretable models that appeal to human logic and intuition because they ensure transparency and simplicity and can be used to make meaningful decisions in real-world applications. A second trend in AI is characterized by important advancements in the realm of causal reasoning. Identifying causal relationships is an important aspect of scientific endeavors in a variety of fields. Causal models and Bayesian inference can help us gain better domain-specific insight and make better data-driven decisions because of their interpretability. The main objective of this dissertation was to adapt theoretically sound AI-based interpretable data-analytic approaches to solve domain-specific problems in the two un-related fields of Storage Systems and Public Policy. For the first task, we considered the well-studied problem of cache replacement problem in computing systems, which can be modeled as a variant of the well-known Multi-Armed Bandit (MAB) problem with delayed feedback and decaying costs, and developed an algorithm called EXP4-DFDC. We proved theoretically that EXP4-DFDC exhibits an important feature called vanishing regret. Based on the theoretical analysis, we designed a machine-learning algorithm called ALeCaR, with adaptive hyperparameters. We used extensive experiments on a wide range of workloads to show that ALeCaR performed better than LeCaR, the best machine learning algorithm for cache replacement at that time. We concluded that reinforcement machine learning can offer an outstanding approach for implementing cache management policies. For the second task, we used Bayesian networks to analyze the service request data from three 311 centers providing non-emergency services in the cities of Miami-Dade, New York City, and San Francisco. Using a causal inference approach, this study investigated the presence of inequities in the quality of the 311 services to neighborhoods with varying demographics and socioeconomic status. We concluded that the services provided by the local governments showed no detectable biases on the basis of race, ethnicity, or socioeconomic status

Designing a human-centred, mobile interface to support real-time flood forecasting and warning system

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonThere is a demand for human-centred technology which improves the management of flood events. This thesis describes the development, design and evaluation of a mobile GIS-based hydrological model. The application provides hydrological forecasts and issues flood warnings. The thesis reports on the usability and practicality of the application. The application, a mobile-based hydrological modelling system, permits the integrated handling of real-time rainfall data from a wireless monitoring network. A spatially distributed GIS-based model integrates this incoming data, approximating real-time, to generate data on catchment hydrology and runoff. The data can be accessed from any android-based mobile computer or mobile phone. It may be further analysed online using several GIS and numerical functions. A human-centred approach to design was taken. Design guidelines for a user-centric application were developed and deployed in the first prototype. There was intensive consultation with potential users. Particular attention was paid to the ease of use of the mobile interface. Users’ needs and attitudes were relevant in the achievement of a highly functional but intuitive interface. The first prototype underwent intensive testing with users. After the initial testing of the first prototype an interactive approach was taken to development. This generated a high-fidelity prototype which was matched to the taxonomy from a user’s mental model. Users were interrogated under controlled laboratory conditions as they performed predefined tasks which were selected to generate data across all aspects of the system and to identify weaknesses. Subsequent to this work there was a major prototype re-design. User test data, identified issues and an improved mental taxonomy closer were used to further refine the application. Of particular note was new functionality which aligned with user expectations and enhanced the applications credibility. The final evaluation of the system was undertaken with diverse subjects. Overall, the subjects considered the system efficient and effective. Users said the system was easy to learn and integrate into their work. Task completion rates were satisfactory. The final interviews with users confirmed that the application was ready to proceed to the implementation phase

Optimal Number of Choices in Rating Contexts

In many settings, people must give numerical scores to entities from a small discrete set—for instance, rating physical attractiveness from 1–5 on dating sites, or papers from 1–10 for conference reviewing. We study the problem of understanding when using a different number of options is optimal. We consider the case when scores are uniform random and Gaussian. We study computationally when using 2, 3, 4, 5, and 10 options out of a total of 100 is optimal in these models (though our theoretical analysis is for a more general setting with k choices from n total options as well as a continuous underlying space). One may expect that using more options would always improve performance in this model, but we show that this is not necessarily the case, and that using fewer choices—even just two—can surprisingly be optimal in certain situations. While in theory for this setting it would be optimal to use all 100 options, in practice, this is prohibitive, and it is preferable to utilize a smaller number of options due to humans’ limited computational resources. Our results could have many potential applications, as settings requiring entities to be ranked by humans are ubiquitous. There could also be applications to other fields such as signal or image processing where input values from a large set must be mapped to output values in a smaller set