Contextual and Ethical Issues with Predictive Process Monitoring

This thesis addresses contextual and ethical issues in the predictive process monitoring framework and several related issues. Regarding contextual issues, even though the importance of case, process, social and external contextual factors in the predictive business process monitoring framework has been acknowledged, few studies have incorporated these into the framework or measured their impact. Regarding ethical issues, we examine how human agents make decisions with the assistance of process monitoring tools and provide recommendation to facilitate the design of tools which enables a user to recognise the presence of algorithmic discrimination in the predictions provided. First, a systematic literature review is undertaken to identify existing studies which adopt a clustering-based remaining-time predictive process monitoring approach, and a comparative analysis is performed to compare and benchmark the output of the identified studies using 5 real-life event logs. This curates the studies which have adopted this important family of predictive process monitoring approaches but also facilitates comparison as the various studies utilised different datasets, parameters, and evaluation measures. Subsequently, the next two chapter investigate the impact of social and spatial contextual factors in the predictive process monitoring framework. Social factors encompass the way humans and automated agents interact within a particular organisation to execute process-related activities. The impact of social contextual features in the predictive process monitoring framework is investigated utilising a survival analysis approach. The proposed approach is benchmarked against existing approaches using five real-life event logs and outperforms these approaches. Spatial context (a type of external context) is also shown to improve the predictive power of business process monitoring models. The penultimate chapter examines the nature of the relationship between workload (a process contextual factor) and stress (a social contextual factor) by utilising a simulation-based approach to investigate the diffusion of workload-induced stress in the workplace. In conclusion, the thesis examines how users utilise predictive process monitoring (and AI) tools to make decisions. Whilst these tools have delivered real benefits in terms of improved service quality and reduction in processing time, among others, they have also raised issues which have real-world ethical implications such as recommending different credit outcomes for individuals who have an identical financial profile but different characteristics (e.g., gender, race). This chapter amalgamates the literature in the fields of ethical decision making and explainable AI and proposes, but does not attempt to validate empirically, propositions and belief statements based on the synthesis of the existing literature, observation, logic, and empirical analogy

Automated Process Discovery: A Literature Review and a Comparative Evaluation with Domain Experts

Äriprotsesside kaeve meetodi võimaldavad analüütikul kasutada logisid saamaks teadmisi protsessi tegeliku toimise kohta. Neist meetodist üks enim uuritud on automaatne äriprotsesside avastamine. Sündmuste logi võetakse kui sisend automaatse äriprotsesside avastamise meetodi poolt ning väljundina toodetakse äriprotsessi mudel, mis kujutab logis talletatud sündmuste kontrollvoogu. Viimase kahe kümnendi jooksul on väljapakutud mitmeidki automaatseid äriprotsessi avastamise meetodeid balansseerides erinevalt toodetavate mudelite skaleeruvuse, täpsuse ning keerukuse vahel. Siiani on automaatsed äriprotsesside avastamise meetodid testitud ad-hoc kombel, kus erinevad autorid kasutavad erinevaid andmestike, seadistusi, hindamismeetrikuid ning alustõdesid, mis viib tihti võrdlematute tulemusteni ning mõnikord ka mittetaastoodetavate tulemusteni suletud andmestike kasutamise tõttu. Eelpool toodu mõistes sooritatakse antud magistritöö raames süstemaatiline kirjanduse ülevaade automaatsete äriprotsesside avastamise meetoditest ja ka süstemaatiline hindav võrdlus üle nelja kvaliteedimeetriku olemasolevate automaatsete äriprotsesside avastamise meetodite kohta koostöös domeeniekspertidega ning kasutades reaalset logi rahvusvahelisest tarkvara firmast. Kirjanduse ülevaate ning hindamise tulemused tõstavad esile puudujääke ning seni uurimata kompromisse mudelite loomiseks nelja kvaliteedimeetriku kontekstis. Antud magistritöö tulemused võimaldavad teaduritel parandada puudujäägid meetodites. Samuti vastatakse küsimusele automaatsete äriprotsesside avastamise meetodite kasutamise kohta väljaspool akadeemilist maailma.Process mining methods allow analysts to use logs of historical executions of business processes in order to gain knowledge about the actual performance of these processes.One of the most widely studied process mining operations is automated process discovery.An event log is taken as input by an automated process discovery method and produces a business process model as output that captures the control-flow relations between tasks that are described by the event log.Several automated process discovery methods have been proposed in the past two decades, striking different tradeoffs between scalability, accuracy and complexity of the resulting models.So far, automated process discovery methods have been evaluated in an ad hoc manner, with different authors employing different datasets, experimental setups, evaluation measures and baselines, often leading to incomparable conclusions and sometimes unreproducible results due to the use of non-publicly available datasets.In this setting, this thesis provides a systematic review of automated process discovery methods and a systematic comparative evaluation of existing implementations of these methods with domain experts by using a real-life event log extracted from a international software engineering company and four quality metrics.The review and evaluation results highlight gaps and unexplored tradeoffs in the field in the context of four business process model quality metrics.The results of this master thesis allows researchers to improve the lacks in the automated process discovery methods and also answers question about the usability of process discovery techniques in industry

Folk Theories, Recommender Systems, and Human-Centered Explainable Artificial Intelligence (HCXAI)

This study uses folk theories to enhance human-centered “explainable AI” (HCXAI). The complexity and opacity of machine learning has compelled the need for explainability. Consumer services like Amazon, Facebook, TikTok, and Spotify have resulted in machine learning becoming ubiquitous in the everyday lives of the non-expert, lay public. The following research questions inform this study: What are the folk theories of users that explain how a recommender system works? Is there a relationship between the folk theories of users and the principles of HCXAI that would facilitate the development of more transparent and explainable recommender systems? Using the Spotify music recommendation system as an example, 19 Spotify users were surveyed and interviewed to elicit their folk theories of how personalized recommendations work in a machine learning system. Seven folk theories emerged: complies, dialogues, decides, surveils, withholds and conceals, empathizes, and exploits. These folk theories support, challenge, and augment the principles of HCXAI. Taken collectively, the folk theories encourage HCXAI to take a broader view of XAI. The objective of HCXAI is to move towards a more user-centered, less technically focused XAI. The elicited folk theories indicate that this will require adopting principles that include policy implications, consumer protection issues, and concerns about intention and the possibility of manipulation. As a window into the complex user beliefs that inform their iii interactions with Spotify, the folk theories offer insights into how HCXAI systems can more effectively provide machine learning explainability to the non-expert, lay public

How to Make Agents and Influence Teammates: Understanding the Social Influence AI Teammates Have in Human-AI Teams

The introduction of computational systems in the last few decades has enabled humans to cross geographical, cultural, and even societal boundaries. Whether it was the invention of telephones or file sharing, new technologies have enabled humans to continuously work better together. Artificial Intelligence (AI) has one of the highest levels of potential as one of these technologies. Although AI has a multitude of functions within teaming, such as improving information sciences and analysis, one specific application of AI that has become a critical topic in recent years is the creation of AI systems that act as teammates alongside humans, in what is known as a human-AI team. However, as AI transitions into teammate roles they will garner new responsibilities and abilities, which ultimately gives them a greater influence over teams\u27 shared goals and resources, otherwise known as teaming influence. Moreover, that increase in teaming influence will provide AI teammates with a level of social influence. Unfortunately, while research has observed the impact of teaming influence by examining humans\u27 perception and performance, an explicit and literal understanding of the social influence that facilitates long-term teaming change has yet to be created. This dissertation uses three studies to create a holistic understanding of the underlying social influence that AI teammates possess. Study 1 identifies the fundamental existence of AI teammate social influence and how it pertains to teaming influence. Qualitative data demonstrates that social influence is naturally created as humans actively adapt around AI teammate teaming influence. Furthermore, mixed-methods results demonstrate that the alignment of AI teammate teaming influence with a human\u27s individual motives is the most critical factor in the acceptance of AI teammate teaming influence in existing teams. Study 2 further examines the acceptance of AI teammate teaming and social influence and how the design of AI teammates and humans\u27 individual differences can impact this acceptance. The findings of Study 2 show that humans have the greatest levels of acceptance of AI teammate teaming influence that is comparative to their own teaming influence on a single task, but the acceptance of AI teammate teaming influence across multiple tasks generally decreases as teaming influence increases. Additionally, coworker endorsements are shown to increase the acceptance of high levels of AI teammate teaming influence, and humans that perceive the capabilities of technology, in general, to be greater are potentially more likely to accept AI teammate teaming influence. Finally, Study 3 explores how the teaming and social influence possessed by AI teammates change when presented in a team that also contains teaming influence from multiple human teammates, which means social influence between humans also exists. Results demonstrate that AI teammate social influence can drive humans to prefer and observe their human teammates over their AI teammates, but humans\u27 behavioral adaptations are more centered around their AI teammates than their human teammates. These effects demonstrate that AI teammate social influence, when in the presence of human-human teaming and social influence, retains potency, but its effects are different when impacting either perception or behavior. The above three studies fill a currently under-served research gap in human-AI teaming, which is both the understanding of AI teammate social influence and humans\u27 acceptance of it. In addition, each study conducted within this dissertation synthesizes its findings and contributions into actionable design recommendations that will serve as foundational design principles to allow the initial acceptance of AI teammates within society. Therefore, not only will the research community benefit from the results discussed throughout this dissertation, but so too will the developers, designers, and human teammates of human-AI teams

Estimating credibility of science claims : analysis of forecasting data from metascience projects : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Statistics at Massey University, Albany, New Zealand

The veracity of scientific claims is not always certain. In fact, sufficient claims have been proven incorrect that many scientists believe that science itself is facing a “replication crisis”. Large scale replication projects provided empirical evidence that only around 50% of published social and behavioral science findings are replicable. Multiple forecasting studies showed that the outcomes of replication projects could be predicted by crowdsourced human evaluators. The research presented in this thesis builds on previous forecasting studies, deriving new findings and exploring new scope and scale. The research is centered around the DARPA SCORE (Systematizing Confidence in Open Research and Evidence) programme, a project aimed at developing measures of credibility for social and behavioral science claims. As part of my contribution to SCORE, myself, along with a international collaboration, elicited forecasts from human experts via surveys and prediction markets to predict the replicability of 3000 claims. I also present research on other forecasting studies. In chapter 2, I pool data from previous studies to analyse the performance of prediction markets and surveys with higher statistical power. I confirm that prediction markets are better at forecasting replication outcomes than surveys. This study also demonstrates the relationship between p-values of original findings and replication outcomes. These findings are used to inform the experimental and statistical design to forecast the replicability of 3000 claims as part of the SCORE programme. A full description of the design including planned statistical analyses is included in chapter 3. Due to COVID-19 restrictions, our generated forecasts could not be validated through direct replication, experiments conducted by other teams within the SCORE collaboration, thereby preventing results being presented in this thesis. The completion of these replications is now scheduled for 2022, and the pre-analysis plan presented in Chapter 3 will provide the basis for the analysis of the resulting data. In chapter 4, an analysis of ‘meta’ forecasts, or forecasts regarding field wide replication rates and year specific replication rates, is presented. We presented and published community expectations that replication rates will differ by field and will increase over time. These forecasts serve as valuable insights into the academic community’s views of the replication crisis, including those research fields for which no large-scale replication studies have been undertaken yet. Once the full results from SCORE are available, there will be additional insights from validations of the community expectations. I also analyse forecaster’s ability to predict replications and effect sizes in Chapters 5 (Creative Destruction in Science) and 6 (A creative destruction approach to replication: Implicit work and sex morality across cultures). In these projects a ‘creative destruction’ approach to replication was used, where a claim is compared not only to the null hypothesis but to alternative contradictory claims. I conclude forecasters can predict the size and direction of effects. Chapter 7 examines the use of forecasting for scientific outcomes beyond replication. In the COVID-19 preprint forecasting project I find that forecasters can predict if a preprint will be published within one year, including the quality of the publishing journal. Forecasters can also predict the number of citations preprints will receive. This thesis demonstrates that information about scientific claims with respect to replicability is dispersed within scientific community. I have helped to develop methodologies and tools to efficiently elicit and aggregate forecasts. Forecasts about scientific outcomes can be used as guides to credibility, to gauge community expectations and to efficiently allocate sparse replication resources

THE IMPACT OF TECHNOLOGICAL CREATIVITY AND ENTREPRENEURSHIP EDUCATION ON THE ENTREPRENEURSHIP INTENTIONS OF STUDENTS AT PARTICULAR TERTIARY INSTITUTIONS IN ZIMBABWE AND SOUTH AFRICA

Pubished ThesisThis study explores the collective influence of technological creativity (TC) and exposure to entrepreneurship education (EE) on the entrepreneurship intentions (EI) of students at particular tertiary institutions in Zimbabwe and South Africa. Drawing on a positivist epistemology, a quantitative case study and total percentage analysis of tertiary education students who were randomly selected from the entrepreneurial programmes at the Central University of Technology, Free State (South Africa) and Kwekwe Polytechnic College (Zimbabwe), the study examines the extent to which these students intended to engage in entrepreneurship careers in the near future. Self-administered questionnaires were used to collect data from respondents. Out of a total of 400 questionnaires which were distributed to students enrolled for entrepreneurship courses at the two campuses, 284 were completed and returned, representing an overall response rate of 71%. A non-parametric test, the Mann-Whitney U test, was employed to establish whether there are any significant differences in the level of entrepreneurship intentions between South African and Zimbabwean tertiary education students. Another non-parametric technique, the Spearman Correlation, was employed to assess the relationships between technological creativity and entrepreneurship education on the one hand, and the direct determinants of entrepreneurship intentions and actual entrepreneurship intentions, on the other. The same technique was also used to assess the relationships between the immediate determinants of entrepreneurship intention and actual entrepreneurship intentions. Multiple regression analysis was conducted to test a number of predictive effects. Firstly, it was used to test the predictive effect of entrepreneurship education and technological creativity on the immediate determinants of entrepreneurship intentions. Secondly, it was used to assess the predictive effects of the immediate precursors of entrepreneurship intentions on the actual entrepreneurship intentions. Lastly, it tested the direct effects of entrepreneurship education and technological creativity on actual entrepreneurship intentions. The results demonstrate that although a majority of the Zimbabwean and South African respondents intended to engage in entrepreneurship in the near future, there were no significant differences in the levels of entrepreneurship intentions amongst both groups. Although some minimum variations were notable across the two groups, some positive and significant correlations between entrepreneurship education and technological creativity, on one hand, and the direct determinants of entrepreneurship intentions and actual entrepreneurship intentions, on the other, were revealed for both groups of students. The findings also provided strong support to the view that entrepreneurship education and technological creativity jointly predict the antecedents of entrepreneurship intention but have a non-significant direct relationship with actual entrepreneurship intentions. Lastly, all the immediate determinants of entrepreneurship intentions, except for subjective norms, significantly predict entrepreneurship intentions of South African and Zimbabwean tertiary students. To a large extent, the results validated the Theory of Planned Behaviour as a guiding tool for estimating any premeditated entrepreneurial behaviour. Thus, the Theory remains an invaluable theoretical lens for academics, educators and policy-makers’ evaluation of effective ways of enhancing the grooming of potential entrepreneurs. The theoretical contribution of this study lies in its introduction of a novel, previously untested construct, technological creativity, to the Theory of Planned Behaviour, to unravel the complexity of entrepreneurial intentions among tertiary students in a comparative country-based study. Based on the significant joint effects of entrepreneurship education and technological creativity on the direct determinants of entrepreneurship intentions, the study recommends the infusion of technological creativity into entrepreneurship education courses at tertiary institutions in Zimbabwe and South Africa

Foundations of Human-Aware Planning -- A Tale of Three Models

abstract: A critical challenge in the design of AI systems that operate with humans in the loop is to be able to model the intentions and capabilities of the humans, as well as their beliefs and expectations of the AI system itself. This allows the AI system to be "human- aware" -- i.e. the human task model enables it to envisage desired roles of the human in joint action, while the human mental model allows it to anticipate how its own actions are perceived from the point of view of the human. In my research, I explore how these concepts of human-awareness manifest themselves in the scope of planning or sequential decision making with humans in the loop. To this end, I will show (1) how the AI agent can leverage the human task model to generate symbiotic behavior; and (2) how the introduction of the human mental model in the deliberative process of the AI agent allows it to generate explanations for a plan or resort to explicable plans when explanations are not desired. The latter is in addition to traditional notions of human-aware planning which typically use the human task model alone and thus enables a new suite of capabilities of a human-aware AI agent. Finally, I will explore how the AI agent can leverage emerging mixed-reality interfaces to realize effective channels of communication with the human in the loop.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Pluralism in Action: Theories and Practices of Financial Markets

The aim of this dissertation is twofold: 1) To add to a meaningful conversation on financial markets by using less conventional, yet empirical, methods, which differ from the dominant statistical empirical methods; 2) To provide a case study of what methodological pluralism can look like: applying various methods and perspectives to one and the same subject or phenomenon. While the various chapters may appear somewhat unrelated, the thread throughout is the interplay of theory and practice in financial markets

Introducing v0.5 of the AI Safety Benchmark from MLCommons

This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-purpose assistant in English), and a limited set of personas (i.e., typical users, malicious users, and vulnerable users). We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0.5 benchmark. We plan to release version 1.0 of the AI Safety Benchmark by the end of 2024. The v1.0 benchmark will provide meaningful insights into the safety of AI systems. However, the v0.5 benchmark should not be used to assess the safety of AI systems. We have sought to fully document the limitations, flaws, and challenges of v0.5. This release of v0.5 of the AI Safety Benchmark includes (1) a principled approach to specifying and constructing the benchmark, which comprises use cases, types of systems under test (SUTs), language and context, personas, tests, and test items; (2) a taxonomy of 13 hazard categories with definitions and subcategories; (3) tests for seven of the hazard categories, each comprising a unique set of test items, i.e., prompts. There are 43,090 test items in total, which we created with templates; (4) a grading system for AI systems against the benchmark; (5) an openly available platform, and downloadable tool, called ModelBench that can be used to evaluate the safety of AI systems on the benchmark; (6) an example evaluation report which benchmarks the performance of over a dozen openly available chat-tuned language models; (7) a test specification for the benchmark

Crowdsourcing User-Centered Teams

Crowdsourcing has become an increasingly important tool for team formation and collaboration. This thesis investigates how User-Centered Design, an iterative process that prioritizes users and their needs, can be applied to improve the efficiency and effectiveness of crowdsourcing systems for teamwork and team formation. To achieve this, we conducted a series of studies to explore the role of various factors in shaping crowd workers' behaviour and preferences in collaborative contexts. The main findings of our research are as follows. In online team formation settings, crowd workers prefer disclosing overt traits (e.g., age, gender, topical interests) and avoid sharing sensitive information (e.g., ethnicity, depression). However, they are willing to share information regarding their personality and values, typically considered deep-level sensitive traits. Well-defined digital nudging interventions, such as a diversity progress bar, can promote diverse team formation. In contrast, subtler forms of nudging may inadvertently trigger biases working against the intended objectives. Ad-hoc crowd teams working under pressure can benefit from systems that account for differences in personality traits, as these can influence collaboration outcomes and perceptions. Designing crowdsourcing systems for emergency response requires modelling communication tools that aid, assist, and monitor the shared load, considering the strictly cooperative roles and task- and user-dependent communication styles between collaborators. When forming teams, crowd workers tend to balance attributes between and within groups, with a preference for Openness to Experience among the Big-5 personality traits. Based on these findings, we recommend applying a User-Centered approach to design collaborative crowdsourcing systems, considering user needs, behaviour, intents, and perceptions of digital environments. Future research should continue to explore and evaluate innovative strategies for promoting effective collaboration and team formation in crowdsourcing contexts