162 research outputs found
Plug-in healthcare:Development, ruination, and repair in health information exchange
This dissertation explores the work done by people and things in emerging infrastructures for health information exchange. It shows how this work relates to processes of development, production, and growth, as well as to abandonment, ruination, and loss. It argues for a revaluation of repair work: a form of articulation work that attends to gaps and disruptions in the margins of technological development. Often ignored by engineers, policy makers, and researchers, repair sensitizes us to different ways of caring for people and things that do not fit, fall in between categories, and resist social norms and conventions. It reminds us that infrastructures emerge in messy and unevenly distributed sociotechnical configurations, and that technological solutions cannot be simply ‘plugged in’ at will, but require all kinds of work. With that, repair emphasizes the need for more democratic, critical, and reflexive engagements with (and interventions in) health information exchange. Empirically, this study aims to understand how ‘integration’ in health information exchange is done in practice, and to develop concepts and insights that may help us to rethink technological development accordingly. It starts from the premise that the introduction of IT in healthcare is all too often regarded as a neutral process, and as a rational implementation challenge. These widespread views among professionals, managers, and policy makers need to be addressed, as they have very real – and mostly undesirable – consequences. Spanning a period of more than ten years, this study traces the birth and demise of an online regional health portal in the Netherlands (2009-2019). Combining ethnographic research with an experimental form of archive work, it describes sociotechnical networks that expanded, collapsed, and reconfigured around a variety of problems – from access to information and data ownership to business cases, financial sustainability, and regional care. It puts a spotlight on the integration of standards, infrastructures, and users in the portal project, and on elements of collapsing networks that quietly resurfaced elsewhere. The reconstruction of these processes foregrounds different instances of repair work in the portal’s development and subsequent abandonment, repurposing, and erasure. Conceptually, this study contributes to academic debates in health information exchange, including the politics of technology, practices of participatory design, and the role of language in emerging information infrastructures. It latches on to ethnographic studies on information systems and infrastructural work, and brings together insights from actor-network theory, science and technology studies, and figurational sociology to rethink and extend current (reflexive and critical) understandings of technological development. It raises three questions: What work is done in the development and demise of an online health portal? How are relations between people and things shaped in that process? And how can insights from this study help us to understand changing sociotechnical figurations in health information exchange? The final analysis includes five key concepts: the act of building network extensions, the method of tracing phantom networks, the notion of sociotechnical figurations, the logic of plug-in healthcare, and repair as a heuristic device.<br/
Improving the validity and usability of decision models: case studies with a focus on physical activity
Background: Health economic evaluation has a crucial role to play in the allocation of scarce societal resources. Economic models used in these evaluations must have a high degree of external validity but must also be usable in order to effectively inform policy. However, there is sometimes a trade-off between the realism of models (external validity) and the ease with which stakeholders can understand and interact with them (usability). This trade-off is particularly relevant in the field of physical activity where modelling is complicated and data availability is limited. The aim of this thesis is to investigate the balance between the external-validity and usability of models used in health economic evaluations of physical activity interventions and develop ways to build models that are more externally valid and usable.
Methods: The study begins by identifying limitations in the external-validity and usability of published physical activity models, with a particular focus on models used to inform National Institute for Health and Care Excellence (NICE) guidance. Three case studies of adaptations to improve external validity are provided, with a discussion of their implications for usability. Additionally, ways to improve the usability of models are examined, with methods proposed to make models more accessible, transparent, secure, and efficient to construct and maintain.
Results: The results of this thesis demonstrate that models can be improved in terms of both external validity and/or usability. The case studies provided show that methodological developments to physical activity models are feasible given new modelling methods and advancements in computing power, but despite improving external validity may reduce usability. Additionally, this thesis outlines methods by which health economic models can be made more accessible, transparent, secure, and efficient to construct and maintain, thereby improving their usability.
Discussion: The overall conclusion of this thesis is that economic evaluation models should be as externally valid and usable as possible. However, a trade-off sometimes exists between the two. With a fixed budget for evaluation, attempts to improve external validity can have an opportunity cost in terms of resources allocated to making models easy to use and understand. The incorporation of methods from computing and data science can help mitigate this trade-off
Understanding Users' Dissatisfaction with ChatGPT Responses: Types, Resolving Tactics, and the Effect of Knowledge Level
Large language models (LLMs) with chat-based capabilities, such as ChatGPT,
are widely used in various workflows. However, due to a limited understanding
of these large-scale models, users struggle to use this technology and
experience different kinds of dissatisfaction. Researchers have introduced
several methods such as prompt engineering to improve model responses. However,
they focus on crafting one prompt, and little has been investigated on how to
deal with the dissatisfaction the user encountered during the conversation.
Therefore, with ChatGPT as the case study, we examine end users'
dissatisfaction along with their strategies to address the dissatisfaction.
After organizing users' dissatisfaction with LLM into seven categories based on
a literature review, we collected 511 instances of dissatisfactory ChatGPT
responses from 107 users and their detailed recollections of dissatisfied
experiences, which we release as a publicly accessible dataset. Our analysis
reveals that users most frequently experience dissatisfaction when ChatGPT
fails to grasp their intentions, while they rate the severity of
dissatisfaction the highest with dissatisfaction related to accuracy. We also
identified four tactics users employ to address their dissatisfaction and their
effectiveness. We found that users often do not use any tactics to address
their dissatisfaction, and even when using tactics, 72% of dissatisfaction
remained unresolved. Moreover, we found that users with low knowledge regarding
LLMs tend to face more dissatisfaction on accuracy while they often put minimal
effort in addressing dissatisfaction. Based on these findings, we propose
design implications for minimizing user dissatisfaction and enhancing the
usability of chat-based LLM services
The Archive Query Log: Mining Millions of Search Result Pages of Hundreds of Search Engines from 25 Years of Web Archives
The Archive Query Log (AQL) is a previously unused, comprehensive query log
collected at the Internet Archive over the last 25 years. Its first version
includes 356 million queries, 166 million search result pages, and 1.7 billion
search results across 550 search providers. Although many query logs have been
studied in the literature, the search providers that own them generally do not
publish their logs to protect user privacy and vital business data. Of the few
query logs publicly available, none combines size, scope, and diversity. The
AQL is the first to do so, enabling research on new retrieval models and
(diachronic) search engine analyses. Provided in a privacy-preserving manner,
it promotes open research as well as more transparency and accountability in
the search industry.Comment: SIGIR 2023 resource paper, 13 page
Building bridges for better machines : from machine ethics to machine explainability and back
Be it nursing robots in Japan, self-driving buses in Germany or automated hiring systems in the USA, complex artificial computing systems have become an indispensable part of our everyday lives. Two major challenges arise from this development: machine ethics and machine explainability. Machine ethics deals with behavioral constraints on systems to ensure restricted, morally acceptable behavior; machine explainability affords the means to satisfactorily explain the actions and decisions of systems so that human users can understand these systems and, thus, be assured of their socially beneficial effects. Machine ethics and explainability prove to be particularly efficient only in symbiosis. In this context, this thesis will demonstrate how machine ethics requires machine explainability and how machine explainability includes machine ethics. We develop these two facets using examples from the scenarios above. Based on these examples, we argue for a specific view of machine ethics and suggest how it can be formalized in a theoretical framework. In terms of machine explainability, we will outline how our proposed framework, by using an argumentation-based approach for decision making, can provide a foundation for machine explanations. Beyond the framework, we will also clarify the notion of machine explainability as a research area, charting its diverse and often confusing literature. To this end, we will outline what, exactly, machine explainability research aims to accomplish. Finally, we will use all these considerations as a starting point for developing evaluation criteria for good explanations, such as comprehensibility, assessability, and fidelity. Evaluating our framework using these criteria shows that it is a promising approach and augurs to outperform many other explainability approaches that have been developed so far.DFG: CRC 248: Center for Perspicuous Computing; VolkswagenStiftung: Explainable Intelligent System
The Effects of a Brief Epistemic Cognition and Metacognition Intervention on the Continued Influence Effect
Individuals rely on accurate information to make important decisions, but in the current environment the vast amount of misinformation present in society is complicating people’s thinking. Many people fall prey to a cognitive bias called the continued influence effect, which occurs when they continue to use misinformation even when they have seen and can acknowledge a correction of the inaccurate messaging. Researchers have started to examine this phenomenon in the context of socioscientific issues such as vaccination, but it is not apparent that it occurs when people engage with less politicized topics. Debunking interventions have also largely been ineffective at helping people avoid the bias. In this dissertation, I conducted three studies examining if people exhibited the continued influence effect when dealing with misinformation about the topic of antioxidant supplements. In the first study, novel materials were reviewed by ten individuals who provided feedback on their accessibility and clarity. In the second study, a randomized control trial (n = 440), the continued influence effect was not detected, but the manipulation of beliefs by misinformation was. After revising the materials, a third study was conducted (n = 572) to examine the efficacy of a prebunking epistemic cognition and metacognition intervention at attenuating the occurrence of the continued influence effect. Again, the bias was not detected. The findings indicated that the continued influence effect may only occur with more politicized and controversial socioscientific issues.Doctor of Philosoph
Mapping the intuitive investigation: Seeking, evaluating and explaining the evidence
The human mind has developed numerous cognitive tools to allow us to navigate the uncertainty of the world and make sense of situations and events. In this thesis I present a descriptive account of some of these tools by probing people’s ability to: evaluate, seek, and explain evidence and information. This was achieved by appraising people’s behaviour in controlled experiments – predominantly representing legal-investigative scenarios – utilising normative causal models (e.g., causal Bayesian networks), and uncovering the alternative strategies that people employed when reasoning under uncertainty.
In Chapter 4, I investigate people’s ability to engage in a pattern of reasoning termed ‘explaining away’ and propose, and find empirical support towards, intuitive theories that address why the observed inference errors were made. In Chapter 5, I outline how people search for, and evaluate, evidence in a sequential investigative information-seeking paradigm – finding that people do not seek information simply to maximize a given utility function but rather are driven by additional strategies which are sensitive to factors such as demands of the task and a novel form of risk aversion. I extend these findings to forensic professionals, and utilise a naturalistic study employing mobile eye-trackers during a mock crime scene investigation to elucidate the key role that ‘asking the right questions’ plays when engaging in sense-making practices ‘in the wild’.
In Chapter 6, I explore people’s preferences for certain types of information relating to opportunity and motive at various stages of the legal-investigative process. Here, I demonstrate that people prefer ‘motive’ accounts of crimes (analogous to a teleology preference) at different stages of the investigative process. In an additional two studies I demonstrate that these preferences are context-sensitive: namely, that ‘motive’ information tends to be moreincriminating and less exculpatory. In a final set of experiments, outlined in Chapter 7, I investigate how drawing causal models of competing explanations of the evidence affects how these same explanations are evaluated – arguing that graphically representing the evidence bolsters people’s understanding of the probabilistic and logical significance of the causal structures drawn.
In sum, this thesis provides a rich descriptive account of how people engage in various aspects of sense-making and decision-making under uncertainty. The work presented in this thesis ultimately aims to increase the ecological and descriptive validity of normative causal frameworks utilised in the cognitive sciences – whilst informing ways to formalise decision-making practices in real-world specialised domains
Unwrapping DIY enquiry: The study of 'enquiry' in DIY practice at individual, community & place levels
Do-It-Yourself (DIY) enquiry represents ownership over learning and action: figuring things
out by oneself, experimenting, and questioning the state of things to find potential solutions
to local concerns. It is an identifiable collective behaviour of self-reliance exhibited
throughout our history but in the digital age and in societies with increasing levels of
education, the way DIY practice unfolds is little understood. Traditional studies on public
engagement in science and technology and perspectives on production of knowledge and
technology have focused primarily on institutionally mediated methods of public
participation and the validity of public contributions to established fields. This thesis research
makes empirical, theoretical, and methodological contributions: using a multi-method
approach and grounded theory for qualitative data analysis to explore DIY enquiry in
practice, community, and place. The three in-depth case studies explore the nature of the
production of knowledge, the role of technologies, and the barriers and opportunities to
public engagement in DIY enquiry. Participant observation of a community of DIY practice
reveals its inner processes, interactions, and framings of science and technology and how
DIY practice is performed through DIY tool use and development. The design and
facilitation of a DIY workshop series demonstrates the initial stages of engagement in DIY
enquiry and reveals that barriers and opportunities to engagement are mediated by frame of
mind, setting, facilitation, and interactions. The observation of place-based citizen initiatives
of DIY enquiry reveals its range of interconnected actions: development of techniques and
strategies for tool development, data interpretation, and leveraging of knowledge and stance
for advocacy. Together the cases reveal the transformative power of DIY enquiry, how it
builds knowledge, culture, and identity and that engagement requires curiosity, courage,
commitment, and foundational competencies. They also reveal an inherent tension between
DIY enquiry framed as a means (seeking collective/organised actionable goals) and as an end
(enabling personal empowerment). This research facilitates a better understanding of the
democratic potential of public engagement in science in our time but it also promotes the
leveraging of knowledge production between professional/institutional science and civil
society
- …