Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making

Today, AI is being increasingly used to help human experts make decisions in high-stakes scenarios. In these scenarios, full automation is often undesirable, not only due to the significance of the outcome, but also because human experts can draw on their domain knowledge complementary to the model's to ensure task success. We refer to these scenarios as AI-assisted decision making, where the individual strengths of the human and the AI come together to optimize the joint decision outcome. A key to their success is to appropriately \textit{calibrate} human trust in the AI on a case-by-case basis; knowing when to trust or distrust the AI allows the human expert to appropriately apply their knowledge, improving decision outcomes in cases where the model is likely to perform poorly. This research conducts a case study of AI-assisted decision making in which humans and AI have comparable performance alone, and explores whether features that reveal case-specific model information can calibrate trust and improve the joint performance of the human and AI. Specifically, we study the effect of showing confidence score and local explanation for a particular prediction. Through two human experiments, we show that confidence score can help calibrate people's trust in an AI model, but trust calibration alone is not sufficient to improve AI-assisted decision making, which may also depend on whether the human can bring in enough unique knowledge to complement the AI's errors. We also highlight the problems in using local explanation for AI-assisted decision making scenarios and invite the research community to explore new approaches to explainability for calibrating human trust in AI

Equilibrium Values in a Competitive Power Exchange Market

We consider an open electricity market with demand uncertainty.In this market, the generators each decide on a bidding price tomaximize profit. Units are dispatched in order of the bid from lowestto highest until demand is satisfied. The market clearing price isthe highest bid among the dispatched units.All dispatched units are then sold at this market clearing price.Under a market stability assumption, we derive Nashequilibrium solutions, i.e., bidders' optimal bidding strategiesand the resulting market clearing price.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/44347/1/10614_2004_Article_250744.pd

Quantifying the impact of climate change on drought regimes using the Standardised Precipitation Index

The study presents a methodology to characterise short- or long-term drought events, designed to aid understanding of how climate change may affect future risk. An indicator of drought magnitude, combining parameters of duration, spatial extent and intensity, is presented based on the Standardised Precipitation Index (SPI). The SPI is applied to observed (1955–2003) and projected (2003–2050) precipitation data from the Community Integrated Assessment System (CIAS). Potential consequences of climate change on drought regimes in Australia, Brazil, China, Ethiopia, India, Spain, Portugal and the USA are quantified. Uncertainty is assessed by emulating a range of global circulation models to project climate change. Further uncertainty is addressed through the use of a high-emission scenario and a low stabilisation scenario representing a stringent mitigation policy. Climate change was shown to have a larger effect on the duration and magnitude of long-term droughts, and Australia, Brazil, Spain, Portugal and the USA were highlighted as being particularly vulnerable to multi-year drought events, with the potential for drought magnitude to exceed historical experience. The study highlights the characteristics of drought which may be more sensitive under climate change. For example, on average, short-term droughts in the USA do not become more intense but are projected to increase in duration. Importantly, the stringent mitigation scenario had limited effect on drought regimes in the first half of the twenty-first century, showing that adaptation to drought risk will be vital in these regions

Death associated protein-3 (DAP3) and DAP3 binding cell death enhancer-1 (DELE1) in human colorectal cancer, and their impacts on clinical outcome and chemoresistance

Death associated protein‑3 (DAP3) was identified as a responsive protein to interferon‑gamma‑induced cell death which possibly exerts this regulation by interacting with DAP3 binding cell Death enhancer‑1 (DELE1), a newly discovered mitochondrial stress protein in response to cell stress signals. Whilst DAP3 has been shown to be aberrantly expressed in several cancer types (i.e. breast cancer), little is known about the relationship between DAP3 and DELE1 in cancers. The present study examined the expression levels of both DAP3 and DELE1 in clinical colorectal cancers (CRCs), as well as their implication on chemoresistance and mechanism behind the action. Firstly, transcript levels of both DAP3 and DELE1 were quantitatively assessed in a clinical cohort of CRC (n=94). Tumour tissues had significantly higher levels of DAP3, but not DELE1 compared with normal tissues. Levels of DAP3 and DELE1 had a significant association with patient's clinical outcomes and local recurrence. DAP3 and DELE1 significantly correlated in normal colorectal tissues but not in tumour tissues. Secondly, the protein levels of DAP3 and DELE1 were evaluated in both normal and tumour colon tissues which showed that both proteins were highly aberrant in CRC tissues. In addition, both DAP3 and DELE1 at transcript and protein levels were identified as prognostic factors for patient's clinical outcomes. Furthermore, in in vitro assays, knocking down DAP3 or DELE1, and in particular both DAP3 and DELE1 together rendered the CRC cells more sensitive to chemotherapy drugs, consistent with clinical findings of the TCGA‑COAD datasets. The acquisition of drug sensitivity following the genetic knockdown was independent of the mitochondrial metabolism, as neither DAP3 knockdown nor DELE1 knockdown showed a significant change. In summary, DAP3 and DELE1 are highly aberrant in CRCs, and both molecules are prognostic factors for patient's clinical outcomes and local recurrence and are indicators for chemoresistance

An assessment of the quality of randomised controlled trials conducted in China

Background: Despite the rapid increase in research in China, little is known about the quality of clinical trials conducted there.Methods: A systematic review and critical appraisal of randomised controlled trials (RCTs) conducted in China and published in 2004 was undertaken to describe their characteristics, assess the quality of their reporting, and where possible, the quality of their conduct. Randomised controlled trials in all disease areas and types of interventions, which took place in China and included Chinese citizens were identified using PubMed and hand searching the Journal Series of the Chinese Medical Association. Quality was assessed against a subset of criteria adapted from the CONSORT statement.Results: Three hundred and seven RCTs were included. One hundred and ninety-nine (64.8%) failed to report methods of randomization and 254 (82.4%) did not mention blinding of either participants or investigators. Reporting of baseline characteristics, primary outcome and length of follow-up was inadequate in a substantial proportion of studies. Fewer than 11% of RCTs mentioned ethical approval and only 18.0% adequately discussed informed consent. However, dropout rates were very favourable with nearly 44% of trials reporting a zero dropout rate.Conclusion: Reporting of RCTs in China requires substantial improvement to meet the targets of the CONSORT statement. The conduct of Chinese RCTs cannot be directly inferred from the standard of reporting; however without good reporting the methods of the trials cannot be clearly ascertained

Differential activation of inflammatory pathways in A549 type II pneumocytes by Streptococcus pneumoniae strains with different adherence properties

BACKGROUND: Adherence of Streptococcus pneumoniae bacteria to lung cells is a first step in the progression from asymptomatic carriage to pneumonia. Adherence abilities vary widely among S. pneumoniae patient isolates. In this study, the binding properties of S. pneumoniae isolates and the effects of binding on activation of the Nuclear Factor-Kappa-B (NFκB) pathway and cytokine secretion by type II pneumocytes were measured. METHODS: Mechanisms of high- and low-binding S. pneumoniae adherence to A549 cells were investigated by blocking putative receptors on bacteria and host cells with antibody and by eluting choline-binding proteins off of bacterial surfaces. NFκB activation was measured by western blot and immunocytochemistry and cytokine secretion was detected by a protein array. RESULTS: This study shows that S. pneumoniae isolates from pneumonia patients (n = 298) can vary by as much as 1000-fold in their ability to bind to human lung epithelial cells. This difference resulted in differential activation of the NFκB pathway. High-, but not low-binding S. pneumoniae used Choline-binding protein A (CbpA) to bind to complement component C3 on epithelial cell surfaces. Interleukin-8 (IL-8) was the only cytokine secreted by cells treated with either low- or high-binding S. pneumoniae. CONCLUSION: These results indicate that S. pneumoniae clinical isolates are not homogeneous in their interaction with host epithelial cells. The differential activation of host cells by high- and low-binding S. pneumoniae strains could have implications for the treatment of pneumococcal pneumonia and for vaccine development