Image quality assessment for machine learning tasks using meta-reinforcement learning

In this paper, we consider image quality assessment (IQA) as a measure of how images are amenable with respect to a given downstream task, or task amenability. When the task is performed using machine learning algorithms, such as a neural-network-based task predictor for image classification or segmentation, the performance of the task predictor provides an objective estimate of task amenability. In this work, we use an IQA controller to predict the task amenability which, itself being parameterised by neural networks, can be trained simultaneously with the task predictor. We further develop a meta-reinforcement learning framework to improve the adaptability for both IQA controllers and task predictors, such that they can be fine-tuned efficiently on new datasets or meta-tasks. We demonstrate the efficacy of the proposed task-specific, adaptable IQA approach, using two clinical applications for ultrasound-guided prostate intervention and pneumonia detection on X-ray images

The impact of inconsistent human annotations on AI driven clinical decision making

In supervised learning model development, domain experts are often used to provide the class labels (annotations). Annotation inconsistencies commonly occur when even highly experienced clinical experts annotate the same phenomenon (e.g., medical image, diagnostics, or prognostic status), due to inherent expert bias, judgments, and slips, among other factors. While their existence is relatively well-known, the implications of such inconsistencies are largely understudied in real-world settings, when supervised learning is applied on such ‘noisy’ labelled data. To shed light on these issues, we conducted extensive experiments and analyses on three real-world Intensive Care Unit (ICU) datasets. Specifically, individual models were built from a common dataset, annotated independently by 11 Glasgow Queen Elizabeth University Hospital ICU consultants, and model performance estimates were compared through internal validation (Fleiss’ κ = 0.383 i.e., fair agreement). Further, broad external validation (on both static and time series datasets) of these 11 classifiers was carried out on a HiRID external dataset, where the models’ classifications were found to have low pairwise agreements (average Cohen’s κ = 0.255 i.e., minimal agreement). Moreover, they tend to disagree more on making discharge decisions (Fleiss’ κ = 0.174) than predicting mortality (Fleiss’ κ = 0.267). Given these inconsistencies, further analyses were conducted to evaluate the current best practices in obtaining gold-standard models and determining consensus. The results suggest that: (a) there may not always be a “super expert” in acute clinical settings (using internal and external validation model performances as a proxy); and (b) standard consensus seeking (such as majority vote) consistently leads to suboptimal models. Further analysis, however, suggests that assessing annotation learnability and using only ‘learnable’ annotated datasets for determining consensus achieves optimal models in most cases

Disruption analytics in urban metro systems with large-scale automated data

Urban metro systems are frequently affected by disruptions such as infrastructure malfunctions, rolling stock breakdowns and accidents. Such disruptions give rise to delays, congestion and inconvenience for public transport users, which in turn, lead to a wider range of negative impacts on the social economy and wellbeing. This PhD thesis aims to improve our understanding of disruption impacts and improve the ability of metro operators to detect and manage disruptions by using large-scale automated data. The crucial precondition of any disruption analytics is to have accurate information about the location, occurrence time, duration and propagation of disruptions. In pursuit of this goal, the thesis develops statistical models to detect disruptions via deviations in trains’ headways relative to their regular services. Our method is a unique contribution in the sense that it is based on automated vehicle location data (data-driven) and the probabilistic framework is effective to detect any type of service interruptions, including minor delays that last just a few minutes. As an important research outcome, the thesis delivers novel analyses of the propagation progress of disruptions along metro lines, thus enabling us to distinguish primary and secondary disruptions as well as recovery interventions performed by operators. The other part of the thesis provides new insights for quantifying disruption impacts and measuring metro vulnerability. One of our key messages is that in metro systems there are factors influencing both the occurrence of disruptions and their outcomes. With such confounding factors, we show that causal inference is a powerful tool to estimate unbiased impacts on passenger demand and journey time, which is also capable of quantifying the spatial-temporal propagation of disruption impacts within metro networks. The causal inference approaches are applied to empirical studies based on the Hong Kong Mass Transit Railway (MTR). Our conclusions can assist researchers and practitioners in two applications: (i) the evaluation of metro performance such as service reliability, system vulnerability and resilience, and (ii) the management of future disruptions.Open Acces

Conceptualizing and Measuring Well-Being Using Statistical Semantics and Numerical Rating Scales

How to define and measure individuals’ well-being is important, as this has an impact on both research and society at large. This thesis concerns how to define and measure the self-reported well-being of individuals, which involves both theorizing as well as developing and applying empirical and statistical methods in order to gain a better understanding of well-being.The first paper critically reviews the literature on well-being. It identifies an individualistic bias in current approaches and accompanying measures related to well-being and happiness; for example, through an over-emphasis on the importance of self-centered aspects of well-being (e.g., the unprecedented focus on satisfaction with life) whilst disregarding the importance of harmony in life, interconnectedness and psychological balance in relation to well- being. It is also discussed how closed-ended well-being measures impose the researchers’ values and limit the ability of respondents to express themselves in regard to their perceived well-being.The second paper addresses concerns regarding this individualistic bias by developing the harmony in life scale, which focuses on interconnectedness and psychological balance. In addition, an open-ended approach is developed in the paper, allowing individuals to freely describe their pursuit of well-being by means of open-ended responses analyzed using statistical semantics (including techniques from artificial intelligence such as natural language processing and machine learning). The results show that the harmony in life scale and the traditional satisfaction with life scale form a two-factor model of well-being, where the harmony in life scale explains more unique variance in measures of psychological well-being, stress, depression and anxiety, but not happiness. It is further demonstrated that participants describe their pursuit of harmony in life using words related to interconnectedness (including words such as: peace, balance, cooperation), whereas they describe their pursuit of satisfaction with life using words related to independence (including words such as: money, achievement, fulfillment). It is concluded that the harmony in life scale complements the satisfaction with life scale for a more comprehensive understanding of subjective well-being.The third paper focuses on developing and evaluating a method for measuring and describing psychological constructs using open-ended questions analyzed by means of statistical semantics rather than closed-ended numerical rating scales. This semantic measures approach is tested and compared with traditional rating scales in nine studies, including two different paradigms involving reports regarding objective stimuli (i.e., the evaluation of facial expressions) and reports regarding subjective states (i.e., the self-reporting of harmony in life, satisfaction with life, depression and worry). The results indicate that semantic measures encompass higher, or competitive, levels of reliability and validity compared to traditional numerical rating scales. In addition, semantic measures appear to be better suited for differentiating between psychological constructs, such as harmony in life versus satisfaction with life as well as depression versus worry.In this thesis, the findings from these three papers are elaborated and integrated into two independent perspectives. The first perspective focuses on the theoretical and empirical differences between harmony in life and satisfaction with life within a context of societal and national progress. It is concluded that harmony in life complements satisfaction with life. The second perspective focuses on the open-ended, statistical semantics approach. It is proposed that statistical semantics may beneficially be used more widely as a research tool within psychological research

A new low-cost technique improves weather forecasts across the world

Computer-generated forecasts divide the earth's surface into gridboxes, each now ~25% of the size of London, and predict one value per gridbox. If weather varies markedly within a gridbox forecasts for specific sites inevitably fail. A completely new statistical post-processing method, using ensemble forecasts as input, anticipates two gridbox-weather-dependant factors: degree of variation in each gridbox, and bias on the gridbox scale. Globally, skill improves substantially; for extreme rainfall, for example, useful forecasts extend 5 days ahead. Without post-processing this limit is < 1 day. Relative to historical forecasting advances this constitutes ground-breaking progress. The key drivers, incorporated during calibration, are meteorological understanding and abandoning classical notions that only local data be used. Instead we simply recognise that "showers are showers, wherever they occur worldwide" which delivers a huge increase in calibration dataset size. Numerous multi-faceted applications include improved flash flood warnings, physics-related insights into model weaknesses and global pointwise re-analyses.Comment: 27 pages, 9 figures. Submitted to Natur

Supplier Selection and Relationship Management: An Application of Machine Learning Techniques

Managing supply chains is an extremely challenging task due to globalization, short product life cycle, and recent advancements in information technology. These changes result in the increasing importance of managing the relationship with suppliers. However, the supplier selection literature mainly focuses on selecting suppliers based on previous performance, environmental and social criteria and ignores supplier relationship management. Moreover, although the explosion of data and the capabilities of machine learning techniques in handling dynamic and fast changing environment show promising results in customer relationship management, especially in customer lifetime value, this area has been untouched in the upstream side of supply chains. This research is an attempt to address this gap by proposing a framework to predict supplier future value, by incorporating the contract history data, relationship value, and supply network properties. The proposed model is empirically tested for suppliers of public works and government services Canada. Methodology wise, this thesis demonstrates the application of machine learning techniques for supplier selection and developing effective strategies for managing relationships. Practically, the proposed framework equips supply chain managers with a proactive and forward-looking approach for managing supplier relationship