Conversational Sensing

Recent developments in sensing technologies, mobile devices and context-aware user interfaces have made it possible to represent information fusion and situational awareness as a conversational process among actors - human and machine agents - at or near the tactical edges of a network. Motivated by use cases in the domain of security, policing and emergency response, this paper presents an approach to information collection, fusion and sense-making based on the use of natural language (NL) and controlled natural language (CNL) to support richer forms of human-machine interaction. The approach uses a conversational protocol to facilitate a flow of collaborative messages from NL to CNL and back again in support of interactions such as: turning eyewitness reports from human observers into actionable information (from both trained and untrained sources); fusing information from humans and physical sensors (with associated quality metadata); and assisting human analysts to make the best use of available sensing assets in an area of interest (governed by management and security policies). CNL is used as a common formal knowledge representation for both machine and human agents to support reasoning, semantic information fusion and generation of rationale for inferences, in ways that remain transparent to human users. Examples are provided of various alternative styles for user feedback, including NL, CNL and graphical feedback. A pilot experiment with human subjects shows that a prototype conversational agent is able to gather usable CNL information from untrained human subjects

Systematic review and narrative synthesis of surgeons’ perception of postoperative outcomes and risk

Background The accuracy with which surgeons can predict outcomes following surgery has not been explored in a systematic way. The aim of this review was to determine how accurately a surgeon's ‘gut feeling’ or perception of risk correlates with patient outcomes and available risk scoring systems. Methods A systematic review was undertaken in accordance with PRISMA guidelines. A narrative synthesis was performed in accordance with the Guidance on the Conduct of Narrative Synthesis In Systematic Reviews. Studies comparing surgeons' preoperative or postoperative assessment of patient outcomes were included. Studies that made comparisons with risk scoring tools were also included. Outcomes evaluated were postoperative mortality, general and operation‐specific morbidity and long‐term outcomes. Results Twenty‐seven studies comprising 20 898 patients undergoing general, gastrointestinal, cardiothoracic, orthopaedic, vascular, urology, endocrine and neurosurgical operations were included. Surgeons consistently overpredicted mortality rates and were outperformed by existing risk scoring tools in six of seven studies comparing area under receiver operating characteristic (ROC) curves (AUC). Surgeons' prediction of general morbidity was good, and was equivalent to, or better than, pre‐existing risk prediction models. Long‐term outcomes were poorly predicted by surgeons, with AUC values ranging from 0·51 to 0·75. Four of five studies found postoperative risk estimates to be more accurate than those made before surgery. Conclusion Surgeons consistently overestimate mortality risk and are outperformed by pre‐existing tools; prediction of longer‐term outcomes is also poor. Surgeons should consider the use of risk prediction tools when available to inform clinical decision‐making. Introduction Surgical procedures all carry associated risks. It is therefore important that surgeons are able to make accurate predictions of potential benefit and risk, including immediate mortality and morbidity, as well as long‐term outcomes, to enable balanced decision‐making and fully informed consent. Risks can also be estimated after surgery, based on additional perioperative and intraoperative data, which allows contemporary prediction of outcome. There are numerous risk prediction models that enable the surgeon to quantify risk based on measurable parameters1-5. However, there are inherent limitations in using a generalized risk prediction model, which may not include clinical data pertinent to the individual case in question, leading to variability in model accuracy6-10. As a result, risk prediction tools are generally used in tandem with the surgeon's ‘gut feeling’ of overall risk and anticipated outcome (‘clinical gestalt’). Several disparate factors influence surgeons' perception of outcome: patient factors, such as their perceived fitness, their pathology and planned procedure; setting factors, such as the experience of other members of staff; and surgeon factors, such as clinical knowledge, operative skill, previous significant surgical complications, and inclinations and attitudes11-13. Anticipating surgical risk is subject to multiple biases, which make it challenging. These include the natural tendency toward anecdotal recall and the availability heuristic (the likelihood of making a decision based on how easily the topic or examples come to mind)14, 15. Some studies16-18 support the accuracy and reproducibility of surgeons' predictions, whereas others19-22 demonstrate less favourable results. The complexity of synthesizing risk perceptions is significant and incompletely understood23, 24. The accuracy of surgeons' prediction has not been explored previously in a systematic manner. The aim of this review was thus to determine, from the available evidence, whether a surgeon's gut feeling or perception of risk correlates with postoperative outcomes, and to compare this prediction with currently available risk scoring systems, where available. Methods This systematic review was undertaken in accordance with the PRISMA guidelines25, 26. MEDLINE (via PubMed), Embase, the Cochrane Library Database, and the Cochrane Collaboration Central Register of Controlled Clinical Trials were searched with no date or language restrictions, with the last search date on 9 July 2018. The search term used was (‘Surgeons’[Mesh] OR ‘General Surgery/manpower*’ [MeSH]) AND (‘perception’ OR ‘intuition’ OR ‘predict*’ OR ‘decision making’ [mesh]). There was no restriction on publication type. This search was complemented by an exhaustive review of the bibliography of key articles, and also by using the Related Articles function in PubMed of included papers. Results were restricted to human research published in English. Inclusion and exclusion criteria All studies of patients undergoing surgery in which a preoperative or postoperative surgeon assessment (or proxy assessment) of a postoperative outcome was performed were included. This included articles that reported general risk (such as mortality) or a surgery‐specific risk (for example anastomotic leakage). Studies that made comparisons with established risk scoring tools were also included. Papers or abstracts in English, or non‐English papers with an English abstract, were included. Papers describing the risk assessment of ‘theoretical’ cases, or patient vignettes in a situation distant from clinical practice (such as a conference), were excluded, as were studies in which surgeons' assessment of risk was compared with an established risk scoring tool, without data on actual patient outcome. Data extraction and assessment of study quality Three authors independently extracted data and assessed the methodological quality of the studies, with all data extraction independently checked by the senior author. The following baseline data were extracted from each study: first author, year of publication, data collection period, geographical location, study design and type (single or multiple centres, number of surgeons involved in risk estimation, whether consecutive patients were enrolled), surgical specialty, whether other risk scoring systems were used for comparison and, if so, whether the assessor was blinded to this result. Data extracted regarding the assessment of risk included: risk outcome assessed; timing of risk estimation (preoperative or postoperative); type of risk assessment by surgeons (qualitative, quantitative, continuous scale such as a visual analogue scale (VAS), or composite score); absolute value of risk event predicted by surgeon and by scoring system; absolute value of risk occurrence rate; summary data on outcome reported, including area under the curve (AUC) of receiver operating characteristic (ROC) curves, observed : expected (O : E) or predicted : observed (P : O) ratios, or any other summary data. When data were available, AUCs were extracted with their 95 per cent confidence intervals. AUCs greater than 0·9 were considered as indicating high performance, 0·7–0·9 as moderate performance, 0·5–0·7 as low performance, and less than 0·5 as indicating risk assessment no better than chance alone27, 28. Risk predictions made by pre‐existing tools, such as the Physiological and Operative Severity Score for the enumeration of Mortality and morbidity (POSSUM)1, Portsmouth‐POSSUM (P‐POSSUM)4 or Continuous Improvement in Cardiac Surgery Program (CICSP)5, were compared with outcome when given. Internal prediction models, where authors would derive significant predictive co‐variables from their data set and assess the accuracy of these co‐variables within the same data set, were not evaluated as they lacked validity. Study quality was assessed using the Newcastle–Ottawa (NO) score29, 30. The NO score assigns points based on: the quality of patient selection (maximum 4 points); comparability of the cohort (maximum 2 points); and outcome assessment (maximum 3 points). Studies that scored 6 points or more were considered to be of higher quality. Outcome measures The following outcome measures were defined a priori and refined during data extraction: postoperative mortality (usually defined as 30 days after surgery); postoperative general morbidity (usually defined as 30 days after surgery); postoperative procedure‐specific morbidity; and long‐term outcome (typically operation‐specific). Further comparative analyses of outcomes included comparison of preoperative and postoperative predictions, and of predictions made by consultants and surgical trainees. Narrative synthesis Given the marked heterogeneity in study design, patient population included, method of assessing risk and outcomes assessed, meta‐analysis was deemed not appropriate. A narrative synthesis was therefore performed according to the Guidance on the Conduct of Narrative Synthesis In Systematic Reviews31. Three authors systematically summarized each article using bullet points to document key aspects of each study, focusing particularly on methods used and results obtained. The validity and certainty of the results were noted (whether appropriate statistical comparisons were used and, if so, their effect size and significance). The senior author identified and grouped common themes, divided larger themes into subthemes, tabulated a combined summary of the paper, and synthesized a common rubric for each theme. Consolidated reviewers' comments can be found in Table S1 (supporting information). Results A total of 584 articles were identified from the literature search, of which 48 were retrieved for evaluation. Papers were excluded on the basis of being duplicates (1) and being irrelevant based on the title (497) and abstract (38) (Fig. 1). Twenty‐seven studies16-24, 32-49 comprising 20 898 patients met the inclusion criteria and were included in the narrative synthesis (Appendix S1, supporting information)

A Preliminary Assessment of Greenhouse Gas Emission Trends in The Production and Consumption of Food in Malaysia

In the past decade, a small but growing body of research has drawn attention to the environmental concerns of rising greenhouse gas emissions associated with the consumption and production of food; this is an issue of increasing importance in Southeast Asia where rapid population growth is leading to year-on-year increases in food demand. To date, countries in Southeast Asia have shown little interest in addressing greenhouse gas emissions across the whole life cycle of food—production, processing, transportation, retailing, consumption, and final disposal—despite a growing awareness of climate change andits effects. This paper serves as a starting point to explore the relatively under-researched topic of greenhouse gas emission trends and the production and consumption of food in Southeast Asia, with particular focus on the Malaysian food sector. Previous research documenting greenhouse gas emissions from specific food products and components in the food supply chain has been used to determine the likely greenhouse gas ‘hotspots’ in Malaysia. The paper concludes by recommending the development of an overarching framework for Sustainable Food Systems in Malaysia and identifies specific areas of research to support this framework

Reasoning and learning services for coalition situational understanding

Situational understanding requires an ability to assess the current situation and anticipate future situations, requiring both pattern recognition and inference. A coalition involves multiple agencies sharing information and analytics. This paper considers how to harness distributed information sources, including multimodal sensors, together with machine learning and reasoning services, to perform situational understanding in a coalition context. To exemplify the approach we focus on a technology integration experiment in which multimodal data — including video and still imagery, geospatial and weather data — is processed and fused in a service-oriented architecture by heterogeneous pattern recognition and inference components. We show how the architecture: (i) provides awareness of the current situation and prediction of future states, (ii) is robust to individual service failure, (iii) supports the generation of ‘why’ explanations for human analysts (including from components based on ‘black box’ deep neural networks which pose particular challenges to explanation generation), and (iv) allows for the imposition of information sharing constraints in a coalition context where there is varying levels of trust between partner agencies

Integrating learning and reasoning services for explainable information fusion

—We present a distributed information fusion system able to integrate heterogeneous information processing services based on machine learning and reasoning approaches. We focus on higher (semantic) levels of information fusion, and highlight the requirement for the component services, and the system as a whole, to generate explanations of its outputs. Using a case study approach in the domain of traffic monitoring, we introduce component services based on (i) deep neural network approaches and (ii) heuristic-based reasoning. We examine methods for explanation generation in each case, including both transparency (e.g, saliency maps, reasoning traces) and post-hoc methods (e.g, explanation in terms of similar examples, identification of relevant semantic objects). We consider trade-offs in terms of the classification performance of the services and the kinds of available explanations, and show how service integration offers more robust performance and explainability

Efficient orchestration of Node-RED IoT workflows using a vector symbolic architecture

Numerous workflow systems span multiple scientific domains and environments, and for the Internet of Things (IoT), Node-RED offers an attractive Web based user interface to execute IoT service-based workflows. However, like most workflow systems, it coordinates the workflow centrally, and cannot run within more transient environments where nodes are mobile. To address this gap, we show how Node-RED workflows can be migrated into a decentralized execution environment for operation on mobile ad-hoc networks, and we demonstrate this by converting a Node-RED based traffic congestion detection workflow to operate in a decentralized environment. The approach uses a Vector Symbolic Architecture (VSA) to dynamically convert Node-Red applications into a compact semantic vector representation that encodes the service interfaces and the workflow in which they are embedded. By extending existing services interfaces, with a simple cognitive layer that can interpret and exchange the vectors, we show how the required services can be dynamically discovered and interconnected into the required workflow in a completely decentralized manner. The resulting system provides a convenient environment where the Node-RED front-end graphical composition tool can be used to orchestrate decentralized workflows. In this paper, we further extend this work by introducing a new dynamic VSA vector compression scheme that compresses vectors for on-the-wire communication, thereby reducing communication bandwidth while maintaining the semantic information content. This algorithm utilizes the holographic properties of the symbolic vectors to perform compression taking into consideration the number of combined vectors along with similarity bounds that determine conflict with other encoded vectors used in the same context. The resulting savings make this approach extremely efficient for discovery in service-based decentralized workflows

Taking action in a changing world

In the last hours of CHI 2017, a group of researchers from universities and businesses across the northern hemisphere sat down together to consider “Taking Action in a Changing World”. The title of the special interest group (SIG) is significant; it speaks of having an impact, of the politics on which we wish to have an impact, and also the dynamism of the structures and systems around us. There is no special mention of technology. In other words, it is a departure from business-as-usual HCI

Trustable service discovery for highly dynamic decentralized workflows

The quantity and capabilities of smart devices and sensors deployed as part of the Internet of Things (IoT) and accessible via remote microservices is set to rise dramatically as the provision of interactive data streaming increases. This introduces opportunities to rapidly construct new applications by interconnecting these microservices in different workflow configurations. The challenge is to discover the required microservices, including those from trusted partners and the wider community, whilst being able to operate robustly under diverse networking conditions. This paper outlines a workflow approach that provides decentralized discovery and orchestration of verifiably trustable services in support of multi-party operations. The approach is based on adoption of patterns from self-sovereign identity research, notably Verifiable Credentials, to share information amongst peers based on attestations of service descriptions and prior service usage in a privacy preserving and secure manner. This provides a dynamic, trust-based framework for ratifying and evaluating the qualities of different services. Collating these new service descriptions and integrating with existing decentralized workflow research based on vector symbolic architecture (VSA) provides an enhanced semantic search space for efficient and trusted service discovery that is necessary to support a diverse range of emerging edge-computing environments. An architecture for a dynamic decentralized service discovery system, is designed, and described through application to a scenario which uses trusted peers’ reported experiences of an anomaly detection service to determine service selection

PrEdiction of Risk and Communication of outcomE followIng major lower limb amputation – a collaboratiVE study (PERCEIVE): Protocol for the PERCEIVE qualitative study

INTRODUCTION: Deciding whether to proceed with a major lower limb amputation is life-changing and complex, and it is crucial that the right decision is made at the right time. However, medical specialists are known to poorly predict risk when assessing patients for major surgery, and there is little guidance and research regarding decisions about amputation. The process of shared decision-making between doctors and patients during surgical consultations is also little understood. Therefore, the aim of this study is to analyse in depth the communication, consent, risk prediction and decision-making process in relation to major lower limb amputation. METHODS AND ANALYSIS: Consultations between patients and surgeons at which major lower limb amputation is discussed will be audio-recorded for 10–15 patients. Semi-structured follow-up interviews with patients (and relatives/carers) will then be conducted at two time points: as soon as possible/appropriate after a decision has been reached regarding surgery, and approximately 6 months later. Semi-structured interviews will also be conducted with 10–15 healthcare professionals working in the UK National Health Service (NHS) involved in amputation decision-making. This will include surgeons, anaesthetists and specialist physiotherapists at 2–4 NHS Health Boards/Trusts in Wales and England. Discourse analysis will be used to analyse the recorded consultations; interviews will be analysed thematically. Finally, workshops will be held with patients and healthcare professionals to help synthesise and interpret findings. ETHICS AND DISSEMINATION: The study has been approved by Wales REC 7 (20/WA/0351). Study findings will be published in international peer-reviewed journal(s) and presented at national and international scientific meetings. Findings will also be disseminated to a wide NHS and lay audience via presentations at meetings and written summaries for key stakeholder groups