The problem of evaluating automated large-scale evidence aggregators

In the biomedical context, policy makers face a large amount of potentially discordant evidence from different sources. This prompts the question of how this evidence should be aggregated in the interests of best-informed policy recommendations. The starting point of our discussion is Hunter and Williams’ recent work on an automated aggregation method for medical evidence. Our negative claim is that it is far from clear what the relevant criteria for evaluating an evidence aggregator of this sort are. What is the appropriate balance between explicitly coded algorithms and implicit reasoning involved, for instance, in the packaging of input evidence? In short: What is the optimal degree of ‘automation’? On the positive side: We propose the ability to perform an adequate robustness analysis as the focal criterion, primarily because it directs efforts to what is most important, namely, the structure of the algorithm and the appropriate extent of automation. Moreover, where there are resource constraints on the aggregation process, one must also consider what balance between volume of evidence and accuracy in the treatment of individual evidence best facilitates inference. There is no prerogative to aggregate the total evidence available if this would in fact reduce overall accuracy

London SynEx Demonstrator Site: Impact Assessment Report

The key ingredients of the SynEx-UCL software components are: 1. A comprehensive and federated electronic healthcare record that can be used to reference or to store all of the necessary healthcare information acquired from a diverse range of clinical databases and patient-held devices. 2. A directory service component to provide a core persons demographic database to search for and authenticate staff users of the system and to anchor patient identification and connection to their federated healthcare record. 3. A clinical record schema management tool (Object Dictionary Client) that enables clinicians or engineers to define and export the data sets mapping to individual feeder systems. 4. An expansible set of clinical management algorithms that provide prompts to the patient or clinician to assist in the management of patient care. CHIME has built up over a decade of experience within Europe on the requirements and information models that are needed to underpin comprehensive multiprofessional electronic healthcare records. The resulting architecture models have influenced new European standards in this area, and CHIME has designed and built prototype EHCR components based on these models. The demonstrator systems described here utilise a directory service and object-oriented engineering approach, and support the secure, mobile and distributed access to federated healthcare records via web-based services. The design and implementation of these software components has been founded on a thorough analysis of the clinical, technical and ethico-legal requirements for comprehensive EHCR systems, published through previous project deliverables and in future planned papers. The clinical demonstrator site described in this report has provided the solid basis from which to establish "proof of concept" verification of the design approach, and a valuable opportunity to install, test and evaluate the results of the component engineering undertaken during the EC funded project. Inevitably, a number of practical implementation and deployment obstacles have been overcome through this journey, each of those having contributed to the time taken to deliver the components but also to the richness of the end products. UCL is fortunate that the Whittington Hospital, and the department of cardiovascular medicine in particular, is committed to a long-term vision built around this work. That vision, outlined within this report, is shared by the Camden and Islington Health Authority and by many other purchaser and provider organisations in the area, and by a number of industrial parties. They are collectively determined to support the Demonstrator Site as an ongoing project well beyond the life of the EC SynEx Project. This report, although a final report as far as the EC project is concerned, is really a description of the first phase in establishing a centre of healthcare excellence. New EC Fifth Framework project funding has already been approved to enable new and innovative technology solutions to be added to the work already established in north London

Computerization of workflows, guidelines and care pathways: a review of implementation challenges for process-oriented health information systems

There is a need to integrate the various theoretical frameworks and formalisms for modeling clinical guidelines, workflows, and pathways, in order to move beyond providing support for individual clinical decisions and toward the provision of process-oriented, patient-centered, health information systems (HIS). In this review, we analyze the challenges in developing process-oriented HIS that formally model guidelines, workflows, and care pathways. A qualitative meta-synthesis was performed on studies published in English between 1995 and 2010 that addressed the modeling process and reported the exposition of a new methodology, model, system implementation, or system architecture. Thematic analysis, principal component analysis (PCA) and data visualisation techniques were used to identify and cluster the underlying implementation ‘challenge’ themes. One hundred and eight relevant studies were selected for review. Twenty-five underlying ‘challenge’ themes were identified. These were clustered into 10 distinct groups, from which a conceptual model of the implementation process was developed. We found that the development of systems supporting individual clinical decisions is evolving toward the implementation of adaptable care pathways on the semantic web, incorporating formal, clinical, and organizational ontologies, and the use of workflow management systems. These architectures now need to be implemented and evaluated on a wider scale within clinical settings

Knowledge-Intensive Processes: Characteristics, Requirements and Analysis of Contemporary Approaches

Engineering of knowledge-intensive processes (KiPs) is far from being mastered, since they are genuinely knowledge- and data-centric, and require substantial flexibility, at both design- and run-time. In this work, starting from a scientific literature analysis in the area of KiPs and from three real-world domains and application scenarios, we provide a precise characterization of KiPs. Furthermore, we devise some general requirements related to KiPs management and execution. Such requirements contribute to the definition of an evaluation framework to assess current system support for KiPs. To this end, we present a critical analysis on a number of existing process-oriented approaches by discussing their efficacy against the requirements

What counts as good evidence

Making better use of evidence is essential if public services are to deliver more for less. Central to this challenge is the need for a clearer understanding about standards of evidence that can be applied to the research informing social policy. This paper reviews the extent to which it is possible to reach a workable consensus on ways of identifying and labelling evidence. It does this by exploring the efforts made to date and the debates that have ensued. Throughout, the focus is on evidence that is underpinned by research, rather than other sources of evidence such as expert opinion or stakeholder views.Publisher PD

Professional Practice Guidelines for Occupationally Mandated Psychological Evaluations

Psychological evaluations are relied on by employers, professional licensing boards, and civil service commissions to make hiring and employment decisions affecting individuals, orga- nizations, and the public. To promote best practices, these professional practice guidelines were developed for use by psychologists who perform clinical evaluations of individuals for occupational purposes, regardless of whether the evaluation is intended to obtain employ- ment, to achieve licensure/certification, or to maintain either. These guidelines were created by the Committee on Professional Practice and Standards (COPPS) to educate and inform the practice of psychologists who conduct occupationally mandated psychological evaluations (OMPEs), as well as to stimulate debate and research in this important area