Randomised controlled trials of complex interventions and large-scale transformation of services

Complex interventions and large-scale transformations of services are necessary to meet the health-care challenges of the 21st century. However, the evaluation of these types of interventions is challenging and requires methodological development. Innovations such as cluster randomised controlled trials, stepped-wedge designs, and non-randomised evaluations provide options to meet the needs of decision-makers. Adoption of theory and logic models can help clarify causal assumptions, and process evaluation can assist in understanding delivery in context. Issues of implementation must also be considered throughout intervention design and evaluation to ensure that results can be scaled for population benefit. Relevance requires evaluations conducted under real-world conditions, which in turn requires a pragmatic attitude to design. The increasing complexity of interventions and evaluations threatens the ability of researchers to meet the needs of decision-makers for rapid results. Improvements in efficiency are thus crucial, with electronic health records offering significant potential

Evaluation of the Wellspring Model for Improving Nursing Home Quality

Examines how successfully the Wellspring model improved the quality of care for residents of eleven nonprofit nursing homes in Wisconsin. Looks at staff turnover, and evaluates the impact on facilities, employees, residents, and cost

Designing and evaluating complex interventions to improve health care

Complex interventions are “built up from a number of components, which may act both independently and interdependently.”1 2 Many health service activities should be considered as complex. Evaluating complex interventions can pose a considerable challenge and requires a substantial investment of time. Unless the trials illuminate processes and mechanisms they often fail to provide useful information. If the result is negative, we are left wondering whether the intervention is inherently ineffective (either because the intervention was inadequately developed or because all similar interventions are ineffective), whether it was inadequately applied or applied in an inappropriate context, or whether the trial used an inappropriate design, comparison groups or outcomes. If there is a positive effect, it can be hard to judge how the results of the trial might be applied to a different context (box 1)

Selection of third party software in Off-The-Shelf-based software development: an interview study with industrial practitioners

The success of software development using third party components highly depends on the ability to select a suitable component for the intended application. The evidence shows that there is limited knowledge about current industrial OTS selection practices. As a result, there is often a gap between theory and practice, and the proposed methods for supporting selection are rarely adopted in the industrial practice. This paper's goal is to investigate the actual industrial practice of component selection in order to provide an initial empirical basis that allows the reconciliation of research and industrial endeavors. The study consisted of semi-structured interviews with 23 employees from 20 different software-intensive companies that mostly develop web information system applications. It provides qualitative information that help to further understand these practices, and emphasize some aspects that have been overlooked by researchers. For instance, although the literature claims that component repositories are important for locating reusable components; these are hardly used in industrial practice. Instead, other resources that have not received considerable attention are used with this aim. Practices and potential market niches for software-intensive companies have been also identified. The results are valuable from both the research and the industrial perspectives as they provide a basis for formulating well-substantiated hypotheses and more effective improvement strategies.Peer ReviewedPostprint (author's final draft

An exploratory cluster randomised controlled trial of knowledge translation strategies to support evidence-informed decision-making in local governments (The KT4LG study)

Background: Childhood overweight and obesity is the most prevalent and, arguably, politically complex child health problem internationally. Governments, communities and industry have important roles to play, and are increasingly expected to deliver an evidence-informed system-wide prevention program. However, efforts are impeded by a lack of organisational access to and use of research evidence. This study aims to identify feasible, acceptable and ideally, effective knowledge translation (KT) strategies to increase evidence-informed decision making in local governments, within the context of childhood obesity prevention as a national policy priority.Methods/Design: This paper describes the methods for KT4LG, a cluster randomised controlled trial which is exploratory in nature, given the limited evidence base and methodological advances. KT4LG aims to examine a program of KT strategies to increase the use of research evidence in informing public health decisions in local governments. KT4LG will also assess the feasibility and acceptability of the intervention. The intervention program comprises a facilitated program of evidence awareness, access to tailored research evidence, critical appraisal skills development, networking and evidence summaries and will be compared to provision of evidence summaries alone in the control program. 28 local governments were randomised to intervention or control, using computer generated numbers, stratified by budget tertile (high, medium or low). Questionnaires will be used to measure impact, costs, and outcomes, and key informant interviews will be used to examine processes, feasibility, and experiences. Policy tracer studies will be included to examine impact of intervention on policies within relevant government policy documents.Discussion: Knowledge translation intervention studies with a focus on public health and prevention are very few in number. Thus, this study will provide essential data on the experience of program implementation and evaluation of a system-integrated intervention program employed within the local government public health context. Standardised programs of system, organisational and individual KT strategies have not been described or rigorously evaluated. As such, the findings will make a significant contribution to understanding whether a facilitated program of KT strategies hold promise for facilitating evidence-informed public health decision making within complex multisectoral government organisations.<br /

The Principles Of Developing A Management Decision Support System For Scientific Employees

Employees engaged in mental work have become the most valuable assets of any organization in the 21st century. The satisfaction of those involved in mental work requires the provision of objectivity and transparency in their decision-making. This, in turn, entails the development of scientifically motivated decision making mechanisms and scientific-methodological approaches to evaluate their performance based on innovative technologies.The main goal of this article is in development of the scientific and methodological framework for the establishment of a decision support system to manage the employees engaged in mental work and operating in uncertainty. In this regard, initially, the question of evaluating the activities of scientific workers is examined, its characteristic features are determined, and the fuzzy relation model is proposed as a multi-criterion issue formed in uncertainty. Taking into consideration the hierarchical structure of the criteria that allows evaluating the activities of scientific workers, a phased solution method based on an additive aggregation method is proposed. In accordance with the methodology, a functional scheme of the decision support system to manage the scientific personnel is developed. The working principle of each block and the interaction of the blocks are described. The rules for the employees\u27 management decisions are shown by referring to the knowledge production model.Based on the proposed methodological approach, the implementation phases of the decision support system for the management of the scientific workers of the Institute of Information Technology of ANAS are described. To evaluate the employees\u27 performance, the tools to collect initial information, evaluate the system of criteria, define their importance coefficients and mathematical descriptions are provided. Some results of the system software are presented. The opportunities of the system based on the proposed methodology to support enterprise mangers to make scientifically justified decisions are provided

A research and evaluation capacity building model in Western Australia

Evaluation of public health programs, services and policies is increasingly required to demonstrate effectiveness. Funding constraints necessitate that existing programs, services and policies be evaluated and their findings disseminated. Evidence-informed practice and policy is also desirable to maximise investments in public health. Partnerships between public health researchers, service providers and policymakers can help address evaluation knowledge and skills gaps. The Western Australian Sexual Health and Blood-borne Virus Applied Research and Evaluation Network (SiREN) aims to build research and evaluation capacity in the sexual health and blood-borne virus sector in Western Australia (WA). Partners’ perspectives of the SiREN model after 2 years were explored. Qualitative written responses from service providers, policymakers and researchers about the SiREN model were analysed thematically. Service providers reported that participation in SiREN prompted them to consider evaluation earlier in the planning process and increased their appreciation of the value of evaluation. Policymakers noted benefits of the model in generating local evidence and highlighting local issues of importance for consideration at a national level. Researchers identified challenges communicating the services available through SiREN and the time investment needed to develop effective collaborative partnerships. Stronger engagement between public health researchers, service providers and policymakers through collaborative partnerships has the potential to improve evidence generation and evidence translation. These outcomes require long-term funding and commitment from all partners to develop and maintain partnerships. Ongoing monitoring and evaluation can ensure the partnership remains responsive to the needs of key stakeholders. The findings are applicable to many sectors

Searching Data: A Review of Observational Data Retrieval Practices in Selected Disciplines

A cross-disciplinary examination of the user behaviours involved in seeking and evaluating data is surprisingly absent from the research data discussion. This review explores the data retrieval literature to identify commonalities in how users search for and evaluate observational research data. Two analytical frameworks rooted in information retrieval and science technology studies are used to identify key similarities in practices as a first step toward developing a model describing data retrieval

Improving the normalization of complex interventions: measure development based on normalization process theory (NoMAD): study protocol

&lt;b&gt;Background&lt;/b&gt; Understanding implementation processes is key to ensuring that complex interventions in healthcare are taken up in practice and thus maximize intended benefits for service provision and (ultimately) care to patients. Normalization Process Theory (NPT) provides a framework for understanding how a new intervention becomes part of normal practice. This study aims to develop and validate simple generic tools derived from NPT, to be used to improve the implementation of complex healthcare interventions.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Objectives&lt;/b&gt; The objectives of this study are to: develop a set of NPT-based measures and formatively evaluate their use for identifying implementation problems and monitoring progress; conduct preliminary evaluation of these measures across a range of interventions and contexts, and identify factors that affect this process; explore the utility of these measures for predicting outcomes; and develop an online users’ manual for the measures.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Methods&lt;/b&gt; A combination of qualitative (workshops, item development, user feedback, cognitive interviews) and quantitative (survey) methods will be used to develop NPT measures, and test the utility of the measures in six healthcare intervention settings.&lt;p&gt;&lt;/p&gt; &lt;b&gt;Discussion&lt;/b&gt; The measures developed in the study will be available for use by those involved in planning, implementing, and evaluating complex interventions in healthcare and have the potential to enhance the chances of their implementation, leading to sustained changes in working practices