SCOPE: Safer care for older persons (in residential) environments: A study protocol

BACKGROUND: The current profile of residents living in Canadian nursing homes includes elder persons with complex physical and social needs. High resident acuity can result in increased staff workload and decreased quality of work life. AIMS: Safer Care for Older Persons [in residential] Environments is a two year (2010 to 2012) proof-of-principle pilot study conducted in seven nursing homes in western Canada. The purpose of the study is to evaluate the feasibility of engaging front line staff to use quality improvement methods to integrate best practices into resident care. The goals of the study are to improve the quality of work life for staff, in particular healthcare aides, and to improve residents' quality of life. METHODS/DESIGN: The study has parallel research and quality improvement intervention arms. It includes an education and support intervention for direct caregivers to improve the safety and quality of their care delivery. We hypothesize that this intervention will improve not only the care provided to residents but also the quality of work life for healthcare aides. The study employs tools adapted from the Institute for Healthcare Improvement's Breakthrough Series: Collaborative Model and Canada's Safer Healthcare Now! improvement campaign. Local improvement teams in each nursing home (1 to 2 per facility) are led by healthcare aides (non-regulated caregivers) and focus on the management of specific areas of resident care. Critical elements of the program include local measurement, virtual and face-to-face learning sessions involving change management, quality improvement methods and clinical expertise, ongoing virtual and in person support, and networking. DISCUSSION: There are two sustainability challenges in this study: ongoing staff and leadership engagement, and organizational infrastructure. Addressing these challenges will require strategic planning with input from key stakeholders for sustaining quality improvement initiatives in the long-term care sector

The relationship between characteristics of context and research utilization in a pediatric setting

BackgroundResearch utilization investigators have called for more focused examination of the influence of context on research utilization behaviors. Yet, up until recently, lack of instrumentation to identify and quantify aspects of organizational context that are integral to research use has significantly hampered these efforts. The Alberta Context Tool (ACT) was developed to assess the relationships between organizational factors and research utilization by a variety of healthcare professional groups. The purpose of this paper is to present findings from a pilot study using the ACT to elicit pediatric and neonatal healthcare professionals\u27 perceptions of the organizational context in which they work and their use of research to inform practice. Specifically, we report on the relationship between dimensions of context, founded on the Promoting Action on Research Implementation in Health Services (PARIHS) framework, and self-reported research use behavior.MethodsA cross-sectional survey approach was employed using a version of the ACT, modified specifically for pediatric settings. The survey was administered to nurses working in three pediatric units in Alberta, Canada. Scores for three dimensions of context (culture, leadership and evaluation) were used to categorize respondent data into one of four context groups (high, moderately high, moderately low and low). We then examined the relationships between nurses` self-reported research use and their perceived context.ResultsA 69% response rate was achieved. Statistically significant differences in nurses\u27 perceptions of culture, leadership and evaluation, and self-reported conceptual research use were found across the three units. Differences in instrumental research use across the three groups of nurses by unit were not significant. Higher self-reported instrumental and conceptual research use by all nurses in the sample was associated with more positive perceptions of their context.ConclusionsOverall, the results of this study lend support to the view that more positive contexts are associated with higher reports of research use in practice. These findings have implications for organizational endeavors to promote evidence-informed practice and maximize the quality of care. Importantly, these findings can be used to guide the development of interventions to target modifiable characteristics of organizational context that are influential in shaping research use behavior.<br /

Reliability and validity of the Alberta context tool (ACT) with professional nurses: Findings from a multi-study analysis

Although organizational context is central to evidence-based practice, underdeveloped measurement hindersitsassessment. The Alberta Context Tool, comprised of 59 items that tap10 modifiable contextual concepts, was developed to address this gap. The purpose of this study to examine the reliability and validity of scores obtained when the Alberta Context Tool is completed by professional nurses across different healthcare settings. Five separate studies (N = 2361 nurses across different care settings) comprised the study sample. Reliability and validity were assessed. Cronbach\u27s alpha exceeded 0.70 for9/10 Alberta Context Tool concepts. Item-total correlations exceeded acceptable standards for 56/59items. Confirmatory Factor Analysescoordinated acceptably with the Alberta Context Tool\u27s proposed latent structure. The mean values for each Alberta Context Tool concept increased from low to high levels of research utilization(as hypothesized) further supporting its validity. This study provides robust evidence forreliability and validity of scores obtained with the Alberta Context Tool when administered to professional nurses

A data quality control program for computer-assisted personal interviews

Researchers strive to optimize data quality in order to ensure that study findings are valid and reliable. In this paper, we describe a data quality control program designed to maximize quality of survey data collected using computer-assisted personal interviews. The quality control program comprised three phases: (1) software development, (2) an interviewer quality control protocol, and (3) a data cleaning and processing protocol. To illustrate the value of the program, we assess its use in the Translating Research in Elder Care Study. We utilize data collected annually for two years from computer-assisted personal interviews with 3004 healthcare aides. Data quality was assessed using both survey and process data. Missing data and data errors were minimal. Mean and median values and standard deviations were within acceptable limits. Process data indicated that in only 3.4% and 4.0% of cases was the interviewer unable to conduct interviews in accordance with the details of the program. Interviewers&rsquo; perceptions of interview quality also significantly improved between Years 1 and 2. While this data quality control program was demanding in terms of time and resources, we found that the benefits clearly outweighed the effort required to achieve high-quality data.<br /

A protocol for advanced psychometric assessment of surveys

Background and Purpose. In this paper, we present a protocol for advanced psychometric assessments of surveys based on the Standards for Educational and Psychological Testing. We use the Alberta Context Tool (ACT) as an exemplar survey to which this protocol can be applied. Methods. Data mapping, acceptability, reliability, and validity are addressed. Acceptability is assessed with missing data frequencies and the time required to complete the survey. Reliability is assessed with internal consistency coefficients and information functions. A unitary approach to validity consisting of accumulating evidence based on instrument content, response processes, internal structure, and relations to other variables is taken. We also address assessing performance of survey data when aggregated to higher levels (e.g., nursing unit). Discussion. In this paper we present a protocol for advanced psychometric assessment of survey data using the Alberta Context Tool (ACT) as an exemplar survey; application of the protocol to the ACT survey is underway. Psychometric assessment of any survey is essential to obtaining reliable and valid research findings. This protocol can be adapted for use with any nursing survey.<br /

Structural equation model testing and the quality of natural killer cell activity measurements

BACKGROUND: Browne et al. [Browne, MacCallum, Kim, Andersen, Glaser: When fit indices and residuals are incompatible. Psychol Methods 2002] employed a structural equation model of measurements of target cell lysing by natural killer cells as an example purportedly demonstrating that small but statistically significant ill model fit can be dismissed as "negligible from a practical point of view". METHODS: Reanalysis of the natural killer cell data reveals that the supposedly negligible ill fit obscured important, systematic, and substantial causal misspecifications. RESULTS: A clean-fitting structural equation model indicates that measurements employing higher natural-killer-cell to target-cell ratios are more strongly influenced by a progressively intrusive factor, whether or not the natural killer cell activity is activated by recombinant interferon γ (rIFN γ). The progressive influence may reflect independent rate limiting steps in cell recognition and attachment, spatial competition for cell attachment points, or the simultaneous lysings of single target cells by multiple natural killer cells. CONCLUSIONS: If the progressively influential factor is ultimately identified as a mere procedural impediment, the substantive conclusion will be that measurements of natural killer cell activity made at lower effector to target ratios are more valid. Alternatively, if the individual variations in the progressively influential factor are modifiable, this may presage a new therapeutic route to enhancing natural killer cell activity. The methodological conclusion is that, when using structural equation models, researchers should attend to significant model ill fit even if the degree of covariance ill fit is small, because small covariance residuals do not imply that the underlying model misspecifications are correspondingly small or inconsequential

What is the influence of randomisation sequence generation and allocation concealment on treatment effects of physical therapy trials? A meta-epidemiological study.

OBJECTIVE To determine if adequacy of randomisation and allocation concealment is associated with changes in effect sizes (ES) when comparing physical therapy (PT) trials with and without these methodological characteristics. DESIGN Meta-epidemiological study. PARTICIPANTS A random sample of randomised controlled trials (RCTs) included in meta-analyses in the PT discipline were identified. INTERVENTION Data extraction including assessments of random sequence generation and allocation concealment was conducted independently by two reviewers. To determine the association between sequence generation, and allocation concealment and ES, a two-level analysis was conducted using a meta-meta-analytic approach. PRIMARY AND SECONDARY OUTCOME MEASURES association between random sequence generation and allocation concealment and ES in PT trials. RESULTS 393 trials included in 43 meta-analyses, analysing 44 622 patients contributed to this study. Adequate random sequence generation and appropriate allocation concealment were accomplished in only 39.7% and 11.5% of PT trials, respectively. Although trials with inappropriate allocation concealment tended to have an overestimate treatment effect when compared with trials with adequate concealment of allocation, the difference was non-statistically significant (ES=0.12; 95% CI -0.06 to 0.30). When pooling our results with those of Nuesch et al, we obtained a pooled statistically significant value (ES=0.14; 95% CI 0.02 to 0.26). There was no difference in ES in trials with appropriate or inappropriate random sequence generation (ES=0.02; 95% CI -0.12 to 0.15). CONCLUSIONS Our results suggest that when evaluating risk of bias of primary RCTs in PT area, systematic reviewers and clinicians implementing research into practice should pay attention to these biases since they could exaggerate treatment effects. Systematic reviewers should perform sensitivity analysis including trials with low risk of bias in these domains as primary analysis and/or in combination with less restrictive analyses. Authors and editors should make sure that allocation concealment and random sequence generation are properly reported in trial reports

Nursing home administrators’ perspectives on a study feedback report : a cross sectional survey

BackgroundThis project is part of the Translating Research in Elder Care (TREC) program of research, a multi-level and longitudinal research program being conducted in 36 nursing homes in three Canadian Prairie Provinces. The overall goal of TREC is to improve the quality of care for older persons living in nursing homes and the quality of work life for care providers. The purpose of this paper is to report on development and evaluation of facility annual reports (FARs) from facility administrators&rsquo; perspectives on the usefulness, meaningfulness, and understandability of selected data from the TREC survey. MethodsA cross sectional survey design was used in this study. The feedback reports were developed in collaboration with participating facility administrators. FARs presented results in four contextual areas: workplace culture, feedback processes, job satisfaction, and staff burnout. Six weeks after FARs were mailed to each administrator, we conducted structured telephone interviews with administrators to elicit their evaluation of the FARs. Administrators were also asked if they had taken any actions as a result of the FAR. Descriptive and inferential statistics, as well as content analysis for open-ended questions, were used to summarize findings. ResultsThirty-one facility administrators (representing thirty-two facilities) participated in the interviews. Six administrators had taken action and 18 were planning on taking action as a result of FARs. The majority found the four contextual areas addressed in FAR to be useful, meaningful, and understandable. They liked the comparisons made between data from years one and two and between their facility and other TREC study sites in their province. Twenty-two indicated that they would like to receive information on additional areas such as aggressive behaviours of residents and information sharing. Twenty-four administrators indicated that FARs contained enough information, while eight found FARs &lsquo;too short&rsquo;. Administrators who reported that the FAR contained enough information were more likely to take action within their facilities than administrators who reported that they needed more information. ConclusionsAlthough the FAR was brief, the presentation of the four contextual areas was relevant to the majority of administrators and prompted them to plan or to take action within their facility. <br /

Interventions aimed at improving the nursing work environment: a systematic review

<p>Abstract</p> <p>Background</p> <p>Nursing work environments (NWEs) in Canada and other Western countries have increasingly received attention following years of restructuring and reported high workloads, high absenteeism, and shortages of nursing staff. Despite numerous efforts to improve NWEs, little is known about the effectiveness of interventions to improve NWEs. The aim of this study was to review systematically the scientific literature on implemented interventions aimed at improving the NWE and their effectiveness.</p> <p>Methods</p> <p>An online search of the databases CINAHL, Medline, Scopus, ABI, Academic Search Complete, HEALTHstar, ERIC, Psychinfo, and Embase, and a manual search of Emerald and Longwoods was conducted. (Quasi-) experimental studies with pre/post measures of interventions aimed at improving the NWE, study populations of nurses, and quantitative outcome measures of the nursing work environment were required for inclusion. Each study was assessed for methodological strength using a quality assessment and validity tool for intervention studies. A taxonomy of NWE characteristics was developed that would allow us to identify on which part of the NWE an intervention targeted for improvement, after which the effects of the interventions were examined.</p> <p>Results</p> <p>Over 9,000 titles and abstracts were screened. Eleven controlled intervention studies met the inclusion criteria, of which eight used a quasi-experimental design and three an experimental design. In total, nine different interventions were reported in the included studies. The most effective interventions at improving the NWE were: primary nursing (two studies), the educational toolbox (one study), the individualized care and clinical supervision (one study), and the violence prevention intervention (one study).</p> <p>Conclusions</p> <p>Little is known about the effectiveness of interventions aimed at improving the NWE, and published studies on this topic show weaknesses in their design. To advance the field, we recommend that investigators use controlled studies with pre/post measures to evaluate interventions that are aimed at improving the NWE. Thereby, more evidence-based knowledge about the implementation of interventions will become available for healthcare leaders to use in rebuilding nursing work environments.</p

Development and assessment of the Alberta Context Tool

<p>Abstract</p> <p>Background</p> <p>The context of healthcare organizations such as hospitals is increasingly accepted as having the potential to influence the use of new knowledge. However, the mechanisms by which the organizational context influences evidence-based practices are not well understood. Current measures of organizational context lack a theory-informed approach, lack construct clarity and generally have modest psychometric properties. This paper presents the development and initial psychometric validation of the Alberta Context Tool (ACT), an eight dimension measure of organizational context for healthcare settings.</p> <p>Methods</p> <p>Three principles guided the development of the ACT: substantive theory, brevity, and modifiability. The Promoting Action on Research Implementation in Health Services (PARiHS) framework and related literature were used to guide selection of items in the ACT. The ACT was required to be brief enough to be tolerated in busy and resource stretched work settings and to assess concepts of organizational context that were potentially <it>modifiable</it>. The English version of the ACT was completed by 764 nurses (752 valid responses) working in seven Canadian pediatric care hospitals as part of its initial validation. Cronbach's alpha, exploratory factor analysis, analysis of variance, and tests of association were used to assess instrument reliability and validity.</p> <p>Results</p> <p>Factor analysis indicated a 13-factor solution (accounting for 59.26% of the variance in 'organizational context'). The composition of the factors was similar to those originally conceptualized. Cronbach's alpha for the 13 factors ranged from .54 to .91 with 4 factors performing below the commonly accepted alpha cut off of .70. Bivariate associations between instrumental research utilization levels (which the ACT was developed to predict) and the ACT's 13 factors were statistically significant at the 5% level for 12 of the 13 factors. Each factor also showed a trend of increasing mean score ranging from the lowest level to the highest level of instrumental research use, indicating construct validity.</p> <p>Conclusions</p> <p>To date, no completely satisfactory measures of organizational context are available for use in healthcare. The ACT assesses several core domains to provide a comprehensive account of organizational context in healthcare settings. The tool's strengths are its brevity (allowing it to be completed in busy healthcare settings) and its focus on dimensions of organizational context that are modifiable. Refinements of the instrument for acute, long term care, and home care settings are ongoing.</p