Improving the efficiency of research proposals evaluation: A two-stage procedure

An important share of research funding is allocated via competitive programs, which entail considerable direct and indirect costs, such as to develop and evaluate the proposals. The goal of this article is to explore whether adopting a two-stage evaluation procedure could improve the efficiency of the process. For this purpose, we study the evaluation system designed by the Foundation Dam (Stiftelsen Dam), one of the largest foundations in Norway supporting health research. In 2020, Foundation Dam adopted a new evaluation procedure consisting in a short proposal for a first stage of selection and, for those selected, a second-stage evaluation of a long proposal. We explore whether such a procedure reduces the evaluation costs and how the evaluation procedures compare in terms of reliability. Survey responses from 94 of the 594 applicants in the one-stage procedure (2018–19) and all the 668 applicants in the two-stage procedure (2020–21) show that the two-stage procedure reduced the average time that applicants spent in drafting the proposal(s) by 38%. According to the reviewers’ estimate, the two-stage procedure also reduced by 28% the time they spent on average to evaluate an applicant’s proposal(s). The analysis of the evaluation data of 594 long proposals in the one-stage procedure and 668 short proposals in the two-stage procedure shows that reviewers’ scores of short proposals display greater reliability and agreement than the reviewers’ scores of long proposals in the old one-stage procedure. Our findings suggest that a two-stage procedure can substantially improve the efficiency of grant writing and review, without harming the reliability of the evaluation.publishedVersio

Reliability and validity of the Norwegian version of the disabilities of the arm, shoulder and hand questionnaire in patients with shoulder impingement syndrome

Background: Patient-rated outcome measures (PROMs) are an important part of clinical decision-making in rehabilitation of patients with shoulder pain. The Disabilities of Arm, Shoulder and Hand (DASH) questionnaire was designed to measure physical disability and symptoms in patients with musculoskeletal disorders of the upper extremity and is one the most commonly used outcome measures for patients with shoulder pain. The purpose of this study was to investigate the reliability and validity of the Norwegian version of the DASH in patients with shoulder impingement syndrome. Methods: Sixty-three patients diagnosed with shoulder impingement syndrome at an orthopaedic outpatient clinic were included in the study. Internal consistency of the DASH was evaluated by the Cronbach’s alpha and item-to-total correlations. Test-retest reliability was analyzed by the intraclass correlation coefficient (ICC) and limits of agreement (LoA) according to the Bland Altman method. Standard error of measurement (SEM) and minimally detectable change (MDC) were calculated for the total DASH score. Construct validity was evaluated by testing six a priori hypotheses for the Pearson’s correlation coefficient between the DASH and the Shoulder Pain and Disability Index (SPADI), the 36-item Short Form Health Survey (SF-36) and a Numeric Pain Rating Scale (NPRS). Results: Reliability: Cronbach’s alpha of the DASH was 0.93 and item-to-total correlations ranged from 0.36 to 0.81. ICC was 0.89. The 95 percent LoA was calculated to be between −11.9 and 14.1. SEM was 4.7 and MDC 13.1. Construct validity: Eighty-three percent of the a priori hypotheses of correlation were confirmed. The DASH showed a high positive correlation of 0.75 with the SPADI, a negative moderate correlation of −0.48 to −0.62 with physical functioning, bodily pain and physical component summary of the SF-36 and a moderate positive correlation of 0.58 with the NPRS. DASH correlated higher with the physical component summary than with the mental component summary of the SF-36. Conclusions: The Norwegian version of the DASH is a reliable and valid outcome measure for patients with shoulder impingement syndrom

Development and Reliability of a Clinician-rated Instrument to Evaluate Function in Individuals with Shoulder Pain: A Preliminary Study

Background and Purpose Subacromial impingement syndrome (SIS) is a common and disabling condition in the population. Interventions are often evaluated with patient-rated outcome measures. The purpose of this study was to develop a simple clinician-rated measure to detect difficulties in the execution of movement-related tasks among patients with subacromial impingement syndrome. Method The steps in the scale development included a review of the clinical literature of shoulder pain to identify condition-specific questionnaires, pilot testing, clinical testing and scale construction. Twenty-one eligible items from thirteen questionnaires were extracted and included in a pilot test. All items were scored on a five-point ordinal scale ranging from 1 (no difficulty) to 5 (cannot perform). Fourteen items were excluded after pilot testing because of difficulties in standardization or other practical considerations. The remaining seven items were included in a clinical test-retest study with outpatients at a hospital. Of these, four were excluded because of psychometric reasons. From the remaining three items, a measure named Shoulder Activity Scale (summed score ranging from 3 to 15) was developed. Results A total of 33 men and 30 women were included in the clinical study; age range 27–80 years. The intraclass correlation coefficient results for inter-rater reliability and test-retest reliability were 0.80 (95% CI = 0.51–0.90) and 0.74 (95% CI = 0.58–0.84), respectively. The standard error of measurement and minimal detectable change were 1.19 and 3.32, respectively. The scale was linked to the International Classification of Functioning, Disability and Health second level categories lifting and carrying objects (d430), dressing (d540), hand and arm use (d445) and control of voluntary movement (b760). Conclusion The Shoulder Activity Scale showed acceptable reliability in a sample of outpatients at a hospital, rated by clinicians experienced in shoulder rehabilitation. The validity of the scale should be investigated in future studies before application to common practice. © 2013 The Authors. Physiotherapy Research International published by John Wiley & Sons Ltd

The Bridge Building Model: connecting evidence-based practice, evidence-based research, public involvement and needs led research

This paper describes a model developed by an interdisciplinary team of research and public engagement specialists, with backgrounds in health and social care research, higher education, evidence-based practice, leadership, commissioning research and public involvement and engagement. The model we propose combines evidence-based practice, evidence-based research, public involvement and needs led research. Our aim is to capitalise on the joining of the rationale and methods for these approaches, which have all been highlighted as important, but for which there has been a lack of models for integration. Our ambition is to argue for and show an effective and evidence-based way of working that bridges health and social care needs identification, evidence-based practice and research

Reliability and validity of the Physical Activity Scale for the Elderly (PASE) in patients with hip osteoarthritis

Background Physical activity (PA) is beneficial in reducing pain and improving function in lower limb osteoarthritis (OA), and is recommended as a first line treatment. Self-administered questionnaires are used to assess PA, but knowledge about reliability and validity of these PA questionnaires are limited, in particular for patients with OA. The purpose of this study was to evaluate the reliability and validity of the Physical Activity Scale for the Elderly (PASE) in patients with hip OA. Methods Forty patients with hip OA (20 men and 20 women, mean age 61.3 ± 10 years) were included. For test-retest reliability PASE was administered twice with a mean time between tests of 9 ± 4 days. Intraclass correlation coefficient (ICC), standard error of measurement (SEM) and minimal detectable change (MDC) were calculated for the total score and for the particular items assessing different PA intensity levels. In addition a Bland-Altman analysis for the total PASE score was performed. Construct validity was evaluated by comparing the PASE results with the Actigraph GT1M accelerometer and the International Physical Activity Questionnaire (IPAQ), using the Spearman rank correlation coefficient. Results ICC for the total PASE score was 0.78, with relatively large error of measurement; SEM = 31 and MDC = 87. ICC for the intensity items was 0.20 for moderate PA intensity, 0.46 for light PA intensity and to 0.68 for vigorous PA intensity. The Spearman rank correlation coefficient between the Actigraph GT1M total counts per minute and the total PASE score was 0.30 (p = 0.089), and ranging from 0.20-0.38 for the different PA intensity categories. The Spearman rank correlation between IPAQ and PASE was 0.61 (p = 0.001) for the total scores. Conclusions In patients with hip OA the test-retest reliability of the total PASE score was moderate, with acceptable ICC, but with large measurement errors. The construct validity of the PASE was poor when compared to the Actigraph GT1M accelerometer. Test-retest reliability and construct validity revealed that the PASE was unable to assess PA intensity levels. PASE is not recommended as a valid tool to examine PA level for patients with hip OA

Development and Reliability of a Clinician-rated Instrument to Evaluate Function in Individuals with Shoulder Pain: A Preliminary Study

Background and Purpose Subacromial impingement syndrome (SIS) is a common and disabling condition in the population. Interventions are often evaluated with patient-rated outcome measures. The purpose of this study was to develop a simple clinician-rated measure to detect difficulties in the execution of movement-related tasks among patients with subacromial impingement syndrome. Method The steps in the scale development included a review of the clinical literature of shoulder pain to identify condition-specific questionnaires, pilot testing, clinical testing and scale construction. Twenty-one eligible items from thirteen questionnaires were extracted and included in a pilot test. All items were scored on a five-point ordinal scale ranging from 1 (no difficulty) to 5 (cannot perform). Fourteen items were excluded after pilot testing because of difficulties in standardization or other practical considerations. The remaining seven items were included in a clinical test-retest study with outpatients at a hospital. Of these, four were excluded because of psychometric reasons. From the remaining three items, a measure named Shoulder Activity Scale (summed score ranging from 3 to 15) was developed. Results A total of 33 men and 30 women were included in the clinical study; age range 27–80 years. The intraclass correlation coefficient results for inter-rater reliability and test-retest reliability were 0.80 (95% CI = 0.51–0.90) and 0.74 (95% CI = 0.58–0.84), respectively. The standard error of measurement and minimal detectable change were 1.19 and 3.32, respectively. The scale was linked to the International Classification of Functioning, Disability and Health second level categories lifting and carrying objects (d430), dressing (d540), hand and arm use (d445) and control of voluntary movement (b760). Conclusion The Shoulder Activity Scale showed acceptable reliability in a sample of outpatients at a hospital, rated by clinicians experienced in shoulder rehabilitation. The validity of the scale should be investigated in future studies before application to common practice. © 2013 The Authors. Physiotherapy Research International published by John Wiley & Sons Ltd