How to conduct valid social science research using MTurk – a checklist

The use of Amazon’s Mechanical Turk (MTurk) for social science research has increased exponentially in recent years. Although there is great excitement about the practical and logistical benefits, there is justifiable skepticism about the validity of research using data collected with MTurk. In this post, Herman Aguinis, Isabel Villamor, and Ravi S. Ramani provide 10 actionable best-practice recommendations and a checklist that can serve as a catalyst for more robust, reproducible, and trustworthy MTurk-based research

The Perioperative Quality Improvement Programme (PQIP patient study): protocol for a UK multicentre, prospective cohort study to measure quality of care and outcomes after major surgery

INTRODUCTION: Major surgery accounts for a substantial proportion of health service activity, due not only to the primary procedure, but the longer-term health implications of poor short-term outcome. Data from small studies or from outside the UK indicate that rates of complications and failure to rescue vary between hospitals, as does compliance with best practice processes. Within the UK, there is currently no system for monitoring postoperative complications (other than short-term mortality) in major non-cardiac surgery. Further, there is variation between national audit programmes, in the emphasis placed on quality assurance versus quality improvement, and therefore the principles of measurement and reporting which are used to design such programmes. METHODS AND ANALYSIS: The PQIP patient study is a multi-centre prospective cohort study which recruits patients undergoing major surgery. Patient provide informed consent and contribute baseline and outcome data from their perspective using a suite of patient-reported outcome tools. Research and clinical staff complete data on patient risk factors and outcomes in-hospital, including two measures of complications. Longer-term outcome data are collected through patient feedback and linkage to national administrative datasets (mortality and readmissions). As well as providing a uniquely granular dataset for research, PQIP provides feedback to participating sites on their compliance with evidence-based processes and their patients' outcomes, with the aim of supporting local quality improvement. ETHICS AND DISSEMINATION: Ethical approval has been granted by the Health Research Authority in the UK. Dissemination of interim findings (non-inferential) will form a part of the improvement methodology and will be provided to participating centres at regular intervals, including near-real time feedback of key process measures. Inferential analyses will be published in the peer-reviewed literature, supported by a comprehensive multi-modal communications strategy including to patients, policy makers and academic audiences as well as clinicians

Duration of androgen deprivation therapy with postoperative radiotherapy for prostate cancer: a comparison of long-course versus short-course androgen deprivation therapy in the RADICALS-HD randomised trial

Background Previous evidence supports androgen deprivation therapy (ADT) with primary radiotherapy as initial treatment for intermediate-risk and high-risk localised prostate cancer. However, the use and optimal duration of ADT with postoperative radiotherapy after radical prostatectomy remains uncertain. Methods RADICALS-HD was a randomised controlled trial of ADT duration within the RADICALS protocol. Here, we report on the comparison of short-course versus long-course ADT. Key eligibility criteria were indication for radiotherapy after previous radical prostatectomy for prostate cancer, prostate-specific antigen less than 5 ng/mL, absence of metastatic disease, and written consent. Participants were randomly assigned (1:1) to add 6 months of ADT (short-course ADT) or 24 months of ADT (long-course ADT) to radiotherapy, using subcutaneous gonadotrophin-releasing hormone analogue (monthly in the short-course ADT group and 3-monthly in the long-course ADT group), daily oral bicalutamide monotherapy 150 mg, or monthly subcutaneous degarelix. Randomisation was done centrally through minimisation with a random element, stratified by Gleason score, positive margins, radiotherapy timing, planned radiotherapy schedule, and planned type of ADT, in a computerised system. The allocated treatment was not masked. The primary outcome measure was metastasis-free survival, defined as metastasis arising from prostate cancer or death from any cause. The comparison had more than 80% power with two-sided α of 5% to detect an absolute increase in 10-year metastasis-free survival from 75% to 81% (hazard ratio [HR] 0·72). Standard time-to-event analyses were used. Analyses followed intention-to-treat principle. The trial is registered with the ISRCTN registry, ISRCTN40814031, and ClinicalTrials.gov , NCT00541047 . Findings Between Jan 30, 2008, and July 7, 2015, 1523 patients (median age 65 years, IQR 60–69) were randomly assigned to receive short-course ADT (n=761) or long-course ADT (n=762) in addition to postoperative radiotherapy at 138 centres in Canada, Denmark, Ireland, and the UK. With a median follow-up of 8·9 years (7·0–10·0), 313 metastasis-free survival events were reported overall (174 in the short-course ADT group and 139 in the long-course ADT group; HR 0·773 [95% CI 0·612–0·975]; p=0·029). 10-year metastasis-free survival was 71·9% (95% CI 67·6–75·7) in the short-course ADT group and 78·1% (74·2–81·5) in the long-course ADT group. Toxicity of grade 3 or higher was reported for 105 (14%) of 753 participants in the short-course ADT group and 142 (19%) of 757 participants in the long-course ADT group (p=0·025), with no treatment-related deaths. Interpretation Compared with adding 6 months of ADT, adding 24 months of ADT improved metastasis-free survival in people receiving postoperative radiotherapy. For individuals who can accept the additional duration of adverse effects, long-course ADT should be offered with postoperative radiotherapy. Funding Cancer Research UK, UK Research and Innovation (formerly Medical Research Council), and Canadian Cancer Society

Adding 6 months of androgen deprivation therapy to postoperative radiotherapy for prostate cancer: a comparison of short-course versus no androgen deprivation therapy in the RADICALS-HD randomised controlled trial

Background Previous evidence indicates that adjuvant, short-course androgen deprivation therapy (ADT) improves metastasis-free survival when given with primary radiotherapy for intermediate-risk and high-risk localised prostate cancer. However, the value of ADT with postoperative radiotherapy after radical prostatectomy is unclear. Methods RADICALS-HD was an international randomised controlled trial to test the efficacy of ADT used in combination with postoperative radiotherapy for prostate cancer. Key eligibility criteria were indication for radiotherapy after radical prostatectomy for prostate cancer, prostate-specific antigen less than 5 ng/mL, absence of metastatic disease, and written consent. Participants were randomly assigned (1:1) to radiotherapy alone (no ADT) or radiotherapy with 6 months of ADT (short-course ADT), using monthly subcutaneous gonadotropin-releasing hormone analogue injections, daily oral bicalutamide monotherapy 150 mg, or monthly subcutaneous degarelix. Randomisation was done centrally through minimisation with a random element, stratified by Gleason score, positive margins, radiotherapy timing, planned radiotherapy schedule, and planned type of ADT, in a computerised system. The allocated treatment was not masked. The primary outcome measure was metastasis-free survival, defined as distant metastasis arising from prostate cancer or death from any cause. Standard survival analysis methods were used, accounting for randomisation stratification factors. The trial had 80% power with two-sided α of 5% to detect an absolute increase in 10-year metastasis-free survival from 80% to 86% (hazard ratio [HR] 0·67). Analyses followed the intention-to-treat principle. The trial is registered with the ISRCTN registry, ISRCTN40814031, and ClinicalTrials.gov, NCT00541047. Findings Between Nov 22, 2007, and June 29, 2015, 1480 patients (median age 66 years [IQR 61–69]) were randomly assigned to receive no ADT (n=737) or short-course ADT (n=743) in addition to postoperative radiotherapy at 121 centres in Canada, Denmark, Ireland, and the UK. With a median follow-up of 9·0 years (IQR 7·1–10·1), metastasis-free survival events were reported for 268 participants (142 in the no ADT group and 126 in the short-course ADT group; HR 0·886 [95% CI 0·688–1·140], p=0·35). 10-year metastasis-free survival was 79·2% (95% CI 75·4–82·5) in the no ADT group and 80·4% (76·6–83·6) in the short-course ADT group. Toxicity of grade 3 or higher was reported for 121 (17%) of 737 participants in the no ADT group and 100 (14%) of 743 in the short-course ADT group (p=0·15), with no treatment-related deaths. Interpretation Metastatic disease is uncommon following postoperative bed radiotherapy after radical prostatectomy. Adding 6 months of ADT to this radiotherapy did not improve metastasis-free survival compared with no ADT. These findings do not support the use of short-course ADT with postoperative radiotherapy in this patient population

Defining, measuring, and rewarding scholarly impact: mind the level of analysis

We address the grossly incorrect inferences that result from using journal impact factor (JIF) as a proxy to assess individual researcher and article scholarly impact. This invalid practice occurs because of confusion about the definition and measurement of impact at different levels of analysis. Specifically, JIF is a journal-level measure of impact, computed by aggregating citations of individual articles (i.e., upward effect), and is therefore inappropriate when measuring impact at lower levels of analysis, such as that of individual researchers, or of individual articles published in a particular journal (i.e., downward effect). We illustrate the severity of the errors that occur when using JIF to evaluate individual scholarly impact, and advocate for an immediate moratorium on the exclusive use of JIF and other journal-level (i.e., higher level of analysis) measures when assessing the impact of individual researchers and individual articles (i.e., lower level of analysis). Given the importance and interest in assessing the scholarly impact of researchers and articles, we delineate level-appropriate and readily available measures

Online_Supplement_for_The_First_20_Years_of_Organizational_Research_Methods_May_30 - The First 20 Years of <i>Organizational Research Methods</i>: Trajectory, Impact, and Predictions for the Future

<p>Online_Supplement_for_The_First_20_Years_of_Organizational_Research_Methods_May_30 for The First 20 Years of <i>Organizational Research Methods</i>: Trajectory, Impact, and Predictions for the Future by Herman Aguinis, Ravi S. Ramani, and Isabel Villamor in Organizational Research Methods</p

Ravi Tej Kumar Maguluri et al,Int.J.Comp.Tech.Appl,Vol 2 (6), 4007-4011 ISSN:2229-6093 Re-Ranking Process for Image Based Retrieval Using Geometric Techniques

We present a fast and efficient geometric re-ranking method that can be incorporated in a feature based image-based retrieval system that utilizes a Vocabulary Tree (VT). We form feature pairs by comparing descriptor classification paths in the VT and calculate geometric similarity score of these pairs. We propose a location geometric similarity scoring method that is invariant to rotation, scale, and translation, and can be easily incorporated in mobile visual search and augmented reality systems. We compare the performance of the location geometric scoring scheme to orientation and scale geometric scoring schemes. We show in our experiments that re-ranking schemes can substantially improve recognition accuracy. We can also reduce the worst case server latency up to 1 sec and still improve the recognition performance. This entails finding the location of a query image in a large dataset containing 3 × 10^4 street side images of a city. We investigate how the traditional invariant feature matching approach falls down as the size of the database grows. In particular we show that by carefully selecting the vocabulary using the most informative features, retrieval performance is significantly improved, allowing us to increase the number of database images by a factor of 10. We also introduce a generalization of the traditional vocabulary tree search algorithm which improves performance by effectively increasing the branching factor of a fixed vocabulary tree. 1

Supplemental Material - When Work Doesn’t Work: An Alternative Framework to Examine Cognitive Factors in Employment-Entrepreneurship Transitions

Supplemental Material for When Work Doesn’t Work: An Alternative Framework to Examine Cognitive Factors in Employment-Entrepreneurship Transitions by Yolanda Christophe, Golshan Javadian, Christopher J. Mathis, and Ravi S. Ramani in Group & Organization Management.</p