Developing a Quality Control Protocol for Evaluation of Recorded Interviews

This presentation will describe the process used at the University of Michigan Survey Research Center for evaluating interviewer performance in survey administration. Within the Survey Research Operations unit, we use an online system for evaluating the interviewer-respondent interaction using recorded interviews. We will present our framework for measuring how well interviewers adhere to General Interviewing Techniques (GIT) - the guidelines in which they were trained. The presentation will describe the question-level and session-level measurement criteria employed, in addition to the selection protocols and the integration of paradata into the selection process. The presentation will include analysis of some evaluation data, with a discussion of how the data were used to inform further development of the evaluation protocol. Although some aggregate data will be shared, the presentation will largely focus on the operational considerations related to the development and implementation of the quality control protocol across projects

Predicting suicides after outpatient mental health visits in the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS).

The 2013 US Veterans Administration/Department of Defense Clinical Practice Guidelines (VA/DoD CPG) require comprehensive suicide risk assessments for VA/DoD patients with mental disorders but provide minimal guidance on how to carry out these assessments. Given that clinician-based assessments are not known to be strong predictors of suicide, we investigated whether a precision medicine model using administrative data after outpatient mental health specialty visits could be developed to predict suicides among outpatients. We focused on male nondeployed Regular US Army soldiers because they account for the vast majority of such suicides. Four machine learning classifiers (naive Bayes, random forests, support vector regression and elastic net penalized regression) were explored. Of the Army suicides in 2004-2009, 41.5% occurred among 12.0% of soldiers seen as outpatient by mental health specialists, with risk especially high within 26 weeks of visits. An elastic net classifier with 10-14 predictors optimized sensitivity (45.6% of suicide deaths occurring after the 15% of visits with highest predicted risk). Good model stability was found for a model using 2004-2007 data to predict 2008-2009 suicides, although stability decreased in a model using 2008-2009 data to predict 2010-2012 suicides. The 5% of visits with highest risk included only 0.1% of soldiers (1047.1 suicides/100 000 person-years in the 5 weeks after the visit). This is a high enough concentration of risk to have implications for targeting preventive interventions. An even better model might be developed in the future by including the enriched information on clinician-evaluated suicide risk mandated by the VA/DoD CPG to be recorded

Developing a Quality Control Protocol for Evaluation of Recorded Interviews

This presentation will describe the process used at the University of Michigan Survey Research Center for evaluating interviewer performance in survey administration. Within the Survey Research Operations unit, we use an online system for evaluating the interviewer-respondent interaction using recorded interviews. We will present our framework for measuring how well interviewers adhere to General Interviewing Techniques (GIT) - the guidelines in which they were trained. The presentation will describe the question-level and session-level measurement criteria employed, in addition to the selection protocols and the integration of paradata into the selection process. The presentation will include analysis of some evaluation data, with a discussion of how the data were used to inform further development of the evaluation protocol. Although some aggregate data will be shared, the presentation will largely focus on the operational considerations related to the development and implementation of the quality control protocol across projects

Sociodemographic and career history predictors of suicide mortality in the United States Army 2004–2009

The US Army suicide rate has increased sharply in recent years. Identifying significant predictors of Army suicides in Army and Department of Defense (DoD) administrative records might help focus prevention efforts and guide intervention content. Previous studies of administrative data, although documenting significant predictors, were based on limited samples and models. A career history perspective is used here to develop more textured models. The analysis was carried out as part of the Historical Administrative Data Study (HADS) of the Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS). De-identified data were combined across numerous Army and DoD administrative data systems for all Regular Army soldiers on active duty in 2004–2009. Multivariate associations of sociodemographics and Army career variables with suicide were examined in subgroups defined by time in service, rank and deployment history. Several novel results were found that could have intervention implications. The most notable of these were significantly elevated suicide rates (69.6–80.0 suicides per 100 000 person-years compared with 18.5 suicides per 100 000 person-years in the total Army) among enlisted soldiers deployed either during their first year of service or with less than expected (based on time in service) junior enlisted rank; a substantially greater rise in suicide among women than men during deployment; and a protective effect of marriage against suicide only during deployment. A career history approach produces several actionable insights missed in less textured analyses of administrative data predictors. Expansion of analyses to a richer set of predictors might help refine understanding of intervention implications.Psycholog

Understanding the elevated suicide risk of female soldiers during deployments

Background The Army Study to Assess Risk and Resilience in Servicemembers (Army STARRS) has found that the proportional elevation in the US Army enlisted soldier suicide rate during deployment (compared with the never-deployed or previously deployed) is significantly higher among women than men, raising the possibility of gender differences in the adverse psychological effects of deployment. Method Person-month survival models based on a consolidated administrative database for active duty enlisted Regular Army soldiers in 2004–2009 (n = 975 057) were used to characterize the gender × deployment interaction predicting suicide. Four explanatory hypotheses were explored involving the proportion of females in each soldier’s occupation, the proportion of same-gender soldiers in each soldier’s unit, whether the soldier reported sexual assault victimization in the previous 12 months, and the soldier’s pre-deployment history of treated mental/behavioral disorders. Results The suicide rate of currently deployed women (14.0/100 000 person-years) was 3.1–3.5 times the rates of other (i.e. never-deployed/previously deployed) women. The suicide rate of currently deployed men (22.6/100 000 person-years) was 0.9–1.2 times the rates of other men. The adjusted (for time trends, sociodemographics, and Army career variables) female:male odds ratio comparing the suicide rates of currently deployed v. other women v. men was 2.8 (95% confidence interval 1.1–6.8), became 2.4 after excluding soldiers with Direct Combat Arms occupations, and remained elevated (in the range 1.9–2.8) after adjusting for the hypothesized explanatory variables. Conclusions These results are valuable in excluding otherwise plausible hypotheses for the elevated suicide rate of deployed women and point to the importance of expanding future research on the psychological challenges of deployment for women.Psycholog

Occupational differences in US Army suicide rates

Background Civilian suicide rates vary by occupation in ways related to occupational stress exposure. Comparable military research finds suicide rates elevated in combat arms occupations. However, no research has evaluated variation in this pattern by deployment history, the indicator of occupation stress widely considered responsible for the recent rise in the military suicide rate. Method The joint associations of Army occupation and deployment history in predicting suicides were analysed in an administrative dataset for the 729 337 male enlisted Regular Army soldiers in the US Army between 2004 and 2009. Results There were 496 suicides over the study period (22.4/100 000 person-years). Only two occupational categories, both in combat arms, had significantly elevated suicide rates: infantrymen (37.2/100 000 person-years) and combat engineers (38.2/100 000 person-years). However, the suicide rates in these two categories were significantly lower when currently deployed (30.6/100 000 person-years) than never deployed or previously deployed (41.2–39.1/100 000 person-years), whereas the suicide rate of other soldiers was significantly higher when currently deployed and previously deployed (20.2–22.4/100 000 person-years) than never deployed (14.5/100 000 person-years), resulting in the adjusted suicide rate of infantrymen and combat engineers being most elevated when never deployed [odds ratio (OR) 2.9, 95% confidence interval (CI) 2.1–4.1], less so when previously deployed (OR 1.6, 95% CI 1.1–2.1), and not at all when currently deployed (OR 1.2, 95% CI 0.8–1.8). Adjustment for a differential ‘healthy warrior effect’ cannot explain this variation in the relative suicide rates of never-deployed infantrymen and combat engineers by deployment status. Conclusions Efforts are needed to elucidate the causal mechanisms underlying this interaction to guide preventive interventions for soldiers at high suicide risk.Psycholog

Predicting Suicides After Psychiatric Hospitalization in US Army Soldiers

IMPORTANCE: The US Army experienced a sharp increase in soldier suicides beginning in 2004. Administrative data reveal that among those at highest risk are soldiers in the 12 months after inpatient treatment of a psychiatric disorder. OBJECTIVE: To develop an actuarial risk algorithm predicting suicide in the 12 months after US Army soldier inpatient treatment of a psychiatric disorder to target expanded posthospitalization care. DESIGN, SETTING, AND PARTICIPANTS: There were 53,769 hospitalizations of active duty soldiers from January 1, 2004, through December 31, 2009, with International Classification of Diseases, Ninth Revision, Clinical Modification psychiatric admission diagnoses. Administrative data available before hospital discharge abstracted from a wide range of data systems (sociodemographic, US Army career, criminal justice, and medical or pharmacy) were used to predict suicides in the subsequent 12 months using machine learning methods (regression trees and penalized regressions) designed to evaluate cross-validated linear, nonlinear, and interactive predictive associations. MAIN OUTCOMES AND MEASURES: Suicides of soldiers hospitalized with psychiatric disorders in the 12 months after hospital discharge. RESULTS: Sixty-eight soldiers died by suicide within 12 months of hospital discharge (12.0% of all US Army suicides), equivalent to 263.9 suicides per 100,000 person-years compared with 18.5 suicides per 100,000 person-years in the total US Army. The strongest predictors included sociodemographics (male sex [odds ratio (OR), 7.9; 95% CI, 1.9-32.6] and late age of enlistment [OR, 1.9; 95% CI, 1.0-3.5]), criminal offenses (verbal violence [OR, 2.2; 95% CI, 1.2-4.0] and weapons possession [OR, 5.6; 95% CI, 1.7-18.3]), prior suicidality [OR, 2.9; 95% CI, 1.7-4.9], aspects of prior psychiatric inpatient and outpatient treatment (eg, number of antidepressant prescriptions filled in the past 12 months [OR, 1.3; 95% CI, 1.1-1.7]), and disorders diagnosed during the focal hospitalizations (eg, nonaffective psychosis [OR, 2.9; 95% CI, 1.2-7.0]). A total of 52.9% of posthospitalization suicides occurred after the 5% of hospitalizations with highest predicted suicide risk (3824.1 suicides per 100,000 person-years). These highest-risk hospitalizations also accounted for significantly elevated proportions of several other adverse posthospitalization outcomes (unintentional injury deaths, suicide attempts, and subsequent hospitalizations). CONCLUSIONS AND RELEVANCE: The high concentration of risk of suicide and other adverse outcomes might justify targeting expanded posthospitalization interventions to soldiers classified as having highest posthospitalization suicide risk, although final determination requires careful consideration of intervention costs, comparative effectiveness, and possible adverse effects