Integrated agricultural riparian stewardship in the Stillaguamish and Snohomish watersheds

The Stillaguamish and Snohomish River watersheds are regionally important to the health of Puget Sound and the Salish Sea and in particular for the recovery of salmon. The habitat gains needed to achieve salmon recovery in these watersheds include much of the agricultural landscape in Snohomish County, a situation that often results in conflicts between salmon recovery and agricultural communities. The Snohomish Conservation District’s National Estuary Program-funded Integrated Riparian Stewardship project is one of several efforts aimed at simultaneously achieving agricultural land preservation and salmon habitat protection and restoration in one of the fastest growing counties in the United States, where pressures from development and climate change threaten both salmon recovery and agriculture, and where agricultural property owners may feel salmon recovery efforts impact their livelihood and way of life. The District and project partners undertook a planning and protection/restoration initiative to identify, protect, and restore high priority riparian agricultural and rural lands in the Stillaguamish and Snohomish watersheds. The District developed an Action Plan that identifies high priority areas for reach-scale riparian protection and restoration and outlines a comprehensive riparian zone management approach for three focus areas in the Stillaguamish and Snohomish watersheds. The Action Plan integrates existing geomorphic, habitat, hydrologic, and water quality studies to identify reaches and parcels on which to purchase conservation easements and complete habitat restoration projects in order to protect or enhance cold water inflows, restore salmon habitat, and improve or protect hydrologic processes. The District is currently implementing the Action Plan. We will share our experience with a precision outreach strategy and our successes in leveraging the easement program with grant funding and the Conservation Reserve Enhancement Program and other grant funding to secure landowner cooperation and achieve reach-scale riparian zone management. Partners include Snohomish County, Washington Department of Ecology, Forterra, and NOAA Restoration Center

Accelerated and interpretable oblique random survival forests

The oblique random survival forest (RSF) is an ensemble supervised learning method for right-censored outcomes. Trees in the oblique RSF are grown using linear combinations of predictors to create branches, whereas in the standard RSF, a single predictor is used. Oblique RSF ensembles often have higher prediction accuracy than standard RSF ensembles. However, assessing all possible linear combinations of predictors induces significant computational overhead that limits applications to large-scale data sets. In addition, few methods have been developed for interpretation of oblique RSF ensembles, and they remain more difficult to interpret compared to their axis-based counterparts. We introduce a method to increase computational efficiency of the oblique RSF and a method to estimate importance of individual predictor variables with the oblique RSF. Our strategy to reduce computational overhead makes use of Newton-Raphson scoring, a classical optimization technique that we apply to the Cox partial likelihood function within each non-leaf node of decision trees. We estimate the importance of individual predictors for the oblique RSF by negating each coefficient used for the given predictor in linear combinations, and then computing the reduction in out-of-bag accuracy. In general benchmarking experiments, we find that our implementation of the oblique RSF is approximately 450 times faster with equivalent discrimination and superior Brier score compared to existing software for oblique RSFs. We find in simulation studies that 'negation importance' discriminates between relevant and irrelevant predictors more reliably than permutation importance, Shapley additive explanations, and a previously introduced technique to measure variable importance with oblique RSFs based on analysis of variance. Methods introduced in the current study are available in the aorsf R package.Comment: 40 pages, 6 figure

Sustaining The Saco Estuary: Final Report 2015

This study focuses on the Saco estuary, the tidal portion of the Saco River, which drains the largest watershed in southern Maine. With headwaters in the White Mountains of New Hampshire, the watershed encompasses more than 4,400 km2, and provides clean healthy drinking water to over 100,000 people living and working in communities in southern Maine. When the study began in 2009, very little was known about the ecology of the Saco estuary. Researchers at the University of New England and the Wells National Estuarine Research Reserve employed the process of collaborative learning to bring together people who care about the estuary in order to identify their concerns. A Stewardship Network composed of people employed by municipal, state and federal governments, water supply organizations and businesses, volunteers from municipal boards making land use decisions, land trusts, property owners and representatives from other organizations that are uniquely focused on the region was formed. The Stewardship Network helped to define the project goals and objectives, and provided input and guidance over the five-year project. This report explains what the researchers discovered about the ecology of the estuary, along with what they learned about its social and economic components. This baseline assessment contributes to the long-term goal of restoring and sustaining the structure and function of the estuary, and supports the efforts of government, businesses and local organizations that value the estuary and depend upon the natural services it provides

Ethanol’s Effect on Coq7 Expression in the Hippocampus of Mice

Coenzyme Q (CoQ) is a well-studied molecule, present in every cell membrane in the body, best known for its roles as a mitochondrial electron transporter and a potent membrane anti-oxidant. Much of the previous work was done in vitro in yeast and more recent work has suggested that CoQ may have additional roles prompting calls for a re-assessment of its role using in vivo systems in mammals. Here we investigated the putative role of Coenzyme Q in ethanol-induced effects in vivo using BXD RI mice. We examined hippocampal expression of Coq7 in saline controls and after an acute ethanol treatment, noting enriched biologic processes and pathways following ethanol administration. We also identified 45 ethanol-related phenotypes that were significantly correlated with Coq7 expression, including six phenotypes related to conditioned taste aversion and ethanol preference. This analysis highlights the need for further investigation of Coq7 and related genes in vivo as well as previously unrecognized roles that it may play in the hippocampus

Proceedings of the 8th Annual Conference on the Science of Dissemination and Implementation

A1 Introduction to the 8(th) Annual Conference on the Science of Dissemination and Implementation: Optimizing Personal and Population Health David Chambers, Lisa Simpson D1 Discussion forum: Population health D&I research Felicia Hill-Briggs D2 Discussion forum: Global health D&I research Gila Neta, Cynthia Vinson D3 Discussion forum: Precision medicine and D&I research David Chambers S1 Predictors of community therapists’ use of therapy techniques in a large public mental health system Rinad Beidas, Steven Marcus, Gregory Aarons, Kimberly Hoagwood, Sonja Schoenwald, Arthur Evans, Matthew Hurford, Ronnie Rubin, Trevor Hadley, Frances Barg, Lucia Walsh, Danielle Adams, David Mandell S2 Implementing brief cognitive behavioral therapy (CBT) in primary care: Clinicians' experiences from the field Lindsey Martin, Joseph Mignogna, Juliette Mott, Natalie Hundt, Michael Kauth, Mark Kunik, Aanand Naik, Jeffrey Cully S3 Clinician competence: Natural variation, factors affecting, and effect on patient outcomes Alan McGuire, Dominique White, Tom Bartholomew, John McGrew, Lauren Luther, Angie Rollins, Michelle Salyers S4 Exploring the multifaceted nature of sustainability in community-based prevention: A mixed-method approach Brittany Cooper, Angie Funaiole S5 Theory informed behavioral health integration in primary care: Mixed methods evaluation of the implementation of routine depression and alcohol screening and assessment Julie Richards, Amy Lee, Gwen Lapham, Ryan Caldeiro, Paula Lozano, Tory Gildred, Carol Achtmeyer, Evette Ludman, Megan Addis, Larry Marx, Katharine Bradley S6 Enhancing the evidence for specialty mental health probation through a hybrid efficacy and implementation study Tonya VanDeinse, Amy Blank Wilson, Burgin Stacey, Byron Powell, Alicia Bunger, Gary Cuddeback S7 Personalizing evidence-based child mental health care within a fiscally mandated policy reform Miya Barnett, Nicole Stadnick, Lauren Brookman-Frazee, Anna Lau S8 Leveraging an existing resource for technical assistance: Community-based supervisors in public mental health Shannon Dorsey, Michael Pullmann S9 SBIRT implementation for adolescents in urban federally qualified health centers: Implementation outcomes Shannon Mitchell, Robert Schwartz, Arethusa Kirk, Kristi Dusek, Marla Oros, Colleen Hosler, Jan Gryczynski, Carolina Barbosa, Laura Dunlap, David Lounsbury, Kevin O'Grady, Barry Brown S10 PANEL: Tailoring Implementation Strategies to Context - Expert recommendations for tailoring strategies to context Laura Damschroder, Thomas Waltz, Byron Powell S11 PANEL: Tailoring Implementation Strategies to Context - Extreme facilitation: Helping challenged healthcare settings implement complex programs Mona Ritchie S12 PANEL: Tailoring Implementation Strategies to Context - Using menu-based choice tasks to obtain expert recommendations for implementing three high-priority practices in the VA Thomas Waltz S13 PANEL: The Use of Technology to Improve Efficient Monitoring of Implementation of Evidence-based Programs - Siri, rate my therapist: Using technology to automate fidelity ratings of motivational interviewing David Atkins, Zac E. Imel, Bo Xiao, Doğan Can, Panayiotis Georgiou, Shrikanth Narayanan S14 PANEL: The Use of Technology to Improve Efficient Monitoring of Implementation of Evidence-based Programs - Identifying indicators of implementation quality for computer-based ratings Cady Berkel, Carlos Gallo, Irwin Sandler, C. Hendricks Brown, Sharlene Wolchik, Anne Marie Mauricio S15 PANEL: The Use of Technology to Improve Efficient Monitoring of Implementation of Evidence-based Programs - Improving implementation of behavioral interventions by monitoring emotion in spoken speech Carlos Gallo, C. Hendricks Brown, Sanjay Mehrotra S16 Scorecards and dashboards to assure data quality of health management information system (HMIS) using R Dharmendra Chandurkar, Siddhartha Bora, Arup Das, Anand Tripathi, Niranjan Saggurti, Anita Raj S17 A big data approach for discovering and implementing patient safety insights Eric Hughes, Brian Jacobs, Eric Kirkendall S18 Improving the efficacy of a depression registry for use in a collaborative care model Danielle Loeb, Katy Trinkley, Michael Yang, Andrew Sprowell, Donald Nease S19 Measurement feedback systems as a strategy to support implementation of measurement-based care in behavioral health Aaron Lyon, Cara Lewis, Meredith Boyd, Abigail Melvin, Semret Nicodimos, Freda Liu, Nathanial Jungbluth S20 PANEL: Implementation Science and Learning Health Systems: Intersections and Commonalities - Common loop assay: Methods of supporting learning collaboratives Allen Flynn S21 PANEL: Implementation Science and Learning Health Systems: Intersections and Commonalities - Innovating audit and feedback using message tailoring models for learning health systems Zach Landis-Lewis S22 PANEL: Implementation Science and Learning Health Systems: Intersections and Commonalities - Implementation science and learning health systems: Connecting the dots Anne Sales S23 Facilitation activities of Critical Access Hospitals during TeamSTEPPS implementation Jure Baloh, Marcia Ward, Xi Zhu S24 Organizational and social context of federally qualified health centers and variation in maternal depression outcomes Ian Bennett, Jurgen Unutzer, Johnny Mao, Enola Proctor, Mindy Vredevoogd, Ya-Fen Chan, Nathaniel Williams, Phillip Green S25 Decision support to enhance treatment of hospitalized smokers: A randomized trial Steven Bernstein, June-Marie Rosner, Michelle DeWitt, Jeanette Tetrault, James Dziura, Allen Hsiao, Scott Sussman, Patrick O’Connor, Benjamin Toll S26 PANEL: Developing Sustainable Strategies for the Implementation of Patient-Centered Care across Diverse US Healthcare Systems - A patient-centered approach to successful community transition after catastrophic injury Michael Jones, Julie Gassaway S27 PANEL: Developing Sustainable Strategies for the Implementation of Patient-Centered Care across Diverse US Healthcare Systems - Conducting PCOR to integrate mental health and cancer screening services in primary care Jonathan Tobin S28 PANEL: Developing Sustainable Strategies for the Implementation of Patient-Centered Care across Diverse US Healthcare Systems - A comparative effectiveness trial of optimal patient-centered care for US trauma care systems Douglas Zatzick S29 Preferences for in-person communication among patients in a multi-center randomized study of in-person versus telephone communication of genetic test results for cancer susceptibility Angela R Bradbury, Linda Patrick-Miller, Brian Egleston, Olufunmilayo I Olopade, Michael J Hall, Mary B Daly, Linda Fleisher, Generosa Grana, Pamela Ganschow, Dominique Fetzer, Amanda Brandt, Dana Farengo-Clark, Andrea Forman, Rikki S Gaber, Cassandra Gulden, Janice Horte, Jessica Long, Rachelle Lorenz Chambers, Terra Lucas, Shreshtha Madaan, Kristin Mattie, Danielle McKenna, Susan Montgomery, Sarah Nielsen, Jacquelyn Powers, Kim Rainey, Christina Rybak, Michelle Savage, Christina Seelaus, Jessica Stoll, Jill Stopfer, Shirley Yao and Susan Domchek S30 Working towards de-implementation: A mixed methods study in breast cancer surveillance care Erin Hahn, Corrine Munoz-Plaza, Jianjin Wang, Jazmine Garcia Delgadillo, Brian Mittman Michael Gould S31Integrating evidence-based practices for increasing cancer screenings in safety-net primary care systems: A multiple case study using the consolidated framework for implementation research Shuting (Lily) Liang, Michelle C. Kegler, Megan Cotter, Emily Phillips, April Hermstad, Rentonia Morton, Derrick Beasley, Jeremy Martinez, Kara Riehman S32 Observations from implementing an mHealth intervention in an FQHC David Gustafson, Lisa Marsch, Louise Mares, Andrew Quanbeck, Fiona McTavish, Helene McDowell, Randall Brown, Chantelle Thomas, Joseph Glass, Joseph Isham, Dhavan Shah S33 A multicomponent intervention to improve primary care provider adherence to chronic opioid therapy guidelines and reduce opioid misuse: A cluster randomized controlled trial protocol Jane Liebschutz, Karen Lasser S34 Implementing collaborative care for substance use disorders in primary care: Preliminary findings from the summit study Katherine Watkins, Allison Ober, Sarah Hunter, Karen Lamp, Brett Ewing S35 Sustaining a task-shifting strategy for blood pressure control in Ghana: A stakeholder analysis Juliet Iwelunmor, Joyce Gyamfi, Sarah Blackstone, Nana Kofi Quakyi, Jacob Plange-Rhule, Gbenga Ogedegbe S36 Contextual adaptation of the consolidated framework for implementation research (CFIR) in a tobacco cessation study in Vietnam Pritika Kumar, Nancy Van Devanter, Nam Nguyen, Linh Nguyen, Trang Nguyen, Nguyet Phuong, Donna Shelley S37 Evidence check: A knowledge brokering approach to systematic reviews for policy Sian Rudge S38 Using Evidence Synthesis to Strengthen Complex Health Systems in Low- and Middle-Income Countries Etienne Langlois S39 Does it matter: timeliness or accuracy of results? The choice of rapid reviews or systematic reviews to inform decision-making Andrea Tricco S40 Evaluation of the veterans choice program using lean six sigma at a VA medical center to identify benefits and overcome obstacles Sherry Ball, Anne Lambert-Kerzner, Christine Sulc, Carol Simmons, Jeneen Shell-Boyd, Taryn Oestreich, Ashley O'Connor, Emily Neely, Marina McCreight, Amy Labebue, Doreen DiFiore, Diana Brostow, P. Michael Ho, David Aron S41 The influence of local context on multi-stakeholder alliance quality improvement activities: A multiple case study Jillian Harvey, Megan McHugh, Dennis Scanlon S42 Increasing physical activity in early care and education: Sustainability via active garden education (SAGE) Rebecca Lee, Erica Soltero, Nathan Parker, Lorna McNeill, Tracey Ledoux S43 Marking a decade of policy implementation: The successes and continuing challenges of a provincial school food and nutrition policy in Canada Jessie-Lee McIsaac, Kate MacLeod, Nicole Ata, Sherry Jarvis, Sara Kirk S44 Use of research evidence among state legislators who prioritize mental health and substance abuse issues Jonathan Purtle, Elizabeth Dodson, Ross Brownson S45 PANEL: Effectiveness-Implementation Hybrid Designs: Clarifications, Refinements, and Additional Guidance Based on a Systematic Review and Reports from the Field - Hybrid type 1 designs Brian Mittman, Geoffrey Curran S46 PANEL: Effectiveness-Implementation Hybrid Designs: Clarifications, Refinements, and Additional Guidance Based on a Systematic Review and Reports from the Field - Hybrid type 2 designs Geoffrey Curran S47 PANEL: Effectiveness-Implementation Hybrid Designs: Clarifications, Refinements, and Additional Guidance Based on a Systematic Review and Reports from the Field - Hybrid type 3 designs Jeffrey Pyne S48 Linking team level implementation leadership and implementation climate to individual level attitudes, behaviors, and implementation outcomes Gregory Aarons, Mark Ehrhart, Elisa Torres S49 Pinpointing the specific elements of local context that matter most to implementation outcomes: Findings from qualitative comparative analysis in the RE-inspire study of VA acute stroke care Edward Miech S50 The GO score: A new context-sensitive instrument to measure group organization level for providing and improving care Edward Miech S51 A research network approach for boosting implementation and improvement Kathleen Stevens, I.S.R.N. Steering Council S52 PANEL: Qualitative methods in D&I Research: Value, rigor and challenge - The value of qualitative methods in implementation research Alison Hamilton S53 PANEL: Qualitative methods in D&I Research: Value, rigor and challenge - Learning evaluation: The role of qualitative methods in dissemination and implementation research Deborah Cohen S54 PANEL: Qualitative methods in D&I Research: Value, rigor and challenge - Qualitative methods in D&I research Deborah Padgett S55 PANEL: Maps & models: The promise of network science for clinical D&I - Hospital network of sharing patients with acute and chronic diseases in California Alexandra Morshed S56 PANEL: Maps & models: The promise of network science for clinical D&I - The use of social network analysis to identify dissemination targets and enhance D&I research study recruitment for pre-exposure prophylaxis for HIV (PrEP) among men who have sex with men Rupa Patel S57 PANEL: Maps & models: The promise of network science for clinical D&I - Network and organizational factors related to the adoption of patient navigation services among rural breast cancer care providers Beth Prusaczyk S58 A theory of de-implementation based on the theory of healthcare professionals’ behavior and intention (THPBI) and the becker model of unlearning David C. Aron, Divya Gupta, Sherry Ball S59 Observation of registered dietitian nutritionist-patient encounters by dietetic interns highlights low awareness and implementation of evidence-based nutrition practice guidelines Rosa Hand, Jenica Abram, Taylor Wolfram S60 Program sustainability action planning: Building capacity for program sustainability using the program sustainability assessment tool Molly Hastings, Sarah Moreland-Russell S61 A review of D&I study designs in published study protocols Rachel Tabak, Alex Ramsey, Ana Baumann, Emily Kryzer, Katherine Montgomery, Ericka Lewis, Margaret Padek, Byron Powell, Ross Brownson S62 PANEL: Geographic variation in the implementation of public health services: Economic, organizational, and network determinants - Model simulation techniques to estimate the cost of implementing foundational public health services Cezar Brian Mamaril, Glen Mays, Keith Branham, Lava Timsina S63 PANEL: Geographic variation in the implementation of public health services: Economic, organizational, and network determinants - Inter-organizational network effects on the implementation of public health services Glen Mays, Rachel Hogg S64 PANEL: Building capacity for implementation and dissemination of the communities that care prevention system at scale to promote evidence-based practices in behavioral health - Implementation fidelity, coalition functioning, and community prevention system transformation using communities that care Abigail Fagan, Valerie Shapiro, Eric Brown S65 PANEL: Building capacity for implementation and dissemination of the communities that care prevention system at scale to promote evidence-based practices in behavioral health - Expanding capacity for implementation of communities that care at scale using a web-based, video-assisted training system Kevin Haggerty, David Hawkins S66 PANEL: Building capacity for implementation and dissemination of the communities that care prevention system at scale to promote evidence-based practices in behavioral health - Effects of communities that care on reducing youth behavioral health problems Sabrina Oesterle, David Hawkins, Richard Catalano S68 When interventions end: the dynamics of intervention de-adoption and replacement Virginia McKay, M. Margaret Dolcini, Lee Hoffer S69 Results from next-d: can a disease specific health plan reduce incident diabetes development among a national sample of working-age adults with pre-diabetes? Tannaz Moin, Jinnan Li, O. Kenrik Duru, Susan Ettner, Norman Turk, Charles Chan, Abigail Keckhafer, Robert Luchs, Sam Ho, Carol Mangione S70 Implementing smoking cessation interventions in primary care settings (STOP): using the interactive systems framework Peter Selby, Laurie Zawertailo, Nadia Minian, Dolly Balliunas, Rosa Dragonetti, Sarwar Hussain, Julia Lecce S71 Testing the Getting To Outcomes implementation support intervention in prevention-oriented, community-based settings Matthew Chinman, Joie Acosta, Patricia Ebener, Patrick S Malone, Mary Slaughter S72 Examining the reach of a multi-component farmers’ market implementation approach among low-income consumers in an urban context Darcy Freedman, Susan Flocke, Eunlye Lee, Kristen Matlack, Erika Trapl, Punam Ohri-Vachaspati, Morgan Taggart, Elaine Borawski S73 Increasing implementation of evidence-based health promotion practices at large workplaces: The CEOs Challenge Amanda Parrish, Jeffrey Harris, Marlana Kohn, Kristen Hammerback, Becca McMillan, Peggy Hannon S74 A qualitative assessment of barriers to nutrition promotion and obesity prevention in childcare Taren Swindle, Geoffrey Curran, Leanne Whiteside-Mansell, Wendy Ward S75 Documenting institutionalization of a health communication intervention in African American churches Cheryl Holt, Sheri Lou Santos, Erin Tagai, Mary Ann Scheirer, Roxanne Carter, Janice Bowie, Muhiuddin Haider, Jimmie Slade, Min Qi Wang S76 Reduction in hospital utilization by underserved patients through use of a community-medical home Andrew Masica, Gerald Ogola, Candice Berryman, Kathleen Richter S77 Sustainability of evidence-based lay health advisor programs in African American communities: A mixed methods investigation of the National Witness Project Rachel Shelton, Lina Jandorf, Deborah Erwin S78 Predicting the long-term uninsured population and analyzing their gaps in physical access to healthcare in South Carolina Khoa Truong S79 Using an evidence-based parenting intervention in churches to prevent behavioral problems among Filipino youth: A randomized pilot study Joyce R. Javier, Dean Coffey, Sheree M. Schrager, Lawrence Palinkas, Jeanne Miranda S80 Sustainability of elementary school-based health centers in three health-disparate southern communities Veda Johnson, Valerie Hutcherson, Ruth Ellis S81 Childhood obesity prevention partnership in Louisville: creative opportunities to engage families in a multifaceted approach to obesity prevention Anna Kharmats, Sandra Marshall-King, Monica LaPradd, Fannie Fonseca-Becker S82 Improvements in cervical cancer prevention found after implementation of evidence-based Latina prevention care management program Deanna Kepka, Julia Bodson, Echo Warner, Brynn Fowler S83 The OneFlorida data trust: Achieving health equity through research & training capacity building Elizabeth Shenkman, William Hogan, Folakami Odedina, Jessica De Leon, Monica Hooper, Olveen Carrasquillo, Renee Reams, Myra Hurt, Steven Smith, Jose Szapocznik, David Nelson, Prabir Mandal S84 Disseminating and sustaining medical-legal partnerships: Shared value and social return on investment James Teufe

A communal catalogue reveals Earth's multiscale microbial diversity

Our growing awareness of the microbial world's importance and diversity contrasts starkly with our limited understanding of its fundamental structure. Despite recent advances in DNA sequencing, a lack of standardized protocols and common analytical frameworks impedes comparisons among studies, hindering the development of global inferences about microbial life on Earth. Here we present a meta-analysis of microbial community samples collected by hundreds of researchers for the Earth Microbiome Project. Coordinated protocols and new analytical methods, particularly the use of exact sequences instead of clustered operational taxonomic units, enable bacterial and archaeal ribosomal RNA gene sequences to be followed across multiple studies and allow us to explore patterns of diversity at an unprecedented scale. The result is both a reference database giving global context to DNA sequence data and a framework for incorporating data from future studies, fostering increasingly complete characterization of Earth's microbial diversity.Peer reviewe

A communal catalogue reveals Earth’s multiscale microbial diversity

Our growing awareness of the microbial world’s importance and diversity contrasts starkly with our limited understanding of its fundamental structure. Despite recent advances in DNA sequencing, a lack of standardized protocols and common analytical frameworks impedes comparisons among studies, hindering the development of global inferences about microbial life on Earth. Here we present a meta-analysis of microbial community samples collected by hundreds of researchers for the Earth Microbiome Project. Coordinated protocols and new analytical methods, particularly the use of exact sequences instead of clustered operational taxonomic units, enable bacterial and archaeal ribosomal RNA gene sequences to be followed across multiple studies and allow us to explore patterns of diversity at an unprecedented scale. The result is both a reference database giving global context to DNA sequence data and a framework for incorporating data from future studies, fostering increasingly complete characterization of Earth’s microbial diversity

Multidimensional signals and analytic flexibility: Estimating degrees of freedom in human speech analyses

Recent empirical studies have highlighted the large degree of analytic flexibility in data analysis which can lead to substantially different conclusions based on the same data set. Thus, researchers have expressed their concerns that these researcher degrees of freedom might facilitate bias and can lead to claims that do not stand the test of time. Even greater flexibility is to be expected in fields in which the primary data lend themselves to a variety of possible operationalizations. The multidimensional, temporally extended nature of speech constitutes an ideal testing ground for assessing the variability in analytic approaches, which derives not only from aspects of statistical modeling, but also from decisions regarding the quantification of the measured behavior. In the present study, we gave the same speech production data set to 46 teams of researchers and asked them to answer the same research question, resulting insubstantial variability in reported effect sizes and their interpretation. Using Bayesian meta-analytic tools, we further find little to no evidence that the observed variability can be explained by analysts’ prior beliefs, expertise or the perceived quality of their analyses. In light of this idiosyncratic variability, we recommend that researchers more transparently share details of their analysis, strengthen the link between theoretical construct and quantitative system and calibrate their (un)certainty in their conclusions