174 Validating a Coding Tool for Translational Science Benefits Model (TSBM) Data: Delphi Methodology

OBJECTIVES/GOALS: To develop and validate a tool to systematically identify benefits accruing to research within the Translational Science Benefits Model (TSBM) framework. We used a Delphi panel to reach consensus among a group of experts on criteria required for a clinical, community, economic, or policy benefit to be verified as coming from research. METHODS/STUDY POPULATION: A coding tool with proposed criteria to verify each of the 30 benefits was created at UCI to confirm the TSBM benefits resulting from funded research. We convened 11 experts from 8 CTSA hubs, who consisted of evaluators (faculty and staff) with experience using the TSBM. A web-based survey was used for Round 1, followed by a panel discussion of remaining unvalidated criteria, and a Round 2 survey as the final decision for inclusion of items in the tool. Response options for each criterion were “yes, required” or “no, not required”. Criteria that reached consensus (>70% agreement) were considered validated for inclusion in the final version. Panelist suggested criteria in Round 1 were also incorporated in the Round 2 survey for consideration by the experts. RESULTS/ANTICIPATED RESULTS: In the web-based survey for Round 1, all 11 experts participated and 92% of criteria reached the determined consensus level (N = 157). The remaining 8% of the criteria (N = 13) were discussed during the panel meeting. The discussion, in which 8 experts participated, was moderated by UCI and took place virtually via Zoom. All experts were sent a recording of the discussion and given the opportunity to post comments online about the remaining criteria before, during, and for a day after the discussion. Round 2 will include 50 newly proposed criteria from panelists and the 13 criteria that did not reach consensus in Round 1. Based on the results of Round 2, the criteria that reach consensus will be included in the final version of the coding tool that can be used across all TSBM benefits. DISCUSSION/SIGNIFICANCE: Using the Delphi Methodology, we will have a standardized set of criteria that may be applied to determine whether a TSBM benefit has resulted from a specific research project or program. This standardization will allow for aggregation and comparison of data across CTSA hubs and further multi-level evaluation of impact

Formative Evaluation of a Student Symptom Decision Tree for COVID-19.

ObjectivesIn support of schools restarting during the COVID-19 pandemic, some schools partnered with local experts in academia, education, community, and public health to provide decision-support tools for determining what actions to take when presented with students at risk for spreading infection at school.MethodsThe Student Symptom Decision Tree, developed in Orange County, California, is a flow chart consisting of branching logic and definitions to assist school personnel in making decisions regarding possible COVID-19 cases in schools which was repeatedly updated to reflect evolving evidence-based guidelines. A survey of 56 school personnel evaluated the frequency of use, acceptability, feasibility, appropriateness, usability, and helpfulness of the Decision Tree.ResultsThe tool was used at least 6 times a week by 66% of respondents. The Decision Tree was generally perceived as acceptable (91%), feasible (70%), appropriate (89%), usable (71%) and helpful (95%). Suggestions for improvement included reducing the complexity in content and formatting of the tool.ConclusionsThe data suggest that school personnel found value in the Decision Tree, which was intended to assist them with making decisions in a challenging and rapidly evolving pandemic