Over the last 45 years, the Nebraska Department of Correctional Services (NDCS) has made two substantial changes to its classification system. The first system was created and implemented in the 1970s. In 2005, Patricia Hardyman was contracted to update and modify the classification and reclassification system. Using statistical analyses of available data, a set of prediction models were created to score inmates on items that predicted future infraction behavior. However, the primary issues of the tools were that inmates’ scores were routinely over-classified and that substantial uses of overrides (approximately 40%) were indicated. After a review of the tools’ development methods, several issues were identified, including: a small development sample, a lack of prediction and outcome item specificity, lack of item weighting, a less than optimal feature select strategy, inefficient validation techniques, and the tool’s inability to assess infraction prediction posttransfer. It was therefore determined that this system required major improvements and a research project was outlined and contracted by the University of Nebraska, Omaha

Hamilton, Zachary

Kigerl, Alex

English

EXECUTIVE SUMMARY Over the last 45 years, the Nebraska Department of Correctional Services (NDCS) has made two substantial changes to its classification system. The first system was created and implemented in the 1970s. In 2005, Patricia Hardyman was contracted to update and modify the classification and reclassification system. Using statistical analyses of available data, a set of prediction models were created to score inmates on items that predicted future infraction behavior. However, the primary issues of the tools were that inmates’ scores were routinely over-classified and that substantial uses of overrides (approximately 40%) were indicated. After a review of the tools’ development methods, several issues were identified, including: a small development sample, a lack of prediction and outcome item specificity, lack of item weighting, a less than optimal feature select strategy, inefficient validation techniques, and the tool’s inability to assess infraction prediction post-transfer. It was therefore determined that this system required major improvements and a research project was outlined and contracted by the University of Nebraska, Omaha. The project was outlined to complete five major project stages. First, a process evaluation was completed consisting of an NDCS document review, classification and reclassification was observed, key staff were interviewed, focus groups of staff, research, and administrative personnel were facilitated to assess key advantages, disadvantages, and potential areas for improvement of the current classification system and processes. Several themes were described around the tool’s functionality and use, including: extensive use of overrides, bed-space driven placement needs, a lack of specialized placement options, mandatory override restrictions, illogical inclusion of certain predictor items, inconsistent scoring across assessment types, and programming availability. The culmination of findings outlined potential issues impacting accuracy and usage of scoring from users as well as methodological limitations of current classification and reclassification model construction. Next, we reviewed available NDCS data and developed an analysis plan for tool construction. Working with NDCS research staff, we identified a sample frame for initial and reclassification analyses. Ultimately, the samples collected consisted of 9,072 male and 1,582 female initial classification and 35,098 male and 2,449 female reclassification assessments of offenders incarcerated and supervised by the NDCS during the study period of August, 1991 and June, 2015. We then sought to develop statistical models for infraction prediction models. First, the current classification and reclassification models were assessed for their ability to predict infraction behavior. Next, feature selection procedures were completed, selecting items that improved prediction of three infractions outcomes – violent, serious, and non-serious. This was completed using several advanced multivariate selection techniques. These models were created in an effort to improve upon, and replace, the current classification and reclassification models. Study Findings Validation procedures were then completed. The new models created were compared to those currently in use based on the industry standard statistic – the Area Under the Curve (AUC). New models demonstrated substantial improvement compared to the previously developed Haryman tools. These findings confirm the predictive improvements gained via the methods and additional data used to develop the new infraction prediction tools. The resulting models identified risk scores for each offender within a given infraction type. A scoring guide is provided, identifying risk points associated with each tool’s items and responses. Offenders are to be scored on each item and their scores summed. The summary score for each of the three infraction models is designed to place them into one of four categories – High Violent, High Serious, Moderate, and Low (see Figure 2). In an effort to improve the classification system, the new tool is designed to inform and support classification staff efforts. Based on themes identified through the process evaluation, staff had indicated several issues that impacted the utility of Haryman tools’ results. In particular, the scored classification designation is often overridden as a result of NDCS or offender needs (i.e. bed space availability and programming). Therefore, staff indicated a need for the ability to move offenders to custody designations based on rationales that are not solely based on security. While infraction risk models were then developed. To ensure that the newly created tools were functional, provide face validity, and, in turn, gain user trust a cross-sectional group of NDCS staff were selected to review the tool. On July 26th the team of Subject Matter Experts (SMEs) gathered to review the tool and assess its functionality. SMEs were encouraged to provide feedback regarding the assessment items, usability, and overall design. Feasible adjustments to the tools were then completed and final models established. Ultimately, the created tool provides categories that indicate an offender’s infraction risk, instead of a one-to-one recommendation of custody designation. When used in conjunction with developed NDCS policy guidelines, the new classification schematic provides staff the flexibility to assign offenders to a lower/higher custody designation when agency or offender need requires. The new classification system also informs staff of an offender’s likely infraction type and risk following a transfer to a new facility, providing the opportunity to differentiate supervision strategies once an offender is residing in their new facility. We feel this categorization system is a novel advancement of prior approaches. However, we also note that it represents a substantial variation from current practice and will require NDCS efforts around training and policy development to operate efficiently. Recommendations Following the initial development of the tools, we have outlined several recommendations. While some will be addressed in the next phases of the project, others identify long-term goals for greater prediction accuracy, more efficient uses of resources, and additional research needing completed. 1. Create an implementation, training, and quality assurance plan 2. Continue improving the tool by adding items and collaborating with recent risk assessment efforts 3. Create efficient uses of assessment labor by identifying assessment redundancies 4. Create an inventory of interventions and forecast agency incarceration needs 5. Evaluate override factors and practices Next Steps There are two remaining phases of the project. Manuals and training materials will need to be developed to adjust the current classification tools and identify any updated policies and new procedures. Materials developed will guide training of new staff as well as refreshers for current staff. Goals for booster training and other quality assurance guidelines will also be developed. All materials will be created in conjunction with NDCS SMEs. Following the development of training materials, an implementation plan should be established. The implementation plan is recommended to include a timeline for a graduated roll out of the new tools, a pilot study to assess system impact and address any modification needs to scoring and cut point placement, a training schedule, and outline for quality assurance checks and future validation analyses

The University of Nebraska, Omaha

Development and Validation of the Nebraska Department of Correctional Services Prison  Classification System

Development and Validation of the Nebraska Department of Correctional Services Prison Classification System

https://digitalcommons.unomaha.edu/cgi/viewcontent.cgi?article=1013&context=ncjrreports

Development and Validation of the Nebraska Department of Correctional Services Prison Classification System

Abstract

Similar works

Full text

Available Versions

The University of Nebraska, Omaha

The University of Nebraska, Omaha