Improving further education provision for learners from minority ethnic groups: a review of the evidence

This paper summarises the findings from a review carried out by the National Research and Development Centre for adult literacy and numeracy (NRDC) for the Quality Improvement Agency (QIA) between April and September 2007. The study sought to summarise what is known about the participation of those from minority ethnic groups in Further Education (FE) and to identify effective practice in helping them to achieve

Best practice statement : use of ankle-foot orthoses following stroke

NHS Quality Improvement Scotland (NHSQIS) leads the use of knowledge to promote improvement in the quality of health care for the people of Scotland and performs three key functions. It provides advice and guidance on effective clinical practice, including setting standards; drives and supports implementation of improvements in quality, and assessing the performance of the NHS, reporting and publishing findings

Best Practice Statement : Use of Ankle-Foot Orthoses Following Stroke

The development of this Best Practice Statement (BPS) was a collaboration between NHS Quality Improvement Scotland (NHS QIS), National Centre for Prosthetics and Orthotics, University of Strathclyde and a multidisciplinary group of relevant specialists. NHS QIS is a strategic health board which has a lead role in supporting the NHS in Scotland to improve the quality of healthcare. It does this by producing advice and evidence in a number of different formats, including BPS. These statements reflect the commitment of NHS QIS to sharing local excellence at a national level, and the current emphasis on delivering care that is patient-centred, cost-effective and fair. As part of a scoping exercise commissioned by NHS QIS in 2007, allied health professionals (AHPs) across Scotland identified the use of AFOs following stroke in adults as a clinical improvement priority. Orthotic intervention following stroke has been recognised as a treatment option for many years, but there is wide variation in current practice, and a lack of evidence-based research to determine the optimal rehabilitation programme for individuals following stroke. Stroke is the most frequent cause of severe adult disability in Scotland, with approximately 8,500 diagnoses of first-ever stroke each year, and more than 70,000 individuals affected by the condition. A recent Scottish Government strategy document confirms stroke as a national clinical priority for the Scottish NHS. In addition to developing a BPS and sharing this with healthcare professionals across Scotland, the initiative also sought to develop resource material to support the implementation of the BPS and to share the work internationally. In order to inform the development of the BPS a systematic literature review on AFO use following stroke was undertaken, including work of both a qualitative and quantitative nature. The full literature review, together with recommendations for future research, was included in the BPS. As it was felt that many medical professionals and AHPs may be unfamiliar with the principles underpinning orthotic practice, additional educational resources that would improve understanding of the reasons why the recommendations were being made were developed and included in the BPS. In addition to the full BPS, the key recommendations were summarised as a two-page 'quick reference guide' for ease of use in a clinical setting

Creating a Workforce Analytics Action Plan

An action plan can be a useful tool for implementing workforce analytics, especially if there are multiple people involved in the process. This resource provides some general advice on action planning, with specific examples for advancing the use of workforce analytics. What is an Action Plan? An action plan outlines the actions, resources, and people needed to achieve goals. It is vital to strategic planning. Action plans include the following components: • A well-defined goal • Tasks/steps needed to reach the goal • Individuals or teams responsible for each task • Task milestones and timeframes • Resources and data needed to complete tasks • Evidence or measures to evaluate progress • Desired long-term outcomes Having this information in one place makes it easier to plan and to track progress. Why Develop an Action Plan? Developing an action plan is essential to project success. An action plan provides credibility to a project and increases efficiency and accountability. In addition, an action plan helps to prepare for potential obstacles. Benefits of an action plan include the following: • Provides clear direction • Helps prioritize tasks • Highlights what steps need to be taken and when they should be completed • Tracks progress towards meeting the goal(s) How to Write an Action Plan 1. Determine what you want to achieve and define your goal Start by thinking about where you are and where you want to be, and write down your goal. Then, ensure that it is a SMART goal: • Specific: well defined and narrowed for more effective planning • Measurable: evidence to show progress or that the goal has been achieved • Attainable: make sure you can reasonably accomplish your goal within your timeframe • Relevant: your goal should align with your mission and desired long-term outcomes • Timely: set a timeline and completion date Use a SMART goal worksheet, like the following example, to assist your team in defining your goal. What do you want to achieve? We want people from Human Resources (HR) and Child Welfare (CW) to work together, break down silos, and get the right people at the table. We want HR and CW to use workforce metrics to make data-informed decisions to address workforce challenges Specific. Once key stakeholders are identified, the first meeting will be scheduled at least 6 weeks in advance to allow time on everyone’s calendars. This will also allow time to pull initial workforce metrics to review or create a catalog of potential metrics to discuss with the new team during the first meeting. After this, meetings should occur at least quarterly, allowing sufficient time for follow-up on recommendations between meetings while ensuring a sustained focus. Ongoing meeting frequency should be guided by the needs of the agency. Your goal should align with your mission and desired long-term outcomes. By creating a multidisciplinary team, the agency can make more informed decisions about the child welfare workforce (e.g., decisions about recruiting, hiring, training, promotion, staff development needs). Identify and contact stakeholders to participate in the initial meeting within the next month. Host the first meeting within the next quarter. This group will convene regularly, and no end date is planned at this time. Rewrite your goal using SMART principles: Establish a multidisciplinary team (comprised of HR and CW leadership and frontline staff) to regularly examine CW workforce metrics to make data-driven workforce decisions. 2. Determine the steps/tasks that are required to meet the goal: Start by clearly defining each task and ensuring they are manageable; break down larger, more complex tasks into smaller ones if necessary. It is important that the entire team be involved in this initial process, to ensure everyone is aware of their roles and responsibilities in achieving the goal(s). A table, such as the one used in the example below, can track tasks, people responsible for each task, and due dates. This will be the foundation of your action plan. 3. Prioritize tasks and establish timeframes Prioritize each task and determine realistic timeframes. Ensure the person or team responsible for task completion knows their capacity before establishing timeframes. For example, in the table above, the HR and CW directors are assigned a number of tasks. However, if these individuals are taking vacation or have other pressing deadlines during these timeframes, then the due dates may need to be adjusted. 4. Establish milestones Milestones are smaller goals that indicate project progress. Milestones motivate the team and provide them an opportunity to recognize their achievements. Milestones can be especially motivating for long-term projects, when the final due date is far away. When establishing milestones, it is helpful to start from the end goal and work backwards. Do not spread milestones too far apart—2 to 4 weeks is best. For example, with the tasks listed above, milestones could include “select members of the project team” or “determine which workforce metrics CW/HR can each provide to the team.” 5. Determine the desired outcome for action steps The overall goal is determined at the beginning of action planning, but there may be several desired outcomes to achieve in pursuit of the overall goal. These outcomes are another way of breaking down the goal into smaller, incremental steps. To establish desired outcomes, determine what changes the project would like to see and link these to the action steps. Sometimes several activities result in achieving one outcome, and sometimes one activity has several outcomes. An example of a desired outcome may be “Establish a multidisciplinary team to examine combined workforce metrics from CW and HR to address CW workforce issues.” 6. Identify what resources are needed Before beginning the project, ensure the team has the necessary resources to complete the tasks. Resources can be financial, human resources (e.g., staff, consultants, volunteers), technology, or other material goods. If the necessary resources are not available, develop a plan to acquire what is needed. Continuing with the example above, resources could include those required for examining workforce metrics, such as a list of HR and CW reports and data available, reporting software, time allocations for analysts, etc. 7. Establish what evidence determines a step has been achieved As action steps are developed, consider the information sources and data collection methods needed to help the team determine when an action step has been successfully completed. Sources of information may include project staff, stakeholders, project documentation, project reports, data from trusted sources, interviews or focus groups, and/or observations. An example of this could be “CW and HR staff from various levels commit to participate in the workgroup.” 8. Create a written action plan Develop a written action plan that everyone understands and that can be shared among the team. The document should be accessible by all team members and should be editable so that any continued adjustments can be made. Ensure that the plan clearly conveys the essential components

Competency-Based Personnel Selection Oklahoma - Summary

Oklahoma Department of Human Services (OKDHS) is a state-operated child welfare system, but hiring has always been done locally. The agency is divided into five field regions serving 27 districts and 77 counties. In 2017, OKDHS employed 1,780 Child Welfare Specialists (caseworkers) and 388 supervisors. That same year, the annual turnover rate was 25%. They applied to be a QIC-WD site with the goal of strengthening their child welfare workforce. When OKDHS started working with the Quality Improvement Center for Workforce Development (QIC-WD), a steering committee was established to participate in a needs assessment process, determine an intervention, and support implementation. The needs assessment took many months and included surveys and a review of Human Resources data. Hiring of new workers, promotion decisions for lead workers and supervisor positions, and staff recognition and rewards were identified as potential areas for intervention. The steering committee recognized that the criteria used for employee selection varied widely across the state and there was no standardized process from office to office. They expressed a need to better understand what characteristics were associated with good performance and how to hire people with those characteristics, and therefore decided to develop a competency-based personnel selection process. A theory of change was developed to explain how buy-in for and use of a standardized hiring process would improve worker retention and performance. The QIC-WD reviewed relevant research and best practices for employee selection. They conducted a job analysis and identified 24 important competencies for the Child Welfare Specialist job through a series of behavioral event interviews and surveys. The QIC-WD team developed ways to assess whether a job candidate possessed these competencies. The standardized hiring process ultimately included a structured interview, a typing test, and a writing assessment. A behaviorally anchored rubric was used to facilitate reliable ratings of the competencies. This video highlights the experience of staff and leaders who were involved in the new hiring process. The QIC-WD began to train potential hiring panel members and Administrative Assistants to implement the standardized personnel selection process in 2020. To provide flexibility and meet agency needs, in-person and virtual interview processes were developed. In addition, all hiring materials and processes were paperless. Two resources were developed to support implementation—a step-by-step guide to assist the Administrative Assistants with their administrative tasks in the process and a rating guide for hiring panels to use during the interview. Training was ongoing and refresher trainings were offered to support implementation. A randomized control trial design was used to test the intervention and key findings also include implementation learnings. Between October 2020 and February 2022, 716 interviews were conducted using the standardized hiring protocol, with most (69%) led by a twoperson hiring panel. Reactions to the process were more positive among those hired via the competency-based selection process; specifically, they were more likely to be satisfied with their experience and to agree that it provided an opportunity to show their capabilities, provided valuable information about the job, and was conducted fairly. After six months in the job, candidates hired using the competency-based selection process reported higher work engagement than those hired with the usual process, but similar levels of self-efficacy, perceived fit, job satisfaction, organizational commitment, and intentions to stay. Nonetheless, those who were hired using the new process were more likely to stay in the job during the study period; specifically, the hiring process lowered the risk of leaving by 22%. Those who chose to leave the agency, however, did so more quickly than those who were hired using the old process. Finally, performance ratings from supervisors at six months after hire did not differ based on which selection process was used, but within the group hired using the new process, higher scores on the selection tools were associated with better performance. Those who scored above certain levels were as much as 5.5 times as likely to have higher than average performance than those below those levels

Frontline Job Redesign Louisiana - Key Findings

Background In 2017, the QIC-WD began working with the Louisiana Department of Children and Family Services (DCFS). At that time DCFS had an average turnover rate of 24% across the state with one region having a rate of 51% and another with a low of 8%. A thorough needs assessment identified that high caseloads and a large number of administrative tasks were barriers to caseworkers’ ability to effectively engage in the clinical aspects of their work and led to the decision to redesign the frontline caseworker job. The intervention created a new job, the Child Welfare Team Specialist (CWTS), responsible for administrative tasks and reorganized workers into Permanency (former Foster Care) and Prevention teams. Implementation began in July 2019 and rolled out in two additional groups in November 2019 and February 2020. The evaluation included an implementation evaluation and a quasi-experimental pretest-posttest nonequivalent groups design comparing workers providing investigative, in-home, and foster care services in three parishes implementing the job redesign (the experimental group) to a group of workers providing the same services in six similar parishes doing business as usual (the comparison group). The goals of the evaluation were to determine whether job redesign reduced overload and increased worker well-being, increased time spent on clinical work with children and families resulting in improved practice, and decreased turnover as outlined in the site logic model and theory of change. Workforce Demographics Data were collected in May 2020 from 270 caseworkers in the study sample, 78 from the redesign group and 192 from comparison parishes. Of these survey respondents 94% were female,72% were African American, 22% were White, 2% were Hispanic, 4% identified as multi-racial or other, and less than 1% identified as Asian or as Indigenous or Pacific Islander. The average age was 38 years. The majority (58%) had bachelor’s degrees. Another 41% had master’s degrees. Approximately 34% had social work degrees. There were no significant differences between experimental and comparison groups related to gender, race, or age. Respondents in the comparison group reported a higher percentage of master’s degrees than those in the experimental group. Of 18 CWTS, 83% were female, 78% were African American and 22% were White; 78%, and all who responded had bachelor’s degrees in fields other than social work. The average age was 35 years. Evaluation Findings Focus group feedback indicated that participants were satisfied with the model and wanted to see it maintained in some form, particularly the role of the CWTS. Participants cited a wide range of benefits for the workforce including less stress and improved morale. They pointed to improvements in timeliness and quality of the work and favorable impacts on children and families, including more timely referrals, earlier service provision, faster case closure, and reductions in cases going to family services or foster care. Surveys were administered in May 2020, three months after full implementation of the redesign, and again in November 2021. Caseworkers in the experimental group scored significantly higher than caseworkers in the comparison group on measures of team cohesion, worklife balance, fit with the organization, and fit with their work group, and significantly lower than the Frontline Job Redesign Key Findings Louisiana Department of Children and Family Services 2 | July 2023 key findings comparison group on role conflict. Expected group differences in self-efficacy, fit with the demands of the job, organizational commitment, work engagement, job involvement, job embeddedness, burnout and role overload did not materialize and there were no significant differences in job satisfaction or intent to stay or leave the agency at either time point. While there were no significant group differences in perceptions of supervisor support or competence on either survey, experimental group participants rated their supervisors more highly on planning and organizing work -unit activities on both the initial and follow-up surveys. Early results of the redesign found that caseworkers in the experimental group rated their work stress significantly lower than the comparison group, however, group differences in ratings of work stress were no longer significant on the follow-up survey in November 2021, which could be related to COVID-19 restrictions and major hurricanes affecting both the experimental and comparison parishes. Following the implementation of the redesign (May 2020), caseworkers in the experimental group rated their jobs significantly lower on job complexity, degree of specialization required, and variety of skills needed than they had in an earlier assessment conducted as part of the needs assessment (in 2018). This suggests that those in redesigned teams perceived their jobs to be more streamlined, however, these differences were not maintained in November 2021, which may be a result of virtual work arrangements. Time study data indicated increases in the percentage of time caseworkers in redesigned units spent on clinical activities, such as interviews, home visits, and case planning, and that the new CWTSs were taking on a wide range of administrative tasks, such as opening and closing cases in the data systems, obtaining consents and archiving records. Specifically, between July 2019 and October 2021, workers in Prevention Units providing investigative and short-term in-home services increased the proportion of time spent on clinical aspects of the job from 46% to 65% and time spent on administrative aspects of the job decreased from 53% to 33%. In the Permanency Units, time spent on clinical aspects of the work increased from 55% to 61%, while time spent on administrative tasks decreased from 45% to 39%. Random samples of cases served by the experimental and comparison groups were evaluated using a customized tool to examine aspects of the work expected to be positively impacted by the job redesign, such as getting services to families more quickly, making transfers to in-home or foster care services more seamless, and improving the quality and timeliness of services to children in out-of-home care and their parents. Reviews showed statistically significant improvements for the experimental group from baseline to full implementation in accuracy of risk and safety assessments; early provision of prevention services; timely involvement of the family services worker in cases identified as requiring in-home family services; and quality and frequency of contacts with children and parents. Significant improvements occurred in diligent efforts to locate absent parents, and timely notifications to participants in Family Team Meetings which were part of the CWTS’s duties. Changes in early provision of prevention services; timely involvement of the family services workers and timeliness of notifications to families were significantly greater for the experimental group than for the comparison group. Between the rollout of the first redesigned units in July 2019 and the state’s move to virtual work in March 2020, admissions to foster care in the experimental parishes decreased, while admissions rose in the comparison parishes. The difference in foster care placements (compared to what would have been expected if the experimental parishes followed the same trend as the comparison parishes) was statistically significant and translated to 103 fewer families experiencing a foster care admission. In the eight months following the move to virtual work the gap Workers in Prevention and Permanency Teams increased time spent on clinical tasks as a result of the redesign. between the expected and actual levels of placements for the experimental group narrowed. DCFS reported that cases accepted for investigation during this time were more serious, making it more difficult to safely maintain children in their homes. Recurrence rates during this time period did not increase suggesting that the lower rates of foster care placements in the experimental parishes were not offset by a rise in subsequent maltreatment. Based on supervisor ratings of behaviors associated with risk of turnover, in May 2020 workers in the experimental group were significantly less likely to exhibit elevated risk than workers in the comparison group. Specifically, the experimental group was significantly less likely than the comparison group to leave early, expend less effort, be less productive, show less focus, show less interest in clients, contribute less in meetings, show less enthusiasm for the agency mission, do only the minimum amount of work, and show less interest in pleasing and more dissatisfaction with their supervisor. However, there were no significant group differences in supervisor’s assessments of their workers’ risk of turnover on the November 2021 survey. Analysis of group differences in actual turnover rates before and after implementation were inconclusive given the small number of data points available and the degree of variation in rates by state fiscal year. Although not all hypothesized changes in worker wellbeing, job satisfaction, and commitment were supported, the job redesign showed promise in reducing perceptions of work stress and role conflict, increasing time available for clinical work with families, and improving case practice. Additional research is needed with respect to intervention effects on turnover. The Team This project would not have been possible without the partnership of DCFS. They identified a statewide team that represented all of the regions and key positions in the organization. They committed to and participated in monthly meetings over multiple years to plan and support the initiative and its evaluation. They also provided essential data and valuable insight throughout the project. We greatly appreciate the work of all members of the steering committee, the training unit, Louisiana Civil Service, human resources, and all of the staff who worked to make this project a success. We are especially indebted to Marketa Garner Walters for having the vision and courage to commit the agency to undertaking a job redesign which had never been done in Child Welfare before and to Rhenda Hodnett and Leslie Calloway for their leadership and dedication to this project. The QIC-WD would also like to extend a special thanks to Brandy Malatesta, Shelly Johnson, Anthony Ellis, Connie Guillory, Fertaeshia Broussard, Leslie Breaux, Melissa Thompson, Leslie Lyons, and Stacey Mire for their many contributions, and to Elizabeth Reveal and Ryan Dodge for their work in providing and interpreting the administrative data used in the evaluation

Creating a Workforce Analytics Team

When conducting workforce analytics, it can be helpful to adopt a team approach and involve various experts and stakeholders, each of which can make a valuable contribution. The Big Picture • Ensure that you have the commitment and buy-in from leaders who control the agency’s budget and policies. • Team membership may vary depending on the analytics goals and the stage of the work, but potential participants include representatives from agency leadership, Human Resources (HR), child welfare, Information Technology (IT), and training. • It’s likely that no one person is going to possess all the skills needed; building a strengths-based, interdisciplinary team can be considered the pooling of cognitive resources. • Focus on who has the specific skills and capacities needed to engage in workforce analytics, rather than focusing on specific roles or titles. Identifying Key Personnel • Agencies will need to consider who will need to be involved to launch a successful workforce analytics partnership by determining: o Who the champions are among HR and child welfare staff to facilitate partnership conversations o The key personnel in HR and child welfare who will need to access, manipulate, and analyze the data o The key personnel in HR and/or child welfare who can present findings to stakeholders in relevant and actionable ways • Agencies will also need to identify the primary staff in HR and child welfare who will need to see, use, and understand the data Key Skills Needed The following skills should be represented in your interdisciplinary team (CIPD, 2018): Macro Skills • Ability to link HR and business strategies • Ability to assess organizational readiness for analytics initiatives • Ability to build the business case for analytics • Ability to demonstrate the credibility and value added by HR • Ability to use analytical insights to improve talent management processes • Ability to define the roles, capabilities, and structures required to maximize analytics effectiveness Micro Skills • Question-driven mindset • Ability to calculate descriptive statistics (e.g., means, ranges, and percentiles) • Ability to analyze trends in the data and interpret the meaningful aspects of that information within the context of the agency • Ability to test plausible hypotheses to establish the root cause of workforce issues • Ability to segment data and metrics to highlight specific areas of concern or opportunity • Ability to effectively and convincingly present insight and conclusions • Ability to examine the correlation, and level of causation, between two or more metrics • Ability to build multivariate models (Note that this is a very advanced skill

Disability Inclusion

The following resources provide information to help agencies create disability-inclusive workplace policies and practices. The Office of Disability Employment Policy (ODEP) is a non-regulatory federal agency that promotes policies and coordinates with employers and all levels of government to increase workplace success for people with disabilities. The Employer Assistance and Resource Network on Disability Inclusion (EARN) is an ODEP technical assistance center that provides resources to help employers recruit, hire, retain, and advance individuals with disabilities and comply with their responsibilities under the ADA. Inclusion@Work: A Framework for Building a Disability-Inclusive Organization allows organizations to explore strategies for creating disability-inclusive workplace cultures and meeting your organization\u27s diversity, equity, inclusion, and accessibility goals. State Exchange on Employment and Disability (SEED) is a state-federal collaboration that supports state and local governments in adopting and implementing inclusive policies and best practices that lead to increased employment opportunities for people with disabilities and a stronger, more inclusive American workforce and economy. Work Matters Checklist: Policy Options for Enhancing the Employment of People with Disabilities outlines policy options for state and local government leaders to enhance the employment of people with disabilities. The Job Accommodation Network (JAN) provides extensive technical assistance on the reasonable accommodations provision of the ADA. JAN provides expert guidance, training, and resources to individuals and employers on all aspects of accommodations. • Employers\u27 Practical Guide to Reasonable Accommodation Under the Americans with Disabilities Act (ADA) is a summary of some of the most frequent issues that employers have regarding accommodations and ADA compliance and ideas for resolving them. • Disability Employment and Inclusion: Your Guide to Success guides readers through the process of creating and executing an effective disability employment and inclusion program. BenchmarkABILITY is a self-assessment tool for organizations that are interested in the representation of people with disabilities in their workforce. Organizations can use this tool to track progress toward the implementation of effective human resource policies and practices around disability in six specific categories: • Recruitment & Hiring • Career Development & Retention • Accessibility & Accommodation • Compensation & Benefits • Diversity & Inclusion • Metrics & Analytics. The National Organization on Disability is a nonprofit that increases work opportunities for people with disabilities • NOD Employment Tracker™ is a free survey that assesses an organization’s current disability employment policies and practices, including strategy, talent outcome metrics, climate and culture, talent sourcing, people practice, and workplace tools and accessibility. Upon completion, the organization will receive a scorecard that can assist them in developing plans and priorities for improving employment practice and policy. • National Organization on Disability: Services provides summaries of resources and services available from NOD. Disability:IN is a nonprofit resource for business disability inclusion worldwide. • The Disability Equality Index (DEI) is a joint initiative with the American Association of People with Disabilities and is a benchmarking tool that helps companies build a roadmap of measurable, tangible actions that they can take to achieve disability inclusion and equality. The DEI provides an objective score and roadmap on disability inclusion policies and practices. Getting to Equal: the Disability Inclusion Advantage: This research conducted by from Accenture, in partnership with Disability:IN and the American Association of People with Disabilities, reveals that companies that embrace best practices for employing and supporting more persons with disabilities have outperformed their peers. Putting the “D” in DEI: Disability Inclusion in Your DEI Efforts. Since 1990, the Americans with Disabilities Act has protected employees from discrimination based on disability. Despite this civil protection, the unemployment rate for people with disabilities remains significantly higher than that of nondisabled individuals. For those with disabilities who are employed, they continue to face significant barriers to gainful employment that ensures a living wage and is free from ableism. In this presentation, Nellie Galindo, MSW, MSPH, discusses ways your organization can better recruit, retain, and accommodate employees with disabilities, as well as dispel common myths about working with a disability

Workforce Metrics Using Child Welfare Data

Workforce metrics that can be constructed from child welfare data fall into three categories: caseload/workload, case continuity, and worker- and unit-level variability. Caseload/Workload Metrics Caseload metrics can help to • Assess compliance with caseload standards (e.g., policy, statute) • Describe and inform case assignment decisions • Describe and address the work burden experienced by staff • Describe and address inequities in caseloads among workers • Inform workforce planning or staffing decisions • Monitor caseloads to assess policy or practices intended to affect caseloads Caseload metrics are among the most commonly used workforce metrics that can typically be obtained from child welfare data. These metrics are usually calculated as caseload per worker as of a specific date. Generally, there are two basic options for obtaining these metrics: Option One – Average Aggregate Estimates1 Example: Total number of active children in care divided by the number of workers serving them Advantages: Relatively easy to compute from basic reports Disadvantages: Can’t be segmented, may produce underestimates if workers are assigned more than one case type Option Two – Average Estimates per Assigned Worker2 Example: Number of children currently assigned to a worker summed across all assigned workers divided by the total of assigned workers Advantages: Can be segmented, accounts for all cases a worker carries, produces a distribution, can be used with workload estimates for weighted caseload analysis Disadvantages: Must be assembled from case-level data where case assignment information regarding worker and worker position data can be attributed to specific time periods (days, months, quarters, etc.) 1 Option One – Aggregate Estimate = ��/Wpt AACCt = Aggregate Average Caseload per Worker at time t ACupt = Number of Active Cases (u = children, families) of type (p = intake, assessment, ongoing, etc.) on (t = date) Wpt = Number of Workers Providing p at t. 2 Option Two – Disaggregated Estimate ACCt = ∑ , . / ACCt = Average Number of Active Cases per Worker at time t. ACuwt = Number of Active Cases (u – children, families) assigned to worker (w) on (t=date). Wt = Assigned Workers at t Continuity Metrics Measures of continuity address the degree to which cases receive attention from the same worker over time. The concept related to outcomes is that greater continuity of involvement of the same workers over time may lead to outcome improvements due to greater levels of family engagement, reduced loss of information, and fewer disruptions of planning and other processes. Case continuity metrics can help to • Identify where (regions, offices, supervisory units) there might be concerns about the level of engagement with children and families • Assess the impact of policies or practices intended to improve case continuity • Support Continuous Quality Improvement (CQI) initiatives focused on reducing variability in continuity Continuity metrics depend on having data about case duration, key case events, and an understanding of the policy and practice related to the type of continuum service provided. Critical to this is the ability to understand which workers were assigned to a case and when. Case continuity measures are a relatively new thing, and their use in child welfare management as indicators is still rare and perhaps limited to a few studies. Thus, there are no standard or common metrics for case continuity. That said, there is some evidence that increased continuity is associated with improved outcomes. Example metrics: • Average number of workers assigned per month • Average largest percent of duration with an assigned worker • Average percent of duration covered by first or last worker assigned The figure below illustrates one measure of case continuity. The idea is that across the entire period of time the service is provided, if one worker is assigned, the level of continuity is 100%. If more than one worker is assigned, the metric is reduced proportional to duration by the amount of time the worker with the longest assignment is assigned. For CPS assessments with short duration (typically less than 30–45 days), continuity may be a minor concern. That said, some disruptions can occur. For placement cases with longer durations (median durations of 9–12 months), continuity is a major concern. Over these types of time spans, there are many potential opportunities and reasons for lower continuity. Knowledge of policy and practice can also inform us about desirable and undesirable loss of continuity. Worker- and Unit-Level Variability Metrics This type of metric is a little more complicated to explain, thought the concept is pretty straightforward. The notion is that a worker or a group of workers (supervisory unit) acquires certain “characteristics” over time as they experience work with children and families. These “characteristics” will be different because the experiences are different. We would expect the experiences of a worker in the Bronx to be different in some ways compared to a worker from rural Colorado. Worker- and unit-level variability metrics can help to • Identify where (regions, offices, supervisory units) there are potential gaps in applying a range of agency policy • Highlight possible explanations for variability that may be systematic and amenable to change • Support continuous quality improvement (CQI) initiatives focused on reducing variability in practice and policy Worker- and unit-level variability metrics depend on • The characteristics of the children and families that are served • The likelihood on average that a decision (e.g., removal) will occur • Worker assignment data • The amount of time we can accumulate information about the range of cases a worker or supervisory unit serve The last point is important because over time the characteristics and decisions for a worker or unit become more stable, and any changes we observe (possibly because of a change in policy or practice) are less subject to small sample biases. Example metrics: • Average age of children served • Average risk level of children served • Proportion of assessments with an open case • Proportion of African American children removed compared to white children. The histogram below shows the distribution of nine workers with varying proportions of decisions to open a case for ongoing services. Proportion of Cases Opened Of the 9 workers, 2 have opened a case less than 15% of the time, 5 have opened a case 15 to 20 percent of the time, and 2 have opened a case 20 to 30 percent of the time. Level of Analysis If assembled well, all three types of the metrics can be segmented by organizational levels and over time. The ingredient of time is critical because these sorts of metrics can play important roles in quality improvement process analysis where the usual goal is to reduce variability in performance over time. Examples of levels of analysis for child welfare workforce metrics: • Average Caseload by Supervisory Unit • Average Worker Placement Proportion by Office • Change in Worker’s Average Case Risk Level Over Time Important Methodological Concept: Position vs. Person The concepts of persons and positions in relation to cases is fundamental to being able to implement the methods needed to produce the sort of metrics presented here. This concept is also important as we consider how metrics obtained from HR data might be related to child welfare outcomes. For example, if we are interested in metrics associated with supervisory unit-level variability, we need to link cases to units, and for that we need to know what position in the agency the worker occupied. If we are interested in case continuity, we are most likely concerned with who was assigned to the case. Ideally, the child welfare information system will allow us to identify both the assigned person and assigned position. Knowing what the system can produce is critical to understanding the range of valid metrics that can be produced. Conclusion In general, the type of metrics just discussed can be used to help understand how agencies can improve outcomes. Ideally, testing to see if and how these metrics might help explain the variability in child welfare outcomes would be one of their primary uses. However, these metrics can also be used to help set HR-related performance targets, with caseload targets being the most common example

Organizational Change Process Milwaukee - Needs Assessment Summary

Exploration of Needs The Quality Improvement Center for Workforce Development (QIC-WD) worked with the Division of Milwaukee Child Protective Services (DMCPS) to conduct a needs assessment. A variety of information was explored to learn about the current workforce and the agency’s workforce practices, including 1) recruitment, hiring, and retention metrics and processes, 2) organizational culture and climate, and 3) other workforce processes and initiatives, such as onboarding, mentoring, supervision, performance management, and employee recognition. More specifically, four major types of information were considered: 1) objective data from various agency databases and reports; 2) subjective perceptions of staff and supervisors, gathered through surveys and focus groups; 3) expertise and input of the project steering committee; and 4) QIC-WD team expertise and feedback. The goal was to identify areas of need within DMCPS that could be addressed through a workforce intervention that also met a set of QIC-WD criteria. Identification of Priority Needs/Relevant Findings One of the surveys that the QIC-WD administered was the Organizational Social Context (OSC) Measure, which assesses organizational culture and climate. Broadly speaking, culture refers to shared expectations and norms that characterize and direct behavior in the workplace, and climate refers to shared employee perceptions regarding how their work environment impacts their psychological well-being and functioning (Glisson et al., 2012). Organizational culture and climate are important because they are associated with a variety of important employee, organizational, and client outcomes, such as job satisfaction, organizational commitment, attitudes toward evidence-based practice, turnover, provision of mental health treatment and child welfare services, and child and youth psychosocial functioning (e.g., Aarons et al., 2012; Aarons & Sawitzky, 2006; Glisson & Green, 2006; Glisson & Green, 2011; Glisson & Hemmelgarn, 1998). Ninety-eight frontline staff and supervisors completed the OSC, which represented a response rate of 83%. The results showed opportunities for improvement in both culture and climate, in all six dimensions of culture and climate measured by the OSC: proficiency, resistance, rigidity, engagement, functionality, and stress, which fell outside of the normal range. Subsequent focus groups among staff, as well as root cause analysis discussions with the steering committee, provided further detail about the dynamics and underlying factors that contributed to the OSC scores. Four worker focus groups and two supervisor focus groups were conducted to support the needs assessment process. Groups were asked: 1) What makes you stay at DMCPS? 2) In what ways have your job expectations been met or not met? How might unmet expectations affect worker retention/turnover at DMCPS? 3) What do you see as the primary sources of your stress at work? What could the agency do to mitigate or help you manage this stress? 4) When a critical incident occurs, what support does DMCPS provide you to cope with the incident? What other types of support or resources would be helpful? 5) How do informal leaders influence organizational culture and climate, and how the work of the agency is accomplished? Organizational Change Process Division of Milwaukee child protective services Needs Assessment summary 2 | may 2022 needs assessment summary 6) In your opinion, what are some of the main reasons that workers leave DMCPS? If you left the agency, would you be inclined to fill out an exit survey? Why or why not? 7) On the topic of worker retention, if you could provide one piece of advice or feedback to DMCPS leadership what would it be? 8) DMCPS will be selecting an intervention to implement and test to improve workforce outcomes? What advice do you have around installing the selected intervention? Similar themes emerged in the focus groups and root cause analysis discussions with the steering committee. Ultimately, organizational culture and climate was identified as the optimal issue to target through the QICWD project. Though there were other areas identified as potential needs, many did not present good opportunities for intervention. For example, though there was interest in reconsidering the minimum qualifications and the hiring process, aspects of those were governed, in part, by state policy and statute, so they were not sufficiently amenable to change through this project. Intervention Selection The QIC-WD identified four alternative interventions to offer DMCPS to address organizational culture and climate: 1) Availability, Responsiveness, and Continuity (ARC; Hemmelgarn & Glisson, 2018), 2) Trauma Resilient Communities (TRC; Crossnore Communities for Children, n.d.), 3) Competing Values Approach (CVA; Cameron & Quinn, 2011), and 4) Civility, Respect, and Engagement at Work (CREW; Osatuke et al., 2013). Purveyors of the ARC and TRC interventions gave virtual presentations to the steering committee prior to the selection meeting; the QIC-WD team presented similar information on CVA and CREW as purveyors were not available to do so. Following these preliminary presentations, the QIC-WD engaged the steering committee in a decision-making process that involved discussion and live rating of each of the four interventions. The QIC-WD team and the steering committee considered the fit of each option with the following QIC-WD criteria: 1) alignment with agency need, 2) level of existing evidence of effectiveness, 3) applicability to other agencies’ needs and circumstances, 4) scope and magnitude, relative to agency and QIC-WD capacity, 5) evaluation potential, 6) contribution to a diverse array of interventions across QIC-WD sites, and 6) agreeable to both the agency and the QIC-WD. Ratings showed CREW to be most widely preferred, followed by TRC, CVA, and ARC. However, when asked to rank their preference of interventions among the four possible interventions using the same survey software, CREW was identified as first, followed by ARC. After reviewing the results, the DMCPS Administrator detected concerns among frontline staff about choosing CREW and then re-opened the conversation amongst the steering committee members. Frontline staff indicated that their perspective was not sufficiently represented and that they preferred an intervention that gave frontline staff greater voice and input into agency operations, that encouraged collaboration between frontline staff and leadership, and that would have an external consultant supporting implementation as an accountability mechanism. After further discussion, the DMCPS Administrator put forward a recommendation that ARC would be the agency’s preferred intervention, because it was more strongly favored by frontline staff. This was discussed by the Implementation Team members who agreed, changing the selection from CREW to ARC. Compared to CREW, which focuses more heavily on behavior change among frontline teams, ARC better involves management, addresses overall organizational functioning, and systematically involves frontline staff in addressing challenges in the organization. In addition, because ARC was developed to improve organizational culture and climate as specifically as measured by the OSC, ARC aligned most tightly with the needs that were identified through the OSC