Faber, David

English

Loyola Marymount University

Masthead Logo Journal of Catholic EducationVolume 22 | Issue 1 Article 155-29-2019NSBECS Program Effectiveness Surveys: A Toolfor Strategic Planning and School ImprovementDavid FaberDiocese of Grand Rapids, MI, dfaber@grcatholicschools.orgFollow this and additional works at: https://digitalcommons.lmu.edu/cePart of the Other Education CommonsThis Focus Section Article is brought to you for free with open access by the School of Education at Digital Commons at Loyola Marymount Universityand Loyola Law School. It has been accepted for publication in Journal of Catholic Education by the journal's editorial board and has been published onthe web by an authorized administrator of Digital Commons at Loyola Marymount University and Loyola Law School. For more information aboutDigital Commons, please contact digitalcommons@lmu.edu. To contact the editorial board of Journal of Catholic Education, please emailCatholicEdJournal@lmu.edu.Recommended CitationFaber, D. (2019). NSBECS Program Effectiveness Surveys: A Tool for Strategic Planning and School Improvement. Journal of CatholicEducation, 22 (1). http://dx.doi.org/10.15365/joce.2201172019218 Journal of Catholic Education / Spring 2019Journal of Catholic Education, Vol. 22, No. 1, Spring 2019, pp. 218-222. This article is licensed under a Creative Commons Attribution 3.0 International License.  https://doi.org/ 10.15365/joce.2201172019NSBECS Program Effectiveness Surveys: A Tool for Strategic Planning and School ImprovementDavid FaberDiocese of Grand Rapids, MichiganFor 20 years as a principal and superintendent of Catholic schools, I was frustrated by the lack of standardized measures available to me to report the effectiveness of the Catholic school(s) I served. With every new fam-ily inquiry, the first question immediately following “How much is tuition?” was “What are your test scores?” While test scores are valuable for formative assessment, student growth, and curriculum planning, I find them to be a woe-fully inadequate measure of the total quality of a school’s program. They are simply one piece of a much larger puzzle, yet for many years in education they were the only standardized measure for reporting the quality of our schools. What was missing were a set of metrics to demonstrate the value and quality of schools other than a simple summary of test scores. When the National Standards and Benchmarks for Effective Catholic Schools (NSBECS) were first developed, I was elated at the possibilities for measuring quality and for addressing total school improvement that the standards would provide. After being introduced to the standards, I began promoting their use within our diocese to benefit our strategic planning and school improvement efforts. I quickly noticed how these standards were better aligned to the needs of our schools than those of our previous accredi-tation process. I began advocating with our Michigan Non-public Schools Accrediting Association (MNSAA) to consider replacing our current stan-dards with the 13 standards and 72 benchmarks of the NSBECS. Fortunately, the seven Catholic dioceses of Michigan make up the largest constituency of the MNSAA, and after much discussion and study, it was decided to replace the previously-used standards with the NSBECS as the basis for future ac-creditations.1As a direct beneficiary of this new system for accreditation in 2013, we were able to take full advantage of the tools and resources provided by the Catholic School Standards Project website not only to assist us through the accreditation process but also as a framework for strategic planning and 1 For more information about the shift by MNSAA to the NSBECS, see the article by Erich and Salas, also in this issue.219Program Effectiveness Surveysschool improvement. We found the program effectiveness surveys to be most valuable because of their close alignment to the standards and the ability to use the data gathered to inform planning and to track school improvement progress. These program effectiveness surveys are available for parents, staff, and students both in English and in Spanish, and have provided for us not only the data needed for strategic planning and school improvement, but also an effective alternative to assess and communicate total school quality.     In the spring of 2014, we launched, by invitation to all our schools, an opportunity for their parents, staff, and students in grades 5-8 and 9-12 to participate in the program effectiveness surveys. We constructed an electronic version of the surveys in both English and Spanish using the content provid-ed by the Catholic School Standards Project (CSSP) website. 2We carefully crafted an introduction to provide context for participants, and added basic demographic information to allow for disaggregation of the data by school. At the close of the survey window, our Office of Catholic Schools collected the data and provided schools with a summary of their results as well as a summary of the diocesan-wide averages, so that each school could bench-mark their own scores against diocesan averages.We have included an open-ended element following the 42 parent and teacher questions and the 27 student questions, which was beneficial. The ad-ditional question simply states, “Use this space to clarify any of your respons-es.” This allows our office and the individual schools to benefit from not only the quantitative data that the surveys deliver, but also the qualitative data which parents, teachers and students provide. As with all anonymous surveys, some of these comments can be harsh, raw, and concerning, but they allow us to learn and grow. While positive word of mouth is our most powerful mar-keting tool, negative word of mouth can have a negative impact that’s equal or greater. Giving parents, teachers and students an outlet to address their concerns through the survey allows school leaders to address these concerns which may not have been previously brought to their attention. The ability to control the message regarding these concerns may ultimately help our schools to reduce the detrimental impact of negative word of mouth.In the first year of the surveys, 2013-14, we collected 872 parent responses, 311 staff responses, and 831 student responses. This represented approximately 25% of parents, 56% of staff, and 28% of students in grades 5-12. We took 2 Beginning in March 2019, an electronic version of these surveys is now available on the CSSP website. 220 Journal of Catholic Education / Spring 2019the 42 parent and teacher questions and the 27 student questions and their corresponding responses and categorized them into the four domains of the NSBECS: Catholic Identity and Mission, Governance and Leadership, Aca-demic Excellence, and Operational Vitality. We tallied the individual scores by school and by question, and we tallied the average of their scores in each of the domains. We also tallied the diocesan average by question and domain. Color-coded graphs were provided to our schools so that they could examine the results of each question and their average by domain, and compare it to the diocesan averages. The information we gained was useful to me and the schools that participated. In fact, it was so insightful that I began to use the individual school results and diocesan averages during my annual visit to each school. The data guided my meetings with principals and pastors, as it poses high-quality questions related to the results. The quality of my conversations with school leaders immediately improved as we examined real data regarding the perceptions of their schools’ parents, teachers and students. In one instance, there was a school where for years I had suspected the need for improvement, but when I attempted to communicate my perceptions, school leaders did not share my concerns. Then, the survey data began to speak for itself. When the principal and pastor viewed the data in Appendix A, I simply asked, “What do you observe?”  Their immediate response was, “We need to make some improve-ments.” These summary graphs have provided “aha” moments for many of our school leaders resulting in their ownership of the present reality.After five years of analyzing and sharing the results of the NSBECS pro-gram effectiveness surveys, the information and insights have greatly impact-ed strategic planning and school improvement efforts both in our schools and at the diocesan level. For example, Parents, staff, and students consistently score our schools the lowest in the operational vitality domain. Thus, we have dedicated two of the four strategic focus areas within our diocesan schools’ strategic plan to operational vitality.The program effectiveness surveys are now an integral tool embedded within our planning and improvement processes. We use them as a 360-de-gree element of our administrator appraisal process. The results are shared each year with both the pastor and the school’s local board of limited juris-diction, and they are used as part of the collection of evidence during the self-study and accreditation school visit. All schools, except for one inde-pendent Catholic school, participate in the surveys; however, that school has committed to implementing them during this upcoming school year. Ap-221Program Effectiveness Surveysproximate diocesan-wide participation has been as high as 60% among par-ents, 84% among staff, and 53% among students, yielding high quality insights into our schools. Appendix B is an unnamed school’s summary report included as an ex-ample. You will see the school’s 2018 parent results by question (color-coded bars) compared to the diocesan average (grey bars). The following graph shows the school’s previous five years of averages in each of the NSBECS domains (color-coded bars), compared to the averages for the diocese (grey bars). This report makes evident the trends that both the school and the diocese are able to use to demonstrate progress or regression. Please note: we have color-coded all the questions on the first graph by NSBECS domain, but the labels for the colors are found on the second page (the five-year trend graph). The sample contains a school’s two-page summary results not only for parents, but also for staff and students. To maintain the anonymity of the actual school community, we did not include the open-ended responses.  Lessons Learned from the Field1.	 The NSBECS Program Effectiveness Surveys have become founda-tional as a measure of quality for our schools.2.	 The information we gather and share provides specific direction and feedback for both strategic planning and school improvement efforts.3.	 Pastors, boards, and school leaders have appreciated the high-level overview that the graphed survey data provides about the perceptions of their parents, staff, and students. This is evident by how they use this data to drive their decision making. Perception is reality for con-stituents, and school leaders must consider these perceptions in their decision making. The year-over-year data also provides the necessary measure for seeing whether their efforts are working. 4.	 The surveys are an invaluable tool to me as superintendent. Having access to all the school scores provides insights that can be utilized for consultation purposes. The awareness of the range of scores alone adds perspective in conversations with pastors, boards, and school leaders about what is possible, what is normal, and what may need attention. I value deeply that our schools have entrusted this informa-tion to the Office of Catholic Schools, and recognize our responsibil-ity to keep it in confidence. 5.	 Our scores in most domains are generally trending up among parents and staff, while students’ scores have not. Our theory among school 222 Journal of Catholic Education / Spring 2019leaders and staff is that raising standards with middle school and high school students may not be as appreciated by the students as it is among the parents and staff. Whether our theory is accurate, notic-ing the trend has caused us to increase our efforts to improve student voice and student agency within our schools. These efforts further reinforce to the students that their feedback through the surveys is important.6.	 The survey of the staff provides the most informed perception of the school’s performance related to the standards. These are perception surveys. They are not the measure of whether the NSBECS are being met. However, in our experience there is a high level of correlation. It is our observation that the staff survey may be the most reliable indicator due to their access and familiarity with policy, procedure, and school operations.   7.	 The surveys are quite long – 42 questions for parents and staff, 27 questions for students. We hear regularly about the concern of survey fatigue, and it has required great effort to maintain, year over year, the highest level of participation. Having collected five years of data, we have decided to use the program effectiveness surveys every other year going forward. In the off years, we are excited to use the shorter NSBECS Defining Characteristics Survey. This will allow our data analysis to be solely focused on our Catholic identity in these off years, while still maintaining a biennial pulse on our overall health. I can’t wait to see what we learn. David Faber is Superintendent of Catholic Schools in the Diocese of Grand Rap-ids, Michigan. Correspondence regarding this article can be sent to Mr. Faber at dfaber@grcatholicschools.org 

NSBECS Program Effectiveness Surveys: A Tool for Strategic Planning and School Improvement

https://digitalcommons.lmu.edu/cgi/viewcontent.cgi?article=2127&amp;context=ce

NSBECS Program Effectiveness Surveys: A Tool for Strategic Planning and School Improvement

Abstract

Similar works

Full text

Available Versions

Loyola Marymount University