As student retention and four-year graduation rates are
of institutional and national interest and frequently referred
metrics for college success, the Supplemental Instruction
(SI) program aims to reduce D’s, F’s and Q drop rates in
historically difficult classes. Although previous work done
by this group revealed that attending SI sessions for a firstyear
course (Introduction to Electrical Engineering)
positively impacted exam scores and subsequent course
grades [1], the program continues to experience low
participation rates. Emerging questions of student
behaviors in relation to attendance at SI sessions are
addressed in this article.
The study utilizes a mixed-methods approach,
incorporating quantitative data relating to grades and
attendance with qualitative data relating to student
awareness, use and perceptions about SI. These analyses
serve to gain an understanding of the effects of SI and
identify components of the program that students value.
Quantitative data was collected in the form of session
attendance logs, grade data, and student demographics.
Qualitative data was collected in the form of pre- and postsurveys
administered during the third and final week of the
semester.Cockrell School of Engineerin

Abraham, Nisha

Telang, Nina

English

Texas ScholarWorks

	Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 Understanding Behaviors of Attendance in Supplemental Instruction and Subsequent Academic Success in a First Year Engineering Course Nisha Abraham, Nina Telang  The University of Texas at Austin 2501 Speedway, Austin, TX 78712, USA E-mail: nisha.abraham@austin.utexas.edu, telang@ece.utexas.edu Abstract      As student retention and four-year graduation rates are of institutional and national interest and frequently referred metrics for college success, the Supplemental Instruction (SI) program aims to reduce D’s, F’s and Q drop rates in historically difficult classes. Although previous work done by this group revealed that attending SI sessions for a first-year course (Introduction to Electrical Engineering) positively impacted exam scores and subsequent course grades [1], the program continues to experience low participation rates. Emerging questions of student behaviors in relation to attendance at SI sessions are addressed in this article.       The study utilizes a mixed-methods approach, incorporating quantitative data relating to grades and attendance with qualitative data relating to student awareness, use and perceptions about SI. These analyses serve to gain an understanding of the effects of SI and identify components of the program that students value. Quantitative data was collected in the form of session attendance logs, grade data, and student demographics. Qualitative data was collected in the form of pre- and post-surveys administered during the third and final week of the semester.  1. Introduction      Supplemental instruction (SI) was created in 1973 at the University of Missouri in Kansas City, to improve grades in traditionally “difficult” classes and in turn promote student retention and graduation rates. In the thirty years since its inception, it has become one of the most widespread and effective academic support models [2].       In response to The University of Texas Task Force on Undergraduate Graduation Rates’ recommendations to increase the four-year graduation rate of first time in college students in the Cockrell School of Engineering (31% in 2011 [1]), the Electrical and Computer Engineering (ECE) department partnered with UT’s Sanger Learning Center in Fall 2015 and piloted SI sessions for the 2015-2016 academic year.       Results from this study revealed that session attendance positively impacted exam scores and DFQW (Ds, Fs, Q-drops, Withdraws) rates, and that participants had an overall favorable perception of the SI program [3]. These results were similar to previously reported studies [2], [4], [5], that have also shown a positive relationship between SI session attendance and overall course grades. However, we have found that attendance in these (optional) SI sessions has remained low.  Furthermore, the student perceptions of the traditional SI model, which uses collaborative group activities and discussions to help students better understand course materials were rated least helpful. This creates a tension with what has been shown in numerous studies [2], which is that SI’s use of collaborative techniques promotes the social interaction within an educational environment that improves learning and retention.      Other works have sought to determine factors that affect attendance in SI sessions, by using qualitative data on students attitudes to predict behaviors of attendance [6]. This work also found that influential individuals may be able to promote attendance to SI sessions. However, this work was performed in an accounting class, whereas our interest is in first-year engineering courses, where little research has been performed. Our study focuses on a first-year engineering course, and uses a mixed method analysis to determine factors that affect attendance and how increased attendance may lead to improved outcomes for students. We hope to use our findings to better address the needs of the student population and to promote attendance in the Supplemental Instruction sessions as an effective intervention to address retention and fail rates. 2. Design and Implementation      The Supplemental Instruction model is a peer-assisted learning model which employs active and collaborative 	Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 learning strategies to review class material and develop transferable study skills. Sanger Learning Center has coordinated SI programming at UT Austin for over 30 years, and has supported departments within the College of Liberal Arts and the College of Natural Sciences.      A partnership between the Sanger Learning Center and the ECE department was established in Fall 2015 to provide SI programming to support students enrolled in EE 302 Introduction to Electrical Engineering. The SI program employed undergraduate upper-class ECE students as SI leaders to lead bi-weekly study sessions. SI leaders were also required to participate in weekly professional development meetings with SI leaders for other courses and the program coordinator within Sanger. These meetings provided the SI leaders with direction and feedback. Regular observations were conducted by Sanger’s SI coordinator. The SI leaders were also responsible for collecting attendance at each session and administering programmatic assessment tools throughout the semester.  In an effort to continually improve the program and boost attendance, the SI program coordinator attended the University of Missouri’s Supplemental Instruction Training program and Conference in the summer of 2017. Based on her experiences and learning, the SI Leader training was revamped for Fall 2017, including a stronger emphasis on ensuring peer-led, collaborative practices inside SI sessions, observations conducted early and often, and the requirement for all SI leaders to plan their sessions with engaging activities that they submit for continual feedback and improvement. We believe these changes could have significantly impacted the attendance of the program and the subsequent student outcomes and would like to investigate further areas for growth.        To assess the impact of SI attendance on student achievement, this study addresses the following research questions: 1] How does students’ initial awareness and perceptions of resources influence SI attendance? 2] How does students’ perceptions of SI and it’s learning model (peer-led collaboration) influence SI attendance? 3] How does SI attendance affect academic performance in current coursework? 4] What is the perceived benefit of SI by participating students?  3. Methodology This study used a mixed methods approach to investigate the research questions. By collecting both quantitative and qualitative data we gained a better understanding of the student population choosing to attend the SI sessions, their motivations for attending, and the perceived value of the sessions. Considering a combination of quantitative and qualitative measures, we took an interpretative approach to examine the relationship between SI attendance and student’s academic performance, and applied these findings to further adapt the program to best meet the needs of the enrolled student populations.  3.1 Quantitative Data Collection Two forms of quantitative data were collected: 1. SI Program Usage: at the beginning of each session, students signed in with both their name and university unique identification number.  2. Grade Data: course letter grades and GPAs for all students enrolled in the course were gathered. Attendance data was documented by the SI leader at the start of each session and reported to the Sanger Learning Center, where the SI coordinator maintained a database that connected with the university’s registrar. The SI coordinator exported additional data regarding student information such as limited demographics and SAT scores. Additional demographics were provided by the School of Engineering’s office of academic affairs.        Students attending zero or one session were categorized as the non-SI group, whereas repeat attendees (those attending two or more sessions) were categorized as the SI group. With this definition of the SI group as those who returned, the quantitative data focuses on the outcomes for students who showed investment in using this resource versus those who did not. To examine the effects of SI on student academic performance, course grades were converted from nominal to ordinal data as per the university’s numerical grade point equivalencies.  3.2  Qualitative Data Collection Qualitative data was used to answer research questions regarding students’ perceptions of SI, its influence on attendance, and the perceived benefits of SI by participating students. The method of qualitative data collection was pre- and post-surveys, which were developed by translating a similar survey created by Goldstein and O’Donnell [6]. The pre-survey comprised four questions and was administered in the third week of the semester; it collected students’ names and university identifier numbers, and then polled the students on their 	Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 initial awareness and intention of using a list of academic support programs offered for the course. Some of these options included: one-on-one tutoring with a undergraduate tutor, study groups, office hours with a graduate TA or professor and Supplemental Instruction sessions led by a SI Leader. The two aspects of interest with each of these services were: 1. the type of expertise offered (high level content expert such as TA or professor versus intermediate expert such as a peer tutor or SI Leader versus low level expert  such as peers in study groups); and 2. the student ratio and interaction (one-on-one interactions such as tutoring and office hours versus group interactions such as study groups and SI sessions). It is our belief that these aspects, either individually or together, impact SI attendance and therefore warranted some investigation.  The post-survey was more extensive; identifying information was once again collected in the form of names and university identifier numbers. Students were then asked to choose which academic support services they had intended to use for the course and which they actually used. Students who used the SI service were asked to rate several aspects of the program, including the SI leaders, the group work model and perceived benefits of SI sessions.   4. Findings and Discussion   The pre-survey responses (n=177) indicated that 60% of the students who were aware of the SI sessions planned on attending. In comparison, more than 80% of the students intended to engage in peer study groups, and attend teaching assistant and professor office hours. In Figure 1, we show the student responses to the question, “Which of the following support services are you aware of?” and in comparison, their answers to the question, “Which of these support services do you plan on using this semester for EE 302?”. A significant decrease is shown between their awareness of one-on-one tutoring and SI sessions and their intention to use either or both of these services. These findings indicate that while students are highly aware of all the academic support services available to them, the aspect of content expertise is strongly valued in an academic support resource, unless students are using their own privately created peer study groups.       In Table 1 we summarize the student responses to our post-survey questions about their perceptions of SI. These student responses seem to indicate that the majority of the students found that the SI sessions helped them with the course content, and with their own perception of their exam performance. Student comments on the surveys corroborated these survey data. 80% of the students commented that they attended SI sessions because they benefited from them. They believed that the SI sessions helped them comprehend the subject matter better, helped them with exam preparation since the session leaders covered old exam problems, and helped with the particularly challenging aspects of the course.     In Table 2 we compare the mean course GPA of the Non-SI and the SI groups for Fall 2015 and Fall 2017. In Fall 2015 there was a 6% difference in mean course GPA between the Non-SI and SI groups. This gap was reduced to 5% between Non-SI and SI attendees in Fall 2017 (see Figure 2). Figure 3 displays the SI session attendance rate difference between Fall 2015 and Fall 2017 semesters. From Fall 2015 to Fall 2017, the attendance rate improved from 37.6% to 40.2%. One potential reason for the Non-SI group’s higher average GPA in both academic years is the optional nature of Supplemental Instruction sessions. Therefore, students who come to the university highly prepared may not seek out additional support. To address this, we looked at the average SAT score for Non-SI vs SI attendees in Fall 2017 and found a 25 point difference. A similar difference was seen in the SAT scores of the student population of Fall 2015 [3]. In Fall 2016, the course was taught by multiple professors without standardization, in terms of course schedule, presentation of topics, and exam questions and level of difficulty; therefore results from that semester have not been included. In an effort to better compare student performance based on predictors of preparedness for college, the student population was divided into five groups, each with a 120 point range of SAT scores and then analyzed for course GPA between non-SI and SI attendees. As seen in Table 3, four of the five groups showed higher course GPAs for SI attendees vs non-SI attendees. These results more accurately reflect our predictions of the impact of SI when comparing similar students. About twenty percent of the student population did not have recorded SAT scores, so were not included in this comparison. Future studies will focus on examining correlations between SI session attendance and final course grades, and to determine whether there is any significant relationship between the distribution of the DFQW percentages amongst the different attendance groups. We also intend to test and implement activities and exercises that are more engaging for the engineering student, since the traditionally used active group work has been consistently rated the least helpful by students in our surveys. 4. Summary 	Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018      From our qualitative survey data, we found that students were aware of the academic support resources offered by the University for this introductory course and their answers of intended use and actual attendance numbers indicate they intentionally made strategic choices in which of those services they took advantage of. We are also consistently finding that students prefer resources led by individuals with a high level of expertise and prefer group work only when they have created their own study groups. This could be a reflection of self-efficacy, control and self-direction that students value, and therefore influence their attendance to SI sessions. The consistency of the grade data comparisons for attendees versus non-attendees is shown over two years, but identifying similar students using standardized scores has allowed for a more accurate picture of which students benefit most from SI. Another important result is the lowering of the percentage of DFQW grades for the SI group. This trend was seen in both academic years. Overall, attendees continued to find SI sessions extremely helpful in clarifying confusing concepts and preparing them for exams.  Acknowledgement  This work was performed as research supported by the Sanger Learning Center and The Cockrell School of Engineering. References [1]	Wilmot, J., Peralez, K., & Telang, N. (2016). Supplemental Instruction Pilot Program for an Introductory Electrical Engineering Course. In Conference proceedings of the First Year Engineering Education Annual Conference, Columbus, Ohio.	[2] Dawson, P.; van der Meer, J.; Skalicky, J.; Cowley, K. (2014). On the Effectiveness of Supplemental Instruction: A Systematic Review of Supplemental Instruction and Peer-Assisted Study Sessions Literature Between 2001 and 2010. Review of Educational Research. 84 (4): 609–639.  [3]  Wilmot, J., & Telang, N. K.(2017). Assessment of Supplemental Instruction Programming on First Year Academic Success. In Conference proceedings of American Society of Engineering Education 2017 annual conference, Columbus, Ohio. [4] Malm, J., Bryngfors, L., & Mörner, L. L. (2012). Supplemental instruction for improving first year results in engineering studies. Studies in Higher education, 37(6), 655-666. [5] Malm, J., Bryngfors, L., & Mörner, L. L. (2016). The potential of supplemental instruction in engineering education: creating additional peer-guided learning opportunities in difficult compulsory courses for first-year students. European Journal of Engineering Education, 41(5), 548-561. [6] Goldstein, J., Sauer, P., & O'Donnell, J. (2014). Understanding factors leading to participation in supplemental instruction programs in introductory accounting courses. Accounting Education, 23(6), 507-526.   	  	Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018  Figure 1. Polled students’ awareness of and intention of use for selected academic support resources (n=177), Fall 2017   	 Agree	 Neutral	 Disagree	SI	sessions	helped	me	to	gain	a	better	understand	of	the	subject	matter	72%	 20%	 16%	SI	sessions	helped	me	gain	good	study	habits	and	self-discipline	32%	 52%	 16%	SI	sessions	helped	me	get	information	about	exam	materials	72%	 20%	 16%	SI	sessions	helped	me	perform	better	on	exams.	60%	 28%	 12%	Table 1. Student attendees rating of perceived benefits of SI sessions, Fall 2017    	Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 GPA	Fall	2015	 Fall	2017	Non	SI	 SI	 Non	SI		 SI	2.79	(N	=	242)	2.62	(N	=	146)	2.74	(N	=	199)	2.6	(N	=	134)	DFWQ%	 11.6	 9.3	 12.7	 9.35	Table 2. Mean course GPA and DFWQ rates for Non SI and SI groups in Fall 2015 and 2017.   Figure 2. Mean course GPA for Non SI and SI groups in Fall 2015 and 2017.  Figure 3: Comparison of SI session attendance in Fall 2015 and Fall 2017 semesters. 	Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018  SAT	scores	 1000-1120	 1130-1250	 1260-1380	 1390-1510	 1520-1600	Mean	GPA	(SI)	 2.22	 1.91	 2.25	 2.61	 3.38	Mean	GPA	(no	SI)	 1.67	 1.87	 2.24	 2.64	 3.18	Table 3. Mean Course GPA for different SAT score ranges (n=266), Fall 2017. 

Understanding Behaviors of Attendance in Supplemental Instruction and Subsequent Academic Success in a First Year Engineering Course

https://repositories.lib.utexas.edu/bitstream/2152/79919/2/understanding-behaviors-of-attendance-in-supplemental-instruction-and-subsequent-academic-success-in-a-first-year-engineering-course.pdf

Understanding Behaviors of Attendance in Supplemental Instruction and Subsequent Academic Success in a First Year Engineering Course

Abstract

Similar works

Full text

Available Versions

Texas ScholarWorks