In Introductory Psychology, 30 graduate TAs/faculty teach 2700 students annually. This year we developed an assessment program to improve student learning and graduate teaching training (Shigeto et al., 2010). Part of the program studied the pedagogical value of using student response systems to answer in-class multiple choice questions. Prior research lacks scientific rigor and provides equivocal evidence that SRSs improve learning (Caldwell, 2007)

Grison, Sarah

Luke, Steven G.

Shigeto, Aya

Watson, Patrick D.K.

English

SPARK: Scholarship at Parkland | Parkland College

Parkland College
Psychology Faculty Psychology
2010
Because You Can't Teach It All and They Won't
Read It All: Student Response Systems Do Improve
Learning
Steven G. Luke
Sarah Grison
Parkland College, sgrison@parkland.edu
Aya Shigeto
Patrick D.K. Watson
Open access to this Poster is brought to you by Parkland College's institutional repository, SPARK: Scholarship at Parkland. For more information,
please contact spark@parkland.edu.
Recommended Citation
Luke, Steven G.; Grison, Sarah; Shigeto, Aya; Watson, Patrick D.K.(2010, January). Because you can't teach it all and they won't read it
all: Student response systems do improve learning. Poster presented the National Institute on the Teaching of Psychology, St. Petersburg
Beach, FL.
Because You Can’t Teach It All and They Won’t Read It All:
Student Response Systems Do Improve Learning  
Steven G. Luke, Sarah Grison, Aya Shigeto, and Patrick D. K. Watson
University of Illinois at Urbana-Champaign
Data Acquisition and Analyses Discussion
1. Having more SRS Qs is associated with 
improved learning. In the short term, exposure to 
more SRS Qs led to greater learning of material not
In Introductory Psychology, 30 graduate 
TAs/faculty teach 2700 students annually. 
This year we developed an assessment 
program to improve student learning and 
Logit Mixed Model Analysis: Created best-fit model 
for binomial post-test data  (correct/incorrect for each 
item) Model accounted for effects of predictors while
Assessing Learning: Students earned participation 
points to complete online multiple choice tests. Post-
test data was analyzed if a student did the pre test
R h Q ti
          
covered in lecture, regardless of whether students 
read the text or not.
2. The benefit of SRS use persists to the end of the 
term. Students who read <40% of text benefitted 
from more SRS Qs even 3 months later, regardless  of 
whether the concepts were lectured on.
3. Low readers benefit more than high readers
graduate teaching training (Shigeto et al.,  
2010). Part of the program studied the pedagogical 
value of using student response systems to answer in-
class multiple choice questions. Prior research lacks 
scientific rigor and provides equivocal  evidence that 
SRSs improve learning (Caldwell, 2007). 
.        
statistically controlling random variables.
Best-Fitting Model: Excluded content chapter, high-
school rank, and ACT scores, as complex models did 
not improve the prediction of post-test data. 
Predictors:  -Number of Qs: (Less: 4 Qs/More: 8 Qs)
-Topic Covered: (Lect/No Lect)
R d Ch (L 40% / Hi h 60%)
         - , 
but not if they got a pre-test item correct or did not 
attend lecture.
Pre-Test:     -2 weeks before lectures, N=620, (38%) 
-16 New Qs: (8 Learning/8 Memory)
Post-Test 1: -2 weeks after lectures, N=390, (24%)
-16 New Qs: (8 Learning/8 Memory)
esearc  ues ons
We investigated the effects of SRS comprehension 
checks on student learning by focusing on 3 issues:
Q 1: Does the number of SRS Qs impact student 
learning?(Preszler et al., 2007)
Q 2: Do effects of SRS Qs on learning persist?
        
from having more SRS Qs. But no other personal 
variables predicted performance (e.g., ACT  score, 
high school percent rank) (Shigeto et al., 2010).Results – Immediate Post-Test
Implications
This research will improve student learning in Intro 
- ea  s: ow: <   g : >  
Random: -Student; Question; Section 
Post-Test 2: -3 months after lectures, N=297, (18%)
-16 New Qs: (8 Learning/8 Memory)
Short-Term Assessment
1
n s%
)
Short-Term Learning Improved for Info Not in Lecture
Experimental Methods
Findings: 
More SRS Qs      Higher ACC: 
Only for topics not in lecture
(z = -5.9, p < 0.001)
Low readers Lower ACC: 
Regardless of Num of SRS Qs   
( 3 07 < 0 005)
Design for SRS Comprehension Checks
          
(Crossgrove & Curran, 2008; Rubio et al, 2008)
Q 3: Do some groups benefit more from SRS Qs? 
(Crossgrove & Curran, 2008; Reay et al., 2007)
Psych by changing SRS training for graduate TAs to 
emphasize best practice in using SRSs:
• For more frequent comprehension checks
• For comprehension checks of material not in lecture
• To orient students to the most important topics.
Future research on the Intro Psych program will build 
f h fi di i i0
0.2
0.4
0.6
0.8
A
c
c
u
r
a
c
y * *
n.s.
. .
P
o
s
t
-
T
e
s
t
 
1
 
A
C
C
 
(
%
z = - . , p  .For 2 years Intro Psych has given TAs standardized 
training on SRS best practice. Student participation 
based on SRS responses is about 10% of their grade.
Subjects:   -1647 students in 30 sections
Materials:    -16 SRS Qs: (8 Learning / 8 Memory)
-1 Learning Objective per Mult Choice Q
Conditions: Number of Qs: (Less=4 Qs / More=8 Qs)
Results – Delayed Post-Test
rom t ese n ngs to nvest gate:
• Do students learn more depending on the type of 
SRS Q (knowledge, comprehension, application)? 
• Do SRS Qs aid learning by orienting to core ideas? 
•Will SRS Q effects be eliminated if we support text 
reading (Freeman et al., 2007,Watson et al., 2010)?  
Not Lectured On Lectured On Not Lectured On Lectured On
Low Readers High Readers
Low # of SRS Qs High # of SRS Qs
P
References -        -Topic Covered: (½ Lect/ ½ No Lect)
Procedures: -Crossed design for TAs in Groups A / B Caldwell, 2007. CBE-Life Sci Ed, 6, 9.
Crossgrove & Curran, 2008. CBE-Life Sci Ed, 7, 46. 
Freeman et al., 2007. CBE-Life Sci Ed, 6, 132.
Preszler, Dawe, Shuster, & Shuster, 2007. CBE-Life Sci, 6, 29.
Reay, Li, & Bao, 2008. Am J Phys, 76, 71–178. 
Rubio, Bassignani, White, & Brant, 2008. AMJ, 190, W319. 
Sect 1 = Less Qs (½ Lect, ½ Not)
Memory 
Lectures
TA Group A: 
15 Sects
9 TAs 
Learning 
Lectures Sect 2 = More Qs (½ Lect, ½ Not)
S t 2 L Q (½ L ½ N )
Sect 1 = More Qs (½ Lect, ½ Not)
Long-Term Assessment
0.4
0.6
0.8
1
A
c
c
u
r
a
c
y * *
n.s. n.s.
Long-Term earning Improved for “Low Readers”
T
e
s
t
 
2
 
A
C
C
 
(
%
)
Findings: 
More SRS Qs      Higher ACC: 
Only for low readers
(z = -2.3, p < 0.05)
ff f f S S Q Shigeto, Grison, Luke, & Watson, 2010. Poster at NITOP, FL. 
Watson, Grison, Luke, Shigeto , 2010. Poster at NITOP, FL. 
Acknowledgments
The authors thank the Psych 100 graduate instructors and 
undergraduates, the Psych Department, the Center for 
Teaching Excellence, the Teaching Advancement Board, and. 
Tim Stelzer, Distinguished Teacher-Scholar.
Memory 
Lectures
Learning 
LecturesTA Group B: 
15 Sects
9 TAs 
ec   = ess s  ect,  ot
Sect 2 = Less Qs (½ Lect, ½ Not)
Sect 1 = More Qs (½ Lect, ½ Not)
Sect 1 = Less Qs (½ Lect, ½ Not)
Sect 2 = More Qs (½ Lect, ½ Not)
0
0.2
Not Lectured On Lectured On Not Lectured On Lectured On
Low Readers High Readers
Low # of SRS Qs High # of SRS Qs
P
o
s
t
-
T No e ect o  num o  R  s x 
lecture coverage:
(p > 0.21)


Because You Can\u27t Teach It All and They Won\u27t Read It All: Student Response Systems Do Improve Learning

Scholarship at Parkland

Parkland CollegePsychology Faculty Psychology2010Because You Can't Teach It All and They Won'tRead It All: Student Response Systems Do ImproveLearningSteven G. LukeSarah GrisonParkland College, sgrison@parkland.eduAya ShigetoPatrick D.K. WatsonOpen access to this Poster is brought to you by Parkland College's institutional repository, SPARK: Scholarship at Parkland. For more information,please contact spark@parkland.edu.Recommended CitationLuke, Steven G.; Grison, Sarah; Shigeto, Aya; Watson, Patrick D.K.(2010, January). Because you can't teach it all and they won't read itall: Student response systems do improve learning. Poster presented the National Institute on the Teaching of Psychology, St. PetersburgBeach, FL.Because You Can’t Teach It All and They Won’t Read It All:Student Response Systems Do Improve Learning  Steven G. Luke, Sarah Grison, Aya Shigeto, and Patrick D. K. WatsonUniversity of Illinois at Urbana-ChampaignData Acquisition and Analyses Discussion1. Having more SRS Qs is associated with improved learning. In the short term, exposure to more SRS Qs led to greater learning of material notIn Introductory Psychology, 30 graduate TAs/faculty teach 2700 students annually. This year we developed an assessment program to improve student learning and Logit Mixed Model Analysis: Created best-fit model for binomial post-test data  (correct/incorrect for each item) Model accounted for effects of predictors whileAssessing Learning: Students earned participation points to complete online multiple choice tests. Post-test data was analyzed if a student did the pre testR h Q ti          covered in lecture, regardless of whether students read the text or not.2. The benefit of SRS use persists to the end of the term. Students who read <40% of text benefitted from more SRS Qs even 3 months later, regardless  of whether the concepts were lectured on.3. Low readers benefit more than high readersgraduate teaching training (Shigeto et al.,  2010). Part of the program studied the pedagogical value of using student response systems to answer in-class multiple choice questions. Prior research lacks scientific rigor and provides equivocal  evidence that SRSs improve learning (Caldwell, 2007). .        statistically controlling random variables.Best-Fitting Model: Excluded content chapter, high-school rank, and ACT scores, as complex models did not improve the prediction of post-test data. Predictors:  -Number of Qs: (Less: 4 Qs/More: 8 Qs)-Topic Covered: (Lect/No Lect)R d Ch (L 40% / Hi h 60%)         - , but not if they got a pre-test item correct or did not attend lecture.Pre-Test:     -2 weeks before lectures, N=620, (38%) -16 New Qs: (8 Learning/8 Memory)Post-Test 1: -2 weeks after lectures, N=390, (24%)-16 New Qs: (8 Learning/8 Memory)esearc  ues onsWe investigated the effects of SRS comprehension checks on student learning by focusing on 3 issues:Q 1: Does the number of SRS Qs impact student learning?(Preszler et al., 2007)Q 2: Do effects of SRS Qs on learning persist?        from having more SRS Qs. But no other personal variables predicted performance (e.g., ACT  score, high school percent rank) (Shigeto et al., 2010).Results – Immediate Post-TestImplicationsThis research will improve student learning in Intro - ea  s: ow: <   g : >  Random: -Student; Question; Section Post-Test 2: -3 months after lectures, N=297, (18%)-16 New Qs: (8 Learning/8 Memory)Short-Term Assessment1n s%)Short-Term Learning Improved for Info Not in LectureExperimental MethodsFindings: More SRS Qs      Higher ACC: Only for topics not in lecture(z = -5.9, p < 0.001)Low readers Lower ACC: Regardless of Num of SRS Qs   ( 3 07 < 0 005)Design for SRS Comprehension Checks          (Crossgrove & Curran, 2008; Rubio et al, 2008)Q 3: Do some groups benefit more from SRS Qs? (Crossgrove & Curran, 2008; Reay et al., 2007)Psych by changing SRS training for graduate TAs to emphasize best practice in using SRSs:• For more frequent comprehension checks• For comprehension checks of material not in lecture• To orient students to the most important topics.Future research on the Intro Psych program will build f h fi di i i00.20.40.60.8Accuracy * *n.s.. .Post-Test 1 ACC (%z = - . , p  .For 2 years Intro Psych has given TAs standardized training on SRS best practice. Student participation based on SRS responses is about 10% of their grade.Subjects:   -1647 students in 30 sectionsMaterials:    -16 SRS Qs: (8 Learning / 8 Memory)-1 Learning Objective per Mult Choice QConditions: Number of Qs: (Less=4 Qs / More=8 Qs)Results – Delayed Post-Testrom t ese n ngs to nvest gate:• Do students learn more depending on the type of SRS Q (knowledge, comprehension, application)? • Do SRS Qs aid learning by orienting to core ideas? •Will SRS Q effects be eliminated if we support text reading (Freeman et al., 2007,Watson et al., 2010)?  Not Lectured On Lectured On Not Lectured On Lectured OnLow Readers High ReadersLow # of SRS Qs High # of SRS QsPReferences -        -Topic Covered: (½ Lect/ ½ No Lect)Procedures: -Crossed design for TAs in Groups A / B Caldwell, 2007. CBE-Life Sci Ed, 6, 9.Crossgrove & Curran, 2008. CBE-Life Sci Ed, 7, 46. Freeman et al., 2007. CBE-Life Sci Ed, 6, 132.Preszler, Dawe, Shuster, & Shuster, 2007. CBE-Life Sci, 6, 29.Reay, Li, & Bao, 2008. Am J Phys, 76, 71–178. Rubio, Bassignani, White, & Brant, 2008. AMJ, 190, W319. Sect 1 = Less Qs (½ Lect, ½ Not)Memory LecturesTA Group A: 15 Sects9 TAs Learning Lectures Sect 2 = More Qs (½ Lect, ½ Not)S t 2 L Q (½ L ½ N )Sect 1 = More Qs (½ Lect, ½ Not)Long-Term Assessment0.40.60.81Accuracy * *n.s. n.s.Long-Term earning Improved for “Low Readers”Test 2 ACC (%)Findings: More SRS Qs      Higher ACC: Only for low readers(z = -2.3, p < 0.05)ff f f S S Q Shigeto, Grison, Luke, & Watson, 2010. Poster at NITOP, FL. Watson, Grison, Luke, Shigeto , 2010. Poster at NITOP, FL. AcknowledgmentsThe authors thank the Psych 100 graduate instructors and undergraduates, the Psych Department, the Center for Teaching Excellence, the Teaching Advancement Board, and. Tim Stelzer, Distinguished Teacher-Scholar.Memory LecturesLearning LecturesTA Group B: 15 Sects9 TAs ec   = ess s  ect,  otSect 2 = Less Qs (½ Lect, ½ Not)Sect 1 = More Qs (½ Lect, ½ Not)Sect 1 = Less Qs (½ Lect, ½ Not)Sect 2 = More Qs (½ Lect, ½ Not)00.2Not Lectured On Lectured On Not Lectured On Lectured OnLow Readers High ReadersLow # of SRS Qs High # of SRS QsPost-T No e ect o  num o  R  s x lecture coverage:(p > 0.21)

https://spark.parkland.edu/cgi/viewcontent.cgi?article=1004&amp;context=psych_fac

Because You Can\u27t Teach It All and They Won\u27t Read It All: Student Response Systems Do Improve Learning

Abstract

Similar works

Full text

Available Versions

SPARK: Scholarship at Parkland | Parkland College

Scholarship at Parkland