Hrepic, Zdeslav

Shaw, Kimberly

English

Columbus State University

Pedagogy and/or Technology: Making Difference in Improving Students’Problem Solving SkillsZdeslav Hrepic, Katherine Lodder, and Kimberly A. Shaw4225 University Avenue, Dept. of Earth and Space Sciences, Columbus State University, Columbus, GA 31907Abstract. Pen input computers combined with interactive software may have substantial potential for promoting activeinstructional methodologies and for facilitating students’ problem solving ability. An excellent example is a study inwhich introductory physics students improved retention, conceptual understanding and problem solving abilities whenone of three weekly lectures was replaced with group problem solving sessions facilitated with Tablet PCs and DyKnowsoftware [1,2]. The research goal of the present study was to isolate the effect of the methodology itself (usingadditional time to teach problem solving) from that of the involved technology.  In Fall 2011 we compared theperformance of students taking the same introductory physics lecture course while enrolled in two separate problem-solving sections.  One section used pen-based computing to facilitate group problem solving while the other section usedlow-tech methods for one third of the semester (covering Kinematics), and then traded technologies for the middle thirdof the term (covering Dynamics). Analysis of quiz, exam and standardized pre-post test results indicated no significantdifference in scores of the two groups. Combining this result with those of previous studies implies primacy of pedagogy(collaborative problem solving itself) over technology for student learning in problem solving recitations.Keywords: Problem solving, Computers in education, Tablet PC, DyKnowPACS: 01.40.Di, 01.40.Fk, 01.40.gb, 01.50.H-, 01.50.htINTRODUCTIONShortcomings of the traditional lecture in helpingstudents learn physics have been well established [3-5]. Prominent physics educators, such as Nobel Prizewinner Carl Wieman, have suggested that technologyin general can serve as a tool in creating courses thatare more interactive and engaging - and consequentlymore effective in promoting student learning [6].Rapid advancement and the popularity of tabletcomputing devices in recent years have beenattracting educators’ interest as instructional devices[7]. Of particular interest for physics and STEMeducators are pen-input computers due to the highlysymbolic nature of annotations in these fields,combined with the complex graphical representationsfor which input without pen writing is inadequate orhighly cumbersome. Tablet PCs, when wirelesslynetworked and combined with interactive software,can open a wide array of possibilities for activeinstructional methodologies [8].This study investigated the instructionaleffectiveness of wirelessly networked tablet PCscombined with software enabling real-timeinformation exchange among all participants(DyKnow) [9]. Tablet PCs can be operated with anelectronic pen (or by touch) in addition to thekeyboard and mouse/pad. Unlike most newlylaunched pen and/or touch slate devices, tablet PCsalso support high-end personal computer processing.Earlier studies indicated a great potential withthis technology in facilitating students’ problemsolving ability, which is a critical aspect of effectivephysics learning [1,2]. In a problem solving context,the use of Tablet PCs and DyKnow, students cansimultaneously annotate their respective group slide,then submit a pen-annotated solution to theinstructor. From his/her computer the instructormonitors the progress of all groups in real time and ifdesired comments and/or annotates on student slides.Instructor can also display individual or groupsubmissions for whole class discussions and candistribute the selected solution to all students with aclick. Sisson1,2 devoted one of the three weeklylecture classes to collaborative problem solvingsupported by this technology and improved students’i) conceptual understanding, ii) test scores(significantly) and iii) retention to the secondsemester course (significantly), all as compared tothe 5-year historical average [1,2].RESEARCH QUESTIONS ANDEXPERIMENTAL DESIGNWhen deploying this instructional technology,Sisson replaced one of the three lecture hours perweek with a substantially different activity - smallgroup problem solving facilitated with tablet PCs andinteractive software [1,2]. The gains she saw in hermetrics (conceptual understanding, problem solvingability and retention) were dramatic. However thestudy included potentially confounding variablesbecause the instructional methodology was changedwith a simultaneous introduction of the technology.Following up on these earlier results, the goal ofthe present study was to investigate whether theobserved changes could have resulted fromreplacement of a weekly lecture hour withcollaborative problem solving sessions alone, orwhether improvements in student outcomes can beattributed to use of the technology deployed tofacilitate the new pedagogy. It is possible thattechnology has been a convenient facilitation tool forthe instructor either with or without a direct effect onstudent learning.To investigate this question we conducted astudy combining lab and lecture parts of calculus-based introductory physics course. During Fall 2011we had two lab sections associated with the courseenabling an exchange of the experimental andcontrol groups. These lab sections were run in three-hour long blocks and we allocated one of these threehours to problem-solving practice. Participants inthis study thus had an hour of collaborative problemsolving on a weekly basis on top of their regularlecture time (which was structured in two 75-minsessions per week). In order to use this setting toexamine the effect of the Tablet PCs and interactivelearning software on student learning, one lab sectionstarted the semester collaboratively using TabletPCs, and the other was allotted the same time towork on the same problems, but used either 3’ x 4’whiteboards or wall whiteboard spaces tocollaboratively solve problems. After a third of thesemester was over and the first exam had beenadministered, lab sections exchanged protocols.After the semester was two-thirds completed and thesecond exam was administered (with the first thirdcorresponding to Kinematics and the second third toDynamics), each section had approximately theequivalent time in using both whiteboards and tabletPCs. Students were then permitted to choose whichtechnique to use when completing problem solvingtasks for the remainder of the term. It was randomlychosen that the Thursday lab group (section 1) be thecontrol during the first part of the study. Theexperimental design thus followed the sequenceshown in Table 1.TABLE 1. Outline of the Experimental Design1st third ofsemester2nd third ofsemesterSection 1(Thursday)Experimental(technology users)Control(whiteboard users)Section 2(Friday)Control(whiteboard users)Experimental(technology users)Students were also randomly assigned tocollaborative groups. In solving problems, studentswere not using the computer’s processing power fortasks different than facilitating classroom interactionto avoid advantaging one of the groups in ways notintended by experiment. During the Tuesday lectureclass following the problem solving sessions in thepreceding week, students were given a 15 minutequiz. Students from both sections were enrolled inthe same lecture class, so they each took the samequiz at the same time. Quiz problems were chosen tobe either directly isomorphic (the problem changednumbers and/or surface features only) or indirectlyisomorphic (the same conceptual problem, but adifferent variable solved for – such as solving akinematic problem for final velocity during therecitation, and for initial velocity on the quiz). Quizproblems were explicitly solved using a structuredproblem solving approach, and prompts for thestructure were included on the quiz itself. In both thefirst and second part of the study, we gave threequizzes. The students regularly solved the same setof problems in both lab sections, with the instructortaking care that equal amount of time was given toeach problem in both sections.Exam scores were also collected, as well as end-of-course grades, as broader measures of studentability in physics problem solving, and these datastreams were examined with respect to the problemsolving technology used, as well as studentdemographic variables. All problem sessions werevideo-taped, although that analysis will be reportedelsewhere.Results and AnalysisThe main findings reported here revolve around threeindependent metrics:a) Student scores achieved on common classquizzes following problem solving sessions.b) Class exams associated with topics ofkinematics and dynamics respectively, each ofwhich corresponded with one part of the studyc) Normalized gains on standardized testsassociated with kinematics (TUG-K test) anddynamics (FCI test).There were three quizzes associated with kinematicsand another three associated with dynamics. Eachsection had 20 students so the possible number ofquizzes that they would have taken was 60 forkinematics and another 60 for dynamics part.Results are presented in Table 2 below. From thequiz scores, we eliminated instances of quizzes takenby students who were absent from the precedingproblem-solving session/lab.According to the quiz results presented in Table2, the Thursday section performed better in bothsegments of the study by a very small margin(average of 72.6% vs. 67.5% for kinematics, and71.9% versus 71.6% for dynamics). Standarddeviations are also similar showing slightly largervariability in Friday group.We ran an ANCOVA test to determine thestatistical significance of these differences (i.e. toshow statistical insignificance). The data satisfied thetest of homogeneity of slopes assumption as theinteraction between students and technology modewas not significant (p=0.802) and also with partialη2=0.000, indicating that in the sample the meandifferences in quiz results are not a function of theindividual student differences or similarities.ANCOVA was performed with individual studentsdefined as covariant, as the same students weretaking multiple quizzes. This test was not significantfor the treatment effect, F(1,206)=0.487,MSE=4.930, p=0.486. Partial η2=0.002 for thetechnology usage (with η2=0.004 for studentvariable).TABLE 2: Student Scores on Quizzes and Tests Associated With Kinematics and Dynamics1st third of semester (Part I)KinematicsTHURSDAY Section used Technology2nd third of semester (Part II)DynamicsFRIDAY Section Used TechnologySection Metric N quizzes(of 60)Average(%)SD(%)N quizzes(of 60)Average(%)SD(%)THU Individual quiz score averages 54 72.6 21.4 53 71.9 21.5FRI Individual quiz score averages 55 67.5 24.7 50 71.6 23.8Section Metric N students(of 20)Average(%)SD(%)N students(of 20)Average(%)SD(%)THU Test Averages 20 70.9 14.7 20 69.7 11.1FRI Test Averages 20 70.5 17.0 20 69.3 10.6Table 2 also shows that the differences in testscore means between the experimental and controlgroups were virtually nonexistent (smaller than 1%for both tests), further supporting the finding that theusage of technology in solving problems did notmake a difference. The Thursday section was againindistinctly better in both segments (70.9% versus70.5% for kinematics and 69.7% versus 69.3% fordynamics). We report next on the findings associatedwith student scores on standardized test forkinematics (TUG-K) and dynamics (FCI) inrespective parts of the study (Table 3).TABLE 3: Student pre-post performance on standardized tests for Kinematics (TUG-K) and dynamics (FCI)1st third of semester (Part I)TUG-K testFRIDAY Section used Technology2nd third of semester (Part II)FCI testTHURSDAY Section Used TechnologySectionData set withinstandardizedtest Pre-PostscoreN (out of 20)Averagenormalized gainSDNormalizedgain for classaverage scoresAverage Posttest score (%)N (out of 20)Averagenormalized gainSDNormalizedgain for classaverage scoresAverage Posttest score (%)THU All PRE-POST Results 19 NA* NA* 39.6 75.9±15.0 19 51.6 24.3 50.3 71.4±20.3Without student withperfect pre-score 18 26.8 54.4 40.3 74.9±14.7 19 51.6 24.3 50.3 71.4±20.3Positive gains only 14 50.5 23.0 49.3 75.5±15.4 19 51.6 24.3 50.3 71.4±20.3FRI All PRE-POST Results 17 45.5 31.6 46.7 75.1±21.3 18 51.0 27.4 49.5 70.9±20.3Positive gains only 16 50.4 25.0 48.2 74.7±20.9 17 54.0 25.0 53.3 73.5±17.5*One student in the Thursday section had a perfect score on the pretest and missed one problem in the posttest which makes thenormalized gain score undefined.Although the technology using group had largergains in both standardized tests (Friday section forkinematics and Thursday section for dynamics) thedifferences are very small and not statisticallysignificant. The ANCOVA test for differences innormalized gain scores is F(1,67)=1.431,MSE=1402.54, p=0.236 with partial η2=0.021 for thetechnology usage which makes the effect oftechnology usage very small, and possibly comparableto factors unaccounted for. Note that this treatmentwas less likely to produce a measurable effect onstandardized tests than on quizzes and exams thatinvolved (to various degrees) isomorphic problems.This was the case in Sisson’s study [1,2] as well.Analysis was further performed on the data fromthe class as a whole, examining it for demographicstrends. As students did not all take the SAT,concordance scores between ACT and SAT wereutilized to extend comparisons based on studentmathematical ability. No significant correlations werefound between gender and any of the following: rawquiz score semester total, quiz totals after the first thirdof the semester, second third of the semester,individual exam scores or final exam scores.CONCLUSIONSEarlier studies showed strong improvements ofstudent learning when a portion of a lecture-basedcourse was dedicated to technology-facilitated groupproblem-solving sessions [1,2,10]. The goal of thisstudy was to determine whether the improvement instudent performance was primarily due to the changein methodology within the course, due to thetechnology usage itself, or possibly both. Wereplicated Sisson’s1,2 earlier successful deployment, inwhich she replaced one of the three hour long lectureclasses per week with technology-facilitatedrecitations. We kept the total of a 3-hour per weeklecture intact and added one hour per week of suchrecitations to the course by using one out of the threehours of the lab sessions per week. Using two labsections for collaborative problem solving we wereable to exchange experimental (technology facilitated)and control (whiteboard facilitated) treatments.We found no difference in students’ performancewhen they did and did not use computing technology.In addition to the possibility of true irrelevance of thecollaborative medium, it is possible that the findingmay be caused by students’ limited exposure totechnology which resulted in insufficient proficiencyin using it effectively. For example the Friday groupwhich started with technology, used it three times forone hour before the switch). Lack of a distinguishableeffect due to insufficient exposure to differenttreatments is possible especially in light of our earlierstudies in which we found that consistent usage of thistechnology in a lecture may have a large positiveimpact on physics learning [11].Our experimental design with two groups did notallow for comparing the effect of these treatmentvariations with the absence of any treatment. Sisson’sresults suggest this difference exist and is large. Forsimultaneous gauging of effects, the third group wouldhave been required for which no additional problemsolving would have been organized. Inclusion of suchcontrol was not logistically feasible. Such omission ofthe extra problem solving opportunity would also haveto be compensated for students otherwise, to avoidethical conflicts. However, as a pointer for futurestudies, this might be a productive direction to furtherdiscern answers to proposed research questions.REFERENCES1. Sisson, C.J., Tablet-based recitations in Physics: Lesslecture, more success, in The impact of Tablet PCs andpen-based technology on education: new horizons, D.A.Berque, L.M. Konkle, and R.H. Reed, Editors. 2009,Purdue University Press: West Lafayette, IN. p. 133-139.2. Sisson, C.J. Trading lecture for learning (Online video).[www] 2010  [cited 2010 May]; Available from:http://coehp.tv/on_demand.php.3. Thornton, R. and D. Sokoloff, Learning motionconcepts using real-time microcomputer-basedlaboratory tools. Am. J. Phys., 1990. 58: p. 858-867.4. Deslauriers, L., E. Schelew, and C. Wieman, ImprovedLearning in a Large-Enrollment Physics Class. Science2011. 332(6031): p. 862-864.5. Hrepic, Z., D. Zollman, and S. Rebello, Comparingstudents’ and experts’ understanding of the content of alecture. Journal of Science Education and Technology,2007. 16(3): p. 213-224.6. Wieman, C. and K. Perkins, Transforming PhysicsEducation. Physics Today, 2005. 58(11).7. The New Media Consortium Horizon Report: 2012 K12Edition Wiki. 2012.8. Hrepic, Z., N.S. Rebello, and D.A. Zollman, RemedyingShortcomings of Lecture-Based Physics InstructionThrough Pen-Based, Wireless Computing And DyKnowSoftware, in Reading: Assessment, Comprehension andTeaching, N.H. Salas and D.D. Peyton, Editors. 2009,Nova Science Publishers; [reprinted in  Journal ofEducation Research, 3(1/2), 161-190 (2009)]. p. 97-129.9. DyKnow. Dyknow Vision and Monitor. [www] 2007[cited 2010 Sept.]; Available from:http://www.dyknow.com/.10. Hrepic, Z., Wireless computers in classrooms:Enhancing interactive physics instruction with TabletPCs and DyKnow software. Latin-American Journal ofPhysics Education, 2011. 5(2): p. 392-401.11. Hrepic, Z. and K. Shaw, Open policy for wirelesscomputers in classrooms: What makes it a good or abad idea?, in The Impact of Tablet PCs and Pen-basedTechnology on Education, R.H. Reed and D.A. Berque,Editors. 2010, Purdue University Press: West Lafayette,IN. p. 83-91.

Methodology and/or Technology: Making Difference in Improving Students\u27 Problem Solving Skills

https://csuepress.columbusstate.edu/cgi/viewcontent.cgi?article=1743&amp;context=bibliography_faculty

Methodology and/or Technology: Making Difference in Improving Students\u27 Problem Solving Skills

Abstract

Similar works

Full text

Available Versions

Columbus State University