Development of a reliable, valid, multi-dimensional measure of student engagement in group projects

The Study Purpose: Assess SGPEQ reliability and internal consistency; establish criterion-related and construct validity, for measure to be usable with confidence for quantitative analysis. Data: All students form 25 ITEC 620 sections and 15 TLMN sections from two semesters in 2012 were invited to participate. Data was gathered from 260 students, from which four declined to fill in the survey and 24 did not answer the questions. Method: • Study on SGPEQ initial data reliability and validity. • Initial item reduction and factors identification through exploratory factor analysis • Further verification of validity through a study on relationships between factors and students’ self-reported engagement and endorsement of self-theories.Development of a Reliable, Valid, Multi-Dimensional Measure of Student Engagement in Group Projects Irena Bojanova, Ph.D. The Graduate School University of Maryland University College Main Result A reliable, valid, multi-dimensional measure of student engagement in group projects was created. Empirical evidence of its usefulness allows the measure to be easily administrated, while providing a comprehensive snapshot of students’ engagement in group projects. Agenda 1.Introduction and Background. 2.The Study 3.Initial Data Reliability and Internal Consistency. 4.Explanatory Factor Analysis and Reliability Estimates. 5.Further Verification of Validity 6.Conclusion Introduction and Background •Graduate programs have to emphasize on group projects •How to optimize students’ learning experiences and projects’ outcomes? •“Student engagement in group projects” was identified as a distinguishable construct and Student Group Project Engagement Questionnaire (SGPEQ) instrument was developed, and face and content validated. Immersive vs. Traditional Project Definitely Not No Maybe Yes Definitely Yes All Yes 3 |3 9 | 7 15 | 22 52 | 43 21 | 26 73 | 69% 3 |4 3 | 12 15 | 19 55 | 42 24 | 23 79 | 65% 3 |3 12 | 7 6 | 11 42 | 46 37 | 34 79 | 80% 3 | 3 9 | 8 30 | 22 33 | 50 25 | 18 58 | 68% 3 | 3 6 | 3 27 | 16 46 | 52 18 | 26 64 | 78% 0 | 1 12 | 5 18 | 16 49 | 54 21 | 23 70 | 77% 0 | 1 9 | 3 12 | 11 46 | 57 34 | 28 79 | 85% 0 | 1 0 | 1 3 | 19 30 | 51 67 | 27 97 | 78% 0 | 1 0 | 8 0 | 32 39 | 39 61 | 19 100 | 58% 0 | 4 3 | 16 15 | 30 36 | 31 46 | 19 82 | 50% Survey Questions Communication/ presentation skills Technical skills Team-building skills Leadership skills Understand better course material Academically challenging Develop critical thinking& problem solving Provoke curiosity and sense of discovery Engaging and fun experience Would like similar in other classes The Study Purpose: •Assess SGPEQ reliability and internal consistency. •Establish criterion-related and construct validity, for measure to be usable with confidence for quantitative analysis. Data: •All students form 25 ITEC 620 sections and 15 TLMN sections from two semesters in 2012 were invited to participate. •Data was gathered from 260 students, from which: four declined to fill in the survey 24 did not answer the questions. Gender Female 24.8% Male 75.2% Age 20-30 years 16.2% 31-40 years 34.7% 41-50 years 36% 51-60 years 10.4% Over 60 years 2.7% Method •Study on SGPEQ initial data reliability and validity. •Initial item reduction and factors identification through exploratory factor analysis •Further verification of validity through a study on relationships between factors and students’ self-reported engagement and endorsement of. Initial Data Reliability and Internal Consistency •Reliability refers to the ability of the measurement instrument to give similar results for similar inputs. •Reliability analysis was conducted based on the following models: Cronbach’s Alpha (model of internal consistency, based on the average inter-item correlation) and Split-Half (model splits the scale into two parts and examines the correlation between the parts). •The Cronbach's alpha (0.940) and Split Half (0.894 vs 0. .884) values for the analyzed survey instrument confirms high level of reliability –Appendix C. •The Inter-Item correlations (783 of the 1225 are larger than 0.30) also confirm suitability of the data for factor analysis – Appendix D. Explanatory Factor Analysis and Reliability Estimates •Item reduction and factors identification: Principal axis factoring with Varimax rotation was performed on the 35 student engagement in group projects items. •The questions were: "How many components (factors) are needed to represent the variables?" and "What do these components represent?" •Analysis of a six-factor (Appendix E) and a four-factor (Appendix F) solution and the inspection of a scree plot (Appendix G) 4 factors were retained. Interpretability was difficult after four factors and the scree plot confirmed the slope decreases a little after four factors. •Only one question, “Stepped in when a teammate was not performing”, was dropped due to low communality and low loading on all factors. Explanatory Factor Analysis and Reliability Estimates •The four factors accounted for 59.15% of the variance. Appendix I shows the factor solution and the items for each factor. •Each factor comprised 19.69%, 14.04%, 11.69%, and 8.23% of the variance correspondingly. •The factors were titled: Effort, Teamwork, Motivation, and Organization. •Factors reliability and preliminary evidence of discriminant validity: The four student engagement in group projects factors show: reasonable reliability (.926, .838, .889, .766 correspondingly) the inter-item correlations (the highest is .088) support the discriminant validity of the SGPEQ measure. Further Verification of Validity SGPEQ validity was further verified through a study on the relationship between the identified factors and: Students’ self-reported engagement Endorsement of self-theories Appendix B and Appendix K. Self-Reported Engagement •Factors predicting absolute engagement (in this group project) and relative engagement (compared with other projects) were determined. •For that two analyses that regressed each of the two dependent variables on the four factors of SGPEQ were performed. Absolute engagement “How engaged were you in this group project?” Regression of relative engagement on the four factors shows: the factors account for 63.1% (R Square from Model Summary table) of the variance in absolute engagement 70% of absolute engagement was explained by the model: F(4, 166)=70.81, p < .001 (F and Sig. from ANOVA table). Factor 1 (β=.611); Factor 2 (β=.174), Factor 3 (β=.325), Factor 4 (β=.236) are all positive predictors of absolute engagement. Self-Reported Engagement Relative engagement “How engaged were you in this group project compared to other group projects you worked on during the same semester?” Regression of relative engagement on the four factors shows: the factors account for 44% of the variance in absolute engagement 33% of relative engagement was explained by the model: F(4, 166)=32.54, p < .05. Factor 1 (β=.387); Factor 2 (β=.259), Factor 3 (β=.326), Factor 4 (β=.233) are all positive predictors of absolute engagement. Incremental Self-Theories •Incremental and entity self-theories: “You have a certain amount of intelligence and you cannot do much to change it.” Regression of belief in incremental theory on the four factors shows: the factors account for 59% of the variance in absolute engagement 3% of belief in incremental theory was explained by the model: F(4,166)=2.6, p < .001 Factor 1 (β=.024); Factor 2 (β=.141)Factor 4 (β=.171) are positive predictors of belief in incremental theory. The Validated SGPEQ Effort •Did good work on my part •Communicated clearly and effectively •Was creative and productive •Was organized and prepared •Fulfilled the assigned role •Presented the final research product clearly and effectively •Contributed to discussions with ideas and opinions •Applied critical thinking and problem solving •Completed all assigned tasks on time •Attended all group meetings •Put forth effort •Developed leadership skills Motivation •Thought about project activities between meetings •Was inspired to learn and contribute •Found ways to make project interesting to me •Found project activities relevant to my life •Found project academically challenging •Was motivated and enthusiastic Teamwork •Had fun during team activities •Trusted teammates will do well on their project parts •Felt presence of team members during meetings/ presentations (as if in person) •Got to know teammates' strengths •Preferred team-work than working on my own •Would like to have similar projects in other classes •Experienced sense of discovery and accomplishment •Was confident that we can learn and do well on the project •Incorporated teammates ideas and opinions •Preferred to work on my own Organization •Helped/ tutored teammates during project activities •Found ways to involve non-participating team members •Took detailed notes during discussion meetings •Rehearsed for project presentation •Wished my teammates were working harder than me •Worked on the project on a regular basis References [1] AERA, APA, NCME (1999). Standards for Educational and Psychological Testing, http://teststandards.org [3] Carini, R., Kuh, G., Klein, S. (2006). Student Engagement and Student Learning: Testing the Linkages, Research in Higher Education, Vol. 47, No. 1, http://gov.alaska.edu/faculty/StudentSuccess/TintoReview-Carini-Kuh-Klein.pdf [4] Clark, L. & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319. [5] Community College Survey of Student Engagement. University of Texas at Austin, www.ccsse.org. [6] Cooperative Institutional Research Program (CIRP) Surveys. Higher Education Research Institute (HERI), http://heri.ucla.edu/herisurveys.php [7] DeVellis, R. (2003). Scale development: Theory and applications. Thousand Oaks, CA: Sage Publications. [8] Dweck, C. (1999). Self-Theories: Their role in motivation, personality, and development. Philadelphia: The Psychology Press. [9] Handelsman, M, Briggs, W., Sullivan, N., & Towler, A. (2005). A Measure of College Student Engagement. Journal of Educational Research, Vol. 98, No. 3, pp. 184-191; http://www.stanford.edu/dept/SUSE/projects/ireport/articles/self-regulation/self-course%20engement%20measure.pdf [10] Hinkin, T. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1, 104–121. [12] Kerlinger, 1F.N., (1986). Foundations of Behavioral Research. Holt, Rinehart and Winston. [13] Likert, R. (1931). A technique for measurement of attitudes. Archives of Psycology. New York: Columbia University Press. [14] Litwin, M. S. (2003). How to assess and interpret survey psychometrics, 2nd edition. Thousand Oaks, CA: Sage Publications. [15] McMillan, J. & Schumacher, S. (2001). Research in education: A conceptual introduction. New York: Longman. [16] Molinari, J. & Huonker, J. (2010). Diagnosing student engagement in the business school classroom. Journal of the Academy of Business Education. [17] National Survey of Student Engagement (NSSE). Center for Postsecondary Research, Indiana University in Bloomington, http://nsse.iub.edu/ [18] O'Donnell, A., Reeve, J., and Smith, J. (2009) Educational Psychology: Reflection for Action, Chapter 11. Wiley. [19] Pike, G., Kuh, G. (2005) A Typology of Student Engagement for American Colleges and Universities, Research in Higher Education, Vol. 46, No. 2,, http://cpr.iub.edu/uploads/Pike,%20Kuh%20(2005)%20A%20Typology%20of%20Student%20Engagement%20for%20American%20Colleges%20and%20Universities.pdf [20] Robinson C. &Hullinger, H. (2008). New Benchmarks in Higher Education: Student Engagement in Online Learning. Journal of Education for Business. http://cyber.law.harvard.edu/communia2010/sites/communia2010/images/Robinson_et_al_2008_New_Benchmarks_in_Higher_Education_Student_Engagement_in_Online_ [21] Schoenfeldt, L. F. (1984). Psychometric properties of organizational research instruments. In T. S. Bateman & G. R. Ferris (Eds.), Method & analysis in organizational research (pp.68-80). Reston, VA: Reston. [22] Skinner, E.A., and Belmont, M.J. (1993). .Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology, 85(4). p. 572. Questions and Answer

Development of a reliable, valid, multi-dimensional measure of student engagement in group projects

The Study Purpose: Assess SGPEQ reliability and internal consistency; establish criterion-related and construct validity, for measure to be usable with confidence for quantitative analysis. Data: All students form 25 ITEC 620 sections and 15 TLMN sections from two semesters in 2012 were invited to participate. Data was gathered from 260 students, from which four declined to fill in the survey and 24 did not answer the questions. Method: • Study on SGPEQ initial data reliability and validity. • Initial item reduction and factors identification through exploratory factor analysis • Further verification of validity through a study on relationships between factors and students’ self-reported engagement and endorsement of self-theories.Development of a Reliable, Valid, Multi-Dimensional Measure of Student Engagement in Group Projects Irena Bojanova Program Director Telecommunications Management, Graduate School, UMUC [email protected], 240-684-2428 Introduction In Summer 2011, student experiences in immersive vs. traditional group projects were examined and “student engagement in group projects” was identified as a distinguishable construct; research on the identified construct and the available measurement instruments were reviewed; and a preliminary scale for a new instrument to measure students engagement in group projects – Student Group Project Engagement Questionnaire (SGPEQ) – was developed, and face and content validated. The Spring 2012 project focused on the theoretical development of SGPEQ as a reliable, valid, multi-dimensional measure of student engagement in group projects. A study on the SGPEQ initial data reliability and validity was conducted in accordance with established psychometric principles for use in survey research. Initial item reduction and factor identification on SGPEQ was performed through exploratory factor analysis and examination of reliability estimates. The validity of the measure was further verified through a study on relationships between the SGPEQ factors and students’ self-reported engagement, endorsement of self-theories, goal orientation and self-reported grades. The Study Purpose: Assess SGPEQ reliability and internal consistency; establish criterion-related and construct validity, for measure to be usable with confidence for quantitative analysis. Data: All students form 25 ITEC 620 sections and 15 TLMN sections from two semesters in 2012 were invited to participate. Data was gathered from 260 students, from which four declined to fill in the survey and 24 did not answer the questions – please refer Appendix A. Method: • Study on SGPEQ initial data reliability and validity. • Initial item reduction and factors identification through exploratory factor analysis • Further verification of validity through a study on relationships between factors and students’ self-reported engagement and endorsement of self-theories. Initial Data Reliability and Internal Consistency Reliability refers to the ability of the measurement instrument to give similar results for similar inputs. Reliability analysis was conducted based on the following models: Cronbach’s Alpha (model of internal consistency, based on the average inter-item correlation) and Split-Half (model splits the scale into two parts and examines the correlation between the parts). The Cronbach's alpha (0.940) and Split Half (0.894 vs 0. .884) values for the analyzed survey instrument confirms high level of reliability – please refer Appendix C. The Inter-Item correlations (783 of the 1225 are larger than 0.30) also confirm suitability of the data for factor analysis – please refer Appendix D. Explanatory Factor Analysis and Reliability Estimates Item reduction and factors identification: Principal axis factoring with Varimax rotation was performed on the 35 student engagement in group projects items. The questions were: "How many components (factors) are needed to represent the variables?" and "What do these components represent?" The analysis of a six-factor (please refer Appendix E) and a four-factor (please refer Appendix F) solution and the inspection of a scree plot (please refer Appendix G), four factors were retained. Interpretability was difficult after four factors and the scree plot confirmed that the slope decreases a little after four factors. The four factors accounted for 59.15% of the variance. Appendix I shows the factor solution and the items (questions) for each factor. Only one question, “Stepped in when a teammate was not performing”, was dropped due to low communality and low loading on all factors. The four factors counted for 53.40% of the variance; each factor comprised 19.69%, 14.04%, 11.69%, and 8.23% of the variance correspondingly. They were titled: Effort, Teamwork, Motivation, and Organization. Factors reliability and preliminary evidence of discriminant validity: The four student engagement in group projects factors show reasonable reliability (.926, .838, .889, .766 correspondingly); and the inter-item correlations (the highest is .088) support the discriminant validity of the SGPEQ measure. Further Verification of Validity SGPEQ validity was further verified through a study on the relationship between the identified factors and the students’ self-reported engagement and endorsement of self-theories – please refer Appendix B and Appendix K. • Self-reported engagement: Factors predicting absolute engagement (engagement in this group project) and relative engagement (engagement compared with other group projects) were determined. For that two analyses that regressed each of the two dependent variables on the four factors of SGPEQ were performed. Absolute engagement (“How engaged were you in this group project How engaged were you in this group project How engaged were you in this group projectHow engaged were you in this group projectHow engaged were you in this group projectHow engaged were you in this group project How engaged were you in this group project How engaged were you in this group project How engaged were you in this group projectHow engaged were you in this group projectHow engaged were you in this group project How engaged were you in this group project How engaged were you in this group project How engaged were you in this group project How engaged were you in this group project How engaged were you in this group projectHow engaged were you in this group project?”): Regression of relative engagement on the four factors shows that the factors account for 63.1% (R Square from Model Summary table) of the variance in absolute engagement; 70% of absolute engagement was explained by the model: F(4, 166)=70.81, p < .001 (F and Sig. from ANOVA table). Factor 1 (β=.611); Factor 2 (β=.174), Factor 3 (β=.325), Factor 4 (β=.236) are all positive predictors of absolute engagement. Relative engagement (“How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other How engaged were you in this group project compared to other group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester?group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester? group projects you worked on during the same semester?group projects you worked on during the same semester? group projects you worked on during the same semester? group projects you worked on during the same semester? group projects you worked on during the same semester? ”): Regression of relative engagement on the four factors shows that the factors account for 44% of the variance in absolute engagement; 33% of relative engagement was explained by the model: F(4, 166)=32.54, p < .05. Factor 1 (β=.387); Factor 2 (β=.259), Factor 3 (β=.326), Factor 4 (β=.233) are all positive predictors of absolute engagement. • Incremental and entity self-theories: Regression of belief in incremental theory (“You You have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it.have a certain amount of intelligence and you cannot do much to change it. have a certain amount of intelligence and you cannot do much to change it.”) on the four factors shows that the factors account for 59% of the variance in absolute engagement; 3% of belief in incremental theory was explained by the model: F(4,166)=2.6, p < .001. Factor 1 (β=.024); Factor 2 (β=.141), and Factor 4 (β=.171) are positive predictors of belief in incremental theory. Conclusion The conducted in Spring and Summer 2012 research resulted in the creation of a reliable, valid, multi-dimensional measure of student engagement in group projects. Empirical evidence of the usefulness of SGPEQ was provided, so it can be easily administrated while providing a comprehensive snapshot of students’ engagement in group projects. References [1] AERA, APA, NCME (1999). Standards for Educational and Psychological Testing, http://teststandards.org [3] Carini, R., Kuh, G., Klein, S. (2006). Student Engagement and Student Learning: Testing the Linkages, Research in Higher Education, Vol. 47, No. 1, http://gov.alaska.edu/faculty/StudentSuccess/TintoReview-Carini-Kuh-Klein.pdf [4] Clark, L. & Watson, D. (1995). Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309-319. [5] Community College Survey of Student Engagement. University of Texas at Austin, www.ccsse.org. [6] Cooperative Institutional Research Program (CIRP) Surveys. Higher Education Research Institute (HERI), http://heri.ucla.edu/herisurveys.php [7] DeVellis, R. (2003). Scale development: Theory and applications. Thousand Oaks, CA: Sage Publications. [8] Dweck, C. (1999). Self-Theories: Their role in motivation, personality, and development. Philadelphia: The Psychology Press. [9] Handelsman, M, Briggs, W., Sullivan, N., & Towler, A. (2005). A Measure of College Student Engagement. Journal of Educational Research, Vol. 98, No. 3, pp. 184-191; http://www.stanford.edu/dept/SUSE/projects/ireport/articles/self-regulation/self-course%20engement%20measure.pdf [10] Hinkin, T. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1, 104–121. [12] Kerlinger, 1F.N., (1986). Foundations of Behavioral Research. Holt, Rinehart and Winston. [13] Likert, R. (1931). A technique for measurement of attitudes. Archives of Psycology. New York: Columbia University Press. [14] Litwin, M. S. (2003). How to assess and interpret survey psychometrics, 2nd edition. Thousand Oaks, CA: Sage Publications. [15] McMillan, J. & Schumacher, S. (2001). Research in education: A conceptual introduction. New York: Longman. [16] Molinari, J. & Huonker, J. (2010). Diagnosing student engagement in the business school classroom. Journal of the Academy of Business Education. [17] National Survey of Student Engagement (NSSE). Center for Postsecondary Research, Indiana University in Bloomington, http://nsse.iub.edu/ [18] O'Donnell, A., Reeve, J., and Smith, J. (2009) Educational Psychology: Reflection for Action, Chapter 11. Wiley. [19] Pike, G., Kuh, G. (2005) A Typology of Student Engagement for American Colleges and Universities, Research in Higher Education, Vol. 46, No. 2,, http://cpr.iub.edu/uploads/Pike,%20Kuh%20(2005)%20A%20Typology%20of%20Student%20Engagement%20for%20American%20Colleges%20and%20Universities.pdf [20] Robinson C. &Hullinger, H. (2008). New Benchmarks in Higher Education: Student Engagement in Online Learning. Journal of Education for Business. http://cyber.law.harvard.edu/communia2010/sites/communia2010/images/Robinson_et_al_2008_New_Benchmarks_in_Higher_Education_Student_Engagement_in_Online_ [21] Schoenfeldt, L. F. (1984). Psychometric properties of organizational research instruments. In T. S. Bateman & G. R. Ferris (Eds.), Method & analysis in organizational research (pp.68-80). Reston, VA: Reston. [22] Skinner, E.A., and Belmont, M.J. (1993). .Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. Journal of Educational Psychology, 85(4). p. 572. Appendix A: Preliminary SGPEQ Scale To what extent do the following behaviors, thought, and feelings described you in this group project? Please rate each of them on the following scale: 1 = Very little; 2 = Some; 3 = Quite a bit’; 4 = Very much Worked on the project on a regular basis Put forth effort Preferred to work on my own Took detailed notes during discussion meetings Wished my teammates were working harder than me Completed all assigned tasks on time Rehearsed for project presentation Was motivated and enthusiastic Found project activities relevant to my life Thought about project activities between meetings Found ways to make project interesting to me Was inspired to learn and contribute Felt presence of team members during meetings/ presentations (as if in person) Found project academically challenging Would like to have similar projects in other classes Fulfilled the assigned role Contributed to discussions with ideas and opinions Got to know teammates' strengths Had fun during team activities Incorporated teammates ideas and opinions Helped/ tutored teammates during project activities Preferred team-work than working on my own Experienced sense of discovery and accomplishment Trusted teammates will do well on their project parts Found ways to involve non-participating team members Stepped in when a teammate was not performing (+ N/A option if everybody participated) Was organized and prepared Communicated clearly and effectively Attended all group meetings Applied critical thinking and problem solving Did good work on my part Was creative and productive Developed leadership skills Presented the final research product clearly and effectively Was confident that we can learn and do well on the project Appendix B: Further validity of the developed measure Global Engagement How engaged were you in this group project? (1=not at all, 6=extremely) How engaged were you in this group project compared to other group projects you worked on during the same semester? (1=less engaged, 6=more engaged) Incremental theory You have a certain amount of intelligence and you cannot do much to change it. (1=do not agree, 6=strongly agree Appendix C: SGPEQ Reliability Ideally, in order to obtain a good estimate of the reliability of a survey, we would like to administer the survey twice to the same group of people and then correlate the two sets of results. However, this is often impractical because bias may be introduced in the second set of answers or because respondents may be unwilling or unable to take the survey a second time. One solution is to compute Cronbach's alpha. Another is to split the items into two groups and then to compare these groups as if they were two separate administrations of the same survey. 1) Cronbach's Alpha Case Processing Summary N % Cases Valid 185 71.2 Excludeda 75 28.8 Total 260 100.0 a. Listwise deletion based on all variables in the procedure. Reliability Statistics Cronbach's Alpha Cronbach's Alpha Based on Standardized Items N of Items .941 .945 35 Item Statistics Mean Std. Deviation N Worked on the project on a regular basis 2.9351 .83803 185 Put forth effort 3.3514 .70001 185 Preferred to work on my own 2.6973 .96978 185 Took detailed notes during discussion meetings 2.3189 1.01131 185 Wished my teammates were working harder than me 1.8703 .99697 185 Completed all assigned tasks on time 3.4811 .70781 185 Rehearsed for project presentation 2.1568 1.08461 185 Was motivated and enthusiastic 2.9405 .92180 185 Found project activities relevant to my life 2.6108 1.08342 185 Thought about project activities between meetings 2.7676 .94122 185 Found ways to make project interesting to me 2.7676 .89990 185 Was inspired to learn and contribute 3.0054 .92371 185 Felt presence of team members during meetings/ presentations (as if in person) 2.3892 1.01612 185 Found project academically challenging 2.7568 .97817 185 Would like to have similar projects in other classes 2.3730 1.12110 185 Fulfilled the assigned role 3.4811 .62634 185 Contributed to discussions with ideas and opinions 3.3946 .73793 185 Got to know teammates' strengths 2.7135 1.00492 185 Had fun during team activities 2.3027 1.09109 185 Incorporated teammates ideas and opinions 3.1405 .76024 185 Helped/ tutored teammates during project activities 2.2432 1.01632 185 Preferred team-work than working on my own 2.0649 1.01944 185 Experienced sense of discovery and accomplishment 2.6162 .99932 185 Trusted teammates will do well on their project parts 2.8216 1.00843 185 Found ways to involve non-participating team members 1.9946 1.01348 185 Stepped in when a teammate was not performing 3.4324 1.37407 185 Was organized and prepared 3.2108 .72524 185 Communicated clearly and effectively 3.3297 .67117 185 Attended all group meetings 3.2649 .82759 185 Applied critical thinking and problem solving 3.2486 .73189 185 Did good work on my part 3.4595 .61656 185 Was creative and productive 3.3081 .69719 185 Developed leadership skills 2.7351 1.06327 185 Presented the final research product clearly and effectively 3.2432 .77338 185 Was confident that we can learn and do well on the project 3.1946 .89380 185 2) Split Half Note: Different splits of the items will produce different estimates of the reliability coefficient. When they are split so that each group contains items that are highly correlated within the group but not between groups (as has been done here), the split-half coefficients will be close to their lowest values. When highly correlated items are paired off and placed into separate groups, then the split-half coefficients will reach their highest values. The following table shows the Split Half-Coefficient: • Correlation between forms - the correlation between the sum