Using exploratory factor analysis to examine self-reported survey data from adult learners, this study identified three factors of cognitive engagement in the adult literacy classroom: program involvement, focus, and independent effort.  A model that demonstrates the impact of independent effort on gains in reading test scores was developed using multivariate analysis

Beder, Harold

Lipnevich, Anastasiya A.

Milton, Judy

Valentine, Thomas

English

Kansas State University

Kansas State University Libraries New Prairie Press Adult Education Research Conference 2006 Conference Proceedings (Minneapolis, MN) Mapping Cognitive Engagement in Adult Literacy Classrooms Thomas Valentine Judy Milton University of Georgia Anastasiya A. Lipnevich Harold Beder Rutgers University See next page for additional authors Follow this and additional works at: https://newprairiepress.org/aerc  Part of the Adult and Continuing Education Administration Commons This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License Recommended Citation Valentine, Thomas; Milton, Judy; Lipnevich, Anastasiya A.; and Beder, Harold (2006). "Mapping Cognitive Engagement in Adult Literacy Classrooms," Adult Education Research Conference. https://newprairiepress.org/aerc/2006/papers/62 This is brought to you for free and open access by the Conferences at New Prairie Press. It has been accepted for inclusion in Adult Education Research Conference by an authorized administrator of New Prairie Press. For more information, please contact cads@k-state.edu. Author Information Thomas Valentine, Judy Milton, Anastasiya A. Lipnevich, and Harold Beder This is available at New Prairie Press: https://newprairiepress.org/aerc/2006/papers/62  421Mapping Cognitive Engagement in Adult Literacy Classrooms   Thomas Valentine, Judy Milton University of Georgia  Anastasiya A. Lipnevich, Harold Beder Rutgers University  Keywords: Adult literacy, cognitive engagement, quantitative research  Abstract:  Using exploratory factor analysis to examine self-reported survey data from adult learners, this study identified three factors of cognitive engagement in the adult literacy classroom: program involvement, focus, and independent effort.  A model that demonstrates the impact of independent effort on gains in reading test scores was developed using multivariate analysis.  Few adult educators would argue against the proposition that cognitive engagement is an essential ingredient for literacy learning.  However, the precise meaning of cognitive engagement is elusive and there is little consensus in this regard.  Moreover, empirical research on the ways in which low-literate adults think about their own learning processes is virtually non-existent.     Theoretical Framework  The core concept, cognitive engagement, appears in many guises in the scholarly literature.  Of interest in this study is cognitive engagement, a learner’s active use of self-regulating strategies in purposeful classroom learning (Corno & Mandinach, 1983).  A number of studies have measured the construct among K-12 and college students (e.g., Helme & Clarke, 2001; Meece, Blumenfeld, & Hoyle, 1988; Platt & Brooks, 2002) using both qualitative and quantitative approaches.  However, there is little research related to cognitive engagement of adult students, and none that advances the type of framework that we could confidently and justifiably impose upon the population of interest: low-literate adults who are seeking to improve their basic skills through participation in adult literacy education.  Consequently, we employed a grounded theory approach (Strauss & Corbin, 1990), in which we studied qualitative data to understand how cognitive engagement manifested itself in literacy classrooms.  Through the analysis of digitized videos and concepts described in the literature (Fredricks, Blumenfeld, & Paris, 2004), we developed a broad definition of cognitive engagement:  “the mental efforts that individuals actively use to focus on tasks that lead to learning.”  This definition served to set the boundaries for the phenomenon under investigation and informed the simple model, shown in Figure 1, that we tested in this study.  422Figure 1. Model of Learner Engagement.    The purpose of this exploratory study was to describe how students perceived their own cognitive engagement in the classroom and the impact of this on learning outcomes.  The research was guided by two research questions:   1. What is the nature of self-reported cognitive engagement in adult literacy students?   2. What is the effect of cognitive engagement on the gain in reading test scores?  Research Design  In order to test the conceptual model, we collected data on a number of variables, using three distinct data sources:  (a) student questionnaires, (b) teacher questionnaires, and (c) pro-gram records.  The specific data for each of the variables in this model are described in Table 1. Table 1. Data sources for simple model  Variable Specific Data Data Source Learning Outcomes  1. Teacher ratings of basic skill improvement (one global rating for each student) 2. Test score evidence of basic skill improvement 3. Additional indicators of goal attainment or skill improvement Teacher questionnaire   Program records  Program records Cognitive  Engagement  Self assessed cognitive engagement (14 multiple choice questions, one scale score) Student questionnaire Teaching Practices  Self report of teaching practices (10 items, 2 scale scores) Teacher questionnaire Personal Characteristics and Circumstances of Learner Self esteem (18 items, 3 scale scores) Age, gender, race, education, etc. Student questionnaire Program records Personal Characteristics and Circumstances of Teacher Age, gender, race, education, training, teaching experience, etc. Teacher questionnaire   Our methodology was designed to collect data from large numbers of low-literate learners.  Consequently, we decided to employ a simply worded questionnaire that would permit efficient data collection.   Development of an instrument to measure cognitive engagement proceeded in three steps.  The first step was construct clarification.  Through wide reading and grounded field work we ultimately decided on the psychological definition stated previously.  Our second step was to build an item pool.  We identified 145 potential items to measure Learner Characteristics Teacher/ Classroom Characteristics Cognitive Engagement Teaching Practices Learning Outcomes  423cognitive engagement in a self-report instrument.  This list went through several refinements, including eliminating redundancies, simplifying and clarifying wording, and review by four expert adult literacy teachers.  We then asked a panel of adult education researchers who had been involved in qualitative research on this topic to select exemplars from the remaining list for a pilot test of the instrument.  We also asked four other adult literacy and ESL teachers to pre-pilot this instrument with their students to critique both the questions and the format.  Particular problems they noted, such as the complexity and apparent redundancy of some items and ambiguous meanings for Spanish speaking ESL students, resulted in changes to the instrument. Our third step was to select a format for the instrument that took into account the low-literacy level of the people who would be asked to complete it.  These decisions resulted in a pilot instrument of 34 items (including demographic information) using simple sentences and clear language.   The instrument asked students to rate their own behaviors using a three-point Likert-type response scale of ‘almost never’, ‘sometimes’, and ‘almost always’.  Using descriptive and correlational analyses, we determined which of the items performed most satisfactorily. The final instrument included 14 items to measure cognitive engagement.   The survey was administered to adult learners in ABE, GED, Spanish GED, advanced ESL, EL Civics, Adult High School, and Workfirst classes at National Labsite for Adult Literacy Education, a partnership between NCSALL at Rutgers University and the New Brunswick Public Schools Adult Learning Center.  Of the 253 learners who completed the survey, 64% were female and 36% were male, and the average age was 32.9 (range was 16-67).  With respect to race/ethnicity, participants self-identified as 60% Hispanic, 25% Black, 9% Caucasian, and 6% Asian.  The mean grade completed prior to participation in their current class was 9.6 (range 3-18).  Learners in the Spanish GED classes (13% of the sample) completed a Spanish version of the instrument that had been developed using a two-step method, first translating from English into Spanish, and then back-translating the Spanish into English to establish validity of the translation.  To administer the instrument to the learners, we developed and field tested a protocol for collecting self-report survey data from low-literate adults. This protocol included the teacher introducing the data collection team in the classroom, the team leader coordinating the survey administration, and other team members being located strategically throughout the classroom to assist learners as needed.   Analysis and Findings The process of determining the factors of cognitive engagement involved two phases of data analysis.  The first phase required us to identify the dimensional structure of cognitive engagement.  We began by examining the means and rank order of each item as reported by the students and then subjected the 14 items to exploratory factor analysis.   Multiple solutions, requesting the extraction and rotation of one through five factors, were examined.   Ultimately, we settled on a three factor orthogonal solution that exhibited both perfect simple structure and conceptual clarity.   The factors and items for each are presented in Table 2.  Factor I, Program Involvement, represents a student’s interest and commitment to the learning program; a high score on this factor suggests that students “buy in” to the learning enterprise and see its relevance to their out-of-school lives.  Factor II, Learning Focus, involves the sustained and intentional application of one’s mental energy to the task at hand; students who score highly on this factor indicate that they successfully filter out the many distractions that characterize adult literacy classrooms in order to optimize their productivity.  Factor III, Independent Effort, is indicative of  424a students’ willingness to take proactive responsibility for the act of learning; it is characterized by self-possession and self-directed problem-solving.   Our second step in understanding these factors was to determine the best way to describe the population with respect to these three factors.   To accomplish this objective, we employed disjoint cluster analysis to construct a typology of students.   After examining numerous solutions, we settled on the typology presented in Table 3.  Proactive Learners exhibit the self-regulating strategies that are at the core of cognitive engagement as described by Corno and Mandinach (1983); they are apt to make the greatest learning progress, particularly in the type of literacy classroom in which one teacher divides her/his attention among many learners working  Table 2. Factors of cognitive engagement  Factor Item Mean SD I find the learning materials in the class very interesting 2.57 0.56 I get excited about the things I am learning in class 2.51 0.60 I use the things I learn in class when I am outside of school 2.45 0.59 I think about my school work when I am not in school 2.18 0.65 I. Program Involvement I get bored in class 1.49 0.62    I pay attention to what I am supposed to be doing in class 2.81 0.45 I stay focused on my work in class 2.69 0.50 I get very involved in the work I do in class 2.60 0.55 I guess the answer instead of figuring it out* 1.60 0.60 I talk with other students in class instead of doing my work 1.52 0.61 II. Learning Focus I have trouble staying awake in class* 1.34 0.61     I work hard to get the right answers 2.72 0.48 If I do not understand something, I try to find another way to learn it 2.55 0.53 III. Independent Effort When I make a mistake, I try to find the right answer by myself 2.39 0.58 *Reverse items on a variety of topics at diverse skill levels.  Inattentive Learners are not resistant to program participation, but they report problems with staying on task.  Uncommitted Learners have not yet found their own reasons for being in the literacy programs; they are apt to measure their success in terms of the time spent in the classroom rather than by the energy they devote to learning.  At best, they are quietly marking time in the classroom; at worst, they resist learning and serve as a distraction to more serious students.  Teacher Dependent Learners report average levels of program involvement and learning focus but somehow fail to take charge of their own learning, relying instead on assigned tasks and teacher-directed strategies.  425Table 3. A typology of adult literacy students with respect to their cognitive engagement Cluster Label % of  Sample Defining Characteristics Proactive Learners 33.9 High scores on Program Involvement High scores on Independent Effort Inattentive Learners 17.4 Very low scores on Learning Focus Uncommitted Learners 24.3 Very low scores on Program Involvement Teacher Dependent Learners 24.3 Low Scores on Independent Effort  Impact of Cognitive Engagement on Reading Gains   In order to answer the second research question, a subset of the full data set was used that included all participants for whom we had valid pre and post test reading scores recorded during a thirty-six month time period.  Scores were considered invalid if post test scores indicated a decrease in reading ability and were recoded to reflect a gain of zero.  We first examined the bivariate relationships between reading gains and each of the engagement factors.  These relationships are shown in Table 4.  Table 4. Results of simple regression analysis: Reading gains on three factors of cognitive engagement (n=76) Statistics for reading gains regressed on each factor Factor Beta p r-square I.  Program Involvement .05 .67 .003 II. Learning Focus .23 .05 .038 III. Independent Effort .25 .03 .064   We then conducted a multivariate analysis that allowed us to test more complex relationships between cognitive engagement and eleven other variables that could affect both engagement and the outcome variable of reading gains.  The most powerful explanatory model is depicted as Figure 2.   Figure 2. Model of the Impact of Cognitive Engagement on Reading Test Scores.    Reading Gain  R2=.27 Factor III Independent Effort Teaching Practices: Group Instruction  R2=.68 β=.27 β=-.41 Teacher’s Gender Number of People in Class Teacher’s  Age β=.53 β=.-29 β=-.39  426Implications for Adult Literacy Theory and Practice Literacy students come to our programs with histories of academic marginalization and educational oppression.   If we truly wish to help them accomplish the learning they will need to optimize their roles as family members, community members, and workers, we need to engage them in self-examination and a redefinition of themselves as learners.   The findings from this research can serve as a framework for authentic dialogue between teachers and students about the relationship between engagement, classroom learning, and lifelong, self-directed learning, thereby empowering students to take charge of their own learning.   Teachers can benefit from a more complete and sophisticated understanding of engagement, as they try to understand why some students make solid progress and others do not.  Certainly, these findings offer a counterpoint to simplistic notions of learning by which students are labeled ‘engaged’ or ‘unengaged,’ with those terms serving as synonyms for ‘good’ and ‘bad’. The success of our research instrumentation and data collection strategies demonstrates the viability of using survey methodology to conduct research with low-literate learners.  Carefully developed self-report measures can be a valid methodology for research in adult literacy classrooms—an option that gives systematic voice to large samples of learners who otherwise would have little opportunity to influence instructional decision-making.     References  Corno, L., & Mandinach, E. B. (1983). The role of cognitive engagement in classroom learning and motivation. Educational Psychologist, 18, 88-108. Fredricks, J., Blumenfeld, P., & Paris, A. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74, 59-109. Helme, S., & Clarke, D. (2001). Identifying cognitive engagement in the mathematics classroom. Mathematics Education Research Journal, 13(2), 133-153. Meece, J. L., Blumenfeld, P.C., Hoyle, R. H. (1988). Students’ goal orientations and cognitive engagement in classroom activities. Journal of Educational Psychology, 80, 514-523. Platt, E., & Brooks, F. B. (2002). Task engagement: A turning point in foreign language development. Language Learning, 52, 365-400. Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage.   

Mapping Cognitive Engagement in Adult Literacy Classrooms

https://newprairiepress.org/cgi/viewcontent.cgi?article=2567&amp;context=aerc

Mapping Cognitive Engagement in Adult Literacy Classrooms

Abstract

Similar works

Full text

Available Versions

Kansas State University