A Composite Framework for ESL Textbook Evaluation

Textbook evaluation practices have not been critically examined to determine effectiveness and value in learning-teaching environments and this is probably the main reason why the literature suggests that textbooks selected have been more of a hindrance than a benefit to teaching. The assumption made by the researcher is that since much of the criticism in selection processes of textbooks is directed towards the checklist, which at this moment seems to be the only instrument used in textbook evaluation practices, then there would be a need to re-evaluate the usefulness of the checklist, identify weak areas and then develop a composite framework where the checklist will be supported by complementary instruments, namely the concordance software and the reflective journal. The researcher suggests a 4-phase procedure in the development of the composite framework. Phase 1 tests the Skierso Evaluation Checklist (SEC) for reliability and item difficulty. Phase 2 tests the capabilities of the concordance software (WordSmith Tools 3.0) to provide analysis of the patterns of presentation of vocabulary and structures in textbooks, to determine the extent to which the software will help discriminate between books in a selection process and to determine to what extent the analysis would provide greater illustration to responses required of by items in Section D of the SEC. Phase 3 tests the capabilities of the reflective journal in providing greater illustration to responses to items in Section E (Exercises and Activities) of the SEC. Finally, in Phase 4 the researcher will assemble aspects of the t, 0 complementary components into a framework which has the checklist as its main instrument. This framework will then be tested for reliability and item difficulty. In Phase 1, the findings revealed that while the overall reliability of the SEC was high, the difficulty analysis of items showed Section) and E of the checklist as having the largest number of difficult items. Phase 2 of the study found that the concordance software is capable of many useful functions it textbook evaluation and is able to provide greater illustration, through computation to 6 items in Section D of the SEC. Phase 3 of the investigation revealed that teaching reflections contributed to input that was beneficial to evaluation, especially the item in Section E of the SEC. The composite framework was assembled and created in Phase 4. It was then compared to the mono-instrument procedure (Phase 1) which consisted of the checklist (SEC). The comparison of the two procedures showed the composite framework to be more reliable at 0.9324 reliability as compared to 0.7675 reliability for the SEC as a standalone instrument. The difficulty analysis of items also showed marked improvement when comparisons were made. Only 4 items were considered difficult within the composite framework as opposed to 14 when the SEC was tested as a standalone. This study has provided an alternative to the checklist dominated procedure by proposing a framework which works on the combined effort of 3 distinct instruments, thus providing for much needed triangulation which is actually expected in an exercise as complex as textbook evaluation. The spin-offs to this research are the added value it provides by way of increased awareness of action research in textbook evaluation, to greater emphasis and attention to retrospective evaluation and adaptation. It has also led to the creation of the first Malaysian Corpus of the Language of Textbooks which has approximately 150,000 words. This corpus will expand when it accommodates the language of more textbooks within the school system

Misinterpretations of real needs in SL and FL writing situations: Is there a need to re-evaluate directions?

Teachers of English in Second language and foreign language situations who teach writing have had to face many challenges, some of which have come about from rapid changes in the teaching of writing in the developed nations. The dilemma that teachers face is a big one – whether to totally embrace the changes (which are advocated by these developed nations as being necessary to achieve better teaching and learning, or to adopt and adapt where necessary. This article provides the over-view of problems faced by writing teachers in SL/FL situations and provides some suggestions as to how teachers can deal with “paradigm shifts” as in the ones that have affected the field of writing

IRREGULARITIES IN VOCABULARY LOAD AND DISTRIBUTION IN SAME LEVEL TEXTBOOKS WRITTEN BY DIFFERENT WRITERS

A syllabus provides a focus for what to be studied and how the contents should be selected and sequenced. In Malaysia, the syllabus has an additional role that is to guide the textbook writers in producing textbooks for school use. The English Language Programmes for Malaysian secondary schools provide more than one textbook for every academic level.  These textbooks must comply with the syllabus in order to achieve the same learning objectives at the end of each programme. There will also be public examinations at the end of Form Three and Form Five. Two Form 2 textbooks were chosen and WordSmith 3.0 was used to analyse these textbooks to identify whether the textbooks were similar and had included all the contents listed in the syllabus. The findings show that one of the textbooks is relatively better that the other in terms of types, distribution of words, presentation of items in the syllabus and repetition of new linguistic items throughout the textbook. This reveals that ad hoc writing of textbooks by different writers for a similar level can lead to irregularities, the most severe problem being non-conformity to syllabus requirements.Keywords: distribution of words, presentation of items, repetition of  new linguistics  items,  textbooks, Wordsmith 3.0, syllabus requirement

Lexical Similarities and Differences in the Mathematics, Science and English Language Textbooks

The teaching of Science and Math in English in Malaysia is an area of great concern to educators and students alike. This study looks, in particular, at the common word classes among keywords identified in the Science, Math and English language Form One textbooks used in Malaysia and the differences in language use identified in the Science and Math textbooks

A corpus-based study of semantic treatment of phrasal verbs in Malaysian ESL secondary school textbooks

Despite their being the most notoriously confusing aspects of English language instruction, phrasal verbs are of high relevance for ESL/EFL learners because knowledge of them is often equated with language proficiency and fluency. With textbooks containing a noticeable number of phrasal verbs, it is of pedagogical significance to see if these combinations are appropriately dealt with in semantic terms. The present corpus-based study was, thus, intended to explore the semantic treatment of these combinations in a pedagogical corpus of Malaysian ESL textbooks. Using WordSmith software and the Oxford Dictionary of Phrasal Verbs as research instruments, the study revealed that despite the overwhelming number of phrasal verbs in the corpus, most of these combinations were presented with a very thin skeleton, as they were repeated in different forms with the same meaning. In addition, some items were presented with their rare and infrequent word meanings. Therefore, the selection and presentation of the word senses of different phrasal verb combinations proved to be more intuitively than empirically motivated

Contrasting audio-taped feedback with minimal marking feedback in EFL writing

In this study, the researchers contrasted audio-taped feedback (ATF) with minimal marking (MM) among 82 EFL female pre-intermediate learners at Kish Air English Language Institute in Tehran-Iran. These 82 students were selected from among 126 pre-intermediate students according to their scores in a pre-intermediate version of Nelson English Language Test. The subjects were randomly divided into two groups of ATF and MM, each with 41 eligible subjects. In each group, they were assigned to write expository paragraphs of about 120 words during 8 sessions, one session for the pre-test, and another one for the post-test; the remaining sessions were allocated for the treatment. One of the groups received ATF and the other MM. A pre-test and a post-test were used before and after the treatment so as to contrast writing performances of the post- and pre-treatments. The results showed that the ATF subjects performed better than the MM in the post-test, while the MM subjects in post-test did not show any progress

Errors and variations of TESL students’ written description

The focus of this study was to highlight different kinds of errors and variations that emerge in the writings of ESL students. The participants of the study were twelve TESL undergraduate students from the Faculty of Educational Studies at Universiti Putra Malaysia. Students’ written descriptions on a picture stimulus were collected and qualitatively analyzed. Then, the emerging errors in their descriptions were categorized into the linguistic and surface structure taxonomies. The findings revealed different grammatical errors in their writings which were basically related to the categories of verb phrases and noun phrases in either simple or compound/complex sentences that resulted in giving misinformation, one of the categories in surface structure. Some variations were also found in the word choice of the students in the three ethnic groups of the study, whereby these choices might be the reflections of their cultural background and world view. The results could help material developers to develop the necessary tasks and exercises and teachers to devise the best teaching strategies for overcoming these kinds of errors in ESL learners’ writings

Incidental focus on form and learner uptake in Iranian EFL classrooms

This study investigated the connection between teachers’ incidental focus on form, namely, corrective feedback and learners’ uptake and immediate repair of errors in communicative English as a Foreign Language classrooms for adults. The data was drawn from the transcripts of oral corrective feedback moves of six audio and video-recorded classrooms at an intermediate level totaling 9 hours. Corrective feedback moves were coded based on Lyster and Ranta (1997) and Panova and Lyster (2002) models. This study investigated the integration of incidental focus on form into six intact communicative EFL classes. A descriptive design which employed qualitative data collection procedure was adopted. The results revealed a significant difference in the ratio of uptake following certain corrective feedback types, which was in sharp contrast to the findings of Lyster and Ranta (1997) and Panova and Lyster (2002). Possible reasons are discussed from different aspects of learners’ age, their motivation, and instructional settings

Refinement of the English language teaching textbook evaluation checklist

The English Language Teaching (ELT) Textbook Evaluation Checklist was developed in response to the need for a reliable, valid and practical instrument to evaluate English language teaching textbooks. The checklist was qualitatively developed by a review of the literature (Mukundan & Ahour, 2010; Mukundan et al., 2011a) and was refined through qualitative (Mukundan et al., 2011b) and quantitative (Mukundan & Nimehchisalem, 2012a) methods. As the validation test results of the checklist (Mukundan & Nimehchisalem, 2012b; Nimehchisalem & Mukundan, 2013) indicated, it could be refined further to improve its validity, reliability and practicality. The present study discusses the modifications made to the checklist following the comments of a panel of experts (n=3), who were sent a copy of the old version of the checklist. They commented on the comprehensiveness, importance and clarity of the domains and items of the checklist independently. The qualitative method was used to collect and analyse the data. The checklist was refined based on the experts' comments; problematic items were removed or revised and a scoring guide was added to it. The refined instrument is more economical than its previous version, and yet further research is required to test its validity empirically