U radu se raspravlja o problematici vrednovanja vještine govorenja u
stranome (engleskome) jeziku te se detaljno prikazuju proces i rezultati vrednovanja
vještine govorenja u sklopu znanstveno-istraživačkoga projekta MZOŠ-a ‘Engleski
jezik u Hrvatskoj’. Rezultati vrednovanja pokazuju da i na kraju osnovnoškolskoga
i srednjoškolskoga obrazovanja učenici mogu komunikacijski učinkovito koristiti
engleski jezik, što jest i glavni cilj suvremene nastave stranoga jezika, no da bi bilo
poželjno dodatno poraditi na točnosti njihova usmenoga iskaza.The paper discusses issues related to assessing the speaking skill
in English as a foreign language. The paper also presents the process and the
results of assessing the speaking skill within the Croatian Ministry of Science,
Education and Sport research project - English in Croatia. Assessment results
show that grade 8 and grade 12 learners are communicatively competent when
speaking English. Although this is the main goal of teaching English as a foreign
language, the author stresses that in the process of teaching more attention should
be payed to the accuracy of learners’ speech

Marta Medved Krajnović

Croatian

Hrčak - Portal of scientific journals of Croatia

182HOW WELL DO CROATIAN LEARNERS SPEAK ENGLISH?Marta Medved KrajnovićFaculty of Humanities and Social Sciences, University of ZagrebSummary - The paper discusses issues related to assessing the speaking skill in English as a foreign language. The paper also presents the process and the results of assessing the speaking skill within the Croatian Ministry of Science, Education and Sport research project - English in Croatia. Assessment results show that grade 8 and grade 12 learners are communicatively competent when speaking English. Although this is the main goal of teaching English as a foreign language, the author stresses that in the process of teaching more attention should be payed to the accuracy of learners’ speech.Key words:  assessment of speaking skill, ELT, speaking skill0. IntroductionNowadays when fast and effective communication is essential for success-ful functioning in all aspects of private and professional life, an individual’s com-municative language competence in both his fi rst and second language is prima-rily assessed by how well he/she speaks that language. Very often, an individual’s communicative language competence also infl uences the attitude that the inter-locutor forms about the speaker’s professional skills. Therefore, modern foreign language teaching sets the development of learners’ speaking skill as one of its main goals. However, that is also one of the most demanding goals, considering the linguistic, psycholinguistic and pragmatic complexity of the speaking skill. In order for a learner to become successful in oral production and interaction, he/she must know the language well and have plenty of opportunities for practicing its meaningful use in different communicative contexts. Croatian learners are exten-sively exposed to English through the media, but they still have not got many op-portunities for its purposeful use in a natural context. Therefore, modern English MetodikaVol. 8, br. 1, 2007, page 182-189Original scientifi c paperReceived: 15.04.2007.UDK: 378.678Medved Krajnović M.: How well do Croatian learners speak english?183language teaching should try to partly compensate for that lack of opportunity and simulate real life communicative situations.The speaking skill is also considered to be one of the most demanding lan-guage skills in terms of its reliable and valid assessment. Therefore, the assess-ment of learners’ speaking skill should be a well designed and carefully carried out process. This process usually consists of several steps:1. well-defi ned assessment aim and purpose,2. assessment material (assessment tasks and their weightings) which is in accordance with the aim and purpose of assessment,3. the actual test administration,4. assessment and interpretations of learners’ results,5. using the results in accordance with the previously set assessment aim and purpose. (Luoma, 2004: 5)The above mentioned steps were the ones followed in the research project English in Croatia, sponsored by the Croatian Ministry of Science, Education and Sports. They will be described in more detail in the chapters that follow.1. The purpose and the aim of assessing the speaking skill within the research project English in CroatiaThe assessment of the speaking skill within the project English in Croatia was part of the assessment of the overall communicative language competence that the Croatian learners have at the end of their primary (year 8) and secondary (year 12) education. The fi nal goal of the whole assessment process was to obtain an indirect insight into the quality of English language teaching in Croatia, and to issue some guidelines for improvement, if found necessary. 2. Assessment materialThe level of communicative language competence that, according to the primary and the secondary school curriculum and the Ministry guidelines, is ex-pected from the Croatian learners at the end of their year 8 and year 12 education corresponds to the Common European Framework of Reference levels A2 and B1, respectively. (For a description of the qualitative aspects of spoken language use at the A2 and B1 levels see Council of Europe CEFR document, 2001: 28-29).The complexity of the speaking skill has already been mentioned. One as-pect of this complexity comes from the linguistic subcomponents (phonological, morpho-syntactic, lexical) and their processing characteristic for the spoken lan-guage use. That is why one often speaks about the ‘spoken language grammar’ 184METODIKA:Vol. 8, br. 14 (1/2007), str. 182-189(Carter and McCarthy, 1997), or the lexical approach in foreign language teach-ing (Nattinger and DeCarrico, 1992; Lewis, 1997) which primarily deals with the lexico-grammatical specifi cities of the spoken language. The other aspect of this complexity is related to the diversity in contexts of the spoken language use. This diversity is two-fold: there is diversity in the topic of conversation, level of formality (present also in other language skills), and there is difference between planned and unplanned speech where the latter very much depends on the reac-tions that one or more interlocutors have to the speaker’s input. In order to pro-vide for this diversity, but also in order to satisfy the principles of validity and re-liability of the whole assessment procedure (Alderson, Clapham and Wall, 1995; Bachman i Palmer, 1996), three types of tasks were used in testing the speaking skill. These tasks differed in terms of topic, type and level of diffi culty. Just for il-lustration, we can add that this diversity of tasks allowed us the following:- to test different levels of linguistic and cognitive processing, from simple reproduction of information to argumentative exposition of personal opinion (Anderson  et al., 2001); - to obey the methodological principle related to task complexity (‘from easier to harder tasks’), which we believe partly reduced learner anxiety and increased test reliability;-  to further increase test validity and reliability by using tasks of different type and content, and thus avoiding learner task bias. Therefore, in the fi rst tasks which was the same for year 8 and year 12 learners, the learners had to answer some of the basic personal questions that they often encountered in the process of English language learning (e.g. What’s your name? Do you like...? What is your hobby?, etc.) The examiner had to ask each learner 7 out of 10 possible questions. In the second task which was also the same for both groups of learners, the examinee had to chose one out of 6 possible pic-tures, describe it and connect it with some personal experience, and then invent a story on the basis of the visual input. The third task differed for the two groups of learners. Year 8 students were presented with 6 different situations (e.g. Your friend is coming to visit you. Give him/her directions from the nearest station or bus stop to your home, or You would like to buy a new pair of jeans at a shop. Ask about makes, sizes, colours and prices, etc. ). They had to choose 3 situations and role-play them together with the examiner. The fi rst interaction was supposed to be initiated by the examiner, while in the other two the learner was supposed to take initiative. Year 12 students  were given to read fi ve different statements (e.g. Young people cannot live without mobile phones, or All Croatian schools should have security guards, ecc.). They had to choose one and give 4 reasons why they believed some people agree and some disagree with the chosen statement.The mentioned tasks were assessed by using two detailed rating scales, one for year 8 and one for year 12 students. (For a detailed discussion about the devel-opment of the speaking tasks and their assessment scales see Luoma, 2004: 29-Medved Krajnović M.: How well do Croatian learners speak english?18595.) After having been developed the scales were used in the training of assessors. After the input from the trainees the scales were somewhat altered and then used as such in the actual assessment procedure.In terms of their structure, the scales were very similar for primary and sec-ondary school learners. Learners performance on each of the three tasks was meas-ured in terms of: 1. Task achievement; 2. Vocabulary; 3. Accuracy; 4. Fluency, pro-nunciation and intonation. For each of these components, in each of three tasks, learners could get a minimum of 0 and a maximum of 4 points. Therefore, the maximum number of points for each task was 16 (4 points for 4 components) and the maximum number of points for the whole test was 46 (3 tasks, maximum of 16 points each). The main difference between the two rating scales were that one was based on CEFR A2 level descriptors, and the other on the B1 level descriptors.3. Test and assessment administration The speaking test was administered to a representative sample of year 8 and year 12 learners from all over Croatia (for details on the methodological issues see J. Mihaljević Djigunović and V. Bagarić this issue). However, due to the complex-ity of the assessment procedure in terms of time needed for examining and scor-ing, technical facilities (good recording equipment and preferably an acoustically isolated room) and human resources (trained examiners and assessors were need-ed), the speaking skill of only several students from each class/school included in the project was tested. The choice of students within a class/school was purely random. Students’ recordings were collected and then assessed centrally (by the members of the research project and trained assessors).4. Results and discussionEND OF PRIMARY SCHOOL (YEAR 8) LEARNERS’ RESULTSA total of 191 (190) students were tested. As expected, learners scored best in Task 1 and worse in Task 3. This is probably due to the linguistic, communica-tive and cognitive complexity of the respective tasks. These results are presented in Table 1.Table 1: The average number of points and variability in the three speaking skill tasksN Mean SD Max.Total for Task 1 191 14,52 2,54 16Total for Task 2 191 13,15 3,33 16Total for Task 3 190 12,96 3,71 16186METODIKA:Vol. 8, br. 14 (1/2007), str. 182-189Table 2 presents results for each task component and for the three tasks.Table 2: The average number of points and variability for each component of each speaking taskN Mean SDTask 1 – task achievement 191 3,68 ,62Task 1 – vocabulary 191 3,65 ,71Task 1 – accuracy 191 3,61 ,69Task 1 – fl uency, pronunciation, intonation 191 3,59 ,72Task 2 – task achievement 191 3,41 ,90Task 2 – vocabulary 191 3,34 ,94Task 2 – accuracy 191 3,15 ,91Task 2 – fl uency, pronunciation, intonation 191 3,26 ,85Task 3 – task achievement 190 3,32 ,97Task 3 – vocabulary 190 3,28 1,02Task 3 – accuracy 190 3,12 ,96Task 3 – fl uency, pronunciation, intonation 190 3,24 ,97When the results for each component in all three tasks are compared, it can be noticed that the examinees scored best in terms of task achievement, then vo-cabulary, and then fl uency, pronunciation and intonation. They scored lowest in terms of grammatical accuracy. These results can be seen in Table 3.Table 3: The average number of points and variability for each component of the speaking skillN Mean SD Max.Total for task achievement 190 10,41 2,28 12Total for vocabulary 190 10,27 2,46 12Total for accuracy 190 9,87 2,33 12Total for fl uency, pronunciation and intonation 190 10,08 2,36 12END OF SECONDARY (YEAR 12) LEARNERS’ RESULTSA total of 80 (79) students were tested. Students did best on Task 1, and worst on Task 2. This is somewhat surprising since Task 3 was more demanding than Task 2, but maybe the examinees found Task 3 more challenging and inter-esting in terms of topic. The average number of points for each component of each task is presented in Table 4. Medved Krajnović M.: How well do Croatian learners speak english?187Table 4: The average number of points and variability for each component of each speaking taskN Mean SDTask 1 – task achievement 80 3,45 ,81Task 1 – vocabulary 80 3,11 ,97Task 1 – accuracy 80 2,99 ,85Task 1 – fl uency, pronunciation, intonation 80 3,25 ,86Task 2 – task achievement 80 3,10 ,94Task 2 – vocabulary 80 2,89 ,97Task 2 – accuracy 80 2,79 ,96Task 2 – fl uency, pronunciation, intonation 79 3,11 ,91Task 3 – task achievement 79 3,13 ,98Task 3 – vocabulary 79 2,95 ,95Task 3 – accuracy 79 2,94 ,98Task 3 – fl uency, pronunciation, intonation 79 3,11 ,93When the results for each component in all three tasks are compared, it can be concluded that the examinees were best in terms of task achievement, then fl u-ency, intonation and pronunciation, then vocabulary, and they were somewhat less successful in terms of grammatical accuracy. This can be seen in Table  5.Table 5: The average number of points and variability for each component of the speaking skillN Mean SD Max.Total for task achievement 79 9,68 2,39 12Total for vocabulary 79 8,95 2,66 12Total for accuracy 79 8,72 2,64 12Total for fl uency, pronunciation and intonation 79 9,47 2,55 12The slight problem in discussing the results for the speaking skill is that we cannot be sure of their statistical relevance because of the fewer number of students included in the assessment of the speaking skill (and the reasons for this have been explained above). However, it has to be stressed that the results for the speaking skill show a similar tendency not only for year 8 and year 12 students, but they also show a similar tendency to the results of the writing skill (i.e. low-est scores in terms of grammatical accuracy), and the writing skill was tested on a 188METODIKA:Vol. 8, br. 14 (1/2007), str. 182-189much larger number of students (for details see V. Josipović-Smojver this issue). Furthermore, the analysis was made between the learners’ results on the speak-ing test and different individual and contextual factors that might infl uence the English language teaching and learning process. These results also show a similar tendency across the all four skills (for details on the listening, reading and writ-ing skill see R. Geld and M.M. Stanojević; V. Josipović-Smojver; L. Zergollern-Miletić this issue).5. Conclusion On the basis of the data obtained during the assessment of the speaking skill, we can conclude that the learners’ results are satisfactory. Therefore, the an-swer to the question in the title of this article could be: Croatian learners speak English well!Year 8 learners, i.e. primary school leavers, showed somewhat better results overall, which could indicate that the quality and intensity of teaching in the sec-ondary school should continue with the primary school practice. The learners scored best in terms of task achievement. We can be satisfi ed with this result which shows that Croatian primary and secondary school students are communicatively competent when using English, and developing the learners’ communicative competence  should be one of the main goals of English language teaching in Croatia. The learner also showed adequate command of vocabulary and relatively fl uent and coherent fl ow of speech. The language component that, according to our research results, could be more insisted on in both primary and secondary school teaching is grammar. However, it has to be stressed that gram-mar is substantially present in the primary and secondary school curriculum and teaching material. Therefore, we suggest that what has to be changed is the way in which grammar is being taught. We suppose that the grammatical structures are still often being presented through isolated grammar tasks, i.e. outside  communi-catively meaningful contexts. As a consequence of such approach, when learners are faced with using grammar under communicative pressure and in communica-tively meaningful tasks – they underachieve. Therefore, the teaching of grammar should be contextualised, and, as already stressed at the beginning of this article, the teacher should try to fi nd opportunities for his/her learners to use English in communicatively meaningful contexts as much as possible. This would allow the learners to notice and correct (either individually or with the teacher’s help) their mistakes, and to speed up and partly automatize the processing of more frequent language structures.We are aware that the proposed guidelines are diffi cult to follow under the pressure of everyday teaching chores and curriculum demands, but they should be kept in mind as the ultimate goal of the teaching of the speaking skill.Medved Krajnović M.: How well do Croatian learners speak english?189REFERENCES:Alderson, J. C., Clapham, C., & Wall, D. (1995). Language Test Construction and Evaluation. Cambridge: Cambridge University Press.Anderson, L.W. et al. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman.Bachman, L. F., & Palmer, A. (1996). Language Testing in Practice. Oxford: Oxford University Press.Carter, R., & McCarthy, M. (1997). Exploring spoken English. Cambridge: Cambridge University Press.Lewis, M. (1997). Implementing the Lexical Approach. London: Language Teaching Publications.Luoma, S. (2004). Assessing Speaking. Cambridge: Cambridge University Press.Nattinger, J., & DeCarrico, J. (1992). Lexical Phrases and Language Teaching. Oxford: Oxford University Press.Council of Europe (2001). Common European Framework of Reference for Languages. Cambridge: Cambridge University Press.

HOW WELL DO CROATIAN LEARNERS SPEAK ENGLISH?

Abstract

Similar works

Full text

Available Versions

Hrčak - Portal of scientific journals of Croatia