Background: The aim was to assess the effectiveness of multiple choice versus short answer questions (SAQs) as assessment tools for evaluating performance of 2nd MBBS students.Methods: The study was observational, retrospective study of written pen and paper type assessment that utilized a sample of 100 2nd year medical students. Study consisted of two parts; part I was multiple-choice questions (MCQs) based on endocrine system where four options were given for a question and the single best answer was to be ticked. MCQ was timed at 20 mins for 30 questions with 1 mark each. There was no negative marking. Part II was SAQ on same system where 16 SAQ were given. SAQ was timed at 60 mins for 30 marks questions. After the test students had to reply on a feedback form.Result: There was a strong correlation between the marks scored in two formats of test and there was no statistical difference between the two set of marks.Conclusion: SAQs are as effective as MCQs in assessing the performance of the students in medical pharmacology

Bhounsule, Sushama A.

Kamat, Amey S.

Walke, Yogeeta Sushant C.

English

International Journal of Basic & Clinical Pharmacology

Print ISSN 2319-2003 | Online ISSN 2279-0780doi: 10.5455/2319-2003.ijbcp20141212IJBCP International Journal of Basic & Clinical Pharmacologywww.ijbcp.com International Journal of Basic & Clinical Pharmacology | November-December 2014 | Vol 3 | Issue 6 Page 1020Research ArticleA retrospective comparative study of multiple choice questions versus short answer questions as assessment tool in evaluating the performance of the students in medical pharmacologyYogeeta Sushant C. Walke*, Amey S. Kamat, Sushama A. BhounsuleINTRODUCTIONAssessment is a powerful driver of innovative changes in education and defines goals for learners and teachers. Assessment is said to drive student learning and define curriculum. However, the assessment of learning is often one of the more difficult and time-consuming aspects of education.1 The performance assessment communicates the students what material is important, motivate the students to study, identify areas of deficiency and the need for further learning. It also identifies areas, where the course or curriculum is weak, and it facilitates modification of the syllabus or curriculum and help in guidance and counselling.2Success in education, in turn, is largely measured by performance on examinations. If students know that that their success on the examination rests entirely on rote recall of facts, they will adopt a learning style that relies purely on memorization. On the other hand, if students are required to integrate, interpret and apply information they will adopt a strategic learning style in order to drive the highest degree of success.3Multiple choice questions (MCQs) are a common type of assessment due to their reliability, validity and ease of scoring.4MCQs offer advantage of being more versatile in measuring rote memory to more complex knowledge. Large amount of information can be measured in a short time. Scoring is objective and can be reliably completed. With 3-5 options, the effects of guessing can be reduced.Another common type of assessment is short or long essay type questions. This format allows students more flexibility in their response and reflects their individuality of approach in which interpretative skills can be evaluated.5ABSTRACTBackground: The aim was to assess the effectiveness of multiple choice versus short answer questions (SAQs) as assessment tools for evaluating performance of 2nd MBBS students.Methods: The study was observational, retrospective study of written pen and paper type assessment that utilized a sample of 100 2nd year medical students. Study consisted of two parts; part I was multiple-choice questions (MCQs) based on endocrine system where four options were given for a question and the single best answer was to be ticked. MCQ was timed at 20 mins for 30 questions with 1 mark each. There was no negative marking. Part II was SAQ on same system where 16 SAQ were given. SAQ was timed at 60 mins for 30 marks questions. After the test students had to reply on a feedback form.Result: There was a strong correlation between the marks scored in two formats of test and there was no statistical difference between the two set of marks.Conclusion: SAQs are as effective as MCQs in assessing the performance of the students in medical pharmacology.Keywords: Short answer, Multiple choice, Assessment, Tool, EffectiveDepartment of Pharmacology, Goa Medical College, Bambolim, GoaReceived: 08 September 2014 Accepted: 24 September 2014*Correspondence to: Dr. Yogeeta Sushant C. Walke, Email: cdcsushant@hotmail.comCopyright: © the author(s), publisher and licensee Medip Academy. This is an open-access article distributed under the terms of the Creative Commons Attribution Non-Commercial License, which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. International Journal of Basic & Clinical Pharmacology | November-December 2014 | Vol 3 | Issue 6 Page 1021Walke YSC et al. Int J Basic Clin Pharmacol 2014 Dec;3(6):1020-1023Students generate responses that have potential to show originality and greater depth of understanding of the topic. It also provides a written record for assessing the thought process of the students.6 Essay questions also allow specific feedback to direct future learning.7Nnodim reported that MCQ papers were less sensitive predictors of the aggregate performance than essay questions,8 whereas Day et al. observed that essay questions although valid, failed to measure aspects of competence over and above those measured by MCQs.9At present, due to overburden of the course of pharmacology, students strategically learn what they perceive as necessary in the face of exams. Hence, assessment’s primary role in the examination should be that of a gold standard test in the diagnosis of incompetence: a test that really sorts the wheat out from the husk and regular performance assessment can definitively improve the gaps within the learning.10MCQs tests are commonly constructed to assess student’s ability to recall isolated pieces of information rapidly. There have been some suggestions, however, that competent students may perform poorly in MCQs because of their ability to read more into the questions than the examiners intend.11Recently, there are endless changes in the assessment methods and strategies. But, MCQs provide an ideal vehicle on which one can assess the body of knowledge.12In this scenario our study was important, for evaluation of attitude and perception of MCQs as performance-assessment tool, of pharmacology versus short answer questions (SAQ) among 2nd MBBS medical students.METHODSThe study was observational, retrospective study of written pen and paper type assessment that utilized a sample of 100 2nd year medical students. Permission of institutional ethics committee was duly sought to carry out the study. Pharmacology course in Goa Medical College is offered during 2nd year of MBBS curriculum whose duration spans a total of 18 months and is divided into three semesters each of 24 weeks. The students received didactic lectures with teaching aids 3 hrs a week and practical sessions inclusive of tutorials and seminars 2 hrs a week. The study consisted of two parts; part I was MCQs based on the endocrine system where four options were given for a question and the single best answer was to be ticked. MCQ was timed at 20 mins for 30 questions with 1 mark each. There was no negative marking. Part II was SAQ on the same system where 16 SAQ were given. SAQ was timed at 60 mins for 30 marks questions.At the end of MCQ and SAQ students had to fill up the feedback form. Every effort was made to eliminate examiners bias when the theory paper was evaluated by the faculty.RESULTSAs seen from the results, there is a strong positive correlation between the marks scored in MCQs and SAQs with Pearson coefficient of 0.766.T-test for paired two sample for means failed to show any significant difference between the marks scored in MCQs and SAQs at 98 degrees of freedom both in terms of the one-tail and two-tail values.Evaluation of feedback•  98% students liked the pattern of questions in MCQ and SAQ• 73% students prefer MCQ over SAQ•  76% students said that the pattern of studies differ for MCQ and SAQ•  96% students would like to have MCQs in their regular examination.DISCUSSIONThe most striking finding of our study was that there was a statistically significant overall correlation between student performance of MCQ and SAQ at 97 degrees of freedom and with p=0.00. This indicated that in general students who performed well in the essay were also likely to do well in the MCQs. Oyebola et al. and Moqattash et al.13,14 noted a similar relationship in the performance of their students.Another interesting finding in our study was that there was no correlation between students who got highest scores in either SAQ or MCQ. This showed that the most competent students had unique strengths in either one or the other examination format. Students with strong factual recall abilities scored higher in the MCQ component, whereas students with strong analytical or interpretative skills and the ability to organize and apply knowledge scored higher in SAQ. Anbar15 also observed a lack of correlation in the performance of competent students in MCQ tests compared with open-ended tests and a positive correlation for less competent students.It was also noted in our study that the overall scores for the SAQ were equivalent to MCQ that may be attributed to some degree of objective marking schemes of SAQ as compared with comparable objective nature of MCQs. A similar observation of higher scores in MCQs and Short essay questions compared with structured integrated long essays was reported by Moqattash et al. who indicated that one of the reasons for lower scores was examiners bias. Walke YSC et al. Int J Basic Clin Pharmacol 2014 Dec;3(6):1020-1023 International Journal of Basic & Clinical Pharmacology | November-December 2014 | Vol 3 | Issue 6 Page 1022However, Moqattash et al. also believed that long essay type assessment is a sensitive test requiring students not only to recall facts, but also to use higher order cognitive skills such as analytical, interpretive and application skills.There was also no significant difference between the marks scored in the two examination formats. t-test for paired two sample for means failed to show any significant difference between the marks scored in MCQs and SAQs at 98 degrees of freedom both in terms of the one-tail and two-tail values.Given the fact that the total number of students being admitted to MBBS course has increased significantly over the last 15 years and also the fact that there are plans to increase the seats further, it is imperative to have some degree of objectivity in assessment patterns. There is a significant inter-individual difference in question paper evaluation amongst different examiners in conventional University exams, wherein the papers are loaded with long answer type questions.Introducing a judicious mix of SAQs and MCQs along with a long answer type questions in various MBBS university exams will provide a much-needed objectivity in the assessment and would reduce the student’s exposure to examiners fallibility.Feedback response from students (in student’s words)SAQSAQ enables the student to think and come to a conclusion about the answer and is not a chance phenomenon. It helps in rationale basis for answering the questions. In LAQ student has to mug up the entire portion. In SAQ something can be written to obtain marks.MCQStudents have to read the entire chapter properly in minute details and most importantly understand the concept and does not need mugging up.MCQs are easier and faster to answer. It is easy to choose the correct answer than remembering everything about it. A student who has not also studied  can top in MCQ. If one don’t know the answer, tick any in the name of God. Also, handwriting is not a problem.CONCLUSIONBased on our study, we conclude that SAQs are as effective as MCQs in assessing the performance of the students in medical pharmacology. We, therefore, recommend suitably incorporating both SAQs and MCQs along with long answer type questions in routine assessment of MBBS students.REFERENCES1. Pepple DJ, Young LE, Carroll RG. A comparison of student performance in multiple-choice and long essay questions in the MBBS stage I physiology examination at the University of the West Indies (Mona Campus). Adv Physiol Educ. 2010;34(2):86-9.2. Debasis B. A study on the evaluation of perception and attitutd of 2nd M.B.B.S. medical students regarding multiple choice questions - MCQS as a performance assessment tool in pharmacology at Burdwan Medical College, West Bengal, India. Rev Prog. 2013;1(13):1-5.3. Davis CW, Donald O. Allen assessment in medical pharmacology: designing MCQs to assess a student’s type-1 problem-solving skills. Pharmacologist. 1999;41(3):107-11.4. Des Marchais JE, Vu NV. Developing and evaluating the student assessment system in the preclinical problem-based curriculum at Sherbrooke. Acad Med. 1996;71(3):274-83.5. Walubo A, Burch V, Parmar P, Raidoo D, Cassimjee M, Onia R, et al. A model for selecting assessment methods for evaluating medical students in African medical schools. Acad Med. 2003;78(9):899-906.6. Bennett RE, Rock DA, Wang M. Equivalence of free response and multiple choice items. J Educ Meas. 1991;28(1):77-92.Table 1: Correlation.SAQ MCQSAQ 1MCQ 1Correlation coefficient 0.7660021.191595 9.848857802t 11.73585df 97p 0.00000SAQ: Short answer question, MCQ: Multiple choice questionTable 2: t‑test: Paired two sample for means.SAQ MCQMean 17.06061 17.51515152Variance 27.95547 19.15027829Observations 99 99Pearson correlation 0.766002Hypothesized mean difference0df 98t-stat −1.32456P (T≤t) one-tail 0.094199t critical one-tail 1.660551P (T≤t) two-tail 0.188398t critical two-tail 1.984467SD 5.287294 4.376103SAQ: Short answer question, MCQ: Multiple choice question, SD: Standard deviation International Journal of Basic & Clinical Pharmacology | November-December 2014 | Vol 3 | Issue 6 Page 1023Walke YSC et al. Int J Basic Clin Pharmacol 2014 Dec;3(6):1020-10237. Freestone N. Drafting and acting on feedback supports student learning when writing essay assignments. Adv Physiol Educ. 2009;33(2):98-102.8. Nnodim JO. Multiple choice testing in anatomy. J Med Educ. 1992;26:301-9.9. Day SC, Norcini JJ, Diserens D, Cebul RD, Schwartz JS, Beck LH, et al. The validity of an essay test of clinical judgment. Acad Med. 1990;65 9 Suppl: S39-40.10. Jain S, Alkhawajah A, Larbi E, Al-Ghamdi M, Al-Mustafa Z. Evaluation of student performance in written examination in medical pharmacology. Sci J King Faisal Univ (Basic Appl Sci). 2005;6(1):1426-35.11. Mujeeb AM, Pardeshi ML, Ghongane BB. Comparative assessment of multiple choice questions versus short essay questions in pharmacology examinations. Indian J Med Sci. 2010;64(3):118-24.12. Anderson J. Multiple choice questions revisited. Med Teach. 2004;26(2):110-3.13. Oyebola DD, Adewoye OE, Iyaniwura JO, Alada AR, Fasanmade AA, Raji Y. A comparative study of students’ performance in preclinical physiology assessed by multiple choice and short essay questions. Afr J Med Med Sci. 2000;29(3-4):201-5.14. Moqattash S, Harris PF, Gumaa KA, Abu-Hijleh MF. Assessment of basic medical sciences in an integrated systems-based curriculum. Clin Anat. 1995;8(2):139-47.15. Anbar M. Comparing assessments of students’ knowledge by computerized open-ended and multiple-choice tests. Acad Med. 1991;66(7):420-2.doi: 10.5455/2319-2003.ijbcp20141212Cite this article as: Walke YSC, Kamat AS, Bhounsule SA. A retrospective comparative study of multiple choice questions versus short answer questions as assessment tool in evaluating the performance of the students in medical pharmacology. Int J Basic Clin Pharmacol 2014;3:1020-3.

A retrospective comparative study of multiple choice questions versus short answer questions as assessment tool in evaluating the performance of the students in medical pharmacology

https://www.ijbcp.com/index.php/ijbcp/article/download/1180/1055

A retrospective comparative study of multiple choice questions versus short answer questions as assessment tool in evaluating the performance of the students in medical pharmacology

Abstract

Similar works

Full text

Available Versions

International Journal of Basic & Clinical Pharmacology