[Abstract]:

A survey was carried out to investigate literature review  skills of students doing their final year research projects. This survey was given before intervention to  their literature review skills. Their literature review skills revealed by this survey were compared with that of a post-intervention survey, which was in turn compared with the literature review skills revealed in their intermittent reports. From the pre- and post-intervention surveys, it   seems that the intervention workshops work and the percentage of students understanding how to conduct a literature review has improved from 11% to 78%. However, the intervention was not so successful if the literature    review knowledge revealed by the project appreciation   was compared with that of pre-intervention survey

Goh, Steven C.

Ku, Harry S.

English

University of Southern Queensland ePrints

Southern Region Engineering Conference 11-12 November 2010, Toowoomba, Australia SREC2010-T4-4 1  SREC2010-T4-4  Preliminary results on literature review skills of students doing capstone projects   H Ku  Faculty of Engineering University of Southern Queensland Toowoomba, Australia e-mail: ku@usq.edu.au S Goh Faculty of Engineering University of Southern Queensland Toowoomba, Australia e-mail: gohs@usq.edu.au  Abstract— A survey was carried out to investigate literature review skills of students doing their final year research projects. This survey was given before intervention to their literature review skills.  Their literature review skills revealed by this survey were compared with that of a post-intervention survey, which was in turn compared with the literature review skills revealed in their intermittent reports.  From the pre- and post-intervention surveys, it seems that the intervention workshops work and the percentage of students understanding how to conduct a literature review has improved from 11% to 78%.  However, the intervention was not so successful if the literature review knowledge revealed by the project appreciation was compared with that of pre-intervention survey.   Keywords- final year engineering research project, pre-intervention, post-intervention, intervention, literature review and project appreciation.  Introduction  A study (Final Project Teaching in Higher Education within Civil Engineering: New Perspective) conducted by Montes et al. [1] showed that 60% of students believed that the most difficult aspects of the project were getting started, defining the methods and content, obtaining data, and preparing and analyzing the data. The remaining 40% pointed to the viability (12%) and the construction (28%) of the project as being the most difficult aspects to deal with. The literature review is one of the main platforms for constructing the research question and establishing the framework of the research project; therefore, it is of critical importance.  The final year undergraduate course, “Research Project”, a form of project-based learning, is considered a critical course in the Engineering and Surveying bachelor degree programs. It provides the opportunity for students to draw comprehensively on the breadth and depth of knowledge and technical skills developed in the first three years of their program. It acts as a confirmation of the tertiary training in engineering or surveying by this final year project in preparation for their future careers [2].  The total number of students enrolled in this course, ENG4111/ENG4112, Research Project in 2009 was 150 with 54 of these as on-campus enrolments.  Students with grade point average (GPA) of 5.5 (out of 7) or above were classified as good students, those between 4.5 and 5.5 being average students and those below 4.5 being weak students.  This is the classification of students from the beginning of the research.  This paper will analyze the pre-intervention survey and post-intervention survey results to evaluate the effect of intervention to improve the literature review skills of students.  In order to ensure that students do really apply the knowledge gained from an intervention program, the literature review skills of post-intervention were compared with the literature review skills revealed by the students in their project appreciation.   The 3-P Model of Learning  Tam [3] outlined the 3-P Model of Learning [4], in which the overall assumption that Biggs had about learning through this 3-P model was that learning outcomes were a result of the effects of the teaching and learning contexts with the student approaches to learning. Both student and teaching presage factors interacted to produce an approach to learning which produced its characteristic outcome. The methodologies used in this study were derived from the 3-P Model [5].  Nine students agreed to participate with the research from start to end: four from Mechanical Engineering, three from Civil Engineering and two from Electrical Engineering disciplines.  Students with a GPA of above 5.5 were considered very good; those with a GPA between 4.5 and 5.5 were considered good and the rest were considered average.  Of the students participating in the research, there were five very good students, two good students and two average students. Pre-intervention results The questions and answers relevant to literature review in the pre-test survey include: i) The “literature review” is one component of the project dissertation. Explain what you think a “literature review” is? If you have no idea what a “literature review” is, please tell us. Summary of answers to i):   Six replied correctly and three answered wrongly, of whom two had a low GPA of slightly above 3 out of 7.  Southern Region Engineering Conference 11-12 November 2010, Toowoomba, Australia SREC2010-T4-4 2  ii) What is the purpose of a literature review? Summary of answers to ii): Only two answered correctly, both of whom had high GPAs. iii) Do you think that your literature review, project methodology and research activities are related? Summary of answers to iii): Three did not know, of which one response was from a very good student and one was from a good student. Three students had no answer. These were also the three who responded negatively in Question i).  Six said ‘yes’, but only one gave a rational answer.  The GPA of this respondent was good.   From the responses to the above questions, it can be seen that only one of the nine students knew anything about a literature review.  Eighty-nine percent of them did not know enough about a literature review. The Intervention Four two-hour workshops for intervention were carried out during lunch.  Workshop 1 was an introduction to the project only. No intervention was begun. Workshop 2 consisted of another two-hour session over lunch in which the following topics would be worked through (Ku and Goh, 2009): • Typical elements of research project; • What literature review is;  • Purposes of literature review; and • Why Engineers Australia wants people who have undertaken a literature review to graduate.  The final two-hour lunch workshops, Workshops 3 and 4, were conducted in a computer laboratory and the following activities were carried out (Ku and Goh, 2009):  • Practical skill-based activities related to finding information; The use of search logs to record processes and results of each activity; • The use of journals; and  • A debrief for students on presence and effect of presage elements.  Exact content of these workshops would depend on responses to the survey and projects being done by students.  Post-intervention results The questions and answers relevant to the literature review in the post-test survey include:  i) The "literature review" is one component of the project dissertation and a major component of your Appreciation. Explain what you think a "literature review" is. Summary of answers to i): Seven answered correctly; the two who answered incorrectly had the lowest GPAs.  ii) What is the purpose of a literature review? Summary of answers to ii): Six gave correct responses; three gave incorrect responses. Of these three, one was a Civil Engineering student and two were from Electrical Engineering.  iii) Has your literature review work influenced how you have undertaken any of your practical research, or will it influence any future practical research? Summary of answers to iii): Seven answered ‘yes’, of whom two had answered Question ii) incorrectly. They did not know the purpose of a literature review but claimed that it had influenced their projects so far and would influence their projects in the future.  Two answered ‘no’, one of whom replied correctly to Question ii). This student knew the purpose of a literature review but claimed that it did not influence his project. This is possible only if he could not find any relevant information about his project from the literature review.  The other one, a very good student, did not know the purpose of a literature review and hence it is logical that he claimed it had not influenced his project so far and would not influence it in the future.  iv) Has your literature review explored information from different situations or applications that could be applied to your own project work? Summary of answers to iv): All but one answered ‘yes’. The student who answered ‘no’ to this question also responded negatively to Question iii).   The student who answered ‘yes’ to this question but ‘no’ to Question iii) confirmed that they could not find any relevant information about this project by conducting a literature review. This was confirmed by the comment ‘because there has been no practical research in my area’.  v) Has your literature review explained your choice of a theory and its relevance to your work? Summary of answers to v): Five answered positively; one answered negatively; one was not sure; two did not respond because they seemed not to understand the question fully.  vi) Has your literature review addressed any social, environmental or ethical issues related to the project? Summary of answers to vi): Five responded positively; the remainder negatively, as they explained that they could not find these issues in the literature.  Despite the intervention, the two students with low GPAs did not know much about a literature review but they agreed that the literature review had influenced their current project, and would influence their projects in the future.  Eight of them commented that they found information relevant to their projects by undertaking the literature review.  However, it can be argued that seven (78%) of them thoroughly understood the purpose and structure of a literature review after the intervention.  No evidence was elicited as to which discipline’s participants performed better than the others.  A Rubric for assessing project appreciation Southern Region Engineering Conference 11-12 November 2010, Toowoomba, Australia SREC2010-T4-4 3  A rubric based on the work of Willison and O’Regan [6] was designed to evaluate the literature review skills from the project appreciation submitted by the students.  Figure 1 shows the literature review assessment rubric.  By reading the first column, it can be found that there are three elements to be considered for assessment: project analysis, use and synthesis of information, and the completed literature review.  All assessment statements were divided into five ratings.  Under the heading Project analysis, the assessment statement includes stating the problem, purpose and aim. A score of 1 is given to ‘No problem, purpose or aim’ and a score of 5 is given to ‘Clear and comprehensive articulation of problem, purpose and aim’.  Under Use and synthesis of information, the assessment statements include demonstrating familiarity with relevant literature, evaluating the credibility of literature use, and reflecting analysis, synthesis and application of literature in project work.  Finally, under Literature review document, the assessment statement includes Communicating knowledge and understanding and Presenting knowledge and understanding.  For each statement, a student should get a score of three to pass.  There were six statements for assessment, hence the maximum total score a student could get would be 5 x 6 = 30 and the passing score was 3 x 6 = 18.  Results of project appreciation and discussions By analyzing the project appreciation of the participants, it was found that only three of the participants passed the assessment with a total score in the range of 20-23. These were marginal passes only.  The two students with the lowest GPAs achieved only 12 out of 30.  The lowest mark was eight, which was awarded to what we regarded as a very good student.  Four participants passed ‘stating the problem, purpose and aim’, including the three who had passed the whole evaluation.  Five participants passed ‘demonstrating familiarity with relevant literature’, of whom three passed the whole evaluation.  Four participants passed ‘evaluating the credibility of literature use’, of whom three passed the whole evaluation.  Six students passed ‘reflecting analysis, synthesis and application of literature in project work’, of whom three passed the whole evaluation.  Five participants passed ‘communicating knowledge and understanding’, of whom three passed the whole evaluation.  Two students passed ‘presenting knowledge and understanding’, of whom one passed the whole evaluation.   From the information provided by the project appreciation, it can be found that only 33% of the participants really passed the assessment rather than 78% as revealed by the post-intervention survey.  Therefore it appears that the intervention was not very successful.  It can be argued that as the assessment of project appreciation is not assessable, and did not influence the final grading of the students, some students did not write the project appreciation whole-heartedly.  This observation is supported by the fact that the student with the highest GPA received a low score of 15.  Conclusions It is not possible to draw a final conclusion as to whether the intervention was successful or not because some of the students did not take the project appreciation exercise seriously.  However, it will be possible to draw a conclusion after students submit their dissertations several months later. These, it would be hoped, would be written seriously.  From this survey, it can be argued that students with high GPAs will benefit from intervention to improve their literature review skills.   References  [1] Montes, G M, Gamez, M C, Escobar, B M, Garcia, Final Project Teaching in Higher Education within Civil Engineering: New Perspective, J of Prof Issues in Eng Ed and Prac, 2007, pp. 94-98.  [2] Vaezi-Nejad, S M, Final Year Project as an Important Component of the Electrical and Electronic Engineering Degree Course at the University of Greenwich, Measurement and Control, Vol. 30, July, 1997, pp.165-166.          [3] Tam, M, Promoting Deep Learning: A Conceptual Model, Learning Matters at Lingnan, 12 April 1999, pp.1-2; http://www.ln.edu.hk/tlc/info/lm/lm0599.htm <viewed on 14 September 2009>  [4] Biggs, J B, Approaches to the Enhancement of Tertiary Teaching, Higher Education Research and Development, Vol. 8, No. 1, 1989, pp. 7-25.  [5] Ku, H and Goh, S, Methodologies for Investigating the Quality of Literature Review in the Thesis of Final Year Engineering Research Project, International Journal of Engineering Education, 2009 (submitted for publication).  [6] Willison, J and O’Regan K, The research skill and development framework, The University of Adelaide, 2006.             Southern Region Engineering Conference 11-12 November 2010, Toowoomba, Australia SREC2010-T4-4 4  Figure 1: Literature review assessment rubric    1 2 3 4 5 Project analysis      States the problem, purpose and aim No problem, purpose or aim evident Limited explanation of problem, purpose and aim Adequate explanation of problem, purpose and aim Comprehensive articulation of problem, purpose and aim articulated Clear and comprehensive articulation of problem, purpose and aim Use and synthesis of information      Demonstrates familiarity with the relevant literature Description of relevant literature seriously lacking, no analysis of relevant literature evident Limited description of relevant literature, no analysis of relevant literature evident Adequate description and minimal analysis of relevant literature Adequate description and analysis of relevant literature Comprehensive description and analysis of relevant literature Evaluates the credibility of literature used Credibility not evaluated Elements of credibility considered but misapplied Credibility evaluated but not linked to proposed work Credibility evaluated and linked to proposed work Credibility evaluated and integrated into proposed work Reflects analysis, synthesis and application of literature in project work Relevant literature not applied to proposed project work Some literature used but not applied to proposed project work Relevant literature applied to proposed work Literature analyzed, synthesized and applied to proposed project work Literature rigorously analyzed, synthesized and applied to proposed project work Literature review document      Communicates knowledge and understanding Research question, research gap and research significance not communicated Limited communication of research question, research gap and research significance Partial communication of research question, research gap and research significance Adequate communication of research question, research gap and research significance Clear and comprehensive communication of research question, research gap and research significance Presents knowledge and understanding Discipline language not used; reference list incomplete and/or inaccurate; in-text citations incomplete or inaccurate; did not adhere to prescribed Appreciation/ Dissertation format Inadequate use of discipline language; reference list incomplete and/or inadequate; in-text citations incomplete or inadequate; did not adhere to prescribed Appreciation/ Dissertation format Some discipline language used; reference list and in-text citations not wholly compliant with Referencing Guide; adhered to prescribed Appreciation/ Dissertation format Discipline language mostly used; reference list complete and mostly compliant with Referencing Guide; in-text citations complete and mostly compliant with Referencing Guide; adhered to prescribed Appreciation/ Dissertation format Discipline language used appropriately at all times; Accurate and complete reference list; accurate and complete in-text citations; adhered fully to prescribed Appreciation/ Dissertation format 

Approaches to the Enhancement of Tertiary Teaching,

Final Year Project as an

Methodologies for Investigating the Quality of Literature Review in the Thesis of Final Year Engineering Research Project,

Promoting Deep Learning: A Conceptual Model,

The research skill and development framework,

Preliminary results on literature review skills of students doing capstone projects

http://eprints.usq.edu.au/18973/1/Ku_Goh_SREC2010_PV.pdf

Preliminary results on literature review skills of students doing capstone projects

Abstract

Similar works

Full text

Available Versions

University of Southern Queensland ePrints