The Bob L. Herd Department of Petroleum Engineering at Texas Tech University has made a lot of significant actions of improvement to its graduate program that was motivated by a systematic SACSCOC assessment plan. This paper shows how the SACSCOC assessment plan aided in making continuous actions of improvement and as a conclusion of these actions, how the current graduate curriculum plan was improved. This paper highlights the details of the graduate department assessment plan, such as how graduate program objectives are assessed, what assessment tools are used, when data are gathered and evaluated, and when actions of improvement are made. This paper will also detail how the analysis of data was utilized in making actions of continuous improvement. At the end of the paper examples of the significant actions of improvement made based on the department assessment and evaluation plan are presented.Cockrell School of Engineerin

Emadi, H.

Gamadi, Talal D.

Stockton, Charlotte

English

Texas ScholarWorks

 Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 Effective Assessment Plan Leading to Strong Reform of Petroleum Engineering Graduate Program Charlotte Stockton, Talal D. Gamadi, and H. Emadi   Texas Tech University/ Bob L. Herd Department of Petroleum Engineering 807 Boston Avenue, Box 43111, Lubbock, Texas, 79409, USA E-mail: talal.gamadi@ttu.edu  Abstract    The Bob L. Herd Department of Petroleum Engineering at Texas Tech University has made a lot of significant actions of improvement to its graduate program that was motivated by a systematic SACSCOC assessment plan. This paper shows how the SACSCOC assessment plan aided in making continuous actions of improvement and as a conclusion of these actions, how the current graduate curriculum plan was improved. This paper highlights the details of the graduate department assessment plan, such as how graduate program objectives are assessed, what assessment tools are used, when data are gathered and evaluated, and when actions of improvement are made. This paper will also detail how the analysis of data was utilized in making actions of continuous improvement. At the end of the paper examples of the significant actions of improvement made based on the department assessment and evaluation plan are presented.    1. Introduction The Bob L. Herd Department of Petroleum Engineering at Texas Tech University is uniquely located in the Permian Basin, where approximately 22% of the nation`s petroleum resources and 68% of Texas` petroleum resources lie a 175-mile radius. The department has been consistently ranked in the top 10 petroleum engineering departments nationwide for both the graduate and undergraduate program. The department offers three degrees: Bachelor of Science in Petroleum Engineering, Master of Science in Petroleum Engineering, and Doctor of Philosophy in Petroleum Engineering. 2. Assessment Plan Methodology     In general, the main objectives of the graduate program in our department are to provide students with the opportunities to reach a critical understanding of the basic scientific and engineering principles underlying their fields of interest and to cultivate their ability to apply these through advanced methods of analysis. In specific, the graduate program administration assigned specific student outcomes to measure the student attainment of the program objectives. The graduate program student outcomes are:     Core Knowledge: Students should demonstrate advanced knowledge in a core area consistent with the focus of their program.  Research Methods and Analysis: Students should demonstrate quantitative and qualitative skills in the design, analysis, and presentation of research projects that are consistent with the focus of their program.   Scholarly Communication: Students should produce written and oral communications of quality, as consistent with the focus of their program.  Pedagogy: Students should participate in classroom pedagogy consistent with undergraduate education in the associated major.  Professionalism: Students should know and participate in the intellectual and organizational aspects of the profession as applicable to the major area of study, including the ethical conduct of research. 3. Assessment Method   3.1 Student`s Performance Indicators  The following assessment methods are used to gather data to evaluate the students’ attainment of the graduate program outcomes:  M.Sc. Thesis: The Students’ ability to demonstrate advanced knowledge in a core area will be assessed through responses indicated on the Petroleum Engineering Thesis Rubric completed by committee members regarding the final presentation and project (see attached corresponding rubric). A response used to assess core knowledge include the following categories listed in the Thesis Rubric: ‘III. Relevance’ (Figure 2).   Ph.D. Dissertation: The Students' ability to demonstrate advanced knowledge in a core area will be assessed through responses indicated on the Petroleum Engineering Dissertation Rubric completed by committee members regarding the final presentation and project (Figure 3). Responses used to assess core knowledge include the following categories listed in the Dissertation Rubric: ‘III. Relevance’ and ‘IV. Results’ Results.  Students will make their final presentation and complete the project during their last semester of enrollment.  Qualifying Exam: Doctoral students’ understanding of the core areas of the program will be assessed through results collected from students' Qualifying Exams which consists of written and oral examinations in the four core areas in Petroleum Engineering (Drilling, Production, Reservoir and Formation Evaluation).  While students can complete the Qualifying Exam twice during their degree  Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 program, only the first set of results will be used for this assessment.   Student Exit Survey: Students understanding of core areas of the program will be assessed through responses indicated on the following portions of the Student Exit Survey.  Patents/Publications: A student’s ability to produce written and oral communication will be assessed by the number of publications or patents produced by a student upon graduation as indicated by the student’s Thesis Committee Chair/Master's Report Chair as well as indicated by the student’s response to the Student Exit Survey (see attached corresponding survey).  Professional Development Activities: Professionalism will be measured by the frequency of professional conference attendance made by students during the completion of their degree program.  3.2 Criterion Used for Each Assessment Method.   M.Sc. Thesis/Dissertation Criterion: 85% of graduate students will receive an average score of 4.0 or higher.  M.Sc. and Ph.D. Student`s performance is assessed and evaluated annually at the end of each academic year.  Student Exit Survey Criterion: Graduate students who complete the survey will respond with an average of 4 or higher. An assessment and evaluation of Student Exit Survey results will occur annually, at the end of each academic year.  Patents/Publications Criterion: 80% of graduate students must have at least one publication. An assessment of student patents and publications will be completed annually at the end of each academic year.  Professional Development Activities Criterion: Graduate students will be measured by the frequency of professional conference attendance made by students during the completion of their degree program. International graduate students are required to take the ITA (International Teaching Assistant) workshop to be considered for a teaching assistant position. More than 35% of our doctoral students are working as a teaching assistant (TAs) and over 90% of our students are international.  Qualifying Exam Criterion: 75% of graduate students, who are taking the exam, will successfully complete 3 of the 4 areas of the Qualifying Exam. An assessment of the Qualifying Exam results will be completed annually at the end of each academic year.  4. Assessment Results and Actions of Improvement Qualifying Exam: Every year, more than 80% of our graduate students pass the QE successfully, however, it has been noticed by faculty that students need more core knowledge of some petroleum engineering areas. Based on that, the graduate program was reformed as following: The graduate program curriculum is organized into four petroleum engineering areas as specified in the Society of Petroleum Engineering (SPE) nomenclature; Drilling Engineering, Formation Evaluation, Reservoir Engineering, and Production Engineering. In each area, the courses are divided into core courses and elective courses. The master`s degree students are required to take one course from each of the four core areas; the doctoral degree students should include at least two courses in each core area.  M.Sc. Thesis and Ph.D. Dissertation:  The department has required all graduate students to attend professional conferences. The goal of this is to help students learn how to present research, as well as how to narrow down research questions to something that is relevant in the field, as well as to ensure the professionalism of our graduates. The department also requires students to publish before their Thesis/Dissertation defenses.  Each year based on the evaluation results, more than 95 % are passing the thesis and dissertation defense with high response value form the survey given to faculty and attendance.  5. Summary  The SACSCOC assessment plan is one method the department uses to assess the graduate program and to determine which actions for improvement that we need to implement. The department has determined five student outcomes to assess each academic year. Data is collected in a variety of forms.  A major action for improvement was the requirement that students take core courses (two in each core area for Ph.D. students) before the Qualifying Exam. This directly correlates to the Core Knowledge student outcome and the high pass rate of the QE shows that our graduates have a solid foundation in the Core Areas as well as a specific focus of their program.  Another significant action for improvement is using the Thesis/Dissertation defense to measure Core Knowledge, Research Methods, and Analysis as well as Scholarly Communication. The department determined that it would be beneficial to our students to require conference attendance and publications before their defenses. The Graduate Committee will continue to analyze these data and look for future actions for improvement. This continuous process allows for the department to adjust to the needs of the industry and our students.   Acknowledgment We would like to thank the department graduate committee for providing valuable feedback to graduate program continues improvement plan.   References [1] Thesis Rubric (2017-2018) [2] Dissertation Rubric (2017-2018) [3] Student Exit Survey (2017)       Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 Figure1 shows the distribution of performance indicators on graduate student outcomes.          M.Sc. Thesis PhD Dissertation Qualifying Exam Student Exit Survey Professional Development Activities:  Patents Publications Core Knowledge X X X X  X Research Methods and Analysis X X     Scholarly Communication X X   X  Pedagogy     X  Professionalism X X    X Category Performance Ratings Exceptional Above Average Average Below Average Very Poor Score 5 4 3 2 1 I. Quality of Oral Communication: Communicates research theory, methodology and results clearly.          II. Quality of Written Dissertation: Communicates research theory, methodology and results clearly.       III. Relevance: Technical contribution of the research completed.       IV. Results: Analyzed and interpreted.       V. Publications: Professional publication(s) resulted/expected or patent(s) filed prior to dissertation. (Patent=1.5 publications.) (4+ pub.) (3 pub.) (2 pub.) (1 pub.) (0 pub.)  Total Score:   Average Score:  Category Performance Ratings Exceptional Above Average Average Below Average Very Poor Score 5 4 3 2 1 I. Quality of Oral Communication: Communicates research theory, methodology and results clearly.          II. Quality of Written Thesis: Communicates research theory, methodology and results clearly.       III. Relevance: Technical contribution of the research completed.       IV. Results: Analyzed and interpreted.       V. Publications: Professional publication(s) resulted/expected or patent(s) filed prior to thesis. (Patent=1.5 publications.) (3+ pub.) (2 pub.) (1 pub.) (0 pub.) (0 pub.)  Total Score:   Average Score:  Fig. 2.1 Thesis Rubric Fig. 2.2 Dissertation Rubric 

Effective Assessment Plan Leading to Strong Reform of Petroleum Engineering Graduate Program

https://repositories.lib.utexas.edu/bitstream/2152/79876/2/effective-assessment-plan-leading-to-strong-reform-of-petroleum-engineering-graduate-program.pdf

Effective Assessment Plan Leading to Strong Reform of Petroleum Engineering Graduate Program

Abstract

Similar works

Full text

Available Versions

Texas ScholarWorks