The undergraduate curriculum committee from the Bob L. Herd Department of Petroleum Engineering at Texas Tech University has made significant modifications that were determined by a systematic student outcomes assessment plan. This paper shows how the department assessment plan facilitated continuous actions of improvement and ultimately provides an example of how a strong undergraduate curriculum plan was constructed. The paper highlights the details of the department assessment plan, such as how ABET student outcomes are mapped to department undergraduate courses, what assessment tools were used, when data were gathered and evaluated, and how the analysis of data was utilized to implement actions of improvement. Finally, the paper provides two examples of significant actions of improvement, made based on the department assessment and evaluation plan.Cockrell School of Engineerin

Disque, Bethany

Gamadi, Talal D.

Heinze, Lloyd

Watson, Marshal

English

Texas ScholarWorks

 Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 Effective Student Outcomes Assessment Plan Reform Strong Undergraduate Curriculum Plan Talal D. Gamadi, Bethany Disque, Marshal Watson, and Lloyd Heinze    Texas Tech University/ Bob L. Herd Department of Petroleum Engineering 807 Boston Avenue, Box 43111, Lubbock, Texas, 79409, USA E-mail: talal.gamadi@ttu.edu  Abstract    The undergraduate curriculum committee from the Bob L. Herd Department of Petroleum Engineering at Texas Tech University has made significant modifications that were determined by a systematic student outcomes assessment plan. This paper shows how the department assessment plan facilitated continuous actions of improvement and ultimately provides an example of how a strong undergraduate curriculum plan was constructed. The paper highlights the details of the department assessment plan, such as how ABET student outcomes are mapped to department undergraduate courses, what assessment tools were used, when data were gathered and evaluated, and how the analysis of data was utilized to implement actions of improvement.  Finally, the paper provides two examples of significant actions of improvement, made based on the department assessment and evaluation plan.    1. Introduction     The Bob L. Herd Department of Petroleum Engineering at Texas Tech University is uniquely located in the Permian Basin, where approximately 22% of the nation`s petroleum resources and 68% of Texas` petroleum resources lie in a 175-mile radius. The department has been consistently ranked in the top 10 petroleum engineering departments nationwide for both the graduate and undergraduate programs. The Bachelor of Science in Petroleum Engineering program began in 1946, graduating its first student in 1948. The program was first accredited by ABET in 1952. The last general review was accomplished in the fall of 2011.  In August 2008, the department was named for Bob L. Herd to recognize his many accomplishments in the industry and his steady and continuing support of this department and Texas Tech University.  The department of petroleum engineering supervises the following degrees:  Bachelor of Science in Petroleum Engineering  Master of Science in Petroleum Engineering  Doctor of Philosophy in Petroleum Engineering The Bachelor of Science in Petroleum Engineering has no options for a minor.  1.1 Facility      In March 2014, the department moved into the new $23.8 million, 42,000 square foot TFPERB. This move allowed the department to invest the following:  $1 million in undergraduate core and rheology laboratories, replacing all undergraduate lab equipment.    $1.5 million drilling simulations lab.  $0.5 million in the production/reservoir visualization lab.  $1 million in A/V system in four classrooms – including 3-D and tele-video capabilities.  $0.3 million on reservoir geo-modeling workstation lab. 1.2 Program Educational Objectives   Continue professional development through participation and leadership in professional organizations (SPE, SPEE, ASEE, API, AADE, SPWLA).  Pursue lifelong learning through continuing education or postgraduate education (professional meetings, short courses, graduate courses).  Progress to professional registration so that some individuals graduate from an ABET-accredited degree plan, pass the Fundamentals of Engineering Exam, work in increasingly responsible engineering positions, and pass the Professional Exam.  1.3 ABET a-k Student Outcomes   Graduates of our BS in Petroleum Engineering department should attain the ABET a-k Student outcomes, listed below, before their graduating. The ABET a-k student outcomes are recently merged into 1-7 ABET student outcomes. All these outcomes were assessed equally using the same performance indicators listed in Table 3, under Metrics.  a. an ability to apply knowledge of mathematics, science, and engineering b. an ability to design and conduct experiments, as well as to analyze and interpret data c. an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability d. an ability to function on multidisciplinary teams e. an ability to identify, formulate, and solve engineering problems  Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 f. an understanding of professional and ethical responsibility g. an ability to communicate effectively h. the broad education necessary to understand the impact of engineering solutions in a global, economic, environmental, and societal context i. a recognition of the need for and an ability to engage in lifelong learning j. a knowledge of contemporary issues k. an ability to use the techniques, skills, and modern engineering tools necessary for engineering practice. To achieve the Program Educational Objectives (PEOs) of the Petroleum Engineering program, graduates of the program must demonstrate that they have achieved the ABET a-k Student Outcomes (SOs). In our assessment process, we relate our SO to our goals as well as our PEO.  2. Continuous Improvement      Our petroleum engineering program is engaged in a process of continuous improvement designed to increase the attainment levels of both student outcomes and program educational objectives.  Specific actions taken to improve the program are guided by the Undergraduate Committee’s evaluation of assessment results and benchmarking these results against other programs. Additionally, actions are guided by faculty discussions at SO and PEO reviews, using input from members of our External Advisory Board, alumni, employers, and students. The Undergraduate Committee presents its recommendation to the faculty, and if needed, votes are taken. Changes are vetted by the External Advisory Board. This section describes the periodic processes of assessing and evaluating the degree of attainment of the SOs and how the results of these processes are used to continuously improve the Bachelor of Science in Petroleum Engineering program. The ABET Undergraduate Committee reviews SOs on a biannual basis, recommended actions and results are presented to members of the External Advisory Board and to the faculty at faculty meetings. The assessment of SOs involves processes of identifying, collecting data and preparing them for evaluation. Data used to assess the SOs include instructor self-assessment of courses, exams from specific courses, a senior survey, and senior exit interviews.     Instructor self-assessment of courses (or Instructor course evaluation, Table 1.  At the end of each semester, instructors perform a self-evaluation of the course(s) they taught in that semester. The instructor self-evaluations include instructor assessment of course outcomes, evaluation of the performance of the students, and strategies for course future improvement.   Senior Capstone Project. Performance in the capstone design courses, PETR4121 & PETR4222.   Senior Exit Survey. This survey is given to students during the last month before their graduation. These students complete a program evaluation survey containing twenty questions related to SOs as well as additional questions querying departmental services, GPA, co-op, undergraduate research experience, and plans for graduate study or employment.   Exit interviews of graduating seniors.  The survey is used to initiate discussion during a 30-minute exit interview with the Department Chair. Results of exit interviews are tabulated annually and are kept in departmental archives.  All SOs are assessed each year (i.e., 2012-13 to 2016-17), Table 3.  To make it less labor intensive, a subset of selected courses that enable each student outcome (as opposed to all eligible courses) was assessed.  Tables 1 shows specific selected courses (in shaded boxes) for evaluating the attainment of each of the SOs (i.e., ABET ak) for the three-year cycle in our assessment plan.   2.1 Evaluation Processes The student outcomes evaluation processes consist of the following steps: 1. Development of performance metrics and criteria for each Student Outcome (SO) that are required to attain Program Educational Objectives (PEOs) 2. Evaluation of attainability of each Student Outcome (SO) using the performance metrics developed in Step 1. This involves obtaining performance evaluation results for each performance metric and its corresponding attainability criterion for each student outcome and aggregating these results to obtain the overall result of the attainability of each SO. 3. Interpretation of the evaluation results on the program including the impact of previous changes to improve the attainability of the PEOs. 4. Examination conclusions and validity of the assessment and evaluation processes. 5. Recommendation of future changes or modification to improve both the results obtained in Step 3 and the processes in Step 4.  Step 2 requires systematic reviews by the undergraduate committee especially for the performance metric based on the instructor self-assessment of courses.  For each SO, the committee first assesses whether the SO is attainable by each course that it enables.  In doing so, evaluation results of each performance metric, in a specified course, must be obtained and aggregated.  Next, the committee aggregates evaluation results from all the courses that enable the SO being considered.  The aggregation of the evaluation results at each level is based on majority votes. Thus, in this report, the evaluation results are binary (e.g., “yes” if the SO is attainable, “no” otherwise).  Steps 3, 4, and 5 are integrated parts of the discussion in this evaluation process. The committee follows the evaluation process by applying the following: as shown in Figure 1, we mapped the courses relevant to each other and asked the faculty in each map (loop) to cooperate with each other. The main objective of these loops is to provide faculty with clearer targets for developing standards-based curriculum, instruction, and assessment. Thus, aiding in mapping the taught curriculum and analyzing its alignment to benchmarks and standards. The Bloom's cognitive  Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 taxonomy (Bloom & Krathwohl, 1984) is applied as a resource to integrate courses with each other. In the sophomore year, faculty utilizes the first level of Bloom’s cognitive taxonomy in each of the courses, which contain the knowledge, remember, and describe elements of the first level. In the junior year, faculty takes the students to an advanced level of the Bloom's cognitive taxonomy by enforcing the topics and applying them to solve related problems, explain why, and apply elements of the second level. In the senior year, faculty utilizes the highest level of the Bloom’s cognitive taxonomy by emphasizing the topics, encouraging students to use these topics to design a system or component, and finally evaluate the outcomes of the system or component, shown in figure 1 and 2.   Table 2 provides a timeline of the detailed activities mentioned above with an explicit schedule in a six-year cycle.  Each activity, however, may occur more often than scheduled, if the committee determines it is needed. Due to the recent implementation of the Bloom’s process, the monitoring cycle is required every two years to ensure that critical data was assessed correctly. Evaluation criteria in the form of performance indicators were developed to assess the attainability of each of the SOs. The expected attainment and summary of results are listed in Table 1-3 below.   3. Results and Actions of Improvement   Table3 shows an example of the assessment and evaluation final results, conducted in each assessment year. Actions of improvement based on assessment and evaluation results process are divided into two categories:   Improvement of Program Educational Objective (PEO) Attainability and Validity with Constituents: deal with continuous improvement of the attainability of the PEOs including improving facilities, attainment of graduates characterized by the PEOs.   Improvement of Student Outcome (SO) Attainability: deal with continuous improvement of subjects and issues that directly impact SOs including curriculum and student outcome assessments.   3.1 Summary of the Actions of Improvement  The department has reviewed and integrated changes in curriculum to meet the technology changes and emphasis in the industry.  These changes were guided by data gathered from the department assessment plan. Revisions to the curriculum are made with input from the members of the faculty curriculum committee, Petroleum Industry Advisory Board (PIAB) curriculum committee, industry surveys and senior exit surveys. Technological advances have also been an integral part of the transitions in the curriculum. The curriculum revisions are identified below:  Added petroleum sophomore course (Petroleum Methods)  Update geology courses and topics.  Implemented more rigorous senior design sequence.  Moved from one elective to four senior electives  Students now can specialize in one of two areas:     Drilling/production operation, and Reservoir engineering  3.2 Two Detailed Examples of the Made Actions of Improvement    The first action of improvement Action Description Moved from multi-course embedded design components in the junior and senior years, to a two-semester design course sequence in the last two semesters of the senior year.  The scenarios containing design are based on industrial experiences including seismic data, drilling, and reservoir and production data. Students are put into teams of three to four individuals based on their selection of senior elective courses, and type of internships (experienced the skills at office engineer, field foreman, pumpers, company men, and servicemen level).  In the standard sequence of design, I and II, the design I class gets a data set on Petra that depicts very early stages of field discovery.  The data set includes logs, scout cards and other related well data including GIS information.  The students design initial drilling, completion, resources, and development.  In design II, students are given substantially more data and tasked with optimizing further development including drilling methods, completions, EOR implementation, etc. Reason/ Justification In response to Texas Tech’s petroleum industry advisory board, senior exit interviews, and surveys from recent graduates; the faculty has altered the senior design course.  Previously, students would design independent projects in multiple courses.  Now students work on a comprehensive design which incorporates a single field data set.  Evidence of Improvement Based on presentations by students of their projects to the PIAB, representing industry positive feedback of the improvements made by this implementation, and the alumni survey results.  The alumni survey results showed the agreement of our alumni on the effectiveness of the action.   Further Actions Students complained about the time spent on the design project compared to hours earned from the courses. We are planning on increasing the design I, PETR 4121 from 1 to 2 credit hours, making a total of 4 credit hours of design I and II instead of 3hrs. Secondly, the department plans to poll recent graduates (May 2015 and 2016) of this new program as they complete their two to three years of field experience.      Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018  The second action of improvement    Acknowledgment       we would like to thank the academic advisory board for their unlimited support that aided in forming these actions of improvement. Furthermore, the author would like to thank the chair of the department for his leadership support toward the implementation of these actions of improvement into the undergraduate curriculum plan.     Figure 1 shows an example of two maps or loops.    Figure 2 shows how the courses are attached to Bloom`s cognitive taxonomy.  Table1 shows the courses used to measure the attainment of the ABET student outcomes.   Action Description PE students take a newly designed course, called Petroleum Methods. This course teaches the basics of each of the fifteen junior and senior petroleum engineering courses.  The sophomore course gives them the technical basis of petroleum engineering.  Three different faculty members are involved in the teaching of this course. Reason/Justification  The main goal was to give the sophomore students the vision and understanding of petroleum engineering in the sophomore year.  The Petroleum Methods course introduces all aspects of petroleum engineering. Students should be prepared for a summer internship in the oil industry.  Furthermore, Students can make an informed decision on whether a petroleum engineering career is for them or change majors before they invest too much time in the program.  Evidence of Improvement Better response from senior surveys, and better student performance in design courses. Students are better prepared for a summer internship in the oil industry after covering the basics of petroleum engineering fundamentals. Industry feedback from those who are hiring interns after completion of this course report that students are out-performing their peers. The alumni survey results showed the agreement of our alumni on the effectiveness of the action, Appendix E.   Further Actions The course content is changed by moving reservoir engineering and formation evaluation parts to other sophomore courses taught in the same semester due to enough faculty and taking over a geology course from geo-sciences (Petroleum Geology).  Proceedings of the 2018 ASEE Gulf-Southwest Section Annual Conference The University of Texas at Austin April 4-6, 2018 Table 2: shows a timeline of the detailed activities of the assessment plan review.     Table3 shows an example of the assessment and evaluation results of two ABET student outcomes.                                                                         Year Activity for each Student Outcome 1 2 3 4 5 6 Review of an assessment method for the instructor course evaluation   X  X  X Review of performance indicators in each course  X  X  X Review of the mapping of the courses used to evaluate the outcome  X  X  X Review of impacts of changes for improvement X  X  X  ABET  a – k Specific Student Outcome Metrics                                                     Met 2012/13 2014/15 2016/2017  a Apply Math, Science, engineering  >3.5 (Sr. Exit Survey)  >70 % (Sr. Exit Interview)   >70 % faculty self-assessment, using courses selected in Table 1.  4.0 80 % 70 %  4.2 85% 75%  4.22 80 % 85 % Yes Yes Yes  b Ability to design and conduct experiments, analyze and interpret data  >3.5 (Sr. Exit Survey)  >70% (Sr. Exit Interview)  >70 % faculty self-assessment using courses selected in Table 1.   3.85 75 % 70 %  4.1290% 80%  3.75 75% 80 % Yes Yes Yes  

Effective Student Outcomes Assessment Plan Reform Strong Undergraduate Curriculum Plan

https://repositories.lib.utexas.edu/bitstream/2152/79877/2/effective-student-outcomes-assessment-plan-reform-strong-undergraduate-curriculum-plan.pdf

Effective Student Outcomes Assessment Plan Reform Strong Undergraduate Curriculum Plan

Abstract

Similar works

Full text

Available Versions

Texas ScholarWorks