This study investigates engineering students’ transitions from academic to professional environments by examining the role capstone design courses play in preparing graduates for the workplace. To better understand how capstone design experiences contribute to graduates’ professional preparation, we are collecting data from participants from four different institutions with project-based capstone courses as they begin post-graduation positions in a variety of engineering workplaces. Through quantitative and qualitative methods, our study is designed to collect insights from participants in their first 12 months on the job. Currently we are collecting and analyzing data from the first of two planned cohorts of participants. Preliminary results for the participants in the first cohort point towards interesting trends regarding participants’ frequency of activities and perception of their preparedness. Professional skills such as team meetings were listed most frequently as activities engaged in by participants, and while there were particular areas such as budgeting where participants felt less prepared, overall their perception of preparedness indicates that capstone design courses and the larger engineering curriculum they are housed within are preparing students for professional careers

Alvarez, Nicholas Emory

Ford, Julie Dyke

Gewirtz, Chris

Hernandez, Cristian

Howe, Susannah

Knight, Daniel

Kotys-Schwartz, Daria A.

Paretti, Marie C.

Rosenbauer, Laura Mae

English

Smith College: Smith ScholarWorks

Smith ScholarWorks Engineering: Faculty Publications Engineering 2018 Preliminary Results from a Study Investigating the Transition from Capstone Design to Industry Susannah Howe Smith College, showe@smith.edu Laura Mae Rosenbauer Smith College Julie Dyke Ford New Mexico Institute of Mining and Technology Nicholas Emory Alvarez New Mexico Institute of Mining and Technology Marie C. Paretti Virginia Polytechnic Institute and State University See next page for additional authors Follow this and additional works at: https://scholarworks.smith.edu/egr_facpubs  Part of the Engineering Commons Recommended Citation Howe, Susannah; Rosenbauer, Laura Mae; Ford, Julie Dyke; Alvarez, Nicholas Emory; Paretti, Marie C.; Gewirtz, Chris; Kotys-Schwartz, Daria A.; Knight, Daniel; and Hernandez, Cristian, "Preliminary Results from a Study Investigating the Transition from Capstone Design to Industry" (2018). Engineering: Faculty Publications, Smith College, Northampton, MA. https://scholarworks.smith.edu/egr_facpubs/56 This Conference Proceeding has been accepted for inclusion in Engineering: Faculty Publications by an authorized administrator of Smith ScholarWorks. For more information, please contact scholarworks@smith.edu Authors Susannah Howe, Laura Mae Rosenbauer, Julie Dyke Ford, Nicholas Emory Alvarez, Marie C. Paretti, Chris Gewirtz, Daria A. Kotys-Schwartz, Daniel Knight, and Cristian Hernandez This conference proceeding is available at Smith ScholarWorks: https://scholarworks.smith.edu/egr_facpubs/56 Preliminary Results from a Study Investigating the Transition from Capstone Design to Industry  Susannah Howe1, Laura Rosenbauer1, Julie Dyke Ford2, Nicholas Alvarez2, Marie Paretti3,  Christopher Gewirtz3, Daria Kotys-Schwartz4, Daniel Knight4, Cristian Hernandez4 1Smith College 2New Mexico Institute of Mining and Technology 3Virginia Tech 4University of Colorado Boulder  This study investigates engineering students’ transitions from academic to professional environments by examining the role capstone design courses play in preparing graduates for the workplace. To better understand how capstone design experiences contribute to graduates’ professional preparation, we are collecting data from participants from four different institutions with project-based capstone courses as they begin post-graduation positions in a variety of engineering workplaces. Through quantitative and qualitative methods, our study is designed to collect insights from participants in their first 12 months on the job. Currently we are collecting and analyzing data from the first of two planned cohorts of participants. Preliminary results for the participants in the first cohort point towards interesting trends regarding participants’ frequency of activities and perception of their preparedness. Professional skills such as team meetings were listed most frequently as activities engaged in by participants, and while there were particular areas such as budgeting where participants felt less prepared, overall their perception of preparedness indicates that capstone design courses and the larger engineering curriculum they are housed within are preparing students for professional careers.  Keywords:  learning transfer, professional preparation, capstone design, school-to-work transition  Corresponding Author: Susannah Howe, showe@smith.edu  Introduction While the motivation of many capstone design courses is to allow culmination of students’ engineering education through a project-based experience that closely replicates the workplace, recent work indicates gaps still exist between school and work1,2. These gaps highlight the need to systematically examine the effectiveness of capstone courses in students’ transition from school to work. Our study is designed to meet this critical need. With a multi-case approach, we ask how and to what extent capstone design courses prepare students to effectively enter communities of practice in engineering workplaces.  Key goals of this work are to enhance capstone courses to better prepare students for work and to provide industry with findings that can help them improve the transition experiences of new graduates. In this paper, we report on weekly survey results from participants’ first 12 weeks of work.  Methodology Data for this project are drawn from a large multi-case study3 across four institutions that uses a sequential explanatory mixed-method design, combining regular interviews with intensive survey data.  The geographically diverse research sites consist of three mechanical engineering programs, and one engineering science program. As one of the largest disciplines nationally and an archetypal design domain, ME offers a useful study focus, although we recognize the study results may not be universally applicable. The sites range in size from a small program graduating 20-30 students annually to larger programs with over 350 graduates per year. All include at least a full-year of senior design; one has a 4-semester sequence that begins in students’ junior year. All include industry-sponsored projects, though most also include faculty-sponsored and competition projects as well. Finally, all use a course coordinator coupled with individual faculty and/or industry mentors for each team. Team sizes are generally 4-6 students. Beginning in late spring 2017, we recruited participants from each program; recruitment included in-person or Skype visits to courses, followed by an email inviting participants to complete a screening survey.  The full data set for Cohort 1 includes 29 females and 33 males (self-reported); for this paper, we use data from the 25 females and 29 males who had started employment by the time  of the data analysis. Of those included in this analysis, 34 participants self-identified as white or Caucasian, 10 as Asian, 3 as Hispanic, 3 as other nationalities, and 4 did not disclose. With respect to sites, across the three large institutions, the data set includes 19, 14, and 11 participants, respectively, with an additional 10 participants from the smaller site.  The full data set includes three forms of data collection for each participant: (1) background interviews conducted at the end of the capstone course, (2) twice-weekly surveys (Likert-type surveys on perceived preparedness and open-ended reflection survey questions) during participants’ first twelve weeks, and (3) interviews after 3, 6, and 12 months of work.  Participants received gift cards for completing the interviews and surveys.  Data analysis for this paper focuses on the weekly surveys: participants received two separate surveys each week: a Likert-type perceived preparedness quantitative survey sent each Tuesday via Qualtrics and a short open-ended reflective survey sent each Thursday via email.   The quantitative survey, informed by Experience Sampling Methodologies (ESM)4,5 asked participants to identify activities in which they had participated within the past week. The list of possible activities, as shown in Figure 1, was selected based on common notions of engineering design activities and refined by the research team to ensure coverage of a wide range of workplace activities. For each activity participants check, the survey asked a follow up question about the degree to which participants felt prepared, using a 7-point sliding scale with 7 being “Completely prepared” and 1 being “Completely unprepared.” Because not every participant completed every survey, the data set includes a total of 432 quantitative survey responses (201 from females and 231 from males). Our analysis focuses on the quantitative data, with the qualitative data providing context and elaboration.  Please check all of the activities you’ve been involved with over the past week: □ Team meetings within your unit or project team □ Project planning □ Writing reports □ Making formal presentations □ Performing engineering calculations □ Generating or refining design concepts □ Prototyping and testing designs □ Computer-aided modeling □ Meeting with clients □ Project budgeting (business financials) □ Other (please provide a short description) Figure 1 - Short Quantitative Survey Items  The reflective survey contained seven questions each week exploring participants’ most significant challenge or accomplishment and the role their capstone experience played in preparing them for that experience.   The prompts, as listed in Figure 2, solicited a thick, rich description of newcomers’ salient challenges.  1.  What was your biggest challenge this week? 2.  What made it so challenging? 3.  How did you approach this challenge? 4. To what extent did you feel prepared for this challenge based on your capstone design experience? Based on other experiences? 5. Is there anything you think your education might have done that would have better prepared you? 6. Are there any other workplace activities this week that you felt particularly well or poorly prepared for? If so, please explain. Figure 2 - Weekly Journal Prompts  Results and Discussion Table 1 summarizes the 54 participants’ responses to the weekly quantitative surveys.  The “N” column lists the number of participants (of 54) who indicated that they had been involved with the given activity at least one (and as many as all twelve) of the weeks.  The “AVG” column was calculated by averaging the perceived preparedness ratings per person (across all weeks) and then averaging across all respondents.  The “MIN” column was calculated by extracting the lowest rating per person (across all weeks) and then averaging across all respondents.  Table 1 - Perceived Prep. Quantitative Survey Results Activity N AVG MIN Team Meetings 50 6.0 4.9 Project Planning 44 5.6 4.7 Report Writing 30 6.0 5.3 Formal Presentations 22 6.0 5.6 Engineering Calculations 39 6.0 5.2 Generating/Refining Concepts 37 5.6 4.8 Prototyping/Testing Designs 22 5.9 4.9 CAD Modeling 35 5.6 4.9 Client Meetings 30 5.6 5.0 Project Budgeting 21 5.0 4.2  As shown in the “N” column, some activities were more prominent during participants’ first twelve weeks than others.  Less than half of participants indicated having been involved with project budgeting, for example, whereas nearly all participated in team meetings, often on multiple weeks.  It is reassuring to see Team Meetings and Project Planning in particular are high frequency topics in the workforce, given the emphasis they are often given in capstone6. Especially of interest are the values in the “AVG” column of Table 1, which provide an indication of how prepared recent graduates perceive themselves to be for their entry-level responsibilities.  While there is some variation across respondents and across weeks, the average values across the entire set of respondents are all between 5 (“Slightly Prepared”) and 6 (“Moderately Prepared”).  These results in themselves suggest that the engineering curriculum, and capstone design courses in particular, are already helping to prepare students for careers after graduation, but have room for improvement. Analyzing the average responses using ANOVA and the Tukey-Kramer post hoc test reveals that there are several statistically significant differences (p=0.0066) across topics: average perceived preparedness for Project Budgeting is lower than that for Team Meetings, Report Writing, and Engineering Calculations.  Given that participants reported meetings and calculations as the two most frequent activities with which they were involved in the first twelve weeks, the fact that participants also feel the most prepared for these is a success.  Moreover, although capstone students commonly view communication and documentation tasks as secondary to the technical portion of their design projects, the emphasis on the topics in capstone design pays off as the graduates enter the workplace.   The results also suggest that capstone design instructors might consider including more exposure to project budgeting; even though it is a less common activity in our data set, it is the one for which participants felt least well prepared.  As suggested in a reflection response by one participant, “This week my biggest challenge had to do with project time-budgeting. This is an issue I had never really run into before and something [capstone] didn't really prepare me for.  I was thinking this would almost be a good exercise for a capstone class to do … being given a budget on a project and how many billable hours they can allot to it while trying to balance that with other project costs and producing a quality product/design.” An analysis of the minimum reported values for perceived preparedness, as shown in the far right column in Table 1, also provides interesting information.  Although some participants did report feeling “Completely unprepared” (rating = 1) for some activities on  some weeks, the minimum reported values on average are between 4.2 and 5.6 for all activities.  Given that 4 = “Neither prepared nor unprepared” and 5 = “Slightly prepared”, these data suggest that as a group, even participants’ minimum perceived preparedness levels lean toward more prepared than not.  Unlike for the average perceived preparedness values, the combined ANOVA and Tukey-Kramer post-hoc test did not reveal any statistically significant difference between activities for minimum reported values (p=0.13). Additional analysis was conducted regarding gender differences in perceived preparedness by activity. Although the sample size from this preliminary data set is not particularly large, two-sided t-tests (unpaired, unequal variances) were possible for some activities (we analyzed all with N ≥ 30). Previous research7,8,9,10 has shown that women report lower self-confidence and self-efficacy especially in technical skills and analytical thinking.  The initial results from this study, however, show no gender difference in perceived preparedness for all but one of the activities. Only Generating/ Refining Design Concepts corresponds to a statistically significant difference (p=0.0014) between average perceived preparedness for males (n=19) and females (n=18), based on t-test  results, with males reporting higher values.  The same outcome holds for minimum perceived preparedness values; p=0.00019 for Generating/Refining Design Concepts (also the only activity with statistically significant difference by gender). A preliminary look into the reflective survey responses provides additional insight into female participants’ lower responses around generating/refining design concepts.  Although the weekly journal prompts did not specifically ask about this activity, multiple participants raised the topic and several mentioned how and why they felt unprepared.  For example, one participant noted “I felt prepared for the task but nothing in my experience had ever given me insight into how to approach things. Most of my previous work and design experience were for one-off or small projects, not production lines that would run for an extended period.”  Another commented “I felt poorly prepared for being careful. I felt like a lot of the design work and FEA would have gone very wrong without the guidance of my mentor.”  A third noted, “A hard part of this project is not knowing a lot about manufacturing processes, and sometimes designing things that wouldn't work. … I think learning more about design for manufacturing would have been very useful for this project.”  Interestingly, one male respondent also mentioned design for manufacturing as an area for improvement - “More exercises on ‘design for manufacturing’ would have helped give a better mindset for how to run a process like this.” - so clearly lack of preparedness for the topic affects both men and women.  More rigorous qualitative data analysis (in progress) of the background interviews, the reflective survey responses, and the quarterly interviews will undoubtedly provide richer insight into gender differences in both perceived preparation and workplace experience. Conclusions While the results presented here are preliminary pending analysis of the full data set, our analysis to date suggests several tentative implications for capstone faculty and engineering employers:  ● The content currently included in capstone is relevant, particularly with respect to the emphasis on professional skills and practices. ● Our participants, on the whole, feel at least somewhat prepared for most of the activities they are faced with on a weekly basis, with capstone experiences playing a key role in that preparation. ● There could be more emphasis in capstone on topics like budgeting and design for manufacturing ● Gender may play a role in participants’ perceived preparation.  Limitations and Future Work The results from Cohort 1’s participants who had completed up to twelve weeks of work offer interesting information regarding participants’ frequency of activities and self-perception of preparedness for these activities.  Limitations of this data set, however, include small Ns for some topics and some respondent segments (gender, institution) that are below the threshold for statistical significance. We look forward to gaining fuller understanding through further data analysis and collection.  Future work includes analyzing both the quantitative and qualitative survey data from the full cohort. We also expect the interviews for Cohort 1 conducted at 3, 6, and 12 month intervals to reveal insightful information to complement quantitative trends. In May 2018 we will begin participant selection and data collection with Cohort 2.   References 1. Stevens, R., Johri, A., and O'Connor, K. (2014). Professional Engineering Work, in Cambridge handbook of engineering education research, A. Johri and B.M. Olds, Editors. Cambridge University Press: Cambridge, UK. p. 119-137. 2. American Society of Mechanical Engineers. (2013). V2030 Research - Drivers, Data, Action and Advocacy. American Society of Mechanical Engineers: New York, NY. p. 19. 3. Yin, R. K. (2014). Case Study Research: Design and Methods (5th ed.). Thousand Oaks, CA: Sage Publications. 4. Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2007). Experience sampling method: Measuring the quality of everyday life. Thousand Oak, CA: Sage Publications. 5. Zirkel, S., Garcia, J. A., & Murphy, M. C. (2015). Experience-Sampling Research Methods and Their Potential for Education Research. Educational Researcher, 0013189X14566879. 6. Howe, S., Rosenbauer, L., Poulos, S. (2017). The 2015 Capstone Design Survey Results: Current Practices and Changes over Time, International Journal of Engineering Education, 33(5): 1393-1421. 7. Ro, H. K. and Knight, D. B. (2016), Gender Differences in Learning Outcomes from the College Experiences of Engineering Students. Journal of Engineering Education, 105: 478–507. doi:10.1002/jee.20125 8. Pinelli, T. E., Hall, C. W., Brush, K. M., & Perry, J. B. (2013).  Are There Gender Differences in How Male and Female Interns and Their Mentors Rate Workforce Skills in STEM Fields? Proceedings of the American Society for Engineering Education Conference, Atlanta, GA. 9. Geisinger, B. N. & Raman, D. R. (2013). Why They Leave: Understanding Attrition from Engineering Majors. International Journal of Engineering Education, 29(4),  914-925. 10. Sheppard, S., Gilmartin, S., Chen, H. L., Donaldson, K., Lichtenstein, G., Eris, Ö., Lande, M., & Toye, G. (2010). Exploring the Engineering Student Experience: Findings from the Academic Pathways of People Learning Engineering Survey (APPLES) (TR-10-01). Seattle, WA: Center for the Advancement for Engineering Education.  

Preliminary Results from a Study Investigating the Transition from Capstone Design to Industry

https://scholarworks.smith.edu/cgi/viewcontent.cgi?article=1054&context=egr_facpubs

Preliminary Results from a Study Investigating the Transition from Capstone Design to Industry

Abstract

Similar works

Full text

Available Versions

Smith College: Smith ScholarWorks