Towards an Integrated Assessment Framework: Using Activity Theory to Understand, Evaluate, and Enhance Programmatic Assessment in Integrated Content and Language Learning

This article uses activity theory to analyse two different portfolio approaches as tools for programmatic assessment of Integrated Content and Language (ICL) programs. The two approaches include a) a model in which students construct portfolios by selecting artifacts from a range of different contexts and provide reflective commentary, and b) a model in which the portfolio consists of major textual artifacts produced across a design project, with no reflective component. Activity theory provides a tool to explore what these models can offer in terms of an assessment of the integration of content and language in disciplinary contexts, where texts serve to mediate the ongoing work of a discipline. By highlighting the work that texts do in context as well as the access to student meta-knowledge afforded by each portfolio, activity theory provides a means to understand the strengths and limitations of both models. Perhaps most importantly, it points to the need for portfolios to include well-designed reflections that can support both student learning and effective programmatic assessment

Learning and Becoming in Design Reviews

Drawing from the prior work of McNair and Paretti (2010), this study investigates how language practices and design artifacts mediate the interactions among novice and expert designers to shape the nature of design, and specifically design learning. By analyzing data collected from two design courses in different fields, this study addresses two research questions: 1) how do language practices mediate the interactions between design mentors and design learners; and 2) how do design artifacts mediate these interactions between mentors and learners? Drawing on activity theory and discourse analysis, we use these questions to explore how students work with experts to make meaning within their design experiences. In doing so, we treat meaning-making as an array of social processes situated within a complex activity system that includes instructors, professionals, team members, and artifacts

What does it mean to be “prepared for work”? Perceptions of new engineers

Background: Engineering education seeks to prepare students for engineering practice, but the concept of preparedness is often ill-defined. Moreover, findings from studies of different populations or in different contexts vary regarding how well new graduates are prepared. These variations, coupled with the lack of clarity, suggest the need to better understand what it means to be prepared for engineering work. Purpose: This study contributes to research on workplace preparation by exploring how new graduates describe being prepared for engineering work. Method: Applying secondary analysis to data from the multi-institution Capstone To Work (C2W) project, we used thematic analysis to explore new engineers\u27 descriptions of preparedness. We analyzed written responses to structured questions about the school-to-work transition collected weekly during participants\u27 first 12 weeks of work; 105 graduates drawn from four universities provided 956 responses, with a mean of 9 (out of 12 possible) responses per participant. Results: Participants\u27 descriptions of preparedness included applying concrete skills, recognizing familiar situations, and having strategies for approaching challenging tasks even when they lacked relevant knowledge or skill. Conclusion: Our findings suggest that although many discussions about workplace preparation implicitly focus narrowly on mastery of skills and knowledge, that focus may not fully capture new graduates\u27 experiences, and may limit discussions about the ways in which school can (and cannot) prepare students for work. A more expansive understanding may better support both student learning and workplace onboarding, though more research is needed across stakeholders to establish shared understanding

Sustaining Engineering Education Research: Sharing Qualitative Research Data For Secondary Analysis

The need for secondary data analysis practices emerges from multiple sources. Qualitative researchers often have rich data sets that far exceed the time available for data analysis, and many of us wish that someone could spend more time with the data. We also recognize that local data sets would benefit from further analysis that linked our data with related data collected in different contexts. Many also grapple with increasing data sharing requirements from funding agencies that raise concerns about participant confidentiality and data integrity. This workshop provides a chance to explore potential responses to these concerns through a robust dialogue around secondary data analysis practices and pitfalls

Designing Hands-On Teaming Activities: Exploring Sustainability Tradeoffs for Courses with Large Enrollments

In this paper, the authors explore sustainability issues that exist in the development of hands-on activities for classes with large enrollments. Specifically, the authors study four different teambuilding activities, all with varying levels of resource commitment, to assess potential tradeoffs between cost, environmental impact, and learning objectives pertaining to design and teaming. Faced with several alternatives and multiple, conflicting objectives, the authors approach this choice from a design context. Specifically, following the identification of activity constraints and objectives, activity alternatives are evaluated against several metrics with post-activity student surveys. Survey data is then translated into an appropriate input for a systematic selection framework, the selection Decision Support Problem. The use of this framework allows the authors to select a teaming activity alternative that offers the best compromise to their multiple design goals

The Effects of a Collaborative Problem-based Learning Experience on Students’ Motivation in Engineering Capstone Courses

We identified and examined how the instructional elements of problem-based learning capstone engineering courses affected students’ motivation to engage in the courses. We employed a two-phase, sequential, explanatory, mixed methods research design. For the quantitative phase, 47 undergraduate students at a large public university completed a questionnaire that measured the components of the MUSIC Model of Academic Motivation (Jones, 2009): empowerment, usefulness, success, situational interest, individual interest, academic caring, and personal caring. For the qualitative phase that followed, 10 students answered questions related to the MUSIC components. We identified several instructional elements that led to motivating opportunities that affected students’ motivation to engage in the courses. We discuss how these motivating opportunities can foster or hinder students’ engagement and provide implications for instruction

Capstone Design Hub: Building the Capstone Design Community

Capstone design courses are common across engineering programs nationwide. Yet, many departments and programs rely on one faculty member or a small handful of faculty members to teach their capstone design course. As a result these faculty members find themselves isolated, with limited mechanisms for sharing ideas and networking with peers who have similar responsibilities and concerns. This paper reports on the ongoing efforts to support the broader capstone design community through the development of the Capstone Design Hub (CDHub) as a web resource for capstone design programs. The features and structure of the CDHub are being developed through capstone faculty input, including results from a survey of the capstone community. To build awareness of the CDHub as well as to solicit additional feedback from the community, this paper describes development of the hub to meet community needs, initial population of the hub with resources focused on communication, and plans for continued expansion of the hub. © 2012 American Society for Engineering Education

Women\u27s Experiences in the Transition from Capstone Design Courses to Engineering Workplaces

Substantial research over the past few decades has documented the challenges women experience both as students in engineering programs and as professionals in engineering workplaces. Few studies, however, have followed women from one context to the other to explore the ways in which school experiences, and particularly capstone experiences designed to facilitate this transition, do and do not prepare women for their work as practicing engineers. To address this gap, we draw on data from a larger multi-institution study to address the question, “How do women experience the transition from engineering school to engineering work?” Participants for this study are drawn from a larger study across four universities (three mechanical engineering programs and one engineering science program). All participants identified as “female” on a screening questionnaire that included options for transgender and gender-nonconforming, as well as an option to skip the question. The full data set includes interviews with the participants conducted at the end of their capstone design course, responses to open-ended questions sent each week during their first 12 weeks of work, and interviews conducted after three, six, and 12 months of work. To answer the research question, we used purposeful sampling to identify four women whose interviews represented different trajectories across this school-to-work transition; we then used constructed narrative analysis to present their individual stories and identify salient points of similarity and difference for discussion. We also present implications for engineering educators, including that life-long learning should be expected, communication and collaboration are as essential workplace skills as technical competencies, and that gender is not necessarily a homogenizing force. Above all, we emphasize the power of the individual voice in better understanding the experiences of our students

Using reflective writing and textual explanations to evaluate students' conceptual knowledge

BACKGROUND OR CONTEXT - Writing is one method used to prompt students to reflect on their own thought processes. Eliciting students’ explanations in the form of text, or writing, also provides lecturers with information about students’ thinking (Goncher, Boles, Jayalath, 2014; Boles, Goncher, Jayalath, 2015). Often in engineering courses, students adopt algorithmic problem-solving approaches without demonstrating conceptual reasoning. Adding a written, or explanatory component, to problems or questions is one approach that can elicit conceptual reasoning. PURPOSE OR GOAL - The purpose of this study was to identify and compare affordances of using students’ written explanations based the type of problem and response. This comparative study sought to answer two research questions, 1) How were the students’ textual answers different for the type of problem and requested explanations? and 2) What does the type and organization of the text of students’ explanations reveal about their conceptual knowledge? APPROACH - We analyzed students’ explanations for procedurally based problems in the statics discipline and conceptually based problems in the signal processing discipline. The first method used “process problems” that required students to explain, using only words, the process that they used to solve a statics homework problem. The second method utilized the Signals and Systems Concept Inventory items, and required students to provide a written explanation for their multiple-choice selection to each item. We categorized responses by the type of problem and structure of the written explanations to evaluate conceptual knowledge. DISCUSSION - We found that the structure of the text and type of problem provided different insights into students’ reasoning. The results showed that students approach learning in statics with varying emphasis placed on procedural and conceptual knowledge, and some students had difficulties explaining underlying concepts in signal processing and reverted to procedural explanations. Regardless of the type of problem, students that are able to get feedback on their thought processes can use the feedback to formatively evaluate their own understanding. RECOMMENDATIONS/IMPLICATIONS/CONCLUSION - Educators who incorporate or require students to reflect on their thinking through textual explanations can promote the revision of incorrect and/or inconsistent knowledge, leading to improved conceptual knowledge development. Assignments or activities that include more incidental writing will engage students in more freethinking and reflection (Essig et al., 2014;Hawkins, Coney, & Bystrom, 1996), and can lead to a richer understanding of technical concepts