I thought this was going to be a waste of time:How portfolio construction can support student learning from project-based experiences

In this work, we sought to understand ways that students experienced a small-scale portfolio assignment provided to them as an opportunity reflect on their experiences in a project-based class. This work was motivated by research in various instructional contexts showing that portfolio construction results in important learning outcomes. We wanted to see if such findings would extend to project-based learning situations in engineering. In addition, our research was motivated by the need to better understand specific portfolio assignments in terms of the effort required of students, the extent and ways in which students value such assignments, and how student overall experiences differ in terms of learning, effort, and value. To address these issues, we interviewed thirteen students who had completed a portfolio assignment as a culminating activity for a project-based manufacturing class. We selected these students because their individual responses to a screening survey suggested that they had had very different experiences with the portfolio assignment. We then conducted open-ended interviews with the students in order to better understand their experiences. In this paper, we show how the themes of epistemically different, manageably effortful, and ultimately valuable can be used to understand three ways of experiencing the assignment (as a significant experience, a muted experience, and a limited experience). This work provides practical information to educators interested in supporting reflection while also contributing the scholarship on how portfolios support learning from experience

The Validity of the Stimulated Retrospective Think-Aloud Method as Measured by Eye Tracking

Retrospective Think aloud (RTA) is a usability method that collects the verbalization of a user’s performance after the performance is over. There has been little work done to investigate the validity and reliability of RTA. This paper reports on an experiment investigating these issues using the method called stimulated RTA. By comparing subjects’ verbalizations with their eye movements, we found stimulated RTA to be valid and reliable: the method provides a valid account of what people attended to in completing tasks, it has a low risk of introducing fabrications, and its validity is unaffected by task complexity. More detailed analysis of RTA shows that it also provides additional information about user’s inferences and strategies in completing tasks. The findings of this study provide valuable support for usability practitioners to use RTA and to trust the users ’ performance information collected by this method in a usability study. Author Keywords Retrospective think aloud, validity, reliability

Does think aloud work? How do we know?

The think aloud method is widely used in usability research to collect user's reports of the experience of interacting with a design so that usability evaluators can find the underlying usability problems. However, concerns remain about the validity and usefulness of think aloud in usability studies. In this panel we will present current studies of the think aloud method, examine and question its usage in the field, discuss the possible pitfalls that may threaten the validity of the method, and provide comments/suggestions on the application of the method. Panel participants will discuss results drawn from both applied research and basic research. We believe that this panel discussion will be useful for HCI designers and usability practitioners in that it will acquaint them with concerns that people have about the think aloud method and provide them with suggestions for improved use of the method. For HCI or usability researchers, this panel discussion will address the importance of formally investigating currently used or newly designed usability methods