Research Report 2013-2014

Introduction From the Chai Research Spotlight: Elena Litvinova, Ph.D. Refereed Publications Invited Presentations Contributed Presentations External Grant Activity (Submitted and Awarded) Notable Awards and Memberships Doctoral Dissertations and Master’s Theses Department Personne

How Engineering Education Guilds are Expanding our Understanding of Propagation in Engineering Education

Background: The National Science Foundation (NSF) and other organizations have spent millions of dollars each year supporting well-designed educational innovations that positively impact the undergraduate engineering students who encounter them. However, many of these pedagogical innovations never experience widespread adoption. To further the ability of innovation developers to advance engineering education practice and achieve sustained adoption of their innovations, this paper focuses on exploring how one community-based model, engineering education guilds, fosters propagation across institutions and individuals. Engineering education guilds seek to work at the forefront of educational innovation by creating networks of instructor change-agents who design and implement a particular innovation in their own context. The guilds of interest are the Consortium to Promote Reflection in Engineering Education (CPREE) and the Kern Entrepreneurial Engineering Network (KEEN). With these guilds as exemplars, this study’s purpose is (1) to articulate how the approaches of engineering education guilds align with existing literature on supporting sustained adoption of educational innovations and (2) to identify how these approaches can advance the STEM education community’s discussion of propagation practices through the use of the Designing for Sustained Adoption Assessment Instrument (DSAAI). The DSAAI is a conceptual framework based on research in sustained adoption of pedagogical innovations. It has previously been previously used in the form of a rubric to analyze dissemination and propagation plans of NSF educational grant recipients and was shown been shown to predict the effectiveness of those propagation plans. Results: Through semi-structured interviews with two leaders from each guild, we observed strong alignment between the structures of CRPEE and KEEN and evidence-based sustained adoption characteristics. For example, both guilds identified their intended audience early in their formation, developed and implemented extensive plans for engaging and supporting potential adopters, and accounted for the complexity of the higher education landscape and their innovations in their propagation plans. Conclusions: Our results suggest that guilds could provide another approach to innovation, as their structures can be aligned with evidence-based methods for propagating pedagogical innovations. Additionally, while the DSAAI captures many of the characteristics of a well-designed propagation strategy, there are additional components that emerged as successful strategies used by the CPREE and KEEN guild leaders. These strategies could and should be considered as educational innovators work to encourage adoption of their innovations, including having mutual accountability among adopters and connecting adoption of innovations to faculty reward structures in the form of recognition and funding

Designing dissemination and validation of a framework for teaching cloud fundamentals

Three previous Working Groups (WGs) met at ITiCSE conferences to explore ways to help educators incorporate cloud computing into their courses and curricula by mapping industry job skills to knowledge areas (KAs). These WGs identified, organized, and grouped together student learning objectives (LOs) and developed these KAs and LOs in a repository of learning materials and course exemplars. This WG focused on the sustainability of the work of its predecessors through dissemination, community building and validation of the framework of KAs and LOs and its contribution to curriculum development. Firstly, a case study is presented which analyzed the implementation of a new Masters program which was based on the KAs and LOs. It was found that these provide a useful basis for program development and approval and demonstrate that successful program development of this nature can provide a valuable opportunity to communicate the work of the previous WGs. Thereafter, a plan was formulated for dissemination of the work done in order to drive adoption and to encourage instructors with an interest in teaching cloud computing to participate and grow the community. While the strategy included a range of dissemination methods, the importance of interaction with users was a guiding principle. Initial pilots of webinar and workshop activities have been implemented. Approaches to validating that a cloud computing course designed around the KAs and LOs can meet the needs of industry have been outlined with further iterations being considered. A research plan has been designed for a study to be implemented over the coming year in order to perform this validation

Designing Dissemination and Validation of a Framework for Teaching Cloud Fundamentals

Three previous Working Groups (WGs) met at ITiCSE conferences to explore ways to help educators incorporate cloud computing into their courses and curricula by mapping industry job skills to knowledge areas (KAs). These WGs identified, organized, and grouped together student learning objectives (LOs) and developed these KAs and LOs in a repository of learning materials and course exemplars. This WG focused on the sustainability of the work of its predecessors through dissemination, community building and validation of the framework of KAs and LOs and its contribution to curriculum development. Firstly, a case study is presented which analyzed the implementation of a new Masters program which was based on the KAs and LOs. It was found that these provide a useful basis for program development and approval and demonstrate that successful program development of this nature can provide a valuable opportunity to communicate the work of the previous WGs. Thereafter, a plan was formulated for dissemination of the work done in order to drive adoption and to encourage instructors with an interest in teaching cloud computing to participate and grow the community. While the strategy included a range of dissemination methods, the importance of interaction with users was a guiding principle. Initial pilots of webinar and workshop activities have been implemented. Approaches to validating that a cloud computing course designed around the KAs and LOs can meet the needs of industry have been outlined with further iterations being considered. A research plan has been designed for a study to be implemented over the coming year in order to perform this validation

Bridging the gap between evidence-informed and actual teaching practices of engineering educators: an AI-enhanced professional learning system

Imagine a classroom where engineering students are challenged to apply what they’re learning, where they interactively explore the complexities of authentic, level-appropriate engineering problems, supported by professors who are aware of and apply evidence-informed teaching practices. Expectations align with the engineering workplace. Learners improve their acquired knowledge and skills through experimentation and deliberate practice. They harness systems thinking as they make connections and see patterns. They are challenged to adapt to whatever scenario they face, to identify problems, think critically, generate and model effective solutions, and to make justifiable decisions. Learners experience the tension between knowing and doing engineering things. They learn firsthand, and in context, what it means to be a practicing engineer. This aspiring approach is very different from the didactic practices reported in most Canadian undergraduate engineering classrooms. The challenge, and the focus of this research, is to encourage and assist engineering educators to stretch their current teaching practices beyond what’s comfortable and customary, to those that are both evidence-informed and truly representative of engineering. This research is a blend of interdisciplinary mixed-methods and design-based research. The interdisciplinary mixed-method research integrates the findings of educational research, learning sciences, professional learning, and systems thinking. Sixteen research studies explore the experiences and practices of educators and students in the Canadian undergraduate engineering system. These findings confirm that a gulf exists between evidence-informed teaching practices and what happens in the typical undergraduate engineering classroom. They clearly establish the need for an educational development model that translates existing educational research into tangible, level-appropriate teaching practices for engineering educators at all levels of experience and skill. This foundational research leads to the design and development of this thesis' first of three contributions, the LENS (Learning Environments Nurture Success) model of engineering faculty development. This model, which is comprised of six lenses that align with an effective learning environment, offers a practical framework to support educational development and planning for all forms of delivery (face-to-face, remote, blended, or hybrid). It can be used independently, in consultation with an educational developer, or in collaboration with colleagues. It threads educator-related threshold concepts associated with learning, pedagogy, assessment, and teaching with technology through each of six lenses, and links myriad interdisciplinary research findings to facilitate the successful education of undergraduate engineering students. The second contribution of this research is a proof-of-concept intelligent Professional Learning System (iPLS). This AI-enhanced learning platform individualizes and guides the development of professional knowledge and skills. The look, feel, and functionality of this proof-of-concept iPLS is shaped by an integration of research findings in professional learning, training and development, technology-based learning, and AI in education. The final contribution of this work is an iPLS application designed to help engineering educators develop their teaching practices. It provides needs-specific recommendations based on an individual's ranking on a novice to expert continuum and achieved teaching-related thresholds. Quantitative and qualitative field test results show the combined LENS, iPLS, and engineering education application (EEA) to be a viable method by which engineering educators can stretch their teaching to include more evidence-informed teaching practices. Using the elements of an elegant design as its measure, the system is determined to be effective and robust with a minimal number of unexpected consequences