PDPP:Projected Diffusion for Procedure Planning in Instructional Videos

In this paper, we study the problem of procedure planning in instructional videos, which aims to make goal-directed plans given the current visual observations in unstructured real-life videos. Previous works cast this problem as a sequence planning problem and leverage either heavy intermediate visual observations or natural language instructions as supervision, resulting in complex learning schemes and expensive annotation costs. In contrast, we treat this problem as a distribution fitting problem. In this sense, we model the whole intermediate action sequence distribution with a diffusion model (PDPP), and thus transform the planning problem to a sampling process from this distribution. In addition, we remove the expensive intermediate supervision, and simply use task labels from instructional videos as supervision instead. Our model is a U-Net based diffusion model, which directly samples action sequences from the learned distribution with the given start and end observations. Furthermore, we apply an efficient projection method to provide accurate conditional guides for our model during the learning and sampling process. Experiments on three datasets with different scales show that our PDPP model can achieve the state-of-the-art performance on multiple metrics, even without the task supervision. Code and trained models are available at https://github.com/MCG-NJU/PDPP.Comment: Accepted as a highlight paper at CVPR 202

Masked Diffusion with Task-awareness for Procedure Planning in Instructional Videos

A key challenge with procedure planning in instructional videos lies in how to handle a large decision space consisting of a multitude of action types that belong to various tasks. To understand real-world video content, an AI agent must proficiently discern these action types (e.g., pour milk, pour water, open lid, close lid, etc.) based on brief visual observation. Moreover, it must adeptly capture the intricate semantic relation of the action types and task goals, along with the variable action sequences. Recently, notable progress has been made via the integration of diffusion models and visual representation learning to address the challenge. However, existing models employ rudimentary mechanisms to utilize task information to manage the decision space. To overcome this limitation, we introduce a simple yet effective enhancement - a masked diffusion model. The introduced mask acts akin to a task-oriented attention filter, enabling the diffusion/denoising process to concentrate on a subset of action types. Furthermore, to bolster the accuracy of task classification, we harness more potent visual representation learning techniques. In particular, we learn a joint visual-text embedding, where a text embedding is generated by prompting a pre-trained vision-language model to focus on human actions. We evaluate the method on three public datasets and achieve state-of-the-art performance on multiple metrics. Code is available at https://github.com/ffzzy840304/Masked-PDPP.Comment: 7 pages (main text excluding references), 3 figures, 7 table

Event-Guided Procedure Planning from Instructional Videos with Text Supervision

In this work, we focus on the task of procedure planning from instructional videos with text supervision, where a model aims to predict an action sequence to transform the initial visual state into the goal visual state. A critical challenge of this task is the large semantic gap between observed visual states and unobserved intermediate actions, which is ignored by previous works. Specifically, this semantic gap refers to that the contents in the observed visual states are semantically different from the elements of some action text labels in a procedure. To bridge this semantic gap, we propose a novel event-guided paradigm, which first infers events from the observed states and then plans out actions based on both the states and predicted events. Our inspiration comes from that planning a procedure from an instructional video is to complete a specific event and a specific event usually involves specific actions. Based on the proposed paradigm, we contribute an Event-guided Prompting-based Procedure Planning (E3P) model, which encodes event information into the sequential modeling process to support procedure planning. To further consider the strong action associations within each event, our E3P adopts a mask-and-predict approach for relation mining, incorporating a probabilistic masking scheme for regularization. Extensive experiments on three datasets demonstrate the effectiveness of our proposed model.Comment: Accepted to ICCV 202

Improving the Quality of Technology-Enhanced Learning for Computer Programming Courses

Teaching computing courses is a major challenge for the majority of lecturers in Libyan higher learning institutions. These courses contain numerous abstract concepts that cannot be easily explained using traditional educational methods. This paper describes the rationale, design, development and implementation stages of an e-learning package (including multimedia resources such as simulations, animations, and videos) using the ASSURE model. This training package can be used by students before they attend practical computer lab sessions, preparing them by developing technical skills and applying concepts and theories presented in lecture through supplementary study and exercises

The production and deployment of an on-line video learning bank in a skills training environment

This paper describes the introduction of videos as aids in clinical skills teaching. Although the process explored focuses on a nursing clinical skills environment it is relevant to many other disciplines. With the introduction of the pre-registration degree in nursing in Ireland in 2002, the formerly hospital-based schools of nursing amalgamated into larger programmes with their affiliated higher education institutes (HEIs). The result was a considerable increase in class sizes. The current average annual cohort in the School of Nursing, Dublin City University (DCU) is 240 students. This has resulted in a need to review the way we teach clinical skills on campus. These skills form a large part of the programme and are taught to students in the school-based simulated nursing environment to prepare them for their practical experience in the clinical environment. Until 2006 the skills had been taught to groups of 25-30 students using a demonstration and practice technique. This teaching method has posed a number of problems: •Learning experiences vary depending on the mix of demonstration and practice in each session. •It can be difficult for students to absorb all of the information presented in a single demonstration. •It is highly resource intensive. It was decided that video technology incorporating a large scale deployment of skills videos over a video web server, in conjunction with a shift in emphasis in the teaching contact sessions could offer a useful tool to aid the teaching process. This paper will discuss the production process, the implementation of the project in the teaching environment and the evaluation findings

Instructional planning and new technologies in teacher education: the initial phase of a research project

The purpose of this work is to present the initial phase of a research project focused on the integration of technologies in the education of kindergarten and primary school student teachers through instructional planning. Firstly, we illustrate a tool designed for planning integrated learning units and describe the training path in which it was used. Secondly, we report the results of a preliminary study conducted with 96 students attending a university course that investigates two personal traits considered as prerequisites for using the tool: perceived proficiency in technology use, and opinions on the importance of each constituent element of the tool. With regard to both traits, some statistically significant variations emerged. The results obtained are an encouragement to continue the research project to verify whether the tool could be suitable to help student teachers develop an integrated planning procedure

Improving Listening and Speaking Achievements of the Eighth Grade Students of SMP Negeri 6 Metro by Using Videos Plus Discussion Method

Listening and speaking are the important skills that have to be mastered by the students. By having these skills, the students can communicate with others easily. However, the teaching and learning of listening and speaking skills are still problematical. The objectives of this study were to find out whether or not videos plus discussion method was effective in improving the students\u27 listening and speaking achievements.  This study involved one hundred and ninety six of the eighth grade students of SMP Negeri 6 Metro and forty of them were selected as the sample by using purposive sampling technique. The results of this study showed that the videos plus discussion method was considered as one of the effective ways to improve the students\u27 listening and speaking achievements