Time-slice analysis of dyadic human activity

La reconnaissance d’activités humaines à partir de données vidéo est utilisée pour la surveillance ainsi que pour des applications d’interaction homme-machine. Le principal objectif est de classer les vidéos dans l’une des k classes d’actions à partir de vidéos entièrement observées. Cependant, de tout temps, les systèmes intelligents sont améliorés afin de prendre des décisions basées sur des incertitudes et ou des informations incomplètes. Ce besoin nous motive à introduire le problème de l’analyse de l’incertitude associée aux activités humaines et de pouvoir passer à un nouveau niveau de généralité lié aux problèmes d’analyse d’actions. Nous allons également présenter le problème de reconnaissance d’activités par intervalle de temps, qui vise à explorer l’activité humaine dans un intervalle de temps court. Il a été démontré que l’analyse par intervalle de temps est utile pour la caractérisation des mouvements et en général pour l’analyse de contenus vidéo. Ces études nous encouragent à utiliser ces intervalles de temps afin d’analyser l’incertitude associée aux activités humaines. Nous allons détailler à quel degré de certitude chaque activité se produit au cours de la vidéo. Dans cette thèse, l’analyse par intervalle de temps d’activités humaines avec incertitudes sera structurée en 3 parties. i) Nous présentons une nouvelle famille de descripteurs spatiotemporels optimisés pour la prédiction précoce avec annotations d’intervalle de temps. Notre représentation prédictive du point d’intérêt spatiotemporel (Predict-STIP) est basée sur l’idée de la contingence entre intervalles de temps. ii) Nous exploitons des techniques de pointe pour extraire des points d’intérêts afin de représenter ces intervalles de temps. iii) Nous utilisons des relations (uniformes et par paires) basées sur les réseaux neuronaux convolutionnels entre les différentes parties du corps de l’individu dans chaque intervalle de temps. Les relations uniformes enregistrent l’apparence locale de la partie du corps tandis que les relations par paires captent les relations contextuelles locales entre les parties du corps. Nous extrayons les spécificités de chaque image dans l’intervalle de temps et examinons différentes façons de les agréger temporellement afin de générer un descripteur pour tout l’intervalle de temps. En outre, nous créons une nouvelle base de données qui est annotée à de multiples intervalles de temps courts, permettant la modélisation de l’incertitude inhérente à la reconnaissance d’activités par intervalle de temps. Les résultats expérimentaux montrent l’efficience de notre stratégie dans l’analyse des mouvements humains avec incertitude.Recognizing human activities from video data is routinely leveraged for surveillance and human-computer interaction applications. The main focus has been classifying videos into one of k action classes from fully observed videos. However, intelligent systems must to make decisions under uncertainty, and based on incomplete information. This need motivates us to introduce the problem of analysing the uncertainty associated with human activities and move to a new level of generality in the action analysis problem. We also present the problem of time-slice activity recognition which aims to explore human activity at a small temporal granularity. Time-slice recognition is able to infer human behaviours from a short temporal window. It has been shown that temporal slice analysis is helpful for motion characterization and for video content representation in general. These studies motivate us to consider timeslices for analysing the uncertainty associated with human activities. We report to what degree of certainty each activity is occurring throughout the video from definitely not occurring to definitely occurring. In this research, we propose three frameworks for time-slice analysis of dyadic human activity under uncertainty. i) We present a new family of spatio-temporal descriptors which are optimized for early prediction with time-slice action annotations. Our predictive spatiotemporal interest point (Predict-STIP) representation is based on the intuition of temporal contingency between time-slices. ii) we exploit state-of-the art techniques to extract interest points in order to represent time-slices. We also present an accumulative uncertainty to depict the uncertainty associated with partially observed videos for the task of early activity recognition. iii) we use Convolutional Neural Networks-based unary and pairwise relations between human body joints in each time-slice. The unary term captures the local appearance of the joints while the pairwise term captures the local contextual relations between the parts. We extract these features from each frame in a time-slice and examine different temporal aggregations to generate a descriptor for the whole time-slice. Furthermore, we create a novel dataset which is annotated at multiple short temporal windows, allowing the modelling of the inherent uncertainty in time-slice activity recognition. All the three methods have been evaluated on TAP dataset. Experimental results demonstrate the effectiveness of our framework in the analysis of dyadic activities under uncertaint

CES-KD: Curriculum-based Expert Selection for Guided Knowledge Distillation

Knowledge distillation (KD) is an effective tool for compressing deep classification models for edge devices. However, the performance of KD is affected by the large capacity gap between the teacher and student networks. Recent methods have resorted to a multiple teacher assistant (TA) setting for KD, which sequentially decreases the size of the teacher model to relatively bridge the size gap between these models. This paper proposes a new technique called Curriculum Expert Selection for Knowledge Distillation (CES-KD) to efficiently enhance the learning of a compact student under the capacity gap problem. This technique is built upon the hypothesis that a student network should be guided gradually using stratified teaching curriculum as it learns easy (hard) data samples better and faster from a lower (higher) capacity teacher network. Specifically, our method is a gradual TA-based KD technique that selects a single teacher per input image based on a curriculum driven by the difficulty in classifying the image. In this work, we empirically verify our hypothesis and rigorously experiment with CIFAR-10, CIFAR-100, CINIC-10, and ImageNet datasets and show improved accuracy on VGG-like models, ResNets, and WideResNets architectures.Comment: ICPR202

Efficient Fine-Tuning of Compressed Language Models with Learners

Fine-tuning BERT-based models is resource-intensive in memory, computation, and time. While many prior works aim to improve inference efficiency via compression techniques, e.g., pruning, these works do not explicitly address the computational challenges of training to downstream tasks. We introduce Learner modules and priming, novel methods for fine-tuning that exploit the overparameterization of pre-trained language models to gain benefits in convergence speed and resource utilization. Learner modules navigate the double bind of 1) training efficiently by fine-tuning a subset of parameters, and 2) training effectively by ensuring quick convergence and high metric scores. Our results on DistilBERT demonstrate that learners perform on par with or surpass the baselines. Learners train 7x fewer parameters than state-of-the-art methods on GLUE. On CoLA, learners fine-tune 20% faster, and have significantly lower resource utilization.Comment: 8 pages, 9 figures, 2 tables, presented at ICML 2022 workshop on Hardware-Aware Efficient Training (HAET 2022

Effective Turning Motion Control of Internally Actuated Autonomous Underwater Vehicles

This paper presents a novel roll mechanism and an efficient control strategy for internally actuated autonomous underwater vehicles (AUVs). The developed control algorithms are tested on Michigan Tech’s custom research glider, ROUGHIE (Research Oriented Underwater Glider for Hands-on Investigative Engineering), in a controlled environment. The ROUGHIE’s design parameters and operational constraints were driven by its requirement to be man portable, expandable, and maneuverable in shallow water. As an underwater glider, the ROUGHIE is underactuated with direct control of only depth, pitch, and roll. A switching control method is implemented on the ROUGHIE to improve its maneuverability, enabling smooth transitions between different motion patterns. This approach uses multiple feedforward-feedback controllers. Different aspects of the roll mechanism and the effectiveness of the controller on turning motion are discussed based on experimental results. The results illustrate that the ROUGHIE is capable of achieving tight turns with a radius of 2.4 meters in less than 3 meters of water, or one order of magnitude improvement on existing internally actuated platforms. The developed roll mechanism is not specific to underwater gliders and is applicable to all AUVs, especially at lower speeds and in shallower water when external rudder is less effective in maneuvering the vehicle

GUPPIE, underwater 3D printed robot a game changer in control design education

This paper presents innovative strategies to teach control and robotic concepts. These strategies include: 1) a real world focus on social/environmental contexts that are meaningful and “make a difference”; 2) continuous design potential and engagement through use of a platform that integrates design with engineering; 3) mission-based versus application-based approaches, where meaningful application justifies the process; and 4) hands-on, inquiry-based problem-solving. For this purpose a Glider for Underwater Problem-solving and Promotion of Interest in Engineering or “GUPPIE” platform and its simulator were utilized. GUPPIE is easy and inexpensive to manufacture, with readily available lightweight and durable components. It is also modular to accommodate a variety of learning activities. This paper describes how GUPPIE and its interdisciplinary nature was used as a pedagogical platform for teaching core control concepts for different age groups. The activities are designed to attract the interest of students as early as middle school and sustain their interest through college. The game changing aspect of this approach is scaffolded learning and the fact that the students will work with the same platform while progressing through the concepts

Learning autonomous systems - An interdisciplinary project-based experience

© 2017 IEEE. With the increased influence of automation into every part of our lives, tomorrow\u27s engineers must be capable working with autonomous systems. The explosion of automation and robotics has created a need for a massive increase in engineers who possess the skills necessary to work with twenty-first century systems. Autonomous Systems (MEEM4707) is a new senior/graduate level elective course with goals of: 1) preparing the next generation of skilled engineers, 2) creating new opportunities for learning and well informed career choices, 3) increasing confidence in career options upon graduation, and 4) connecting academic research to the students world. Presented in this paper is the developed curricula, key concepts of the project-based approach, and resources for other educators to implement a similar course at their institution. In the course, we cover the fundamentals of autonomous robots in a hands-on manner through the use of a low-cost mobile robot. Each student builds and programs their own robot, culminating in operation of their autonomous mobile robot in a miniature city environment. The concepts covered in the course are scalable from middle school through graduate school. Evaluation of student learning is completed using pre/post surveys, student progress in the laboratory environment, and conceptual examinations

Co-robotics hands-on activities: A gateway to engineering design and STEM learning

© 2017 Elsevier B.V. This paper presents the effect of meaningful learning contexts and hands-on activities, facilitated using two robots that work with people (co-robots), in broadening and sustaining pre-college student engagement in Science, Technology, Engineering, and Mathematics (STEM). The two co-robots are: (1) a Glider for Underwater Problem-solving and Promotion of Interest in Engineering or GUPPIE and (2) a Neurally controlled manipulator called Neu-pulator. The co-robots are easy and inexpensive to manufacture, with readily available lightweight and durable components. They are also modular to accommodate a variety of learning activities that help young students to learn crosscutting concepts and engineering practice. The early assessment results show that students’ interests in activities related to robotics depend on their perception of the difficulty and their confidence level. The key is to start early when the students are young. The challenge is to break the barriers and define tasks as fun activities with a learn and play approach that can be rewarding. In this work, using a meaningful context – as in co-robots that help humans – in a hands-on project-based program that integrates different aspect of design, science, and technology is found effective in increasing students’ enthusiasm and participation. The co-robots and the hands-on activities can be easily adopted in classrooms by teachers with no engineering background who seek innovative ways to connect interdisciplinary core ideas and standards to the concepts they need to teach

Robotics education to and through college

Robotics education has made great strides to enable the next generation of engineers and workers with early education and outreach. This early education effort is able to engage students and promote interest, however an integrated pathway to and through college is needed. This pathway needs to build upon early experiences with opportunities to advance across age groups. This paper presents the authors experience developing robotics curriculum across age groups. Middle and high school education has been implemented in a summer camp environment utilizing two co-robotic platforms, a water sensing robot called GUPPIE and an assistive robot named Neu-pulator, engaging 201 total students between Summer 2014–2017. The university course is a senior level technical elective introducing autonomous systems through a mobile robotic platform, a smart car, with 72 total students in Spring 2017 and 2018. In this work, the survey results gathered from Summer 2017 pre-college and Spring 2018 college level activities are presented. Overall observations and lessons learned across age groups are also discussed to better create a pathway from young learners to practicing engineers. The key to success of robotics programs at any age are hands-on, exciting activities with sufficient expert support so that students are able to learn in a frustration free environment