Neural Task Programming: Learning to Generalize Across Hierarchical Tasks

In this work, we propose a novel robot learning framework called Neural Task Programming (NTP), which bridges the idea of few-shot learning from demonstration and neural program induction. NTP takes as input a task specification (e.g., video demonstration of a task) and recursively decomposes it into finer sub-task specifications. These specifications are fed to a hierarchical neural program, where bottom-level programs are callable subroutines that interact with the environment. We validate our method in three robot manipulation tasks. NTP achieves strong generalization across sequential tasks that exhibit hierarchal and compositional structures. The experimental results show that NTP learns to generalize well to- wards unseen tasks with increasing lengths, variable topologies, and changing objectives.Comment: ICRA 201

A Portable Augmented Reality Science Laboratory

Augmented Reality (AR) is a technology which overlays virtual objects on the real world; generates three-dimensional (3D) virtual objects and provides an interactive interface which people can work in the real world and interact with 3D virtual objects at the same time. AR has the potential to engage and motivate learners to explore material from a variety of differing perspective, and has been shown to be particularly useful for teaching subject matter that students could not possibly experience first hand in the real world. This report provides a conceptual framework of a simulated augmented reality lab which could be used in teaching science in classrooms. The recent years, the importance of lab-based courses and its significant role in the science education is irrefutable. The use of AR in formal education could prove a key component in future learning environments that are richly populated with a blend of hardware and software applications. The aim of this project is to enhance the teaching and learning of science by complementing the existing traditional lab with the use of a simulated augmented reality lab. The system architecture and the technical aspects of the proposed project will be described. Implementation issues and benefits of the proposed AR Lab will be highlighted

Spartan Daily, December 8, 2003

Volume 121, Issue 67https://scholarworks.sjsu.edu/spartandaily/9933/thumbnail.jp

Spartan Daily, December 8, 2003

Volume 121, Issue 67https://scholarworks.sjsu.edu/spartandaily/9933/thumbnail.jp

Spartan Daily, December 8, 2003

Volume 121, Issue 67https://scholarworks.sjsu.edu/spartandaily/9933/thumbnail.jp

Aplicação para smartphone ’Practice As You Walk’: mobile learning e gamification em ensaio coral

With the worldwide massification of mobile devices, the use of technology for pedagogical purposes in the context of music learning has proven to be an indispensable tool for ensuring motivation among students. By exploring the implementation of concepts such as gamification and mobile learning in music education, mentioning relevant case studies in this field, this dissertation culminates with the development of an application for Android smartphones entitled ’Practice As You Walk’. As the name implies, this learning tool consists in the reproduction of musical excerpts at the user’s walking pace, being a form of musical practice that stimulates the memorization of music pieces and synchronization ability of the individual. For the development of the application, this work explores innovative methods used for step detection through integrated sensors on mobile devices, such as the accelerometer and the gyroscope, and also presents the fundamentals of the MIDI communication protocol for the digital transmission of events related with musical performance. Two methods for smartphone-based step detection are proposed, with the rulebased method attaining an F1-score of 99% and the machine learning method attaining an F1-score of 95.84%. The development of the application, initially in the Unity platform, consists of integrating classes for MIDI file manipulation and processing with the ability to interpret and reproduce them at the user’s walking pace. Due to some faults identified in the music playback mechanism, migration to the Android Studio IDE took place through a third-party library that integrates the Sonivox EAS synthesizer. This abstraction from the playback mechanism allowed direct incorporation of the core functionalities developed in Unity and focus on the construction of a captivating user interface. Finally, within the pedagogical purpose of the present work, the application was tested by members of a children and youth choir. The questionnaire revealed general satisfaction with the application, allowing collection of opinions and suggestions on potential future improvements.Com a massificação global dos dispositivos móveis, o uso da tecnologia para fins pedagógicos no contexto da aprendizagem musical revelou ser uma ferramenta imprescindível para assegurar a motivação dos estudantes. Explorando a implementação dos conceitos de gamification e mobile learning no ensino da música, referindo casos de estudo relevantes neste campo, esta dissertação culmina com o desenvolvimento de uma aplicação para smartphones Android denominada ’Practice As You Walk’. Conforme indica o nome, esta ferramenta de aprendizagem consiste na reprodução de excertos musicais ao ritmo determinado pelo passo do utilizador, sendo uma forma de prática musical que estimula no indivíduo a memorização de obras musicais e a capacidade de sincronização. Com vista ao desenvolvimento da aplicação, são explorados neste trabalho métodos inovadores utilizados na deteção de passo através de sensores incorporados nos dispositivos móveis, tais como o acelerómetro e o giroscópio, e também apresentados os fundamentos do protocolo de comunicação MIDI para a transmissão digital de eventos relacionados com a interpretação musical. São propostos dois métodos para a deteção de passo com recurso a um smartphone, tendo o método baseado em regras atingido um F1-score de 99% e o método baseado em aprendizagem automática um F1-score de 95.84%. O desenvolvimento da aplicação, inicialmente na plataforma Unity, consiste na integração de classes para a manipulação e processamento de ficheiros MIDI com a capacidade de leitura e reprodução dos mesmos ao ritmo do passo do utilizador. Devido a alguns defeitos identificados no mecanismo de reprodução, segue-se a migração para o ambiente de desenvolvimento Android Studio recorrendo a uma biblioteca que integra o sintetizador Sonivox EAS. Esta abstração do mecanismo de reprodução permite a integração direta das funcionalidades desenvolvidas em Unity e um maior foco na construção de uma interface de utilizador cativante. Por fim, no âmbito pedagógico deste trabalho, a aplicação foi testada por membros de um coro infanto-juvenil. Um questionário revelou satisfação geral com a aplicação e permitiu a recolha de opiniões e sugestões tendo em vista potenciais melhorias.Mestrado em Engenharia Eletrónica e Telecomunicaçõe

FROM A RESEARCH PROTOTYPE TO A DEPLOYED SYSTEM: THE CASE OF THE HECHT MUSEUM VISITOR\u27S GUIDE

Cultural heritage, Deployment, Mobile guides, Research prototype