Observing, Coaching and Reflecting: A Multi-modal Natural Language-based Dialogue System in a Learning Context

The Metalogue project aims to develop a multi-modal, multi-party dialogue system with metacognitive abilities that will advance our understanding of natural conversational human-machine interaction and dialogue interfaces. This paper introduces the vision for the system and discusses its application in the context of debate skills training where it has the potential to provide learners with a rich, immersive experience. In particular, it considers a potentially powerful learning analytics tool in the form of a performance reflection dashboard

GAME EDUKASI BAGI ANAK AUTIS BERTEMA ANGGOTA KELUARGA BERBASIS KINECT XBOX 360

 Pengenalan obyek merupakan salah satu materi pembelajaran yang masuk dalam kurikulum buku kompetensi belajar untuk anak autis tingkat sekolah dasar yang digunakan di sekolah terpadu Rumah Pintar Salatiga. Selama ini pengenalan obyek dilakukan dengan cara guru menyiapkan gambar  obyek seperti yang terdapat dalam buku panduan kemudian ditunjukkan pada siswa. Metode seperti ini selain kurang menarik, memungkinkan siswa autis mengalami kesulitan untuk mengenali obyek , aktifitas gerak motorik mereka kurang dan akan cepat merasa bosan. Hal ini sesuai dengan karakter anak autis.  Penelitian ini bertujuan untuk memperkenalkan obyek anggota keluarga dalam lingkup yang kecil yaitu meliputi ayah, ibu, kakak dan adik serta beberapa obyek benda di sekitar keluarga sesuai kurikulum. Pengenalan obyek dikemas dalam sebuah game edukasi yang berbasis kinect xbox 360, sehingga menarik dan interaktif sekaligus melatih gerak motorik para siswa. Wawancara terhadap para guru dilakukan untuk mengetahui metode pembelajaran pengenalan obyek, metode melatih gerak motorik, mengetahui tingkat kemampuan berkomunikasi  dan menangkap informasi dari anak autis. Pembuatan game ini menggunakan software utama  Unity 2D/3D dan Kinect SDK. Software Inkscape digunakan untuk membuat aset-aset obyek yang terdapat dalam game, sedangkan animasi dibuat menggunakan software DragonBones. Pengujian dilakukan dengan mendemokan game edukasi ini kepada para siswa  kemudian mereka diberi kesempatan untuk mencoba dengan didampingi beberapa guru. Hasil kuisioner yang diisi oleh para guru dapat dinyatakan bahwa game ini mudah dimainkan, sangat menarik dan dapat digunakan untuk merehabilitasi gerak motorik anak autis, serta dapat digunakan sebagai salah satu media pembelajaran bagi anak autis tingkat sekolah dasar. Kata kunci: Autis, Game Edukasi,  Kinect Xbox, Motori

Game Edukasi Anak Autis Guna Melatih dan Merehabilitasi Gerak Motorik Berbasis Kinect Bertema Anggota Keluarga

Children with autism spectrum disorder or autism, require more intensive guidance to learn new subject. Besides having difficulty in focus, unpredictable and uncontrollable motoric movements are main factors which hinder their studies. Sometimes children with autism are unable to distinguish between multiple object. The aim of this research is to introduce family members and daily objects to children with autism by using game based on Kinect. This educational game is expected to be able to present primary family members which are father, mother, older brother, and little sister to autistic children in more pleasant way. This game can also be used to train focus ability and motoric movements of autistic children thus they can easily absorb the subject given in the game. This research is held in "Sekolah Rumah PintarAnak Berkebutuhan Khusus (ABK)", Salatiga and the subject used is based on the syllabus for elementary school of children with autism spectrum disorder. Autistic children get bored quickly when paying attention to a single repetitive stimulus for long periods of time or when facing the subject which is not interesting for them. Introducing family members in the different way like using animation with supported sound will make them more excited in learning the subject. This Kinect-base educational game was designed in reference to the results of interviews with the teacher and observations in Sekolah Rumah Pintar (ABK). The primary software used to built this game are Unity 2D/3D and Kinect SDK. The assets of the game were made using Inkscape software and the animations are made using DragonBones software. In a trial stage of this application, the students of Sekolah Rumah Pintar accompanied by their teachers are asked to play this game. From the questioner filled by the teachers of Sekolah Rumah Pintar (ABK), this game greatly attracts the attention of children with autism. This game is not only can help rehabilitating the motoric movements of children with autism but also can be used as a studying material for autistic children in elementary school

Pembuatan Game Berbasis Kinect Sebagai Media Pembelajaran Anak Berkebutuhan Khusus (Autis) Bertema Keluarga Besarku Untuk Tingkat Sekolah Dasar

Autism is one of the children with special needs (ABK), a birth defect and has the ability below average with compared to the normal children in general. Special schools (SLB) is an educational institution that handles the children with special needs, each SLB has a specific subject matter and teaching methods required special or different than the normal children learning methods in general. The purpose of this research is to create educational games based Kinect Xbox 360 that can assist in delivering the material, attract the children, and train the motor nerves of children, where the contents of the game by book of Autism Children Curriculum 2013 in SLB (Rumah Pintar Salatiga ), specifically for children with autism light level in Elementary School grade 2. This research is begins with visiting the Rumah Pintar SLB Salatiga and interviews by several teachers, and observed the process of teaching and learning in the SLB. The material in this game is Keluarga Besarku ,in the form of the introduction of the extended family like grandfather, grandmother, uncle and aunt which is there are game to learn to count, matching the images, compare the images, and choose the images. This game was made using the Unity 3D software, Kinect SDK, DragonBones Pro for making animation and Inscape Vector for image editing. Testing this game is done by demonstrating the game in the classroom and autism children who play the game, and during the testing was accompanied by several teachers. From the results of questionnaires that have been filled respondents stated that the game in accordance with the curriculum in special schools was the result of 79% answered agree, to see exciting game with the result 76% disagree, and be interactive with the results of 95% agreed

The Application of KINECT Motion Sensing Technology in Game-Oriented Study

The learning environment based on the KINECT Motion Sensing technology is able to fully mobilize the learners' multi-sensory organs, closely combine study with sports and enhance human-computer interactions, which can be conducive to the learners' health, greatly increase the relishes of learning and promote effective learning in the game, and finally compensate for the shortage of human-computer interactions in the traditional mouse and keyboard mode. The article elaborates on the KINECT Motion Sensing Technology and its educational applications status by analyzing its effective supports for game-oriented studying environment, based on which the article establishes a game-oriented learning environment. Eventually the article reveals an applicable case of game-oriented teaching and learning as a reference for related researches