271 research outputs found
Enhanced visualisation of dance performance from automatically synchronised multimodal recordings
The Huawei/3DLife Grand Challenge Dataset provides multimodal recordings of Salsa dancing, consisting of audiovisual streams along with depth maps and inertial measurements. In this paper, we propose a system for augmented reality-based evaluations of Salsa dancer performances. An essential step for such a system is the automatic temporal synchronisation of the multiple modalities captured from different sensors, for which we propose efficient solutions. Furthermore, we contribute modules for the automatic analysis of dance performances and present an original software application, specifically designed for the evaluation scenario considered, which enables an enhanced dance visualisation experience, through the augmentation of the original media with the results of our automatic analyses
An advanced virtual dance performance evaluator
The ever increasing availability of high speed Internet access has led to a leap in technologies that support real-time realistic interaction between humans in online virtual environments. In the context of this work, we wish to realise the vision of an online dance studio where a dance class is to be provided by an expert dance teacher and to be delivered to online students via the web. In this paper we study some of the technical issues that need to be addressed in this challenging scenario. In particular, we describe an automatic dance analysis tool that would be used to evaluate a student's performance and provide him/her with meaningful feedback to aid improvement
Recommended from our members
3d augmented mirror: a multimodal interface for string instrument learning and teaching with gesture support
Multimodal interfaces can open up new possibilities for music education, where the traditional model of teaching is based predominantly on verbal feedback. This paper explores the development and use of multimodal interfaces in novel tools to support music practice training. The design of multimodal interfaces for music education presents a challenge in several respects. One is the integration of multimodal technology into the music learning process. The other is the technological development, where we present a solution that aims to support string practice training with visual and auditory feedback. Building on the traditional function of a physical mirror as a teaching aid, we describe the concept and development of an "augmented mirror" using 3D motion capture technology
The Multimodal Tutor: Adaptive Feedback from Multimodal Experiences
This doctoral thesis describes the journey of ideation, prototyping and empirical testing of the Multimodal Tutor, a system designed for providing digital feedback that supports psychomotor skills acquisition using learning and multimodal data capturing. The feedback is given in real-time with machine-driven assessment of the learner's task execution. The predictions are tailored by supervised machine learning models trained with human annotated samples. The main contributions of this thesis are: a literature survey on multimodal data for learning, a conceptual model (the Multimodal Learning Analytics Model), a technological framework (the Multimodal Pipeline), a data annotation tool (the Visual Inspection Tool) and a case study in Cardiopulmonary Resuscitation training (CPR Tutor). The CPR Tutor generates real-time, adaptive feedback using kinematic and myographic data and neural networks
Ubiquitous Integration and Temporal Synchronisation (UbilTS) framework : a solution for building complex multimodal data capture and interactive systems
Contemporary Data Capture and Interactive Systems (DCIS) systems are tied in with various
technical complexities such as multimodal data types, diverse hardware and software
components, time synchronisation issues and distributed deployment configurations. Building
these systems is inherently difficult and requires addressing of these complexities before the
intended and purposeful functionalities can be attained. The technical issues are often
common and similar among diverse applications.
This thesis presents the Ubiquitous Integration and Temporal Synchronisation (UbiITS)
framework, a generic solution to address the technical complexities in building DCISs. The
proposed solution is an abstract software framework that can be extended and customised to
any application requirements. UbiITS includes all fundamental software components,
techniques, system level layer abstractions and reference architecture as a collection to enable
the systematic construction of complex DCISs.
This work details four case studies to showcase the versatility and extensibility of UbiITS
frameworkâs functionalities and demonstrate how it was employed to successfully solve a
range of technical requirements. In each case UbiITS operated as the core element of each
application. Additionally, these case studies are novel systems by themselves in each of their
domains. Longstanding technical issues such as flexibly integrating and interoperating
multimodal tools, precise time synchronisation, etc., were resolved in each application by
employing UbiITS. The framework enabled establishing a functional system infrastructure in
these cases, essentially opening up new lines of research in each discipline where these
research approaches would not have been possible without the infrastructure provided by the
framework. The thesis further presents a sample implementation of the framework on a
device firmware exhibiting its capability to be directly implemented on a hardware platform.
Summary metrics are also produced to establish the complexity, reusability, extendibility,
implementation and maintainability characteristics of the framework.Engineering and Physical Sciences Research Council (EPSRC) grants - EP/F02553X/1, 114433 and 11394
MIROR-Musical Interaction Relying On Reflexion. Project Final Report
open7siIl progetto è stato valutato dalla Commissione Europea con il massimo del punteggio (15/15) con la seguente valutazione sintetica: Excellent. The proposal successfully addresses all relevant aspects of the criterion in question. Any shortcomings are minor". Ha superato le tre valutazioni annuali con giudizio molto positivo (good) e con brillante giudizio finale. Progetto co-finanziato dalla ComunitĂ Europea, 7° Programma Quadro, ICT-Challenge 4.2, Technology enhanced-learning, Programma Cooperation, no 258338. Il Programma âCOOPERATIONâ è identificato dal DM del 1/7/2011 (Identificazione dei programmi di ricerca di alta qualificazione, finanziati dall'Unione europea o dal Ministero dell'istruzione, dell'universita' e della ricerca di cui all'articolo 29, comma 7, della legge n. 240/2010) come uno dei due programmi di ricerca di alta qualificazione finanziati dallâUE. In dettaglio, dal Coordinatore sono stati ricoperti i seguenti ruoli:
⢠Preparazione della proposta e Coordinameto scientifico del Progetto
⢠Responsabile dei contatti con la Commissione Europea
⢠Supervisione e monitoraggio del lavoro svolto dal Consorzio, attraverso workshops, report tecnici e scientifici e coordinamento dei deliverable
⢠Lieder dei seguenti Work-Packages:
⢠WP1. Project Management
⢠WP5. Psychological Experiments
⢠WP8. Dissemination and Exploitation
⢠Coordinatore dellâALB
⢠Coordinamento scientifico e organizzativo del gruppo di ricerca dell'Università di Bologna, composto da:
o 2 assegni di ricerca post-dottorato
o 1 assegno di ricerca
o 2 contratti di assistenza alla ricerca
o 1 contratto per il sito web
o 15 studenti-collaboratori
o 3 insegnanti collaboratori
⢠Responsabile delle collaborazioni e convenzioni con lâIstituto Comprensivo di Casalecchio di Reno, la Nuova Scuola di Musica Baroncini di Imola, Il Centro Danza Musikè-Bologna.
⢠Supervisione del Project Management (Dipartimento della Ricerca Europea dell'UniversitĂ di Bologna - ARIC).The MIROR (Musical Interaction Relying On Reflexion) project is coâfunded by the European Commission under the 7th Framework Programme, Theme ICTâ2009.4.2, Technologyâenhanced learning. MIROR is a threeâyears project and started on September 1st, 2010. All information regarding MIROR is available through the MIROR Portal at http://www.mirorproject.eu.
The MIROR Project-Final Report deals with the description of the development of an adaptive system for music learning and teaching based on the âreflexive interactionâ paradigm. The system is developed in the context of early childhood music education. It acts as an advanced cognitive tutor, designed to promote specific cognitive abilities in the field of music improvisation, both in formal learning contexts (kindergartens, primary schools, music schools) and informal ones (at home, kinder centres, etc.). The reflexive interaction paradigm is based on the idea of letting users manipulate virtual copies of themselves, through specifically designed machineâlearning software referred to as âInteractive Reflexive Musical Systemsâ (IRMS). By definition IRMS are able to learn and configure themselves according to their understanding of the learner's behaviour. In MIROR the IRMS paradigm is extended with the analysis and synthesis of multisensory expressive gesture to increase its impact on the musical pedagogy of young children, by developing new multimodal interfaces. The project is based on a spiral design approach involving coupled interactions between technical and psychopedagogical partners. MIROR integrates both psychological caseâstudy experiments, aiming to investigate cognitive hypotheses concerning the mirroring behaviour and the learning efficacy of the platform, and validation studies aiming at developing the software in concrete educational settings. The project contributes to promoting the reflexive interaction paradigm not only in the field of music learning, but more generally as a new paradigm for establishing a synergy between learning and cognition in the context of child/machine interaction.openopenA. R. Addessi ; C. Anagnostopoulou; S. Newman; B. Olsson; F. Pachet; G. Volpe; S. YoungA. R. Addessi ; C. Anagnostopoulou; S. Newman; B. Olsson; F. Pachet; G. Volpe; S. Youn
Landscapes of Affective Interaction: Young Children's Enactive Engagement with Body Metaphors
Empirical research into embodied meaning making suggests specific
sensorimotor experiences can support childrenâs understanding of abstract
science ideas. This view is aligned with enactive and grounded cognition
perspectives, both centred in the view that our ability to conceptualise emerges
from our experiences of interaction with our environment. While much of this
research has focused on understanding action and action processes in
individual children or children in pairs, less attention has been paid to affective
dimensions of young childrenâs group interaction, and how this relates to
meaning making with body metaphors. Indeed, Gallagher describes how no
action exists in a vacuum, but rather revolves around a complex web of
affective-pragmatic features comprising a âLandscape of Interactionâ (2020,
p.42).
This research project addresses gaps in research in understanding young
childrenâs affective engagement from an enactivist cognition perspective. It
takes a Design-Based Research approach with an iterative design orientation
to examine young childrenâs interaction with multisensory body-based
metaphors through an embodied participation framework. A series of empirical
studies with young children, aged 2-7 years, comprising of experiential
workshops, build iteratively upon each other. A novel theoretically informed
method, Affective Imagination in Motion, is developed involving several
purpose-built multisensory body metaphors prompts to enable access to
dimensions of young childrenâs affective engagement.
This research makes theoretical and methodological contributions. It extends
the theoretical notion of âaffectâ from enactive and grounded cognition
perspectives through identifying key interactive processes in young childrenâs
engagement with multisensory action metaphors. In addition, the novel
method offers a contribution as a way of âlookingâ at affect within a group
situation from affective-pragmatic and social embodiment perspectives.
Finally, the research contributes to embodied learning design frameworks
offering a guideline for designers wishing to inform their work from enactive
cognition perspective
- âŚ