Using computer vision in security applications

In this paper we present projects developed in the Computer Vision Laboratory, which address the issue of safety. First, we present the Internet Video Server (IVS) monitoring system [5] that sends live video stream over the Internet and enables remote camera control. Its extension GlobalView [1,6], which incorporates intuitive user interface for remote camera control, is based on panoramic image. Then we describe our method for automatic face detection [3] based on color segmentation and feature extraction. Finally, we introduce our SecurityAgent system [4] for automatic surveillance of observed location

Gestures as an interface of performers’ intentionality : a case study of Western Embodiment of Karnatic Music in piano performance

Background Music performance has a strong corporeal dimension, involving different types of gestures (technical gestures, expressive gestures, etc.) that performers employ to transform a written score into live music (Leman 2007). This transformation is based on the musical intentions that arise from the performers’ personal interpretation of the composition as an outcome of their artistic praxis, which leads to decisions on how to play the music in terms of its structure, articulation of the phrases, dynamics, timber and the necessary motor strategies to realize these decisions in the sounding music. Aims This research wants to investigate musical gestures as an interface of performers’ intentionality, i.e. an outcome of the artistic praxis and the process of embodiment, in the light of the recent theories on enactment and embodied music cognition (Leman 2016). For this reason, we considered a case study based on the interpretation of a piece that includes the acquisition and embodiment of musical knowledge quite knew to the performer in order to map the modifications in the corporeal engagement from an intuitive approach to a conscious approach. The composition chosen was a contemporary piano piece based on a non-western music tradition: the Karnatic modes from South India. Method To assist the performer in (re)framing the phases of her artistic process, a methodology, called performer based analysis method (Caruso et al 2016), was developed to establish also the procedures of a performative experiment where the performance of the 8th cycle from the 72 Etudes Karnatiques pour piano by Jacques Charpentier (b.1933) was taken as a case study. The performative experiment required a period of preparation, which concerns the performer/researcher’s artistic praxis (to embody the piece) and the self-observation of a video recording archive of her performances in order to map and describe the artistic praxis. The pianist conducted a musicological research on the influence of Indian music in the French contemporary piano repertoire to enrich specifically her current competences in Karnatic Music and had a collaborative three years practice with the composer and with two experts in Karnatic music, a singer and a dancer (see the Re-Orient project: http://re-orient.wixsite.com/indiandream). A retrospective thinking-aloud procedure (Van den Haak & De Jong, 2003) was used during the experiment to allow the performer in rendering explicit and systematic the artistic reflections. The experiment was recorded by a video camera, a microphone and the Motion Capture System. Results To catch the development between the initial intuitive performance and the final embodied performance, two recordings of one fragment from the piece played with these two different approaches (intuitive and conscious) were compared. The analysis of these fragments was based on an alignment between qualitative data acquired through subjective descriptions - based on a performance model and a score annotation - and quantitative data (objective measurements) produced by the audio-video and motion capture recordings. The qualitative and quantitative data (audio and video) were processed through the ELAN software. Gestural similarities and differences between the intuitive and conscious versions were detected by comparing the kinematic and audio measurements (quantitative data) with the performer’s subjective annotations (qualitative data) concerning the motor strategies and the interpretative cues. The results show a different corporeal engagement of the pianist related to the different intentions through a parallel configuration of these two different subjective and objective layers. Conclusions The actual investigation wants to present musical gestures as vehicles of idiosyncratic intentions and expressions by linking performers’ corporeal engagement to the embodiment of their interpretation in order to better understand the connection between musical intentions, goal actions and sound. The role of the technology-mediated approach (thirdperson’s perspective) gives the opportunity to study, as in a mirror-like tool, some aspects, which imply the performer’s subjective involvement (first-person’s perspective). This method provides specifically to musicians/researchers an easier access to music performance analysis. Furthermore, with the implementation of a transdisciplinary and collaborative practice (with the composer, the Indians singer and dancer) plus the aid of technology with the motion and audio analysis, the actual study adds an alternative perspective concerning the exploration and the design of new paths within the field of artistic research

Sketching-out virtual humans: From 2d storyboarding to immediate 3d character animation

Virtual beings are playing a remarkable role in today’s public entertainment, while ordinary users are still treated as audiences due to the lack of appropriate expertise, equipment, and computer skills. In this paper, we present a fast and intuitive storyboarding interface, which enables users to sketch-out 3D virtual humans, 2D/3D animations, and character intercommunication. We devised an intuitive “stick figurefleshing-outskin mapping” graphical animation pipeline, which realises the whole process of key framing, 3D pose reconstruction, virtual human modelling, motion path/timing control, and the final animation synthesis by almost pure 2D sketching. A “creative model-based method” is developed, which emulates a human perception process, to generate the 3D human bodies of variational sizes, shapes, and fat distributions. Meanwhile, our current system also supports the sketch-based crowd animation and the storyboarding of the 3D multiple character intercommunication. This system has been formally tested by various users on Tablet PC. After minimal training, even a beginner can create vivid virtual humans and animate them within minutes

Human-Machine Interface for Remote Training of Robot Tasks

Regardless of their industrial or research application, the streamlining of robot operations is limited by the proximity of experienced users to the actual hardware. Be it massive open online robotics courses, crowd-sourcing of robot task training, or remote research on massive robot farms for machine learning, the need to create an apt remote Human-Machine Interface is quite prevalent. The paper at hand proposes a novel solution to the programming/training of remote robots employing an intuitive and accurate user-interface which offers all the benefits of working with real robots without imposing delays and inefficiency. The system includes: a vision-based 3D hand detection and gesture recognition subsystem, a simulated digital twin of a robot as visual feedback, and the "remote" robot learning/executing trajectories using dynamic motion primitives. Our results indicate that the system is a promising solution to the problem of remote training of robot tasks.Comment: Accepted in IEEE International Conference on Imaging Systems and Techniques - IST201