制度:新 ; 報告番号:甲3337号 ; 学位の種類:博士(工学) ; 授与年月日:2011/2/25 ; 早大学位記番号:新564

Petersen Klaus

English

DSpace at Waseda University

	 ????????  ????????     ?? ?? ?? ?? ?? ?          ? 	 ? 	 ? 	 ?    Study on Perception-Action Scheme for Human-Robot Musical Interaction in Wind Instrumental Play ????????????????	 ?????????????????	 ??????????	   ?	 	 ?	 	 ? 	 Klaus 	 Petersen	       ??????	 ????????????	   2010 ? 12 ?      	  ???1     In recent years, there have been many attempts, to introduce robots to areas where they come into direct contact with people.  As soon as a robot is expected to operate in an environment that is shared with humans, the robot needs to be able to interact.  Sensors allow the robot to find its area of operation and define its task space. It  needs to be able to detect the proximity of people and give humans the possibility to operate it  through an appropriate interface. The people being in the same environment with the robot,  regardless if  they passively or actively use the device,  need to have the possibility to control the machine's actions. The amount of operability of a robot depends on the character of the HRI (Human Robot Interaction) interface that is used for interaction. The more natural interaction the interface allows, the easier it  is  for untrained users to interact with the robot.  The optimum solution is completely human-like interaction with the machine. Recently there have been many research efforts to develop interaction systems, which allow natural communication between human and robot.   Through the approach presented in this thesis,  an anthropomorphic robot is to be enabled to act in the same environment as its human partner with the intention to perform a certain task. In contrast to interaction through predefined actions, the robot is to dynamically adapt to the human partner behavior to reach a common goal.  Through perceiving information via various communication channels,  the robot should be aware of its situation within a certain environment and act accordingly (perception-action scheme).  By the development of a multi-modal interface with learning capabilities,  anticipated communication patterns related to the target task are to be used to provide natural interaction to the human partner of the robot.  Robots that perform a very rich communication with humans are musical performance robots.  Anthropomorphic musical performance robots are robots that have the ability to mechanically emulate the human way of playing a musical instrument. These technically very complex robots reach high performance levels that are comparable to the skill  level of  professional human musicians. A feature that in most cases they still  lack however, is  the ability to interact with other musicians. Playing a fixed, invariable sequence, they might be able to perform together with other players,  but as soon as there is a spontaneous variation in ???2     the musical performance, the human and the robot performance will  be de-synchronized.  In particular,  a Musical-based Interaction System (MbIS) that allows human musicians to interact with the Flutist Robot WF-4RV is proposed. The system focuses on musical interaction with the purpose of creating novel ways of musical expression. The humanoid robot is not merely used as an instrument-playing machine, but it  is  enabled to act as a musical partner. This is achieved by processing acoustic as well  as visual sensor input and by mapping those incoming inputs into musical outputs. The system has three specific  novelties that separate it  from other musical interaction systems: It  does not only use acoustic sensor data to render new musical content, but also has a vision recognition system to react to gestures by other musicians. The music is performed by a humanoid robot that is designed to emulate the human way of playing the flute as closely as possible.  This incorporates that the robot has, l ike a human player, certain physical l imitations that need to be taken into account within the interaction system. As the interaction with a complex robot l ike the WF-4RV is considered to be a challenge for a musician, the interaction system is adjustable to a beginner and an advanced skill  level.  The system consists of  several modules with different tasks, for each task, there is one specific module that analyses the output from the camera and microphone of the robot and maps the extracted information to parameters that modify the musical performance. From these parameters MIDI data is generated and sent to the robot.  The robot itself  has a motor control module that receives the MIDI information and adjusts the movement of the motors accordingly. The separation of the interaction system into two stages takes place in the sound and video analysis part of  the system.   The basic level of  interaction uses virtual faders and buttons to sense the instrument movements of the robot ’s  partner musician. The data sent from the input sensor processing to the robot control module is modulated by the physical  state of the robot.  I  use histogram-based audio analysis to verify,  i f  the performance output of the robot is according to the intended result.  In the extended level interaction interface, my goal is to give the user the possibility to interact with the robot more freely (compared to the basic level) .  To achieve this,  ???3     I  propose a teaching system, that allows the user to l ink instrument gestures with musical patterns. Here, the correlation of sensor input to sensor output is not fixed. Furthermore, I  allow for more degrees-of-freedom in the instrument movements of the user. As a result this level is  more suitable for advanced level players. For this task I use a particle f i lter-based instrument gesture detection system. A Bayesian mapping algorithm is employed in order to ensure, that if  the teaching musician does not account for all  combinations of instrument orientation and musical output in the teaching phase, in the performance phase the robot will  automatically play the most closely matching answer modulation to a given instrument state. In both interaction levels the proposed controllers can be used to control the onset of  the musical performance.  To verify the proposed contribution of the described interaction system to the research field of HRI, several experiments were performed. With these experiments the conceptual functionality of the interaction system as an interface for musical interaction with a performance robot is to be demonstrated. To evaluate the system, musicians of different skill  levels were asked to interact with the robot,  following their personal intention for musical expression. I  analyzed the result of  these experiments practically by recording incoming sensor data and musical output. This recorded data was examined in order to investigate the accuracy and robustness of the proposed method. The system was evaluated in terms of  the practicability of  the HRI concept by surveying the personal performance experience of the human partner musicians. The results from a comparative, technical and non-technical user-oriented evaluation are used to verify the contribution of the proposed interaction system to the HRI research field.  The system can be regarded as being functional in respect to its purpose from a technical as well  as from a user-experience point of view. From the experiments, it  is  to be verified, that the system satisfies the requirements of the research objective: It  allows communication through multiple perception channels.  It  modulates the interaction to comply with the physical constraints of the robot.  It  allows adjustment to different user skill  levels.  It  is  applicable not only to one certain type of robot,  but works as a general purpose concept.  Therefore, the system as a whole can be, due to its novel components, considered as a conceptual as well  as a practical contribution to the HRI field.      ????	 ?????	 ??????	 ????	 ?????	 	 ?	 ?	 ??????	 ????	 ?	 ??????	 ??	 ???	 ?	 ?	 ?	 	 	 ???	 	 ??????????	 	 ????????	 	 ??????????	 1. ??    	 	 	 	 	 	 	 	 	 	 	 	 	  Implementation of a Musical Performance Interaction System for the Waseda Flutist Robot: Combining Visual and Acoustic Sensor Input based on Sequential Bayesian Filtering, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.  2010?10?, Klaus Petersen, Jorge Solis, Atsuo Takanishi.  Development of a Visual	 Interface for Sound Parameter Calibration of	 the Waseda Flutist Robot WF-4RIV, ROMANSY 2010 18th CISM-IFToMM Symposium on Robot Design, Dynamics, and Control, 2010?7?, Klaus Petersen, Jorge Solis, Atsuo Takanishi.  Musical-Based Interaction System for the Waseda Flutist Robot: Implementation of the Visual Tracking Interaction Module, Autonomous Robots Journal. 2010? 3? , Klaus Petersen, Jorge Solis, Atsuo Takanishi.  Implementation of an Auditory Feedback Control System on an Antropomorphic Flutist Robot Inspired on the Performance of a Professional Flutist, Advanced Robotics Journal. 2009?12?, Jorge Solis, Koichi Taniguchi, Takeshi Ninomiya, Klaus Petersen, Tetsuro Yamamoto, Atsuo Takanishi.  Development of the Anthropomorphic Saxophonist Robot WAS-1: Mechanical Design of the Simulated Organs and Implementation of Air Pressure, Advanced Robotics Journal. 2009?12?, Jorge Solis, N. Takeshi, Klaus Petersen, Masaki Takeuchi, Atsuo Takanishi.  Development of a Aural Real-Time Rhythmical and Harmonic Tracking to Enable the Musical Interaction with the Waseda Flutist Robot, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2009? 10?, Klaus Petersen, Jorge Solis, Atsuo Takanishi.  Development of Anthropomorphic Musical Performance Robots: From Understanding the Nature of Music Performance to Its Application in Entertainment Robotics?2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009?10? , Jorge Solis, Klaus Petersen, Takeshi Ninomiya, Masaki Takeuchi, Atsuo Takanishi.  Anthropomorphic Musical Performance Robots at Waseda University: Increasing Understanding of the Nature of Human Musical Interaction, 9th International Conference New Interfaces for Musical Expression, 2009?10?, Jorge Solis, Takeshi Ninomiya, Klaus Petersen, Tetsuro Yamamoto.  Comparison of the Temperature Dependence of Irreversible and Reversible Magnetization of Basalt Samples", 2009? ???	 Journal of Physics of the Earth and Planetary Interiors, Nikolai Petersen, Klaus Petersen.  Development of the Anthropomorphic Saxophonist Robot WAS-1: Mechanical Design of the Lip, Tonguing, Fingers and Air Pump Mechanisms, International Conference on Robotics and Automation. 2009 ? 5 ?, Klaus Petersen, Jorge Solis, Ninomiya Takeshi, Masaki Takeuchi, Yamamoto Tetsuro, Takanishi Atsuo.	     ????	 ?????	 ??????	 ????	 ?????	 	 ?	 ?	 ?	 	 	 ???	 	 ??????????	 	 ????????	 	 ??????????	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	  2. ??	 	 	 	 	 	 	 	 	 	 	 	 The Waseda Flutist Robot No.4 Refined IV: From a Musical Partner to a Musical Teaching Too??? IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics. 2008 ? 10 ?, Jorge Solis, Koichi Taniguchi, Takeshi Nimomiya, Klaus Petersen, Tetsuro Yamamoto, Atsuo Takanishi?? Development of the Waseda Flutist Robot No. 4 Refined IV: Implementation of a Real-Time Interaction System with Human Partners, Proceedings of IEEE BioRob 2008 Conference. 2008? 10?, Klaus Petersen, Jorge Solis, Atsuo Takanishi.  Improved Musical Performance Control of WF-4RIV: Implementation of an expressive music generator and an automated sound quality detection, 17th International Symposium on Robot and Human Interactive Communication. 2008? 8?, Jorge Solis, Koichi Taniguchi, Takeshi Ninomiya, Klaus Petersen, Tetsuro Yamamoto, Atsuo Takanishi.  Development of a Real-Time Instrument Tracking System for Enabling the Musical Interaction with the Waseda Flutist Robot, Proceedings of IEEE IROS 2008 Conference. 2008? 5?, Klaus Petersen, Jorge Solis, Atsuo Takanishi.  Development of a Real-Time Gestural Interface for Hands-Free Musical Performance Control, Proceedings of ICMC 2008 Conference. 2008? 4?, Klaus Petersen, Jorge Solis, Atsuo Takanishi.  Toward Enabling a Natural Interaction between Human Musicians and Musical Performance Robots: Implementation of a Real-Time Gestural Interface, Proceedings of IEEE RO-MAN Conference 2008. 2008? 3?, Klaus Petersen, Jorge Solis, Atsuo Takanishi.  Autonomous Line tracking Car with Infineon XC866 / XC164 MCUs and Power Devices, Embedded World Exhibition and Conference 2006, Nürnberg, Germany. 2006? 4?, Patrick Leteinturier and Klaus Petersen.   ??????????????????-??????????????????????????????-?? 27???????????????2009? 9??????, ????, ????, ??????? ????? ??? ???, ????, ?????  ??????????????????−??????????????????????????????−??? IFToMM? ???????????2009 ? 7 ??????, ????, ????, Klaus Petersen, Jorge Solis, ?????  ??????????????????-??????????????????-?? 26 ???????????????2008? 9??????, ????, ????, Klaus Petersen, Jorge Solis, ?????  Towards enabling the interaction between the Waseda Flutist Robot and Human Players within the context of a musical band, ? 25??????????????, ???, ??, 2007???	 (????). 2007? 9?, Klaus Petersen, Jorge Solis, Ninomiya Takeshi, Yamamoto Tetsuro, Takanishi Atsuo.	     ????	 ?????	 ??????	 ????	 ?????	 	 ?	 ?	 ?	 	 	 ???	 	 ??????????	 	 ????????	 	 ??????????	 	 	 	 	 3. ?? 	   4. ???	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	  	  	 Automatic simulation modeling of vehicle navigation dynamics using video images correlated to MDP state-action transitions, ?24??????????????, ??, ??, 2006, 2E23. 2006? 9?, Klaus Petersen.   Variable Field Translation Balance, Encyclopedia of Earth Sciences Series, 2006, Springer Pub., 2006? 6?, David Krasa, Klaus Petersen, Nikolai Petersen.    Design, Construction and Programming of a Soccer Robot according to the Guidelines of the RoboCup F-180 Class, 2000? 9?, Klaus Petersen.	  

Study on Perception-Action Scheme for Human-Robot Musical Interaction in Wind Instrumental Play

制度:新 ; 報告番号:甲3337号 ; 学位の種類:博士(工学) ; 授与年月日:2011/2/25 ; 早大学位記番号:新5641textthesi

Petersen, Klaus

32142

Waseda University Repository

人間とロボットとの吹鳴楽器演奏のインタラクションにおける知覚・行動スキームに関する研究

https://waseda.repo.nii.ac.jp/?action=repository_action_common_download&item_id=19409&item_no=1&attribute_id=20&file_no=3

Study on Perception-Action Scheme for Human-Robot Musical Interaction in Wind Instrumental Play

Abstract

Similar works

Full text

Available Versions

DSpace at Waseda University

Waseda University Repository