Hülsmann F, Waltemate T, Pfeiffer T, et al. The ICSPACE Platform: A Virtual Reality Setup for Experiments in Motor Learning. Presented at the Virtual Environments: Current Topics in Psychological Research (VECTOR), Tübingen, Germany

Botsch, Mario 

Frank, Cornelia 

Hülsmann, Felix 

Kopp, Stefan 

Pfeiffer, Thies 

Schack, Thomas

Waltemate, Thomas

Publications at Bielefeld University

3D Scanner Template model and scanned point cloud Posture fitting Shape fitting Joint correction Texture32 simultaneously triggered DSLR cameras.The point cloud is computed by multi-view stereo reconstruction.28 corresponding landmarks (red dots) are manually selected on template model and point cloud.The template model can be animated by its skeleton. By template fitting we transfer this attribute to the scan.Rigid registration alternating with inverse kinematicsThis step scales, rotates and translates the point cloud and adapts the posture of the template model based on the selected landmarks.Non-linear optimization with three energy terms After the fitting procedure the joints of the skeleton are not in the correct position anymore. This is corrected by exploiting generalized barycentric coordinates.Landmarks Point-to-point Regularization Cluster of ExcellenceCognitive Interaction TechnologyThe ICSPACE Platform: A Virtual Reality Setup for Experiments in Motor LearningFelix Hülsmann1,3, Thomas Waltemate1, Thies Pfeiffer2, Cornelia Frank4, Thomas Schack4, Mario Botsch1, Stefan Kopp31 Graphics and Geometry, 2 Central Lab Facilities, 3 Social Cognitive Systems,  4 Neurocognition and ActionCenter of Excellence Cognitive Interaction Technology (CITEC), Bielefeld University, GermanyVirtual reality (VR) environments allow us to develop coaching strategies for motor learning, which exceed the limits of real-world coaching. To allow for motor learning in VR, a highly responsive virtual environment is needed which provides at least similar feedback as a real world coaching environment would do. We designed such a VR platform – the Intelligent Coaching Space (ICSPACE) – that enables full-body motor learning of complex actions. The whole system is designed to provide very low end-to-end latency of 42ms. The system consists of the following parts: Two-sided CAVE, optical motion capture system and self-developed software components which include rendering, 3D cloning and motion analysis. See Poster “Latency, Sensorimotor Feedback and Virtual Agents: Feedback Channels for Motor Learning Using the ICSPACE Platform” for further information on psychological experiments we already conducted using ICSPACE.Virtual Environments: Current Topics in Psychological Research WorkshopJuly 27th to 29th in 2016, Tübingen, GERGraduate School Research Retreat29th of November – 30th of November 2010OBJECTIVE• Hülsmann, F., Frank, C., Schack, T., Kopp, S., & Botsch, M. (2016). Multi-level Analysis of Motor Actions as a Basis for Effective Coaching in Virtual Reality. In Proceedings of the 10th International Symposium on Computer Science in Sports (ISCSS) (pp. 211-214). Springer International Publishing. • de Kok, I., Hough, J., Hülsmann, F., Botsch, M., Schlangen, D., & Kopp, S. (2015, November). A Multimodal System for Real-Time Action Instruction in Motor Skill Learning. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction (pp. 355-362). ACM.This research/work was supported by the Cluster of Excellence Cognitive Interaction Technology 'CITEC' (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG).Technical EnvironmentSelected Technical PublicationsRendering Engine3D Cloning by Template FittingMotor Performance Analysis and Analysis of Mental Representation• Achenbach, J., Zell, E., & Botsch, M. (2015). Accurate Face Reconstruction through Anisotropic Fitting and Eye Correction. Proceedings of Vision, Modeling and Visualization• Waltemate, T., Hülsmann, F., Pfeiffer, T., Kopp, S., & Botsch, M. (2015, November). Realizing a Low-latency Virtual Reality Environment for Motor Learning. In Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology (pp. 139-147). ACM.Graphics environment ● Two-sided CAVE (front and floor), Rearprojection for both walls● 2100 x 1600 pixels per side ● Passive stereo (INFITEC)● Four Projectors: Projection Design F35 WQ ● Single Computer with two NVIDIA QuadroK5000 graphics cardsMotor Performance AnalysisGoals: Build motion representation which combines levels of hierarchy: • Joint angles / 3D coordinates • Movement Primitives • Prototypical Error Patterns Motion capture• 10 camera OptiTrack System: Prime 13W• FoV: 82° x 70°• Spatial Resolution: 1.3 MP• Temporal Resolution: up to 240 Hz• Customized motion capture suit: Markers are partly glued to the skin, partly attached on the suit using Velcro• 44 markers are used to reconstruct up to  21 jointsTop: Marker Setup. Right: User performs squats inside the ICSPACE: He can observe own motion inside a virtual mirror, due to fast MoCap, 3D cloning and effective rendering.Mental Representation:• Structural-Dimensional Analysis, SDA-M (Schack, 2012)• Split task on set of basic action concepts (e.g., legs bent, knees behind toes, ...)Clusters of BACs by an unskilled user determined by the split test. The results for a skilled user would contain more distinctive clusters. Our system first determines the performed motor action (e.g., a squat) and then the current movement primitive (e.g., going-down). Then, prototypical performance errors and their interpretations are identified. This is realized via comparison of the motion trajectory with sequential descriptions of prototypical error patterns. The hierarchy which represents motion in the ICSPACE system consists of four levels: Pure joint data, higher level features, movement primitives and motor actions.Low latency virtual mirror Multi-GPU rendering Sufficient visual qualtiyTo visualize the scene together with our virtual mirror inside the CAVE, we employ a self-developed rendering engine. This engine runs on a single computer exploiting multi-GPU rendering (middle image). The engine is optimized to achieve a low end-to-end latency of only 42 ms in the basic virtual mirror scenario (left image). For a more advanced scenario with a richer scene and shadow mapping enabled (right image) the latency is still low at around 60 ms. High performance and low latency are achieved by ● employing the multi-GPU approach using a single computer,● offloading all expensive computations to the GPU,● controlling the data flow inside the application.Split task used to perform the SDA-M. Instruction: Do the two movements relate to each other during the movement execution?

The ICSPACE Platform: A Virtual Reality Setup for Experiments in Motor Learning

https://pub.uni-bielefeld.de/download/2905377/2905759

The ICSPACE Platform: A Virtual Reality Setup for Experiments in Motor Learning

Abstract

Similar works

Full text

Available Versions

Publications at Bielefeld University