Walking with virtual humans : understanding human response to virtual humanoids' appearance and behaviour while navigating in immersive VR

In this thesis, we present a set of studies whose results have allowed us to analyze how to improve the realism, navigation, and behaviour of the avatars in an immersive virtual reality environment. In our simulations, participants must perform a series of tasks and we have analyzed perceptual and behavioural data. The results of the studies have allowed us to deduce what improvements are needed to be incorporated to the original simulations, in order to enhance the perception of realism, the navigation technique, the rendering of the avatars, their behaviour or their animations. The most reliable technique for simulating avatars’ behaviour in a virtual reality environment should be based on the study of how humans behave within the environment. For this purpose, it is necessary to build virtual environments where participants can navigate safely and comfortably with a proper metaphor and, if the environment is populated with avatars, simulate their behaviour accurately. All these aspects together will make the participants behave in a way that is closer to how they would behave in the real world. Besides, the integration of these concepts could provide an ideal platform to develop different types of applications with and without collaborative virtual reality such as emergency simulations, teaching, architecture, or designing. In the first contribution of this thesis, we carried out an experiment to study human decision making during an evacuation. We were interested to evaluate to what extent the behaviour of a virtual crowd can affect individuals' decisions. From the second contribution, in which we studied the perception of realism with bots and humans performing just locomotion or varied animations, we can conclude that the combination of having human-like avatars with animation variety can increase the overall realism of a crowd simulation, trajectories and animation. The preliminary study presented in the third contribution of this thesis showed that realistic rendering of the environment and the avatars do not appear to increase the perception of realism in the participants, which is consistent with works presented previously. The preliminary results in our walk-in-place contribution showed a seamless and natural transition between walk-in-place and normal walk. Our system provided a velocity mapping function that closely resembles natural walk. We observed through a pilot study that the system successfully reduces motion sickness and enhances immersion. Finally, the results of the contribution related to locomotion in collaborative virtual reality showed that animation synchronism and footstep sound of the avatars representing the participants do not seem to have a strong impact in terms of presence and feeling of avatar control. However, in our experiment, incorporating natural animations and footstep sound resulted in smaller clearance values in VR than previous work in the literature. The main objective of this thesis was to improve different factors related to virtual reality experiences to make the participants feel more comfortable in the virtual environment. These factors include the behaviour and appearance of the virtual avatars and the navigation through the simulated space in the experience. By increasing the realism of the avatars and facilitating navigation, high scores in presence are achieved during the simulations. This provides an ideal framework for developing collaborative virtual reality applications or emergency simulations that require participants to feel as if they were in real life.En aquesta tesi, es presenta un conjunt d'estudis els resultats dels quals ens han permès analitzar com millorar el realisme, la navegació i el comportament dels avatars en un entorn de realitat virtual immersiu. En les nostres simulacions, els participants han de realitzar una sèrie de tasques i hem analitzat dades perceptives i de comportament mentre les feien. Els resultats dels estudis ens han permès deduir quines millores són necessàries per a ser incorporades a les simulacions originals, amb la finalitat de millorar la percepció del realisme, la tècnica de navegació, la representació dels avatars, el seu comportament o les seves animacions. La tècnica més fiable per simular el comportament dels avatars en un entorn de realitat virtual hauria de basar-se en l'estudi de com es comporten els humans dins de l¿entorn virtual. Per a aquest propòsit, és necessari construir entorns virtuals on els participants poden navegar amb seguretat i comoditat amb una metàfora adequada i, si l¿entorn està poblat amb avatars, simular el seu comportament amb precisió. Tots aquests aspectes junts fan que els participants es comportin d'una manera més pròxima a com es comportarien en el món real. A més, la integració d'aquests conceptes podria proporcionar una plataforma ideal per desenvolupar diferents tipus d'aplicacions amb i sense realitat virtual col·laborativa com simulacions d'emergència, ensenyament, arquitectura o disseny. En la primera contribució d'aquesta tesi, vam realitzar un experiment per estudiar la presa de decisions durant una evacuació. Estàvem interessats a avaluar en quina mesura el comportament d'una multitud virtual pot afectar les decisions dels participants. A partir de la segona contribució, en la qual estudiem la percepció del realisme amb robots i humans que realitzen només una animació de caminar o bé realitzen diverses animacions, vam arribar a la conclusió que la combinació de tenir avatars semblants als humans amb animacions variades pot augmentar la percepció del realisme general de la simulació de la multitud, les seves trajectòries i animacions. L'estudi preliminar presentat en la tercera contribució d'aquesta tesi va demostrar que la representació realista de l¿entorn i dels avatars no semblen augmentar la percepció del realisme en els participants, que és coherent amb treballs presentats anteriorment. Els resultats preliminars de la nostra contribució de walk-in-place van mostrar una transició suau i natural entre les metàfores de walk-in-place i caminar normal. El nostre sistema va proporcionar una funció de mapatge de velocitat que s'assembla molt al caminar natural. Hem observat a través d'un estudi pilot que el sistema redueix amb èxit el motion sickness i millora la immersió. Finalment, els resultats de la contribució relacionada amb locomoció en realitat virtual col·laborativa van mostrar que el sincronisme de l'animació i el so dels avatars que representen els participants no semblen tenir un fort impacte en termes de presència i sensació de control de l'avatar. No obstant això, en el nostre experiment, la incorporació d'animacions naturals i el so de passos va donar lloc a valors de clearance més petits en RV que treballs anteriors ja publicats. L'objectiu principal d'aquesta tesi ha estat millorar els diferents factors relacionats amb experiències de realitat virtual immersiva per fer que els participants se sentin més còmodes en l'entorn virtual. Aquests factors inclouen el comportament i l'aparença dels avatars i la navegació a través de l'entorn virtual. En augmentar el realisme dels avatars i facilitar la navegació, s'aconsegueixen altes puntuacions en presència durant les simulacions. Això proporciona un marc ideal per desenvolupar aplicacions col·laboratives de realitat virtual o simulacions d'emergència que requereixen que els participants se sentin com si estiguessin en la vida realPostprint (published version

Investigation of a mechatronic device for the remedial treatment of brain injured children.

To speed the recovery of brain injured children using the method of patterning; it must be made efficient. Efficiency can be achieved by automating the manual method, which will provide the patients with the necessary stimuli needed to help them enhance/restore their natural mobility. This thesis describes research into a novel moderate-cost single-axis Mechatronics device for the remedial treatment of brain injured patients. The device will enhance and/or improve their natural mobility by stimulating the undamaged brain cells responsible for mobility in the central nervous system through physical activity. A detailed review of rehabilitation robotics was undertaken, covering more than seventy projects relating to disabled people. This review helped to identify the main areas of this research regarding the most suitable structure of the machine and setting up the design specifications for the device. A critical investigation of past and present patterning machines and workstations helped avoid the mistakes made by previous designers in not including brain-injured patients in the initial stages of the design. Use of high technology video equipment has made practicable the development of mathematical expressions based on experimental data for the movements of human arms, feet and head. Measurements taken and ergonomic data used made it possible to implement a realistic practical novel kinematic arrangement for the patterning machine. A thorough review of direct drive electrical actuators, and surveys and measurements of the human body with respect to the kinematic arrangements, resulted in the selection of the most appropriate actuator for each axis. The selection of the motor and gearbox was based on the mass of each part of the human body in the prone position, the criteria of high peak torque to motor ratio, low cost, minimum maintenance, safety and compatibility. A computer model of the kinematic arrangement designed was created including the necessary motion constrains, using ADAMS and 3D Working Model simulation packages to test, verify and analyse the static and dynamic stability of the kinematic arrangements and the force interaction between the system and the patient. The simulation results led to some modification in the design regarding the kinematics and dynamic stability of the system by varying different design variables. A walking model of a human was created to simulate the real patient. The model was placed on two units where the feet were the only contact points with the moving belts; the model torso was supported by a harness to hold it in the upright standing position. The results obtained showed the movements of both feet (knees. hips and ankles) in addition to the right and left elbows. The system hardware was designed and implemented using custom-made safety critical software to control the device to carry out the desired tasks. Safety is considered to be one of the main issues that this research program has developed and implemented. An optimal control strategy was developed to drive the prototype. Smooth movements of the system were achieved through a PD control system enhanced with velocity feed forward gain with position accuracy of ± 0.168 mm. The desired positional accuracy of the Patterner Machine was ± 0.632 mm

Development of an Innovative Biomechanical Model for Clinical Gait Analysis

Work contained in this thesis describes a new biomechanical model for clinical gait analysis. Identified problems with current models are related to the subjective reasonableness of their underlying modelling assumptions and used to guide development of a new model. Gait scores were developed to evaluate alternative modelling solutions in terms of cycle average position, range, inter-cycle variability and movement pattern. An overly simple pro-forma model was developed to facilitate evaluation of the effects of complexity from alternative modelling approaches, both existing and newly developed. A clinical interpretation of Euler angles, widely used to describe joint orientations and movements, is presented. Leading directly from this interpretation, simple, joint-specific rules are derived that ensure calculated angles match clinical terminology. Other identified concerns are tackled on a per body-segment basis, with each segment presenting a different challenge. Identified problems with current trunk models were related to difficulties of orientating and tracking movement of the whole based on a selected sub-region. This was solved via the development of a more holistic solution, which additionally reduces the need for patient upper body exposure. For the pelvis and thigh segments, excessive soft tissue cover was identified as the major issue and resolved by the amalgamation of published methods with the novel introduction of fixed length thigh segments and a bespoke axial alignment calibration procedure. Foot model accuracy was improved by the application of bespoke calibration(s) that relates a defining flat foot posture to the position of surface markers, thus reducing the requirement for accurate placement over bony landmarks. Existing multi-segment foot models were grouped by a novel complexity index. Analysis of each group revealed an optimal balance with hind, medial forefoot, and lateral forefoot divisions. A model with this configuration was developed and outputs related to existing clinical terminology describing the foot shape

Swayed by sound: sonic guidance as a neurorehabilitation strategy in the cerebellar ataxias

Cerebellar disease leads to problems in controlling movement. The most common difficulties are dysmetria and instability when standing. Recent understanding of cerebellar function has expanded to include non -motor aspects such as emotional, cognitive and sensory processing. Deficits in the acquisition and processing of sensory information are one explanation for the movement problems observed in cerebellar ataxia. Sensory deficits result in an inability to make predictions about future events; a primary function of the cerebellum. A question therefore, is whether augmenting or replacing sensory information can improve motor performance in cerebellar disease. This question is tested in this thesis by augmenting sensory information through the provision of an auditory movement guide.A variable described in motor control theory (tau) was used to develop auditory guides that were continuous and dynamic. A reaching experiment using healthy individuals showed that the timing of peak velocity, audiomotor coordination accuracy, and velocity of approach, could be altered in line with the movement parameters embedded in the auditory guides. The thesis then investigated the use of these sonic guides in a clinical population with cerebellar disease. Performance on neurorehabilitation exercises for balance control was tested in twenty people with cerebellar atrophy, with and without auditory guides. Results suggested that continuous, predictive, dynamic auditory guidance is an effective way of improving iii movement smoothness in ataxia (as measured by jerk). In addition, generating and swaying with imaginary auditory guides was also found to increase movement smoothness in cerebellar disease.Following the tests of instantaneous effects, the thesis then investigated the longterm consequences on motor behaviour of following a two -month exercise with auditory guide programme. Seven people with cerebellar atrophy were assessed pre - and post -intervention using two measures, weight -shifting and walking. The results of the weight -shifting test indicated that the sonic -guide exercise programme does not initiate long -term changes in motor behaviour. Whilst there were minor, improvements in walking, because of the weight -shifting results, these could not be attributed to the sonic guides. This finding confirms the difficulties of motor rehabilitation in people with cerebellar disease.This thesis contributes original findings to the field of neurorehabilitation by first showing that on -going and predictive stimuli are an appropriate tool for improving motor behaviour. In addition, the thesis is the first of its kind to apply externally presented guides that convey continuous meaningful information within a clinical population. Finally, findings show that sensory augmentation using the auditory domain is an effective way of improving motor coordination in some forms of cerebellar disease

From motion capture to interactive virtual worlds : towards unconstrained motion-capture algorithms for real-time performance-driven character animation

This dissertation takes performance-driven character animation as a representative application and advances motion capture algorithms and animation methods to meet its high demands. Existing approaches have either coarse resolution and restricted capture volume, require expensive and complex multi-camera systems, or use intrusive suits and controllers. For motion capture, set-up time is reduced using fewer cameras, accuracy is increased despite occlusions and general environments, initialization is automated, and free roaming is enabled by egocentric cameras. For animation, increased robustness enables the use of low-cost sensors input, custom control gesture definition is guided to support novice users, and animation expressiveness is increased. The important contributions are: 1) an analytic and differentiable visibility model for pose optimization under strong occlusions, 2) a volumetric contour model for automatic actor initialization in general scenes, 3) a method to annotate and augment image-pose databases automatically, 4) the utilization of unlabeled examples for character control, and 5) the generalization and disambiguation of cyclical gestures for faithful character animation. In summary, the whole process of human motion capture, processing, and application to animation is advanced. These advances on the state of the art have the potential to improve many interactive applications, within and outside virtual reality.Diese Arbeit befasst sich mit Performance-driven Character Animation, insbesondere werden Motion Capture-Algorithmen entwickelt um den hohen Anforderungen dieser Beispielanwendung gerecht zu werden. Existierende Methoden haben entweder eine geringe Genauigkeit und einen eingeschränkten Aufnahmebereich oder benötigen teure Multi-Kamera-Systeme, oder benutzen störende Controller und spezielle Anzüge. Für Motion Capture wird die Setup-Zeit verkürzt, die Genauigkeit für Verdeckungen und generelle Umgebungen erhöht, die Initialisierung automatisiert, und Bewegungseinschränkung verringert. Für Character Animation wird die Robustheit für ungenaue Sensoren erhöht, Hilfe für benutzerdefinierte Gestendefinition geboten, und die Ausdrucksstärke der Animation verbessert. Die wichtigsten Beiträge sind: 1) ein analytisches und differenzierbares Sichtbarkeitsmodell für Rekonstruktionen unter starken Verdeckungen, 2) ein volumetrisches Konturenmodell für automatische Körpermodellinitialisierung in genereller Umgebung, 3) eine Methode zur automatischen Annotation von Posen und Augmentation von Bildern in großen Datenbanken, 4) das Nutzen von Beispielbewegungen für Character Animation, und 5) die Generalisierung und Übertragung von zyklischen Gesten für genaue Charakteranimation. Es wird der gesamte Prozess erweitert, von Motion Capture bis hin zu Charakteranimation. Die Verbesserungen sind für viele interaktive Anwendungen geeignet, innerhalb und außerhalb von virtueller Realität

Engineering for a changing world: 60th Ilmenau Scientific Colloquium, Technische Universität Ilmenau, September 04-08, 2023 : programme

In 2023, the Ilmenau Scientific Colloquium is once more organised by the Department of Mechanical Engineering. The title of this year’s conference “Engineering for a Changing World” refers to limited natural resources of our planet, to massive changes in cooperation between continents, countries, institutions and people – enabled by the increased implementation of information technology as the probably most dominant driver in many fields. The Colloquium, supplemented by workshops, is characterised but not limited to the following topics: – Precision engineering and measurement technology Nanofabrication – Industry 4.0 and digitalisation in mechanical engineering – Mechatronics, biomechatronics and mechanism technology – Systems engineering – Productive teaming - Human-machine collaboration in the production environment The topics are oriented on key strategic aspects of research and teaching in Mechanical Engineering at our university