The maximum output force controller and its application to a virtual surgery system

It is difficult to achieve ideal virtual surgery transparency and stability when virtual tissue stiffness and damping are high. Typically, the stability of the surgery system is improved, while its transparency is sacrificed. In order to achieve high transparency in virtual surgical interactions, a maximum output force controller based on passive theory is proposed in this work. This controller is then applied in a virtual surgery system. The maximum output force controller predicts the maximum allowable output force above which the system passivity is broken and limits the force presented to the operator to this amount. The main contributions of this work include the following two parts: firstly, the maximum output force controller is developed and applied to a virtual surgery system; secondly, a new criterion for transparency is presented and analyzed for the level of transparency that can be achieved for a virtual surgical system when the stability is guaranteed. Experimental results show that the maximum output force controller can guarantee stability of the virtual surgical interaction with maximum transparency even when the virtual tissue stiffness and damping are high. In addition, the maximum output force controller is a self-adaptive controller. It works well without modification, regardless of the virtual tissue stiffness and damping

The Hand-Held Force Magnifier: Surgical Tools to Augment the Sense of Touch

Modern surgeons routinely perform procedures with noisy, sub-threshold, or obscured visual and haptic feedback,either due to the necessary approach, or because the systems on which they are operating are exceeding delicate. For example, in cataract extraction, ophthalmic surgeons must peel away thin membranes in order to access and replace the lens of the eye. Elsewhere, dissection is now commonly performed with energy-delivering tools – rather than sharp blades – and damage to deep structures is possible if tissue contact is not well controlled. Surgeons compensate for their lack of tactile sensibility by relying solely on visual feedback, observing tissue deformation and other visual cues through surgical microscopes or cameras. Using visual information alone can make a procedure more difficult, because cognitive mediation is required to convert visual feedback into motor action. We call this the “haptic problem” in surgery because the human sensorimotor loop is deprived of critical tactile afferent information, increasing the chance for intraoperative injury and requiring extensive training before clinicians reach independent proficiency. Tools that enhance the surgeon’s direct perception of tool-tissue forces can therefore potentially reduce the risk of iatrogenic complications and improve patient outcomes. Towards this end, we have developed and characterized a new robotic surgical tool, the Hand-Held Force Magnifier (HHFM), which amplifies forces at the tool tip so they may be readily perceived by the user, a paradigm we call “in-situ” force feedback. In this dissertation, we describe the development of successive generations of HHFM prototypes, and the evaluation of a proposed human-in-the-loop control framework using the methods of psychophysics. Using these techniques, we have verified that our tool can reduce sensory perception thresholds, augmenting the user’s abilities beyond what is normally possible. Further, we have created models of human motor control in surgically relevant tasks such as membrane puncture, which have shown to be sensitive to push-pull direction and handedness effects. Force augmentation has also demonstrated improvements to force control in isometric force generation tasks. Finally, in support of future psychophysics work, we have developed an inexpensive, high-bandwidth, single axis haptic renderer using a commercial audio speaker

Development and evaluation of a haptic framework supporting telerehabilitation robotics and group interaction

Telerehabilitation robotics has grown remarkably in the past few years. It can provide intensive training to people with special needs remotely while facilitating therapists to observe the whole process. Telerehabilitation robotics is a promising solution supporting routine care which can help to transform face-to-face and one-on-one treatment sessions that require not only intensive human resource but are also restricted to some specialised care centres to treatments that are technology-based (less human involvement) and easy to access remotely from anywhere. However, there are some limitations such as network latency, jitter, and delay of the internet that can affect negatively user experience and quality of the treatment session. Moreover, the lack of social interaction since all treatments are performed over the internet can reduce motivation of the patients. As a result, these limitations are making it very difficult to deliver an efficient recovery plan. This thesis developed and evaluated a new framework designed to facilitate telerehabilitation robotics. The framework integrates multiple cutting-edge technologies to generate playful activities that involve group interaction with binaural audio, visual, and haptic feedback with robot interaction in a variety of environments. The research questions asked were: 1) Can activity mediated by technology motivate and influence the behaviour of users, so that they engage in the activity and sustain a good level of motivation? 2) Will working as a group enhance users’ motivation and interaction? 3) Can we transfer real life activity involving group interaction to virtual domain and deliver it reliably via the internet? There were three goals in this work: first was to compare people’s behaviours and motivations while doing the task in a group and on their own; second was to determine whether group interaction in virtual and reala environments was different from each other in terms of performance, engagement and strategy to complete the task; finally was to test out the effectiveness of the framework based on the benchmarks generated from socially assistive robotics literature. Three studies have been conducted to achieve the first goal, two with healthy participants and one with seven autistic children. The first study observed how people react in a challenging group task while the other two studies compared group and individual interactions. The results obtained from these studies showed that the group interactions were more enjoyable than individual interactions and most likely had more positive effects in terms of user behaviours. This suggests that the group interaction approach has the potential to motivate individuals to make more movements and be more active and could be applied in the future for more serious therapy. Another study has been conducted to measure group interaction’s performance in virtual and real environments and pointed out which aspect influences users’ strategy for dealing with the task. The results from this study helped to form a better understanding to predict a user’s behaviour in a collaborative task. A simulation has been run to compare the results generated from the predictor and the real data. It has shown that, with an appropriate training method, the predictor can perform very well. This thesis has demonstrated the feasibility of group interaction via the internet using robotic technology which could be beneficial for people who require social interaction (e.g. stroke patients and autistic children) in their treatments without regular visits to the clinical centres

Application of ultrasonic motors to MR-compatible haptic interfaces

Functional Magnetic Resonance Imagery (fMRI) is an imaging technique allowing the observation of brain activity. Haptic interfaces can be used in conjunction with fMRI to stimulate the subject while measuring brain activity. Using robotic stimulation over conventional methods offers repeatability, flexibility and the possibility of logging of different experiment variables. Such system becomes a powerful tool for neuroscience study, diagnostic and rehabilitation. The MR scanner with its high magnetic fields and radio frequency pulses is a harsh environment for a robotic system. Robots that can operate safely and do not induce disturbances in the imaging of the scanner are qualified as MR-compatible. The actuation of these robots is an important issue. Electrical power brought to the actuator represents an important source of interferences with the scanner. Since electrical motors cannot be introduced in the MR room, haptic interfaces are conventionally remotely actuated over a long transmission with the actuators placed outside of the MR room. In particular cases, such as the study of finger motion, small haptic interfaces with limited force ranges are required. Remote actuation methods impose transmission lengths and means that cannot be reduced nor scaled down thus imposing a trade-off between performances and size reduction in these applications. This work investigates an alternative actuator that can achieve high-quality force-interactions with the fingers. The Ultrasonic Motor (USM) is MR-compatible and offers good performances. But it is not well suited for force-feedback and may be hazardous for the users. To address these issues, mechanical solutions are investigated by using an electrical analogy applied to mechanical systems. A novel actuation system using the USM as a power source and a clutch to control the output torque is proposed: the Hybrid USM Clutch Actuator (HUCA). The first prototype validates the different mechanical concepts developed in this work. The second, MR-compatible, integrates a clutch based on electrorheological fluids (ER). MR-compatibility has been validated and performances evaluated. Since the HUCA has the unique property of behaving both like a force source and a velocity source, dedicated control schemes have been developed to implement impedance and admittance force control. These enable the display of stiff walls and the rendering of a wide range of impedances thanks to the overlap of their range of displayable impedances. Compared to the hydrostatic transmission actuation, the HUCA shows higher performances and user safety. Furthermore, the powering through electrical wires allows developments of multi-DOF interfaces

Human Motor Control and the Design and Control of Backdriveable Actuators for Human-Robot Interaction

The design of the control and hardware systems for a robot intended for interaction with a human user can profit from a critical analysis of the human neuromotor system and human biomechanics. The primary observation to be made about the human control and ``hardware’’ systems is that they work well together, perhaps because they were designed for each other. Despite the limited force production and elasticity of muscle, and despite slow information transmission, the sensorimotor system is adept at an impressive range of motor behaviors. In this thesis I present three explorations on the manners in which the human and hardware systems work together, hoping to inform the design of robots suitable for human-robot interaction. First, I used the serial reaction time (SRT) task with cuing from lights and motorized keys to assess the relative contribution of visual and haptic stimuli to the formation of motor and perceptual memories. Motorized keys were used to deliver brief pulse-like displacements to the resting fingers, with the expectation that the proximity and similarity of these cues to the response motor actions (finger-activated key-presses) would strengthen the motor memory trace in particular. Error rate results demonstrate that haptic cues promote motor learning over perceptual learning. The second exploration involves the design of an actuator specialized for human-robot interaction. Like muscle, it features series elasticity and thus displays good backdrivability. The elasticity arises from the use of a compressible fluid while hinged rigid plates are used to convert fluid power into mechanical power. I also propose impedance control with dynamics compensation to further reduce the driving-point impedance. The controller is robust to all kinds of uncertainties. The third exploration involves human control in interaction with the environment. I propose a framework that accommodates delays and does not require an explicit model of the musculoskeletal system and environment. Instead, loads from the biomechanics and coupled environment are estimated using the relationship between the motor command and its responses. Delays inherent in sensory feedback are accommodated by taking the form of the Smith predictor. Agreements between simulation results and empirical movements suggests that the framework is viable.PhDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/120675/1/gloryn_1.pd

XR, music and neurodiversity: design and application of new mixed reality technologies that facilitate musical intervention for children with autism spectrum conditions

This thesis, accompanied by the practice outputs,investigates sensory integration, social interaction and creativity through a newly developed VR-musical interface designed exclusively for children with a high-functioning autism spectrum condition (ASC).The results aim to contribute to the limited expanse of literature and research surrounding Virtual Reality (VR) musical interventions and Immersive Virtual Environments (IVEs) designed to support individuals with neurodevelopmental conditions. The author has developed bespoke hardware, software and a new methodology to conduct field investigations. These outputs include a Virtual Immersive Musical Reality Intervention (ViMRI) protocol, a Supplemental Personalised, immersive Musical Experience(SPiME) programme, the Assisted Real-time Three-dimensional Immersive Musical Intervention System’ (ARTIMIS) and a bespoke (and fully configurable) ‘Creative immersive interactive Musical Software’ application (CiiMS). The outputs are each implemented within a series of institutional investigations of 18 autistic child participants. Four groups are evaluated using newly developed virtual assessment and scoring mechanisms devised exclusively from long-established rating scales. Key quantitative indicators from the datasets demonstrate consistent findings and significant improvements for individual preferences (likes), fear reduction efficacy, and social interaction. Six individual case studies present positive qualitative results demonstrating improved decision-making and sensorimotor processing. The preliminary research trials further indicate that using this virtual-reality music technology system and newly developed protocols produces notable improvements for participants with an ASC. More significantly, there is evidence that the supplemental technology facilitates a reduction in psychological anxiety and improvements in dexterity. The virtual music composition and improvisation system presented here require further extensive testing in different spheres for proof of concept

Uma arquitetura de telerreabilitação baseada em realidade aumentada para apoiar o treinamento de usuários de cadeiras de rodas motorizadas

Many people worldwide have been experimenting a decrease in their mobility as a result of aging, accidents and degenerative diseases. In many cases, a Powered Wheelchair (PW) is an alternative help. Currently, in Brazil, patients can receive a PW from the Unified Health System, following prescription criteria. However, they do not have an appropriate previous training for driving the PW. Consequently, users might suffer accidents since a customized training protocol is not available. Nevertheless, due to financial and/or health limitations, many users are unable to attend a rehabilitation center. To overcome these limitations, we developed an Augmented Reality (AR) Telerehabilitation System Architecture based on the Power Mobility Road Test (PMRT), for supporting PW user’s training. In this system, the therapists can remotely customize and evaluate training tasks and the user can perform the training in safer conditions. Video stream and data transfer between each environment were made possible through UDP (User Datagram Protocol). To evaluate and present the system architecture potential, a preliminary test was conducted with 3 spinal cord injury participants. They performed 3 basic training protocols defined by a therapist. The following metrics were adopted for evaluation: number of control commands; elapsed time; number of collisions; biosignals and a questionary was used to evaluate system features by participants. Results demonstrate the specific needs of individuals using a PW, thanks to adopted (qualitative and emotional) metrics. Also, the results have shown the potential of the training system with customizable protocols to fulfill these needs. User’s evaluation demonstrates that the combination of AR techniques with PMRT adaptations, increases user’s well-being after training sessions. Furthermore, a training experience helps users to overcome their displacement problems, as well as for appointing challenges before large scale use. The proposed system architecture allows further studies on telerehabilitation of PW users.CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível SuperiorTese (Doutorado)Muitas pessoas em todo o mundo estão vivenciando uma diminuição de sua mobili- dade como resultado de envelhecimento, acidentes e doenças degenerativas. Em muitos casos, uma cadeira de rodas motorizada (CRM) é uma ajuda alternativa. Atualmente, no Brasil, os pacientes podem receber uma CRM do Sistema Único de Saúde, seguindo os critérios de prescrição. No entanto, eles não têm um treinamento prévio apropriado para dirigir a CRM. Conseqüentemente, os usuários podem sofrer acidentes, pois um protocolo de treinamento personalizado não está disponível. Além disto, devido a limi- tações financeiras e / ou de saúde, muitos usuários não podem comparecer a um centro de reabilitação. Para superar essas limitações, desenvolvemos uma arquitetura de sistema de telereabilitação com Realidade Aumentada (RA) baseado no PMRT (Power Mobility Road Test), para apoiar o treinamento de usuários de CRM. Nesse sistema, os terapeutas podem personalizar e avaliar remotamente as tarefas de treinamento e o usuário pode realizar o treinamento em condições mais seguras. O fluxo de vídeo e a transferência de dados entre cada ambiente foram possíveis através do UDP (User Datagram Protocol). Para avaliar e apresentar o potencial da arquitetura do sistema, foi realizado um teste preliminar de três participantes com lesão medular. Eles realizaram três protocolos básicos de treinamento definidos por um terapeuta. As seguintes métricas adotadas para avaliação foram: número de comandos de controle; tempo decorrido; número de colisões e biossinais. Além disso, um questionário foi usado para avaliar os recursos do sistema. Os resultados demonstram as necessidades específicas dos indivíduos que usam uma CRM, graças às métricas adotadas (qualitativas e emocionais). Além disso, os resultados mostraram o potencial do sistema de treinamento com protocolos personalizáveis para atender a essas necessidades. A avaliação do usuário demonstra que a combinação de técnicas de RA com adaptações PMRT aumenta o bem-estar do usuário após as sessões de treinamento. Além disso, esta experiência de treinamento ajuda os usuários a superar seus problemas de deslocamento, bem como a apontar desafios antes do uso em larga escala. A arquitetura de sistema proposta, permite estudos adicionais sobre a telerreabilitação de usuários de CRM

Aerial Vehicles

This book contains 35 chapters written by experts in developing techniques for making aerial vehicles more intelligent, more reliable, more flexible in use, and safer in operation.It will also serve as an inspiration for further improvement of the design and application of aeral vehicles. The advanced techniques and research described here may also be applicable to other high-tech areas such as robotics, avionics, vetronics, and space

Motion cueing in driving simulators for research applications

This research investigated the perception of self-motion in driving simulation, focussing on the dynamic cues produced by a motion platform. The study was undertaken in three stages, evaluating various motion cueing techniques based on both subjective ratings of realism and objective measures of driver performance. Using a Just Noticeable Difference methodology, Stage 1 determined the maximum perceptible motion scaling for platform movement in both translation and tilt. Motion cues scaled by 90% or more could not be perceptibly differentiated from unscaled motion. This result was used in Stage 2‟s examination of the most appropriate point in space at which the platform translations and rotations should be centred (Motion Reference Point, MRP). Participants undertook two tracking tasks requiring both longitudinal (braking) and lateral (steering) vehicle control. Whilst drivers appeared unable to perceive a change in MRP from head level to a point 1.1m lower, the higher position (closer to the vestibular organs) did result in marginally smoother braking, corresponding to the given requirements of the longitudinal driving task. Stage 3 explored the perceptual trade-off between the specific force error and tilt rate error generated by the platform. Three independent experimental factors were manipulated: motion scale-factor, platform tilt rate and additional platform displacement afforded by a XY-table. For the longitudinal task, slow tilt that remained sub-threshold was perceived as the most realistic, especially when supplemented by the extra surge of the XY-table. However, braking task performance was superior when a more rapid tilt was experienced. For the lateral task, perceived realism was enhanced when motion cues were scaled by 50%, particularly with added XY-sway. This preference was also supported by improvements in task accuracy. Participants ratings were unmoved by changing tilt rate, although rapid tilt did result in more precise lane control. Several interactions were also observed, most notably between platform tilt rate and XY-table availability. When the XY-table was operational, driving task performance varied little between sub-threshold and more rapid tilt. However, while the XY-table was inactive, both driving tasks were better achieved in conditions of high tilt rate. An interpretation of these results suggests that without the benefit of significant extra translational capability, priority should be given to the minimisation of specific force error through motion cues presented at a perceptibly high tilt rate. However, XY-table availability affords the simulator engineer the luxury of attaining a slower tilt that provides both accurate driving task performance and accomplishes maximum perceived realism