A Hybrid Visual Control Scheme to Assist the Visually Impaired with Guided Reaching Tasks

In recent years, numerous researchers have been working towards adapting technology developed for robotic control to use in the creation of high-technology assistive devices for the visually impaired. These types of devices have been proven to help visually impaired people live with a greater degree of confidence and independence. However, most prior work has focused primarily on a single problem from mobile robotics, namely navigation in an unknown environment. In this work we address the issue of the design and performance of an assistive device application to aid the visually-impaired with a guided reaching task. The device follows an eye-in-hand, IBLM visual servoing configuration with a single camera and vibrotactile feedback to the user to direct guided tracking during the reaching task. We present a model for the system that employs a hybrid control scheme based on a Discrete Event System (DES) approach. This approach avoids significant problems inherent in the competing classical control or conventional visual servoing models for upper limb movement found in the literature. The proposed hybrid model parameterizes the partitioning of the image state-space that produces a variable size targeting window for compensatory tracking in the reaching task. The partitioning is created through the positioning of hypersurface boundaries within the state space, which when crossed trigger events that cause DES-controller state transition that enable differing control laws. A set of metrics encompassing, accuracy ($D$), precision ($\theta_{e}$), and overall tracking performance ($\psi$) are also proposed to quantity system performance so that the effect of parameter variations and alternate controller configurations can be compared. To this end, a prototype called \texttt{aiReach} was constructed and experiments were conducted testing the functional use of the system and other supporting aspects of the system behaviour using participant volunteers. Results are presented validating the system design and demonstrating effective use of a two parameter partitioning scheme that utilizes a targeting window with additional hysteresis region to filtering perturbations due to natural proprioceptive limitations for precise control of upper limb movement. Results from the experiments show that accuracy performance increased with the use of the dual parameter hysteresis target window model ($0.91 \leq D \leq 1$, $\mu(D)=0.9644$, $\sigma(D)=0.0172$) over the single parameter fixed window model ($0.82 \leq D \leq 0.98$, $\mu(D)=0.9205$, $\sigma(D)=0.0297$) while the precision metric, $\theta_{e}$, remained relatively unchanged. In addition, the overall tracking performance metric produces scores which correctly rank the performance of the guided reaching tasks form most difficult to easiest

HUMAN CONTROL OF ROBOTIC MECHANISMS: MODELLING AND ASSESSMENT OF ASSISTIVE DEVICES

The prescription and use of Assistive Technology, particularly teleprostheses, may be enhanced by the use of standard assessment techniques. For input devices, in particular, existing assessment studies, most of which are based on Fitts' Law, have produced contradictory results. This thesis has made contributions to these and related fields, particularly in the following four areas. Fitts' Law (and background information theory) is examined. The inability of this paradigm to match experimental results is noted and explained. Following a review of the contributing fields, a new method of assessing input devices is proposed, based on Fitts' Law, classical control and the concept of 'profiling'. To determine the suitability of the proposed method, it is applied to the results of over 2000 trials. The resulting analysis emphasises the importance of interaction effects and their influence on general comparison techniques for input devices. The process of verification has highlighted gain susceptability as a performance criterion which reflects user susceptability; a technique which may be particularly applicable to Assistive Technology.Dept. of Mechanical and Marine Engineerin

The BUMP model of response planning: a neuroengineering account of speed-accuracy tradeoffs, velocity profiles, and physiological tremor in movement

Speed-accuracy tradeoffs, velocity profiles, and physiological tremor are fundamental characteristics of human movement. The principles underlying these phenomena have long attracted major interest and controversy. Each is well established experimentally but as yet they have no common theoretical basis. It is proposed that these three phenomena occur as the direct consequence of a movement response planning system that acts as an intermittent optimal controller operating at discrete intervals of ~100 ms. The BUMP model of response planning describes such a system. It forms the kernel of adaptive model theory which defines, in computational terms, a basic unit of motor production or BUMP. Each BUMP consists of three processes: (i) analysing sensory information, (ii) planning a desired optimal response, and (iii) executing that response. These processes operate in parallel across successive sequential BUMPs. The response planning process requires a discrete time interval in which to generate a minimum acceleration trajectory of variable duration, or horizon, to connect the actual response with the predicted future state of the target and compensate for executional error. BUMP model simulation studies show that intermittent adaptive optimal control employing two extremes of variable horizon predictive control reproduces almost exactly findings from several authoritative human experiments. On the one extreme, simulating spatially-constrained movements, a receding horizon strategy results in a logarithmic speed-accuracy tradeoff and accompanying asymmetrical velocity profiles. On the other extreme, simulating temporally-constrained movements, a fixed horizon strategy results in a linear speed-accuracy tradeoff and accompanying symmetrical velocity profiles. Furthermore, simulating ramp movements, a receding horizon strategy closely reproduces experimental observations of 10 Hz physiological tremor. A 100 ms planning interval yields waveforms and power spectra equivalent to those of joint-angle, angular velocity and electromyogram signals recorded for several speeds, directions, and skill levels of finger movement. While other models of response planning account for one or other set of experimentally observed features of speed-accuracy tradeoffs, velocity profiles, and physiological tremor, none accounts for all three. The BUMP model succeeds in explaining these disparate movement phenomena within a single framework, strengthening this approach as the foundation for a unified theory of motor control and planning

Modelling of Human Control and Performance Evaluation using Artificial Neural Network and Brainwave

Conventionally, a human has to learn to operate a machine by himself / herself. Human Adaptive Mechatronics (HAM) aims to investigate a machine that has the capability to learn its operator skills in order to provide assistance and guidance appropriately. Therefore, the understanding of human behaviour during the human-machine interaction (HMI) from the machine’s side is essential. The focus of this research is to propose a model of human-machine control strategy and performance evaluation from the machine’s point of view. Various HAM simulation scenarios are developed for the investigations of the HMI. The first case study that utilises the classic pendulum-driven capsule system reveals that a human can learn to control the unfamiliar system and summarise the control strategy as a set of rules. Further investigation of the case study is conducted with nine participants to explore the performance differences and control characteristics among them. High performers tend to control the pendulum at high frequency in the right portion of the angle range while the low performers perform inconsistent control behaviour. This control information is used to develop a human-machine control model by adopting an Artificial Neural Network (ANN) and 10-time- 10-fold cross-validation. Two models of capsule direction and position predictions are obtained with 88.3% and 79.1% accuracies, respectively. An Electroencephalogram (EEG) headset is integrated into the platform for monitoring brain activity during HMI. A number of preliminary studies reveal that the brain has a specific response pattern to particular stimuli compared to normal brainwaves. A novel human-machine performance evaluation based on the EEG brainwaves is developed by utilising a classical target hitting task as a case study of HMI. Six models are obtained for the evaluation of the corresponding performance aspects including the Fitts index of performance. The averaged evaluation accuracy of the models is 72.35%. However, the accuracy drops to 65.81% when the models are applied to unseen data. In general, it can be claimed that the accuracy is satisfactory since it is very challenging to evaluate the HMI performance based only on the EEG brainwave activity

Augmenting user capabilities through an adaptive assistive manipulator

Mención Internacional en el título de doctorAssistive robot manipulators have the potential to increase the independence of disabled persons in activities of daily living. The current designs are mainly limited to pure teleoperation by the user, given the need for keeping the user in the control loop, and the complexity of the tasks and environments in which they operate. This thesis aims to augment the user’s capabilities for performing such tasks by adapting the robot, and its level of assistance, to the user. Methodologies for modeling and benchmarking the complete human-robot system were established, which helped drive the development of different approaches to adaptation. This included a task-oriented optimization of the robot physical structure, approaches for low-level adaptive shared control, and work on interactive learning of, and assistance on completing, simple object manipulation tasks. Three experimental platforms were used: The ASIBOT manipulator of Universidad Carlos III de Madrid (UC3M), the AMOR manipulator of Exact Dynamics, and the iCub humanoid robot.Los manipuladores asistenciales tienen el potencial de incrementar la independencia de personas discapacitadas en sus actividades de la vida diaria. Los diseños actuales se limitan principalmente a una pura teleoperación, pues dada la complejidad de las tareas y del entorno, se necesita mantener al usuario en el lazo de control. Esta tesis pretende mejorar las capacidades del usuario para realizar estas tareas, adaptando el robot y su nivel de asistencia a las necesidades del usuario. Se han establecido metodologías para el modelado y evaluación del comportamiento del sistema formado por humano y robot, lo que ha permitido el desarrollo de diferentes aproximaciones a la adaptación. Esto incluye desde la optimización de la estructura del robot atendiendo a las tareas, la evaluación de diversas aproximaciones al control compartido adaptativo a bajo nivel, al aprendizaje interactivo y el desarrollo de asistencias para completar tareas sencillas de manipulación. Se ha hecho uso de tres plataformas experimentales: el manipulador ASIBOT de la Universidad Carlos III de Madrid (UC3M), el manipulador AMOR de Exact Dynamics y el humanoide iCub.Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Alberto Sanfeliú.- Secretario: Concepción Alicia Monje Micharet.- Vocal: Yiannis Demiri

Haptic communication between partner dancers and swing as a finite state machine

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2005.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Vita.Includes bibliographical references (p. 129-138).To see two expert partners, one leading and the other following, swing dance together is to watch a remarkable two-agent communication and control system in action. Even blindfolded, the follower can decode the leader's moves from haptic cues. The leader composes the dance from the vocabulary of known moves so as to complement the music he is dancing to. Systematically addressing questions about partner dance communication is of scientific interest and could improve human-robotic interaction, and imitating the leader's choreographic skill is an engineering problem with applications beyond the dance domain. Swing dance choreography is a finite state machine, with moves that transition between a small number of poses. Two automated choreographers are presented. One uses an optimization and randomization scheme to compose dances by a sequence of shortest path problems, with edge lengths measuring the dissimilarity of dance moves to each bar of music. The other solves a two-player zero-sum game between the choreographer and a judge. Choosing moves at random from among moves that are good enough is rational under the game model.(cont.) Further, experiments presenting conflicting musical environments to two partners demonstrate that although musical expression clearly guides the leader's choice of moves, the follower need not hear the same music to properly decode the leader's signals. Dancers embody gentle interaction, in which each participant extends the capabilities of the other, and their cooperation is facilitated by a shared understanding of the motions to be performed. To demonstrate that followers use their understanding of the move vocabulary to interact better with their leaders, an experiment paired a haptic robot leader with human followers in a haptically cued dance to a swing music soundtrack. The subjects' performance differed significantly between instances when the subjects could determine which move was being led and instances when the subjects could not determine what the next move would be. Also, two-person teams that cooperated haptically to perform cyclical aiming tasks showed improvements in the Fitts' law or Schmidt's law speed-accuracy tradeoff consistent with a novel endpoint compromise hypothesis about haptic collaboration.by Sommer Elizabeth Gentry.Ph.D

Modelling Human-Driver Behaviour Using a Biofidelic Approach

This dissertation is concerned with the subject of modelling human steering control of ground vehicles. Special care has been taken with respect to designing a model that is biofidelic, i.e., a model that operates according to the principles of human control. With this aim, first classical human control theory has been revisited, both from a literature review and an experimental perspective; data have been recorded from test subjects in compensatory and pursuit tracking tasks. The tracking experiments are the first ever to be performed with fractional order plants, which are plants suitable to represent system memory. From the data, an extension of the Crossover model by McRuer’s is designed, to include the control of such category of plants. The proposed model is referred to as the Fractional Crossover Model. This is followed by a study on modelling memory in human-machine systems from a classical control theory viewpoint. These results broaden the existing array of manual control modelling techniques and can be employed in a modular manner, combined with current models. More significantly – and still with respect to the domain of generic human control and human-machine systems – a new approach for modelling the human-operator is proposed. This approach consists in treating the problem from a statistical viewpoint. With this methodology a novel human control model based on multiplicative dynamics is presented. The model, which was inspired on actual results in neuroscience, is validated with the tracking data obtained from test subjects and by comparing it to classical models in the literature. Hence the model is useful to analyse human performance or to reproduce human control in simulation, field tests or in the video game industry. With respect to steering control modelling, which is the main topic of this dissertation, additional experiments with test subjects were conducted in a simple vehicle simulator – with hardware and software specifically developed during this research program to test multiple hypotheses. The data were analysed with the intent of identifying which optical variables drivers employ while controlling a vehicle on public roads; it is seen that the splay angles– which are the projections of the road lines on the retina – are likely candidates for lane keeping at low speeds. This brings on a novel human-centred driver model first proposed here. This model includes multiplicative human control over the splay angles, and far-point error perception for lane keeping at higher speeds. The human-centred model has its domain of applicability in the intelligent transportation industry, in particular for the development of shared control systems and advanced driver-assistance systems for semi-autonomous ground vehicles. Additionally, the model can be employed in field testing of ground vehicles – for example, in vehicle durability tests. Furthermore, the topic of alternative steering devices for driving autonomous and semi-autonomous vehicles is investigated. This leads to another of the contributions in this dissertation. Here it is proposed that for such vehicles, and for the control of systems with a shared control perspective, anisometric steering wheel can be advantageous under certain schemes – tight rein or loose rein modes according to the H-metaphor. This is supported by additional data collected in the driving simulation experiments. Resulting from this, fractional order transfer functions are employed to increment steering stability and control accuracy with the isometric device. This prototypical steering system is applicable for the control of ground vehicles with the so-called by-wire controls, which are already incorporated in some commercially available vehicles

Human-robot interaction using a behavioural control strategy

PhD ThesisA topical and important aspect of robotics research is in the area of human-robot interaction (HRI), which addresses the issue of cooperation between a human and a robot to allow tasks to be shared in a safe and reliable manner. This thesis focuses on the design and development of an appropriate set of behaviour strategies for human-robot interactive control by first understanding how an equivalent human-human interaction (HHI) can be used to establish a framework for a robotic behaviour-based approach. To achieve the above goal, two preliminary HHI experimental investigations were initiated in this study. The first of which was designed to evaluate the human dynamic response using a one degree-of-freedom (DOF) HHI rectilinear test where the handler passes a compliant object to the receiver along a constrained horizontal path. The human dynamic response while executing the HHI rectilinear task has been investigated using a Box-Behnken design of experiments [Box and Hunter, 1957] and was based on the McRuer crossover model [McRuer et al. 1995]. To mimic a real-world human-human object handover task where the handler is able to pass an object to the receiver in a 3D workspace, a second more substantive one DOF HHI baton handover task has been developed. The HHI object handover tests were designed to understand the dynamic behavioural characteristics of the human participants, in which the handler was required to dexterously pass an object to the receiver in a timely and natural manner. The profiles of interactive forces between the handler and receiver were measured as a function of time, and how they are modulated whilst performing the tasks, was evaluated. Three key parameters were used to identify the physical characteristics of the human participants, including: peak interactive force (fmax), transfer time (Ttrf), and work done (W). These variables were subsequently used to design and develop an appropriate set of force and velocity control strategies for a six DOF Stäubli robot manipulator arm (TX60) working in a human-robot interactive environment. The optimal design of the software and hardware controller implementation for the robot system has been successfully established in keeping with a behaviour-based approach. External force control based on proportional plus integral (PI) and fuzzy logic control (FLC) algorithms were adopted to control the robot end effector velocity and interactive force in real-time. ii The results of interactive experiments with human-to-robot and robot-to-human handover tasks allowed a comparison of the PI and FLC control strategies. It can be concluded that the quantitative measurement of the performance of robot velocity and force control can be considered acceptable for human-robot interaction. These can provide effective performance during the robot-human object handover tasks, where the robot was able to successfully pass the object from/to the human in a safe, reliable and timely manner. However, after careful analysis with regard to human-robot handover test results, the FLC scheme was shown to be superior to PI control by actively compensating for the dynamics in the non-linear system and demonstrated better overall performance and stability. The FLC also shows superior performance in terms of improved sensitivity to small error changes compared to PI control, which is an advantage in establishing effective robot force control. The results of survey responses from the participants were in agreement with the parallel test outcomes, demonstrating significant satisfaction with the overall performance of the human-robot interactive system, as measured by an average rating of 4.06 on a five point scale. In brief, this research has contributed the foundations for long-term research, particularly in the development of an interactive real-time robot-force control system, which enables the robot manipulator arm to cooperate with a human to facilitate the dextrous transfer of objects in a safe and speedy manner.Thai government and Prince of Songkla University (PSU

Nineteenth Annual Conference on Manual Control

No abstract availabl