Haptic Feedback Effects on Human Control of a UAV in a Remote Teleoperation Flight Task

The remote manual teleoperation of an unmanned aerial vehicle (UAV) by a human operator creates a human-in-the loop system that is of great concern. In a remote teleoperation task, a human pilot must make control decisions based upon sensory information provided by the governed system. Often, this information consists of limited visual feedback provided by onboard cameras that do not provide an operator with an accurate portrayal of their immediate surroundings compromising the safety of the mobile robot. Due to this shortfall, haptic force feedback is often provided to the human in an effort to increase their perceptual awareness of the surrounding world. To investigate the effects of this additional sensory information provided to the human op-erator, we consider two haptic force feedback strategies. They were designed to provide either an attractive force to influence control behavior towards a reference trajectory along a flight path, or a repulsive force directing operators away from obstacles to prevent collision. Subject tests were con-ducted where human operators manually operated a remote UAV through a corridor environment under the conditions of the two strategies. For comparison, the conditions of no haptic feedback and the liner combination of both attractive and repulsive strategies were included in the study. Experi-mental results dictate that haptic force feedback in general (including both attractive and repulsive force feedback) improves the average distance from surrounding obstacles up to 21%. Further statis-tical comparison of repulsive and attractive feedback modalities reveal that even though a repulsive strategy is based directly on obstacles, an attractive strategy towards a reference trajectory is more suitable across all performance metrics. To further examine the effects of haptic aides in a UAV teleoperation task, the behavior of the human system as part of the control loop was also investigated. Through a novel device placed on the end effector of the haptic device, human-haptic interaction forces were captured and further analyzed. With this information, system identification techniques were carried out to determine the plausibility of deriving a human control model for the system. By defining lateral motion as a one-dimensional compensatory tracking task the results show that general human control behavior can be identified where lead compensation in invoked to counteract second-order UAV dynamics

Evaluation of Haptic and Visual Cues for Repulsive or Attractive Guidance in Nonholonomic Steering Tasks.

Remote control of vehicles is a difficult task for operators. Support systems that present additional task information may assist operators, but their usefulness is expected to depend on several factors such as 1) the nature of conveyed information, 2) what modality it is conveyed through, and 3) the task difficulty. In an exploratory experiment, these three factors were manipulated to quantify their effects on operator behavior. Subjects ( $n = {{15}}$) used a haptic manipulator to steer a virtual nonholonomic vehicle through abstract environments, in which obstacles needed to be avoided. Both a simple support conveying near-future predictions of the trajectory of the vehicle and a more elaborate support that continuously suggests the path to be taken were designed (factor 1). These types of information were offered either with visual or haptic cues (factor 2). These four support systems were tested in four different abstracted environments with decreasing amount of allowed variability in realized trajectories (factor 3). The results show improvements for the simple support only when this information was presented visually, but not when offered haptically. For the elaborate support, equally large improvements for both modalities were found. This suggests that the elaborate support is better: additional information is key in improving performance in nonholonomic steering tasks

Aerial Vehicles

This book contains 35 chapters written by experts in developing techniques for making aerial vehicles more intelligent, more reliable, more flexible in use, and safer in operation.It will also serve as an inspiration for further improvement of the design and application of aeral vehicles. The advanced techniques and research described here may also be applicable to other high-tech areas such as robotics, avionics, vetronics, and space

SWORD: um dispositivo vibratório vestível para uma rabilitação mais eficiente em doentes com AVC

Doutoramento em Engenharia ElectrotécnicaAnualmente ocorrem cerca de 16 milhões AVCs em todo o mundo. Cerca de metade dos sobreviventes irá apresentar défice motor que necessitará de reabilitação na janela dos 3 aos 6 meses depois do AVC. Nos países desenvolvidos, é estimado que os custos com AVCs representem cerca de 0.27% do Produto Interno Bruto de cada País. Esta situação implica um enorme peso social e financeiro. Paradoxalmente a esta situação, é aceite na comunidade médica a necessidade de serviços de reabilitação motora mais intensivos e centrados no doente. Na revisão do estado da arte, demonstra-se o arquétipo que relaciona metodologias terapêuticas mais intensivas com uma mais proficiente reabilitação motora do doente. Revelam-se também as falhas nas soluções tecnológicas existentes que apresentam uma elevada complexidade e custo associado de aquisição e manutenção. Desta forma, a pergunta que suporta o trabalho de doutoramento seguido inquire a possibilidade de criar um novo dispositivo de simples utilização e de baixo custo, capaz de apoiar uma recuperação motora mais eficiente de um doente após AVC, aliando intensidade com determinação da correcção dos movimentos realizados relativamente aos prescritos. Propondo o uso do estímulo vibratório como uma ferramenta proprioceptiva de intervenção terapêutica a usar no novo dispositivo, demonstra-se a tolerabilidade a este tipo de estímulos através do teste duma primeira versão do sistema apenas com a componente de estimulação num primeiro grupo de 5 doentes. Esta fase validará o subsequente desenvolvimento do sistema SWORD. Projectando o sistema SWORD como uma ferramenta complementar que integra as componentes de avaliação motora e intervenção proprioceptiva por estimulação, é descrito o desenvolvimento da componente de quantificação de movimento que o integra. São apresentadas as diversas soluções estudadas e o algoritmo que representa a implementação final baseada na fusão sensorial das medidas provenientes de três sensores: acelerómetro, giroscópio e magnetómetro. O teste ao sistema SWORD, quando comparado com o método de reabilitação tradicional, mostrou um ganho considerável de intensidade e qualidade na execução motora para 4 dos 5 doentes testados num segundo grupo experimental. É mostrada a versatilidade do sistema SWORD através do desenvolvimento do módulo de Tele-Reabilitação que complementa a componente de quantificação de movimento com uma interface gráfica de feedback e uma ferramenta de análise remota da evolução motora do doente. Finalmente, a partir da componente de quantificação de movimento, foi ainda desenvolvida uma versão para avaliação motora automatizada, implementada a partir da escala WMFT, que visa retirar o factor subjectivo da avaliação humana presente nas escalas de avaliação motora usadas em Neurologia. Esta versão do sistema foi testada num terceiro grupo experimental de cinco doentes.About 16 million first ever-strokes occur worldwide every year. Half of stroke survivors are left with some degree of physical impairment that needs rehabilitation in the 3 to 6 month after-stroke time window. This situation implies a high economic and social burden. In developed countries, stroke cost is estimated to represent an average of 0.27% of each country’s gross domestic product. Paradoxically, it is accepted in the medical community the need for more intensive and patient-centered rehabilitation services. In the state of art review, it is demonstrated the archetype that relates the intensity on rehabilitation with a proficient motor recovery of the patient. Additionally, it is shown that the major pitfalls in current technological solutions in the field of motor rehabilitation are due to their intrinsic complexity and associated cost. Given this state of the art, the research question that supports this thesis, inquiries the possibility of creating a novel low-cost device targeted at the motor rehabilitation of stroke patients, capable of providing a more efficient treatment through enabling higher intensity and automated determination of the correctness of the movements performed by the recovering patient. The validity of the vibratory stimulus is presented from an historic and neurophysiologic point of view. Furthermore, a state of art review of motion capture systems is presented. Intending the use of the vibratory stimulus as a proprioceptive therapeutic tool to be integrated in the new device, it is demonstrated the tolerability of the stimulus from the experimental test of a first version of the device, incorporating the stimulation component, in a first group of five patients. Projecting the SWORD device as a tool that combines both features of motor function evaluation with proprioceptive intervention through vibratory stimulation, it is described the development of the motion capture component. Several solutions were studied and the final algorithm, based on the sensory fusion of the measures from three sensors (accelerometer, gyroscope and magnetometer), is described in detail. The experimental test of the SWORD system on a second group of patients showed that, when compared with a typical treatment, it is capable of providing a more intensive intervention and with a higher quality in 4 out of 5 patients. To demonstrate the versatility of the SWORD system, it was developed the tele-rehabilitation module that complements the motion capture component with a graphical feedback interface and a remote tool for the clinician to evaluate the performance of the patient through out the time he uses the system in his home or any other remote environment. Finally, from the motion capture component, a motor function evaluation version of the system was deployed. Implemented from the WMFT scale, it aims to eliminate the human subjectivity present in the traditional evaluation scales used in the neurology medical area. This system was evaluated on a third group of five patients

Limited Information Shared Control and its Applications to Large Vehicle Manipulators

Diese Dissertation beschäftigt sich mit der kooperativen Regelung einer mobilen Arbeitsmaschine, welche aus einem Nutzfahrzeug und einem oder mehreren hydraulischen Manipulatoren besteht. Solche Maschinen werden für Aufgaben in der Straßenunterhaltungsaufgaben eingesetzt. Die Arbeitsumgebung des Manipulators ist unstrukturiert, was die Bestimmung einer Referenztrajektorie erschwert oder unmöglich macht. Deshalb wird in dieser Arbeit ein Ansatz vorgeschlagen, welcher nur das Fahrzeug automatisiert, während der menschliche Bediener ein Teil des Systems bleibt und den Manipulator steuert. Eine solche Teilautomatisierung des Gesamtsystems führt zu einer speziellen Klasse von Mensch-Maschine-Interaktionen, welche in der Literatur noch nicht untersucht wurde: Eine kooperative Regelung zwischen zwei Teilsystemen, bei der die Automatisierung keine Informationen von dem vom Menschen gesteuerten Teilsystem hat. Deswegen wird in dieser Arbeit ein systematischer Ansatz der kooperativen Regelung mit begrenzter Information vorgestellt, der den menschlichen Bediener unterstützen kann, ohne die Referenzen oder die Systemzustände des Manipulators zu messen. Außerdem wird ein systematisches Entwurfskonzept für die kooperative Regelung mit begrenzter Information vorgestellt. Für diese Entwurfsmethode werden zwei neue Unterklassen der sogenannten Potenzialspiele eingeführt, die eine systematische Berechnung der Parameter der entwickelten kooperativen Regelung ohne manuelle Abstimmung ermöglichen. Schließlich wird das entwickelte Konzept der kooperativen Regelung am Beispiel einer großen mobilen Arbeitsmaschine angewandt, um seine Vorteile zu ermitteln und zu bewerten. Nach der Analyse in Simulationen wird die praktische Anwendbarkeit der Methode in drei Experimenten mit menschlichen Probanden an einem Simulator untersucht. Die Ergebnisse zeigen die Überlegenheit des entwickelten kooperativen Regelungskonzepts gegenüber der manuellen Steuerung und der nicht-kooperativen Steuerung hinsichtlich sowohl der objektiven Performanz als auch der subjektiven Bewertung der Probanden. Somit zeigt diese Dissertation, dass die kooperative Regelung mobiler Arbeitsmaschinen mit den entwickelten theoretischen Konzepten sowohl hilfreich als auch praktisch anwendbar ist

Addressing training data sparsity and interpretability challenges in AI based cellular networks

To meet the diverse and stringent communication requirements for emerging networks use cases, zero-touch arti cial intelligence (AI) based deep automation in cellular networks is envisioned. However, the full potential of AI in cellular networks remains hindered by two key challenges: (i) training data is not as freely available in cellular networks as in other fields where AI has made a profound impact and (ii) current AI models tend to have black box behavior making operators reluctant to entrust the operation of multibillion mission critical networks to a black box AI engine, which allow little insights and discovery of relationships between the configuration and optimization parameters and key performance indicators. This dissertation systematically addresses and proposes solutions to these two key problems faced by emerging networks. A framework towards addressing the training data sparsity challenge in cellular networks is developed, that can assist network operators and researchers in choosing the optimal data enrichment technique for different network scenarios, based on the available information. The framework encompasses classical interpolation techniques, like inverse distance weighted and kriging to more advanced ML-based methods, like transfer learning and generative adversarial networks, several new techniques, such as matrix completion theory and leveraging different types of network geometries, and simulators and testbeds, among others. The proposed framework will lead to more accurate ML models, that rely on sufficient amount of representative training data. Moreover, solutions are proposed to address the data sparsity challenge specifically in Minimization of drive test (MDT) based automation approaches. MDT allows coverage to be estimated at the base station by exploiting measurement reports gathered by the user equipment without the need for drive tests. Thus, MDT is a key enabling feature for data and artificial intelligence driven autonomous operation and optimization in current and emerging cellular networks. However, to date, the utility of MDT feature remains thwarted by issues such as sparsity of user reports and user positioning inaccuracy. For the first time, this dissertation reveals the existence of an optimal bin width for coverage estimation in the presence of inaccurate user positioning, scarcity of user reports and quantization error. The presented framework can enable network operators to configure the bin size for given positioning accuracy and user density that results in the most accurate MDT based coverage estimation. The lack of interpretability in AI-enabled networks is addressed by proposing a first of its kind novel neural network architecture leveraging analytical modeling, domain knowledge, big data and machine learning to turn black box machine learning models into more interpretable models. The proposed approach combines analytical modeling and domain knowledge to custom design machine learning models with the aim of moving towards interpretable machine learning models, that not only require a lesser training time, but can also deal with issues such as sparsity of training data and determination of model hyperparameters. The approach is tested using both simulated data and real data and results show that the proposed approach outperforms existing mathematical models, while also remaining interpretable when compared with black-box ML models. Thus, the proposed approach can be used to derive better mathematical models of complex systems. The findings from this dissertation can help solve the challenges in emerging AI-based cellular networks and thus aid in their design, operation and optimization

Kooperative Regelungskonzepte auf Basis der Spieltheorie und deren Anwendung auf Fahrerassistenzsysteme

Cooperative control loops in which human and a technical automation system perform a control task in close cooperation are investigated. A control framework is proposed which is based on a formal description of the cooperative control problem. The main idea of the control algorithm is to solve a differential game on a sliding horizon. The concept has been applied to design two cooperative advanced driver-assistance systems. One for the longitudinal driving task, one for the lateral driving task

Kooperative Regelungskonzepte auf Basis der Spieltheorie und deren Anwendung auf Fahrerassistenzsysteme

Diese Arbeit betrachtet Regelkreise, in denen die Regelaufgabe von Menschen und maschinellen Reglern gemeinsam ausgeführt wird. Für diese maschinellen Regler wird systematisch ein formalisiertes Regelkonzept abgeleitet. Ein wesentlicher Teil der Arbeit besteht in der Entwicklung von Algorithmen für die Implementierung. Als Anwendungsbeispiel werden zwei kooperative Fahrerassistenzsysteme vorgestellt. Am Fahrsimulator durchgeführte Studien zeigen eine deutliche Verbesserung des Fahrverhaltens aber auch des Kraftstoffverbrauchs