Perception-motivated parallel algorithms for haptics

Negli ultimi anni l\u2019utilizzo di dispositivi aptici, atti cio\ue8 a riprodurre l\u2019interazione fisica con l\u2019ambiente remoto o virtuale, si sta diffondendo in vari ambiti della robotica e dell\u2019informatica, dai videogiochi alla chirurgia robotizzata eseguita in teleoperazione, dai cellulari alla riabilitazione. In questo lavoro di tesi abbiamo voluto considerare nuovi punti di vista sull\u2019argomento, allo scopo di comprendere meglio come riportare l\u2019essere umano, che \ue8 l\u2019unico fruitore del ritorno di forza, tattile e di telepresenza, al centro della ricerca sui dispositivi aptici. Allo scopo ci siamo focalizzati su due aspetti: una manipolazione del segnale di forza mutuata dalla percezione umana e l\u2019utilizzo di architetture multicore per l\u2019implementazione di algoritmi aptici e robotici. Con l\u2019aiuto di un setup sperimentale creato ad hoc e attraverso l\u2019utilizzo di un joystick con ritorno di forza a 6 gradi di libert\ue0, abbiamo progettato degli esperimenti psicofisici atti all\u2019identificazione di soglie differenziali di forze/coppie nel sistema mano-braccio. Sulla base dei risultati ottenuti abbiamo determinato una serie di funzioni di scalatura del segnale di forza, una per ogni grado di libert\ue0, che permettono di aumentare l\u2019abilit\ue0 umana nel discriminare stimoli differenti. L\u2019utilizzo di tali funzioni, ad esempio in teleoperazione, richiede la possibilit\ue0 di variare il segnale di feedback e il controllo del dispositivo sia in relazione al lavoro da svolgere, sia alle peculiari capacit\ue0 dell\u2019utilizzatore. La gestione del dispositivo deve quindi essere in grado di soddisfare due obbiettivi tendenzialmente in contrasto, e cio\ue8 il raggiungimento di alte prestazioni in termini di velocit\ue0, stabilit\ue0 e precisione, abbinato alla flessibilit\ue0 tipica del software. Una soluzione consiste nell\u2019affidare il controllo del dispositivo ai nuovi sistemi multicore che si stanno sempre pi\uf9 prepotentemente affacciando sul panorama informatico. Per far ci\uf2 una serie di algoritmi consolidati deve essere portata su sistemi paralleli. In questo lavoro abbiamo dimostrato che \ue8 possibile convertire facilmente vecchi algoritmi gi\ue0 implementati in hardware, e quindi intrinsecamente paralleli. Un punto da definire rimane per\uf2 quanto costa portare degli algoritmi solitamente descritti in VLSI e schemi in un linguaggio di programmazione ad alto livello. Focalizzando la nostra attenzione su un problema specifico, la pseudoinversione di matrici che \ue8 presente in molti algoritmi di dinamica e cinematica, abbiamo mostrato che un\u2019attenta progettazione e decomposizione del problema permette una mappatura diretta sulle unit\ue0 di calcolo disponibili. In aggiunta, l\u2019uso di parallelismo a livello di dati su macchine SIMD permette di ottenere buone prestazioni utilizzando semplici operazioni vettoriali come addizioni e shift. Dato che di solito tali istruzioni fanno parte delle implementazioni hardware la migrazione del codice risulta agevole. Abbiamo testato il nostro approccio su una Sony PlayStation 3 equipaggiata con un processore IBM Cell Broadband Engine.In the last years the use of haptic feedback has been used in several applications, from mobile phones to rehabilitation, from video games to robotic aided surgery. The haptic devices, that are the interfaces that create the stimulation and reproduce the physical interaction with virtual or remote environments, have been studied, analyzed and developed in many ways. Every innovation in the mechanics, electronics and technical design of the device it is valuable, however it is important to maintain the focus of the haptic interaction on the human being, who is the only user of force feedback. In this thesis we worked on two main topics that are relevant to this aim: a perception based force signal manipulation and the use of modern multicore architectures for the implementation of the haptic controller. With the help of a specific experimental setup and using a 6 dof haptic device we designed a psychophysical experiment aimed at identifying of the force/torque differential thresholds applied to the hand-arm system. On the basis of the results obtained we determined a set of task dependent scaling functions, one for each degree of freedom of the three-dimensional space, that can be used to enhance the human abilities in discriminating different stimuli. The perception based manipulation of the force feedback requires a fast, stable and configurable controller of the haptic interface. Thus a solution is to use new available multicore architectures for the implementation of the controller, but many consolidated algorithms have to be ported to these parallel systems. Focusing on specific problem, i.e. the matrix pseudoinversion, that is part of the robotics dynamic and kinematic computation, we showed that it is possible to migrate code that was already implemented in hardware, and in particular old algorithms that were inherently parallel and thus not competitive on sequential processors. The main question that still lies open is how much effort is required in order to write these algorithms, usually described in VLSI or schematics, in a modern programming language. We show that a careful task decomposition and design permit a mapping of the code on the available cores. In addition, the use of data parallelism on SIMD machines can give good performance when simple vector instructions such as add and shift operations are used. Since these instructions are present also in hardware implementations the migration can be easily performed. We tested our approach on a Sony PlayStation 3 game console equipped with IBM Cell Broadband Engine processor

A Contribution to Resource-Aware Architectures for Humanoid Robots

The goal of this work is to provide building blocks for resource-aware robot architectures. The topic of these blocks are data-driven generation of context-sensitive resource models, prediction of future resource utilizations, and resource-aware computer vision and motion planning algorithms. The implementation of these algorithms is based on resource-aware concepts and methodologies originating from the Transregional Collaborative Research Center "Invasive Computing" (SFB/TR 89)

Reconfigurable Computing Systems for Robotics using a Component-Oriented Approach

Robotic platforms are becoming more complex due to the wide range of modern applications, including multiple heterogeneous sensors and actuators. In order to comply with real-time and power-consumption constraints, these systems need to process a large amount of heterogeneous data from multiple sensors and take action (via actuators), which represents a problem as the resources of these systems have limitations in memory storage, bandwidth, and computational power. Field Programmable Gate Arrays (FPGAs) are programmable logic devices that offer high-speed parallel processing. FPGAs are particularly well-suited for applications that require real-time processing, high bandwidth, and low latency. One of the fundamental advantages of FPGAs is their flexibility in designing hardware tailored to specific needs, making them adaptable to a wide range of applications. They can be programmed to pre-process data close to sensors, which reduces the amount of data that needs to be transferred to other computing resources, improving overall system efficiency. Additionally, the reprogrammability of FPGAs enables them to be repurposed for different applications, providing a cost-effective solution that needs to adapt quickly to changing demands. FPGAs' performance per watt is close to that of Application-Specific Integrated Circuits (ASICs), with the added advantage of being reprogrammable. Despite all the advantages of FPGAs (e.g., energy efficiency, computing capabilities), the robotics community has not fully included them so far as part of their systems for several reasons. First, designing FPGA-based solutions requires hardware knowledge and longer development times as their programmability is more challenging than Central Processing Units (CPUs) or Graphics Processing Units (GPUs). Second, porting a robotics application (or parts of it) from software to an accelerator requires adequate interfaces between software and FPGAs. Third, the robotics workflow is already complex on its own, combining several fields such as mechanics, electronics, and software. There have been partial contributions in the state-of-the-art for FPGAs as part of robotics systems. However, a study of FPGAs as a whole for robotics systems is missing in the literature, which is the primary goal of this dissertation. Three main objectives have been established to accomplish this. (1) Define all components required for an FPGAs-based system for robotics applications as a whole. (2) Establish how all the defined components are related. (3) With the help of Model-Driven Engineering (MDE) techniques, generate these components, deploy them, and integrate them into existing solutions. The component-oriented approach proposed in this dissertation provides a proper solution for designing and implementing FPGA-based designs for robotics applications. The modular architecture, the tool 'FPGA Interfaces for Robotics Middlewares' (FIRM), and the toolchain 'FPGA Architectures for Robotics' (FAR) provide a set of tools and a comprehensive design process that enables the development of complex FPGA-based designs more straightforwardly and efficiently. The component-oriented approach contributed to the state-of-the-art in FPGA-based designs significantly for robotics applications and helps to promote their wider adoption and use by specialists with little FPGA knowledge

Event-Driven Technologies for Reactive Motion Planning: Neuromorphic Stereo Vision and Robot Path Planning and Their Application on Parallel Hardware

Die Robotik wird immer mehr zu einem Schlüsselfaktor des technischen Aufschwungs. Trotz beeindruckender Fortschritte in den letzten Jahrzehnten, übertreffen Gehirne von Säugetieren in den Bereichen Sehen und Bewegungsplanung noch immer selbst die leistungsfähigsten Maschinen. Industrieroboter sind sehr schnell und präzise, aber ihre Planungsalgorithmen sind in hochdynamischen Umgebungen, wie sie für die Mensch-Roboter-Kollaboration (MRK) erforderlich sind, nicht leistungsfähig genug. Ohne schnelle und adaptive Bewegungsplanung kann sichere MRK nicht garantiert werden. Neuromorphe Technologien, einschließlich visueller Sensoren und Hardware-Chips, arbeiten asynchron und verarbeiten so raum-zeitliche Informationen sehr effizient. Insbesondere ereignisbasierte visuelle Sensoren sind konventionellen, synchronen Kameras bei vielen Anwendungen bereits überlegen. Daher haben ereignisbasierte Methoden ein großes Potenzial, schnellere und energieeffizientere Algorithmen zur Bewegungssteuerung in der MRK zu ermöglichen. In dieser Arbeit wird ein Ansatz zur flexiblen reaktiven Bewegungssteuerung eines Roboterarms vorgestellt. Dabei wird die Exterozeption durch ereignisbasiertes Stereosehen erreicht und die Pfadplanung ist in einer neuronalen Repräsentation des Konfigurationsraums implementiert. Die Multiview-3D-Rekonstruktion wird durch eine qualitative Analyse in Simulation evaluiert und auf ein Stereo-System ereignisbasierter Kameras übertragen. Zur Evaluierung der reaktiven kollisionsfreien Online-Planung wird ein Demonstrator mit einem industriellen Roboter genutzt. Dieser wird auch für eine vergleichende Studie zu sample-basierten Planern verwendet. Ergänzt wird dies durch einen Benchmark von parallelen Hardwarelösungen wozu als Testszenario Bahnplanung in der Robotik gewählt wurde. Die Ergebnisse zeigen, dass die vorgeschlagenen neuronalen Lösungen einen effektiven Weg zur Realisierung einer Robotersteuerung für dynamische Szenarien darstellen. Diese Arbeit schafft eine Grundlage für neuronale Lösungen bei adaptiven Fertigungsprozesse, auch in Zusammenarbeit mit dem Menschen, ohne Einbußen bei Geschwindigkeit und Sicherheit. Damit ebnet sie den Weg für die Integration von dem Gehirn nachempfundener Hardware und Algorithmen in die Industrierobotik und MRK

Cross-Layer System Design for Autonomous Driving

Autonomous driving has gained tremendous popularity and becomes one of the most emerging applications recently, which allows the vehicle to drive by itself without requiring help from a human. The demand of this application continues to grow leading to ever increasing investment from industry in the last decade. Unfortunately, autonomous driving systems remain unavailable to the public and are still under development even with the recent considerable advancement achieved in our community. Several key challenges are observed across the stack of autonomous driving systems and must be addressed to bridge the gap. This dissertation investigates cross-layer autonomous driving systems from hardware architecture, software algorithms to human-vehicle interaction. In the hardware architecture layer, we investigate and present the design constraints of autonomous driving systems. With an end-to-end autonomous driving system framework we built, we accelerate the computational bottlenecks identified and thoroughly investigate the implications and trade-offs across various accelerator platforms. In the software algorithm layer, we propose an accelerating technique for object recognition, which is one of the critical bottlenecks in autonomous driving systems. We exploit the similarity across frames in streaming videos for autonomous vehicles and reuse the intermediate outputs computed in the algorithm to reduce the computation required and improve the performance. In the human-vehicle interaction layer, we design a conversational in-vehicle interface framework which enables drivers to interact with vehicles by using natural human language to improve the usability of autonomous driving features. We also integrate this framework into a commercially available vehicle and conduct a real-world driving study.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/149951/1/shihclin_1.pd

Hardware, software and control design considerations towards low-cost compliant quadruped robots

Quadrupedal robots have been a field of interest the last few years, with many new maturing platforms. Many of these projects have in common the use of state of the art actuation and sensing, and therefore are able to handle difficult locomotion tasks very effectively. This work focuses on another trend of low-cost, quadrupedal robots, that features less precise actuators and sensors, but overcomes their limitations with strong bio-inspired designs to achieve state of the art locomotion. We aim here to further extend the achievements of this approach to handle more complex tasks and that require anticipation, We would like also to verify to which extent a close synergy between clever mechanics, sensorimotor coordination, and Central Pattern Generator models is able to handle these tasks. This thesis presents supporting work that was required to pursue this goal. A software architecture for the development of real-time drivers and low-level control for robotic applications, based on a clear separation of concerns is presented. An implementation of this architecture able to handle the specific requirements for small compliant quadruped robots is proposed. Furthermore, the development and integration of a communication protocol for inter-electronic devices communication on the Oncilla robot is discussed. As leg load is a key quantity in some of the sensory-motor coordination this thesis want to explore, a novel tactile sensing approach for its estimation is proposed, based on an Extended Kalman Filter data fusion of static and dynamic tactile sensor information. Then, to support the design of efficient interactions between the control and the bio-inspired mechanics, accurate dynamic modeling of the Advanced Spring Loaded Pantographic leg, equipping all robots considered here, is presented. We propose two approaches to this modeling with the presentation of their benefits and limitations. Finally, two Central Pattern Generator architectures are proposed, based on biologically inspired foot trajectories. The first is using a well-known method for inter-limb coordination with strong neural coupling, and the second, the Tegotae rule, relies only on limb physical coupling and strong sensory-motor coordination. These two approaches are compared on their capacity to handle dynamic footstep placement and it let to the conclusion that strong sensory-motor coordination is required for this task

MATLAB

This excellent book represents the final part of three-volumes regarding MATLAB-based applications in almost every branch of science. The book consists of 19 excellent, insightful articles and the readers will find the results very useful to their work. In particular, the book consists of three parts, the first one is devoted to mathematical methods in the applied sciences by using MATLAB, the second is devoted to MATLAB applications of general interest and the third one discusses MATLAB for educational purposes. This collection of high quality articles, refers to a large range of professional fields and can be used for science as well as for various educational purposes

Sistema embarcado baseado em arquiteturas reconfiguráveis do controle dinâmico de uma mão robótica sintonizado com algoritmos bioinspirados

Dissertação (mestrado)—Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Mecânica, 2017.Nos últimos anos grandes avanços tecnológicos formam feitos no campo da computação e áreas correlatas, o que permitiu o desenvolvimento de sistemas robóticos sofisticados como robôs biomiméticos. Esses robôs imitam sistemas biológicos que decorrem robustez e e ciência maiores se comparados com robôs convencionais quando usados em ambientes não estruturados. Por exemplo, uma mão robótica biomimética tem uma destreza e agilidade maior para executar tarefas de manipulação e agarres do que pinças convencionais. Desde os anos oitenta, o desenvolvimento de mãos robóticas biomiméticas é o foco de pesquisa de várias equipes de investigação no mundo. Na atualidade há um número vasto de trabalhos encaminhados à construção e controle dos mesmos, os quais tem o intuito de melhorar a destreza e o desempenho das mãos e incluem tópicos como projeto da mão, mecanismos de movimento para as juntas, plataformas embarcadas e estratégias de controle. Existem várias abordagens a nível computacional que ainda não têm sido exploradas neste tipo de robôs, por exemplo o uso de um chip FPGA para o aumento de desempenho das estratégias de controle dinâmico usadas nos mesmos. O presente trabalho descreve o desenvolvimento de uma arquitetura em hardware baseada em FPGA do controlador dinâmico de uma mão robótica, o qual é sintonizado usando algoritmos de otimização bioinspirada visando para atingir estabilidade de agarre. O projeto da mão robótica realizado neste trabalho inclui o uso de mecanismos para emular os movimentos de exão-extensão dos dedos. Os mecanismos foram otimizados visando minimizar o erro de trajetória usando a mão humana como referência. Os algoritmos bioinspirados PSO, DE e GA foram implementados para otimizar o mecanismo. 32 experimentos foram realizados para cada algoritmo a m de realizar uma análise estatística para determinar o mecanismo com o melhor resultado, o qual é implementado no projeto nal do mecanismo do dedo incluído no CAD da mão completa, o qual é descrito junto com o projeto eletrônico da plataforma. O projeto nal da mão é avaliado com análise cinemática e adaptações do teste de Kapandji. Em seguida o protótipo é fabricado e montado usando diversos processos de fabricação e prototipagem, tais como corte a jato de água, torneamento e impressão 3D. Logo após, foi projetado na plataforma Matlab/Simulink em alto nível o esquema de controle de impedância dos dedos o qual foi validado usando um simulador numérico do dedo para estudar o efeito do controlador no sistema sem colocar em risco a plataforma física. A sintonização do controlador é realizada usando o algoritmo de otimização bioinspirado PSO visando reduzir o tempo de estabilidade, o sobreimpulso e o tempo de subida. Seguidamente, esses resultados foram implementados em plataformas embarcadas nas linguagens de programação C e na linguagem de descrição de hardware HDL. Após ser avaliado, o esquema de controle foi implementado em C na plataforma Arduino e mapeado em FPGA na placa de desenvolvimento ZedBoard. Uma comparação numérica e analítica foi realizada em termos do desempenho e precisão das duas abordagens. O resultado da otimização do mecanismo de exão-extensão produziu um erro de 0.2660% e o uso deste mecanismo permitiu fabricar um protótipo com dimensões e peso similar a uma mão humana real. Além disso, o protótipo atingiu os dez níveis da adaptação do teste de Kapandji. Adicionalmente, a sintonização da estratégia de controle resultou no comportamento desejado, o qual é subamortecido e com um tempo de estabilização 355ms. Similarmente, os resultados da implementação em FPGA foram satisfatórios no sentido do desempenho do tempo de execução da estratégia de controle, o qual melhorou os resultados da implementação em Arduino e outros trabalhos correlatos no estado da arte.In recent years, huge technological advances have been made in the eld of computing sciences and related areas, which has allowed the development of sophisticated robotic systems such as biomimetic robots. These robots mimic biological systems that result in greater robustness and e ciency compared to conventional robots when used in unstructured environments. For instance, a biomimetic robotic hand has greater dexterity and agility to perform manipulation tasks and grasp than conventional grippers. Since the 1980s, the development of biomimetic robotic hands has been the focus of many research teams all over the world. Nowadays several contributions regarding the construction and control of said systems, which aim to improve the dexterity and performance of the hands and include topics such as hand design, joint mechanisms, embedded platforms and control strategies. However, even with the great available knowledge, there are still no perfect robotic hands, therefore, there is still knowledge to be contributed in the scienti c community. There are several approaches at the computational level that have not yet been explored in this type of robots, for example, the use of a single FPGA chip to increase the performance of the dynamic control schemes used. This work describes the development of an FPGA-based hardware architecture of the dynamic controller in a robotic hand, which is tuned using bioinspired optimization algorithms applied to achieve stability of grasps. The robotic hand design performed in this work includes the use of mechanisms to emulate the exionextension movements of the ngers. The mechanisms were optimized to minimize the trajectory error using the human hand as reference. The bioinspired algorithms PSO, DE and GA were implemented to optimize the mechanism. 32 experiments are performed for each algorithm to perform a statistical analysis to determine 2 the best result. This optimized mechanism was implemented in the nal design of the nger mechanism included in the CAD of the complete hand, which is described together with the electronic design of the platform. The nal hand design is evaluated with kinematic analysis and Kapandji clinical test adaptations. The prototype was manufactured and assembled using various manufacturing and prototyping processes, such as water-jet cutting, turning and 3D printing. Afterwards, the nger impedance control scheme was designed on a high level platform using Matlab/Simulink, in addition to a numerical simulator of the nger for the study of the controller e ect on the system avoiding physical damage to the system. The controller was tuned using the PSO optimization algorithm aiming to reduce the stability time, the overshoot and the rise time. These results are then implemented on embedded platforms in both C and VHDL languages. After being evaluated, the control scheme is implemented in C on the Arduino platform and was manually mapped to FPGA on the ZedBoard development board. A numerical comparison between the two approaches was done in terms of performance and accuracy. The result of the optimization of the exion-extension mechanism produced an error of 0.2660% and the use of this mechanism allowed for manufacturing the prototype with dimensions and weight similar to a real human hand. The prototype reached the ten levels of the Kapandji test tting. In addition, the tuning of the control strategy resulted in the desired behavior, which is underdamped and with a stabilization time of 355ms. Similarly, the FPGA implementation results were satisfactory in the sense of the execution-time performance of the control strategy, which improved the implementation results in Arduino and other related work in the state of the art