Evaluation of a robot-assisted therapy for children with autism and intellectual disability

It is well established that robots can be suitable assistants in the care and treatment of children with Autism Spectrum Disorder (ASD). However, the majority of the research focuses on stand-alone interventions, high-functioning individuals and the success is evaluated via qualitative analysis of videos recorded during the interaction. In this paper, we present a preliminary evaluation of our on-going research on integrating robot-assisted therapy in the treatment of children with ASD and Intellectual Disability (ID), which is the most common case. The experiment described here integrates a robot-assisted imitation training in the standard treat‐ ment of six hospitalised children with various level of ID, who were engaged by a robot on imitative tasks and their progress assessed via a quantitative psycho- diagnostic tool. Results show success in the training and encourage the use of a robotic assistant in the care of children with ASD and ID with the exception of those with profound ID, who may need a different approach

Pilot Study: A Visuotactile Haptic Primary Colors Sensor

In this paper, we present our preliminary studies on how to make a unified skin-like visuotactile sensor capable of sensing haptic primary colors, namely: force, vibration, and temperature. Our sensor is based on GelSight technology that has proven its worth in the field of haptics, robotics, and computer vision. In our previous studies, we proposed switchable UltraViolet (UV) markers that can be turned on using UV light. These markers can be tracked using an optical flow algorithm to visualize forces related to gel deformation. In this study, we introduced layers of thermochromic pigments on the reflective layer, making our visuotactile sensor capable of sensing not only force and vibration inferred from gel deformation but can also sense the temperature of the contacted object by analyzing the change of hue in the reflective coatin

Low-cost GelSight with UV Markings: Feature Extraction of Objects Using AlexNet and Optical Flow without 3D Image Reconstruction

GelSight sensor has been used to study microgeometry of objects since 2009 in tactile sensing applications. Elastomer, reflective coating, lighting, and camera were the main challenges of making a GelSight sensor within a short period. The recent addition of permanent markers to the GelSight was a new era in shear/slip studies. In our previous studies, we introduced Ultraviolet (UV) ink and UV LEDs as a new form of marker and lighting respectively. UV ink markers are invisible using ordinary LED but can be made visible using UV LED. Currently, recognition of objects or surface textures using GelSight sensor is done using fusion of camera-only images and GelSight captured images with permanent markings. Those images are fed to Convolutional Neural Networks (CNN) to classify objects. However, our novel approach in using low-cost GelSight sensor with UV markings, the 3D height map to 2D image conversion, and the additional non-Gelsight captured images for training the CNN can be eliminated. AlexNet and optical flow algorithm have been used for feature recognition of five coins without UV markings and shear/slip of the coin in GelSight with UV markings respectively. Our results on confusion matrix show that, on average coin recognition can reach 93.4% without UV markings using AlexNet. Therefore, our novel method of using GelSight with UV markings would be useful to recognize full/partial object, shear/slip, and force applied to the objects without any 3D image reconstruction

Visuotactile Sensors with Emphasis on GelSight Sensor: A Review

This review paper focuses on vision and touch-based sensors known as visuotactile. The study of visuotactile sensation and perception became a multidisciplinary field of study by philosophers, psychologists, biologists, engineers, technologists, and roboticists in the fields of haptics, machine vision, and artificial intelligence and it dates back centuries. To the best of our knowledge, the earliest records of visuotactile sensor was not applied to robotics and was not even for hand or finger imprint analysis yet for recording the foot pressure distribution of a walking or standing human known as pedobarograph. Our review paper presents the different literature related to visuotactile sensors that lead to a high-resolution miniature pedobarographlike sensor known as the GelSight sensor. Moreover, this review paper focuses on architecture, different techniques, hardware, and software development of GelSight sensor since 2009 with its applications in haptics, robotics, and computer vision

Control and benchmarking of a 7-DOF robotic arm using Gazebo and ROS

Robot controller plays an important role in controlling the robot. The controller mainly aims to eliminate or suppress the influence of uncertain factors on the control robot. Furthermore, there are many types of controllers, and different types of controllers have different features. To explore the differences between controllers of the same category, this paper studies some controllers from basic controllers and advanced controllers. This paper conducts the benchmarking of the selected controller through pre-set tests. The test task is the most commonly used pick and place. Furthermore, to complete the robustness test, a task of external force interference is also set to observe whether the controller can control the robot arm to return to a normal state. Subsequently, the accuracy, control efficiency, jitter and robustness of the robot arm controlled by the controller are analyzed by comparing the Position and Effort data. Finally, some future works of the benchmarking and reasonable improvement methods are discussed

HaptiTemp: A Next-Generation Thermosensitive GelSight-like Visuotactile Sensor

This study describes the creation of a new type of compact skin-like silicone-based thermosensitive visuotactile sensor based on GelSight technology. The easy integration of this novel sensor into a complex visuotactile system capable of very rapid detection of temperature change (30°C/s) is unique in providing a system that parallels the withdrawal reflex of the human autonomic system to extreme heat. To the best of authors’ awareness, this is the first time a sensor that can trigger a sensory impulse like a withdrawal reflex of humans in robotic community. To attain this, we used thermochromic pigments color blue, orange, and black with a threshold of 31°C, 43°C, and 50°C, respectively on the gel material. Each pigment has the property of becoming translucent when its temperature threshold is reached, making it possible to stack thermochromic pigments of different colors and thresholds. The pigments were air-brushed on a low-cost commercially available transparent silicone sponge. We used MobileNetV2 and transfer learning to simulate tactile preprocessing in order to recognize five different objects. The new thermosensitive visuotactile sensor helped to achieve 97.3% tactile image classification accuracy of five different objects. Our novel thermosensitive visuotactile sensor could be of benefit in material texture analysis, telerobotics, space exploration, and medical applications

Self-assembly and Self-repair during Motion with Modular Robots

Self-reconfigurable modular robots consist of multiple modular elements and have the potential to enable future autonomous systems to adapt themselves to handle unstructured environments, novel tasks, or damage to their constituent elements. This paper considers methods of self-assembly, bringing together robotic modules to form larger organism-like structures, and self-repair, removing and replacing faulty modules damaged by internal events or environmental phenomena, which allow group tasks for the multi-robot organism to continue to progress while assembly and repair take place. We showthat such “inmotion" strategies can successfully assemble and repair a range of structures. Previously developed self-assembly and self-repair strategies have required group tasks to be halted before they could begin. This paper finds that self-assembly and self-repair methods able to operate during group tasks can enable faster completion of the task than previous strategies, and provide reliability benefits in some circumstances. The practicality of these new methods is shown with physical hardware demonstrations. These results show the feasibility of assembling and repairing modular robots whilst other tasks are in progress

Design, fabrication and stiffening of soft pneumatic robots

Although compliance allows the soft robot to be under-actuated and generalise its control, it also impacts the ability of the robot to exert forces on the environment. There is a trade-off between robots being compliant or precise and strong. Many mechanisms that change robots' stiffness on demand have been proposed, but none are perfect, usually compromising the device's compliance and restricting its motion capabilities. Keeping the above issues in mind, this thesis focuses on creating robust and reliable pneumatic actuators, that are designed to be easily manufactured with simple tools. They are optimised towards linear behaviour, which simplifies modelling and improve control strategies. The principle idea in relation to linearisation is a reinforcement strategy designed to amplify the desired, and limit the unwanted, deformation of the device. Such reinforcement can be achieved using fibres or 3D printed structures. I have shown that the linearity of the actuation is, among others, a function of the reinforcement density and shape, in that the response of dense fibre-reinforced actuators with a circular cross-section is significantly more linear than that of non-reinforced or non-circular actuators. I have explored moulding manufacturing techniques and a mixture of 3D printing and moulding. Many aspects of these techniques have been optimised for reliability, repeatability, and process simplification. I have proposed and implemented a novel moulding technique that uses disposable moulds and can easily be used by an inexperienced operator. I also tried to address the compliance-stiffness trade-off issue. As a result, I have proposed an intelligent structure that behaves differently depending on the conditions. Thanks to its properties, such a structure could be used in applications that require flexibility, but also the ability to resist external disturbances when necessary. Due to its nature, individual cells of the proposed system could be used to implement physical logic elements, resulting in embodied intelligent behaviours. As a proof-of-concept, I have demonstrated use of my actuators in several applications including prosthetic hands, octopus, and fish robots. Each of those devices benefits from a slightly different actuation system but each is based on the same core idea - fibre reinforced actuators. I have shown that the proposed design and manufacturing techniques have several advantages over the methods used so far. The manufacturing methods I developed are more reliable, repeatable, and require less manual work than the various other methods described in the literature. I have also shown that the proposed actuators can be successfully used in real-life applications. Finally, one of the most important outcomes of my research is a contribution to an orthotic device based on soft pneumatic actuators. The device has been successfully deployed, and, at the time of submission of this thesis, has been used for several months, with good results reported, by a patient