A Review on Human-Computer Interaction and Intelligent Robots

In the field of artificial intelligence, human–computer interaction (HCI) technology and its related intelligent robot technologies are essential and interesting contents of research. From the perspective of software algorithm and hardware system, these above-mentioned technologies study and try to build a natural HCI environment. The purpose of this research is to provide an overview of HCI and intelligent robots. This research highlights the existing technologies of listening, speaking, reading, writing, and other senses, which are widely used in human interaction. Based on these same technologies, this research introduces some intelligent robot systems and platforms. This paper also forecasts some vital challenges of researching HCI and intelligent robots. The authors hope that this work will help researchers in the field to acquire the necessary information and technologies to further conduct more advanced research

Soft Biomimetic Finger with Tactile Sensing and Sensory Feedback Capabilities

The compliant nature of soft fingers allows for safe and dexterous manipulation of objects by humans in an unstructured environment. A soft prosthetic finger design with tactile sensing capabilities for texture discrimination and subsequent sensory stimulation has the potential to create a more natural experience for an amputee. In this work, a pneumatically actuated soft biomimetic finger is integrated with a textile neuromorphic tactile sensor array for a texture discrimination task. The tactile sensor outputs were converted into neuromorphic spike trains, which emulate the firing pattern of biological mechanoreceptors. Spike-based features from each taxel compressed the information and were then used as inputs for the support vector machine (SVM) classifier to differentiate the textures. Our soft biomimetic finger with neuromorphic encoding was able to achieve an average overall classification accuracy of 99.57% over sixteen independent parameters when tested on thirteen standardized textured surfaces. The sixteen parameters were the combination of four angles of flexion of the soft finger and four speeds of palpation. To aid in the perception of more natural objects and their manipulation, subjects were provided with transcutaneous electrical nerve stimulation (TENS) to convey a subset of four textures with varied textural information. Three able-bodied subjects successfully distinguished two or three textures with the applied stimuli. This work paves the way for a more human-like prosthesis through a soft biomimetic finger with texture discrimination capabilities using neuromorphic techniques that provides sensory feedback; furthermore, texture feedback has the potential to enhance the user experience when interacting with their surroundings. Additionally, this work showed that an inexpensive, soft biomimetic finger combined with a flexible tactile sensor array can potentially help users perceive their environment better

Embedded Sensing and Control for High-Speed Electrohydraulic Soft Robots

Soft robotics is a field of robotic system design characterized by materials and structures that exhibit large-scale deformation, high compliance, and rich multifunctionality. The incorporation of soft and deformable structures endows soft robotic systems with the compliance and resiliency that makes them well-adapted for unstructured and dynamic environments. While actuation mechanisms for soft robots vary widely, soft electrostatic transducers such as dielectric elastomer actuators (DEAs) and hydraulically amplified self-healing electrostatic (HASEL) actuators have demonstrated promise due to their muscle-like performance. Despite significant leaps in design and modeling of HASEL actuators thus far, the actuators in and of themselves are limited in terms of estimating their own states and individually lacks useful system-level functionalities. To address the mentioned shortcomings, this body of research has enabled embedded perception and intelligence to electromechanical systems driven by electro-hydraulic actuators, by (i) developing capacitive self-sensing and magnetic sensing techniques to estimate shape changes of electro-hydraulic actuators in real-time; (ii) designing and implementing hardware and software controllers for an array of electrohydraulic actuators to enable complex motions and their collective useful functionalities; and (iii) demonstrating the integration of embedded sensing and control to bring soft robots driven by these actuators closer to practical, real-world applications. While not exhaustive, the introduced system technologies provide the fundamental building blocks for more advanced functionalities and features that can potentially integrate HASEL actuators to augment and enrich our daily lives

Modular robots for sorting

Current industrial sorting systems allow for low error, high throughput sorts with tightly constrained properties. These sorters, however, are often hardware limited to certain items and criteria. There is a need for more adaptive sorting systems for processes that involve a high throughput of heterogeneous items such as import testing by port authorities, warehouse sorting for online retailers, and sorting recycling. The variety of goods that need to be sorted in these applications mean that existing sorting systems are unsuitable, and the objects often need to be sorted by hand. In this work I detail my design and control of a modular, robotic sorting system using linear actuating robots to create both low-frequency vibrations and peristaltic waves for sorting. The adaptability of the system allows for multimodal sorting and can improve heterogeneous sorting systems. I have designed a novel modular robot called the Linbot. These Linbots are based on an actuator design utilising a voice coil for linear motion. I designed this actuator to be part of a modular robot by adding on-board computation, sensing and communication. I demonstrate the individual characteristics of these robots through a series of experiments in order to give a comprehensive overview of their abilities. By combining multiple Linbots in a collective I demonstrate their ability to move and sort objects using cooperative peristaltic motion. In order to allow for rapid optimisation of control schemes for Linbot collectives I required a peristaltic table simulator. I designed and implemented a peristaltic table simulator, called PeriSim, due to a lack of existing solutions. Controllers optimised in simulation often suffer from a reduction in performance when moved to a real-world system due to the inaccuracies in the simulation, this effect is called the reality gap. I used a method for reducing the reality gap called the radical envelope of noise hypothesis, whereby I only modelled the key aspects of peristalsis in PeriSim and then varied the underlying physics of the simulation between runs. I used PeriSim to optimise controllers in simulation that worked on a real-world system. I demonstrate the how the Linbots and PeriSim can be used to build and control an adaptive sorter. I built an adaptive sorter made of a 5x5 grid of Linbots with a soft sheet covering them. I demonstrate that the sorter can grade produce and move objects of varying shapes and sizes. My work can guide the future design of industrial adaptive sorting systems

Data-driven Tactile Sensing using Spatially Overlapping Signals

Providing robots with distributed, robust and accurate tactile feedback is a fundamental problem in robotics because of the large number of tasks that require physical interaction with objects. Tactile sensors can provide robots with information about the location of each point of contact with the manipulated object, an estimation of the contact forces applied (normal and shear) and even slip detection. Despite significant advances in touch and force transduction, tactile sensing is still far from ubiquitous in robotic manipulation. Existing methods for building touch sensors have proven difficult to integrate into robot fingers due to multiple challenges, including difficulty in covering multicurved surfaces, high wire count, or packaging constrains preventing their use in dexterous hands. In this dissertation, we focus on the development of soft tactile systems that can be deployed over complex, three-dimensional surfaces with a low wire count and using easily accessible manufacturing methods. To this effect, we present a general methodology called spatially overlapping signals. The key idea behind our method is to embed multiple sensing terminals in a volume of soft material which can be deployed over arbitrary, non-developable surfaces. Unlike a traditional taxel, these sensing terminals are not capable of measuring strain on their own. Instead, we take measurements across pairs of sensing terminals. Applying strain in the receptive field of this terminal pair should measurably affect the signal associated with it. As we embed multiple sensing terminals in this soft material, a significant overlap of these receptive fields occurs across the whole active sensing area, providing us with a very rich dataset characterizing the contact event. The use of an all-pairs approach, where all possible combinations of sensing terminals pairs are used, maximizes the number of signals extracted while reducing the total number of wires for the overall sensor, which in turn facilitates its integration. Building an analytical model for how this rich signal set relates to various contacts events can be very challenging. Further, any such model would depend on knowing the exact locations of the terminals in the sensor, thus requiring very precise manufacturing. Instead, we build forward models of our sensors from data. We collect training data using a dataset of controlled indentations of known characteristics, directly learning the mapping between our signals and the variables characterizing a contact event. This approach allows for accessible, cheap manufacturing while enabling extensive coverage of curved surfaces. The concept of spatially overlapping signals can be realized using various transduction methods; we demonstrate sensors using piezoresistance, pressure transducers and optics. With piezoresistivity we measure resistance values across various electrodes embedded in a carbon nanotubes infused elastomer to determine the location of touch. Using commercially available pressure transducers embedded in various configurations inside a soft volume of rubber, we show its possible to localize contacts across a curved surface. Finally, using optics, we measure light transport between LEDs and photodiodes inside a clear elastomer which makes up our sensor. Our optical sensors are able to detect both the location and depth of an indentation very accurately on both planar and multicurved surfaces. Our Distributed Interleaved Signals for Contact via Optics or D.I.S.C.O Finger is the culmination of this methodology: a fully integrated, sensorized robot finger, with a low wire count and designed for easy integration into dexterous manipulators. Our DISCO Finger can generally determine contact location with sub-millimeter accuracy, and contact force to within 10% (and often with 5%) of the true value without the need for analytical models. While our data-driven method requires training data representative of the final operational conditions that the system will encounter, we show our finger can be robust to novel contact scenarios where the shape of the indenter has not been seen during training. Moreover, the forward model that predicts contact locations and applied normal force can be transfered to new fingers with minimal loss of performance, eliminating the need to collect training data for each individual finger. We believe that rich tactile information, in a highly functional form with limited blind spots and a simple integration path into complete systems, like we demonstrate in this dissertation, will prove to be an important enabler for data-driven complex robotic motor skills, such as dexterous manipulation

The Development and Control of Soft Robotic Materials Driven by Hydraulically Amplified Self-Healing Electrostatic (HASEL) Actuators

Soft robotics is a growing research area focused on the development of compliant, adaptable, and bio-inspired robotic systems. Compared to traditional robotic solutions, soft robots are better suited for medical devices, wearable electronics, human-robot interaction, and other unique applications. The use of compliant materials enables design simplicity and bio-inspiration as well as entirely new functionalities not present in rigid robotic solutions.Electrostatic actuators are an effective way to drive soft robotic motion because of their low cost, mechanical simplicity, and high actuation bandwidth. A specific class of electrostatic actuator, the Hydraulically Amplified Self-healing Electrostatic (HASEL) actuator, further improves performance. However, system integration of HASEL actuator-driven robots is lacking. One approach to solving this problem is through the use of robotic materials which integrate actuation, sensing, communication, and control through a scalable constituent unit. Developing a HASEL actuator-driven soft robotic material would enable the creation of high degree of freedom robots with increased functionality. To do so, several challenges related to the sensing and feedback control of HASEL actuators must first be addressed.This thesis describes research efforts in the design and control of HASEL actuator-driven systems. Chapter 1 presents a literature review of soft robotics, soft actuators, and introduce the concept of robotic materials. Chapter 2 then presents a system identification and control technique for a single HASEL actuator. The system integrates a soft capacitive sensor onto the actuator. Then the control of a multi-HASEL-actuator robot using a novel magnetic sensing mechanism is presented in Chapter 3. Chapter 4 builds upon these results by introducing sTISSUE, a soft robotic material using HASEL actuators and the magnetic sensing mechanism. Multiple advanced demonstrations highlight the capabilities of this robotic material. Chapter 5 presents a multi-functional artificial potential field control method to enable highly controllable object manipulation with actuator arrays. Finally, Chapter 6 provides concluding statements and suggested next steps

Robotic Haptic Exploration of Shape and Symmetry

This thesis presents research on the use of symmetric models during haptic exploration procedures that have the objective of determining an object’s shape. These haptic exploration techniques, and their subsequent determination of a surface’s geometric properties, are crucial to allow robots to interact with a greater variety of objects, especially as the field of robotics transitions into unstructured environments. Symmetry is an extremely frequent shape property, especially in man-made objects, and it provides shape information that becomes useful in grasping and manipulation tasks, as well as enriching shape information for the aforementioned haptic exploration tasks. In this work, we present an improvement to Gaussian Process-driven exploration tasks. This method allows to describe symmetry to obtain a more precise shape estimation during active exploration, and can even be discovered in real time during the exploration procedure itself. This work involved the creation of a custom software resource to perform Gaussian Process regression with the addition of symmetries, and include a novel method of representing rotational symmetries. These novel models were then used in shape exploration procedures of 2D and 3D surfaces, both in a simulated environment and in an actual robotic task, using a series of custom-made contact sensors. These procedures are able to discover symmetry of each particular object in real time. This property can also be exploited, resulting in shape estimations that have a lower surface error and uncertainty. Additionally, exploration experiments that use these symmetry-finding procedures also require a lower total number of physical contacts and take less time to finish