Embodied active tactile perception

Tactile perception plays an important role in an agent safely interacting with the environment while acquiring information about it. Bio-inspired robotics opens up possibilities for a new paradigm leveraging the morphology of the body, which filters the tactile information in physical interactions and enables investigations of new designs for embodied active tactile perception. The subjects of morphology embodied active perception and motor embodied active perception is defined and discussed in this chapter. In the scope of morphology embodied active perception, sensor optimization and sensor adaptation are further defined to describe the change of sensor morphology in the design phase and the interacting phase, respectively. More specifically, the concept of online and offline sensor adjustment is presented. Sensor optimization is solely considered in the offline process for optimization and evolution design of the sensor structure and characteristics. Sensor adaptation and motor embodied active perception are considered in the online process to actively shape the sensing process with the morphology change of the sensors themselves and the action of the body where the sensors are placed, respectively. "Design as a whole" is proposed as an inverse problem to address the sensing tasks. The design of new tactile sensors should not focus on the sensor per se but should also include design parameters for sensor optimization, sensor adaptation, and motor actions

Soft Morphological Computation

Soft Robotics is a relatively new area of research, where progress in material science has powered the next generation of robots, exhibiting biological-like properties such as soft/elastic tissues, compliance, resilience and more besides. One of the issues when employing soft robotics technologies is the soft nature of the interactions arising between the robot and its environment. These interactions are complex, and the their dynamics are non-linear and hard to capture with known models. In this thesis we argue that complex soft interactions can actually be beneficial to the robot, and give rise to rich stimuli which can be used for the resolution of robot tasks. We further argue that the usefulness of these interactions depends on statistical regularities, or structure, that appear in the stimuli. To this end, robots should appropriately employ their morphology and their actions, to influence the system-environment interactions such that structure can arise in the stimuli. In this thesis we show that learning processes can be used to perform such a task. Following this rationale, this thesis proposes and supports the theory of Soft Morphological Computation (SoMComp), by which a soft robot should appropriately condition, or ‘affect’, the soft interactions to improve the quality of the physical stimuli arising from it. SoMComp is composed of four main principles, i.e.: Soft Proprioception, Soft Sensing, Soft Morphology and Soft Actuation. Each of these principles is explored in the context of haptic object recognition or object handling in soft robots. Finally, this thesis provides an overview of this research and its future directions.AHDB CP17

Adaptation of sensor morphology: an integrative view of perception from biologically inspired robotics perspective

Sensor morphology, the morphology of a sensing mechanism which plays a role of shaping the desired response from physical stimuli from surroundings to generate signals usable as sensory information, is one of the key common aspects of sensing processes. This paper presents a structured review of researches on bioinspired sensor morphology implemented in robotic systems, and discusses the fundamental design principles. Based on literature review, we propose two key arguments: first, owing to its synthetic nature, biologically inspired robotics approach is a unique and powerful methodology to understand the role of sensor morphology and how it can evolve and adapt to its task and environment. Second, a consideration of an integrative view of perception by looking into multidisciplinary and overarching mechanisms of sensor morphology adaptation across biology and engineering enables us to extract relevant design principles that are important to extend our understanding of the unfinished concepts in sensing and perceptionThis study was supported by the European Commission with the RoboSoft CA (A Coordination Action for Soft Robotics, contract #619319). SGN was supported by School of Engineering seed funding (2016), Malaysia Campus, Monash University

Environment induced emergence of collective behaviour in evolving swarms with limited sensing

Designing controllers for robot swarms is challenging, because human developers have typically no good understanding of the link between the details of a controller that governs individual robots and the swarm behavior that is an indirect result of the interactions between swarm members and the environment. In this paper we investigate whether an evolutionary approach can mitigate this problem. We consider a very challenging task where robots with limited sensing and communication abilities must follow the gradient of an environmental feature and use Differential Evolution to evolve a neural network controller for simulated robots. We conduct a systematic study to measure the flexibility and scalability of the method by varying the size of the arena and number of robots in the swarm. The experiments confirm the feasibility of our approach, the evolved robot controllers induced swarm behavior that solved the task. We found that solutions evolved under the harshest conditions (where the environmental clues were the weakest) were the most flexible and that there is a sweet spot regarding the swarm size. Furthermore, we observed collective motion of the swarm, showcasing truly emergent behavior that was not represented in- and selected for during evolution.Comment: (1) Three authors contributed equally to this researc

Pulsating fluidic sensor for sensing of location, pressure and contact area

Designing information-rich and space-efficient sensors is a key challenge for soft robotics, and crucial for the development of safe soft robots. Sensing and understanding the environmental interactions with a minimal footprint is especially important in the medical context, where portability and unhindered patient/user movement is a priority, to move towards personalized and decentralized healthcare solutions. In this work, a pulsating fluidic soft sensor (PFS) capable of determining location, pressure and contact area of press events is shown. The sensor relies on spatio-temporal resistance changes driven by a pulsating conductive fluid. The sensor demonstrates good repeatability and distinction of single and multiple press events, detecting single indents of sizes greater than 1 cm, forces larger than 2 N, and various locations across the sensor, as well as multiple indents spaced 2 cm apart. Furthermore, the sensor is demonstrated in two applications to detect foot placement and grip location. Overall, the sensor represents an improvement towards minimizing electronic hardware, and cost of the sensing solution, without sacrificing the richness of the sensing information in the field of soft fluidic sensors

Instrumentation, Data, And Algorithms For Visually Understanding Haptic Surface Properties

Autonomous robots need to efficiently walk over varied surfaces and grasp diverse objects. We hypothesize that the association between how such surfaces look and how they physically feel during contact can be learned from a database of matched haptic and visual data recorded from various end-effectors\u27 interactions with hundreds of real-world surfaces. Testing this hypothesis required the creation of a new multimodal sensing apparatus, the collection of a large multimodal dataset, and development of a machine-learning pipeline. This thesis begins by describing the design and construction of the Portable Robotic Optical/Tactile ObservatioN PACKage (PROTONPACK, or Proton for short), an untethered handheld sensing device that emulates the capabilities of the human senses of vision and touch. Its sensory modalities include RGBD vision, egomotion, contact force, and contact vibration. Three interchangeable end-effectors (a steel tooling ball, an OptoForce three-axis force sensor, and a SynTouch BioTac artificial fingertip) allow for different material properties at the contact point and provide additional tactile data. We then detail the calibration process for the motion and force sensing systems, as well as several proof-of-concept surface discrimination experiments that demonstrate the reliability of the device and the utility of the data it collects. This thesis then presents a large-scale dataset of multimodal surface interaction recordings, including 357 unique surfaces such as furniture, fabrics, outdoor fixtures, and items from several private and public material sample collections. Each surface was touched with one, two, or three end-effectors, comprising approximately one minute per end-effector of tapping and dragging at various forces and speeds. We hope that the larger community of robotics researchers will find broad applications for the published dataset. Lastly, we demonstrate an algorithm that learns to estimate haptic surface properties given visual input. Surfaces were rated on hardness, roughness, stickiness, and temperature by the human experimenter and by a pool of purely visual observers. Then we trained an algorithm to perform the same task as well as infer quantitative properties calculated from the haptic data. Overall, the task of predicting haptic properties from vision alone proved difficult for both humans and computers, but a hybrid algorithm using a deep neural network and a support vector machine achieved a correlation between expected and actual regression output between approximately ρ = 0.3 and ρ = 0.5 on previously unseen surfaces

Power-efficient adaptive behavior through a shape-changing elastic robot

The adaptive morphology of a robot, such as shape adaptation, plays a significant role in adapting its behaviors. Shape adaptation should ideally be achieved without considerable cost, like the power required to deform the robot’s body, and therefore, it is reasonably considered as the last resort in classical rigid robots. However, the last decade has seen an increasing interest in soft robots: robots that can achieve deformability through their inherent material properties or structural compliance. Nevertheless, the dynamics of these types of robots is often complex and therefore it is difficult to substantiate whether the cost like the required power for changing its shape will be worthwhile to achieve the desired behavior. This article presents an approach in the development and analysis of a shape-changing locomoting robot, which relies on the ability of elastic beams to deform and vibrate. Through a proper use of elastic materials and the robot’s vibration-based dynamics, it will be shown both analytically and experimentally how shape adaptation can be designed such that it leads to desirable behaviors, with better power efficiency compared to when the robot solely relies on changing its control input. The results encourage emerging direction in robotics that investigates approaches to change robots’ behaviors through their adaptive morphology. </jats:p

Engineering derivatives from biological systems for advanced aerospace applications

The present study consisted of a literature survey, a survey of researchers, and a workshop on bionics. These tasks produced an extensive annotated bibliography of bionics research (282 citations), a directory of bionics researchers, and a workshop report on specific bionics research topics applicable to space technology. These deliverables are included as Appendix A, Appendix B, and Section 5.0, respectively. To provide organization to this highly interdisciplinary field and to serve as a guide for interested researchers, we have also prepared a taxonomy or classification of the various subelements of natural engineering systems. Finally, we have synthesized the results of the various components of this study into a discussion of the most promising opportunities for accelerated research, seeking solutions which apply engineering principles from natural systems to advanced aerospace problems. A discussion of opportunities within the areas of materials, structures, sensors, information processing, robotics, autonomous systems, life support systems, and aeronautics is given. Following the conclusions are six discipline summaries that highlight the potential benefits of research in these areas for NASA's space technology programs

Biomechanics of a parasitic wasp ovipositor : Probing for answers

Insects such as mosquitoes, true bugs, and parasitic wasps, probe for resources hidden in various substrates. The resources are often, located deep within the substrate and can only be reached with long and thin (slender) probes. Such probes can, however, easily bend or break (buckle) when pushed inside the substrate, which makes probing a challenging task. Nevertheless, the mentioned insects use their probes repeatedly throughout their lifetime without apparent damage. Furthermore, the probes are also used for sensing the targets, can be steered during insertion, and can transport both fluids (e.g. blood, phloem sap) and eggs. Insect probes seem highly versatile structures that satisfy many functional requirements, including buckling avoidance, steering, sensing, and transport. Similar requirements also hold for minimally invasive medical procedures, where slender tools are used to minimize damage to the patient. Understanding the probing process in insects can bring insights in the insect ecology and evolution and it may also help in the development of novel surgical tools. In this thesis, I focus on the mechanical and motor adaptations of insect probing, while other aspects are only briefly discussed. In chapter 2, we review the literature on the probing structures and their operating principles across mosquitoes, parasitic wasps, and hemipterans. Probes are either modified mouthparts (mosquitoes, true bugs) or special tubular outgrowths of the abdomen (parasitic wasps). Despite having different developmental origins, the probes share three major morphological characteristics, which may reflect the shared functional requirements of buckling avoidance and steering: (i) the probes consist of multiple, interconnected elements that can slide along each other, (ii) the probe diameters are very small, which leaves no space for internal musculature, and (iii) the distal ends (tips) of the probe elements are asymmetric and often bear various serrations, hooks, bulges, or notches. How such slender multi-element probes avoid buckling during insertion has been hypothesized in the so-called push–pull mechanism. According to this mechanism, the probe is inserted into the substrate by reciprocal movements of the elements. The insects therefore simultaneously push on some of the probe elements, while pulling on the others. The tip serrations are directed such, that they primarily increase the friction upon pulling of the elements. This puts the pulled elements under tension and makes them effectively stiffer in bending (like when pulling a rope). The elements under tension can serve as guides along which the other elements are pushed inside the substrate without the risk of buckling. The insect alternates the pushing and pulling between the elements to incrementally insert the probe in the substrate. This mechanism has, however, never been quantified in insects and it was hitherto unknown whether the animals rely on it during probing. The probe tip asymmetry presumably facilitates steering. The asymmetric tip geometry leads to asymmetric reaction forces from the substrate on the tip during insertion, which push the probe tip sideways into a curved path. Controlling the tip geometry therefore allows for control of probing direction. Although offsetting the elements by sliding already changes the shape of the probe tip, these changes might be too small to induce the necessary change of probing direction. A number of mechanisms that enhance the tip asymmetry during the sliding of the elements have been suggested. However, few mechanisms have been observed or studied in vivo, so it is not completely clear how insects steer with their probes. Additionally, the effect of the substrate on both the steering and insertion mechanisms is unknown. To understand the biomechanics of insect probing, we investigated the probing behaviour of the braconid parasitic wasp Diachasmimorpha longicaudata. This is an ideal species for studying the buckling avoidance and steering, because it: (i) possess a slender ovipositor several millimetres in length, (ii) probes into solid material (e.g. citrus fruits), and (iii) attack fruit-fly larvae that are freely moving within the substrate (i.e. steering can be expected). The ovipositor of D. longicaudata is similar to other hymenopterans and consists of three interconnected elements (valves), one dorsal and two ventral ones. The interconnection is a tongue-and-groove mechanism, which allows for sliding of the valves, but prevents their separation. The ovipositor has an asymmetric tip—the distal end of the dorsal valve is enlarged (bulge), while the ventral valve tips have harpoon-like serrations. Additionally, just proximal to the bulge of the dorsal valve, the ovipositor is characteristically bent in an S-shape. This seems to be a feature present only in D. longicaudata and closely related species. The wasps also possess a pair of sheaths that envelop the ovipositor at rest and throughout most of the probing process, but do not penetrate into the substrate. In chapter 3, we studied the kinematics of ovipositor insertion into translucent, artificial substrates of various stiffnesses. Ovipositor insertion was filmed in a three camera setup, which allowed us to reconstruct the ovipositor insertion in 3D, while also monitoring the orientation of the insect’s body. We discovered that the wasps can explore a wide range of the substrate by probing in any direction with respect to their body orientation from a single puncture point. Probing range and speed decreased with increasing substrate stiffness. Wasps used two strategies of ovipositor insertion. In soft substrates, all ovipositor valves were pushed inside the substrate at the same time. In stiff substrates, wasps always moved the valves alternatively, presumably employing the hypothesized push–pull mechanism. We observed that ovipositors can follow curved trajectories inside the substrate. Detailed kinematic analysis revealed that the ovipositors followed a curved path during probing with protracted ventral valve(s). In contrast, probing with protracted dorsal valve resulted in straight trajectories. We linked the changes in the probing direction to the shape changes in the ovipositor tip. When the ventral valves were protracted, they curved towards the dorsal valve, resulting in an enhanced bevel which presumably caused a change in insertion direction. In chapter 4, we investigated the above described steering mechanism by quantifying the bending stiffness (three point bend test) and the geometry (high-resolution computer tomography) of the ovipositor in D. longicaudata. Additionally, we qualitatively assessed the material composition of the valves using fluorescence imaging. The thick dorsal valve bulge might be stiff and could straighten the S-shaped region of the ovipositor during the valve offset, causing bending of the tip. We discovered that the S-shaped region of the ovipositor is significantly softer than its neighbouring regions, which is mostly due to the presence of resilin in the S-shaped region of the ventral valve. Resilin is a rubber-like protein and reduces the stiffness of the otherwise heavily sclerotized valves. Additionally, we showed that the ventral valves have a higher bending stiffness than the dorsal valve along most of their length. The exception is presumably the bulge on the dorsal valve—although we could not directly measure its bending stiffness, its geometrical properties show that it is the thickest (and therefore stiffest) region in the distal end of the ovipositor. Outside the substrate, offsetting of the valves in any direction (i.e. pro- or retraction of the ventral valves) caused a straightening of the S-shaped region of the ovipositor and a curving towards the dorsal side. However, during probing in a substrate, such curving was only observed upon protraction of the ventral valves. We hypothesize this is due to the interaction of the ovipositor with the substrate. Namely, the bevelled ventral valve tips generate substrate reaction forces that promote dorsal curving, while the bevelled tip of the dorsal valve generates substrate forces that promote ventral bending. The interaction between the ventral and dorsal valves straightens the S-shaped region of the ovipositor and enhances dorsal curving. This therefore facilitates strong shape changes of the tip only upon protraction of the ventral valves, while counteracting the ventral curving of the dorsal valve. These opposing mechanisms presumably result in an approximately straight protraction of the dorsal valve. In chapters 2 and 3 we describe how the wasps use the reciprocal valve movements when probing in stiff substrates. As such substrates presumably require strong forces during insertion, the reciprocal valve movements may indeed serve to avoid buckling. However, how the valves are actuated or the forces generated during probing have never been quantified. In chapter 5, we therefore investigated the ovipositor base and the muscles driving the movements of the valves. At the base, the valves attach to plate-like structures that are interconnected with a series of linkages. The muscles attach to these plates and can move them with respect to each other. Such movements also result in the movements of the valves. To analyse the mechanics of this linked system, we performed high-resolution computer tomography scans of wasps in different stages of the probing cycle. This allowed us to compare the configurational changes of the basal plates to the valve offset, and measure the muscle cross-sections and attachment sites. We also calculated the muscle moment arms and estimated the forces and moments of the most relevant musculature actuating the ovipositor movements, by assuming a tensile muscle stress previously reported for insect muscles. For the ventral valves only, we also calculated the forces the valves can exert onto the substrate. The dorsal valve can only be moved by moving the base that is linked inside the abdomen, and therefore force estimation could not be made. The displacement magnitude of the basal plates corresponded to the valve offset, indicating that the valves are indeed moved due to the changes in the arrangement of the basal plates. We also showed that the ventral valve plates move most during the probing cycle, while the magnitude of the dorsal valve plate movements is much smaller. This suggests that the ventral valves move along the dorsal valve, while the dorsal valve moves together with the abdomen during probing. Additionally, in the situation where the animal keeps its abdomen stationary, we estimated the maximal forces actuating the ventral valves. The estimated maximal pushing forces can be higher than the estimated buckling load of the unsupported ovipositor outside the substrate. Assuming the maximal pushing forces are required during probing, antibuckling mechanisms are needed to avoid damaging the ovipositor. Buckling can be limited (prevented) by either supporting the ovipositor outside the substrate with additional sheaths, employing the push–pull mechanism, or both. Subtracting the maximal estimated pushing and pulling forces on the ventral valves, results in a net pushing force that is very close to the buckling threshold of the ovipositor, albeit still slightly higher. The sheaths, although being flexible, might provide the additional support if needed. In this thesis, I show that multi-element probes are inserted into the substrate using reciprocal movements of the individual elements. These movements appear to be necessary in stiff substrates, which presumably require high pushing forces on a single element during probing. This is in accordance with the hypothesis that reciprocal valve movements serve as an anti-buckling mechanism. Additionally, such valve movements are also important for steering of the probe during insertion. The valve offset controls the shape of the probe tip and therefore the net substrate reaction forces that result in bending of the probe. Wasps evolved special structures that enhance the shape changes of their ovipositor tips and facilitate steering. Our findings may be interesting for a broad range of audiences. Entomologists, evolutionary biologists, and ecologists may find them useful when studying the diversification of probing insects, their evolutionary success, or their ecological interactions (e.g. insect–plant, parasite–host). The anti-buckling and steering mechanisms may be helpful when developing novel, man-made probes. These mechanisms allow for minimization of the probe thickness and accurate steering control, which minimizes substrate damage during probing. Our findings may be particularly useful in the development of slender, steerable needles for minimally invasive surgery.</p