Optimizing Reinforcement Learning Control Model in Furuta Pendulum and Transferring it to Real-World

Reinforcement learning does not require explicit robot modeling as it learns on its own based on data, but it has temporal and spatial constraints when transferred to real-world environments. In this research, we trained a balancing Furuta pendulum problem, which is difficult to model, in a virtual environment (Unity) and transferred it to the real world. The challenge of the balancing Furuta pendulum problem is to maintain the pendulum’s end effector in a vertical position. We resolved the temporal and spatial constraints by performing reinforcement learning in a virtual environment. Furthermore, we designed a novel reward function that enabled faster and more stable problem-solving compared to the two existing reward functions. We validate each reward function by applying it to the soft actor-critic (SAC) and proximal policy optimization (PPO). The experimental result shows that cosine reward function is trained faster and more stable. Finally, SAC algorithm model using a cosine reward function in the virtual environment is an optimized controller. Additionally, we evaluated the robustness of this model by transferring it to the real environment

Design of Polarization-Independent and Wide-Angle Broadband Absorbers for Highly Efficient Reflective Structural Color Filters

We propose a design of angle-insensitive and polarization-independent reflective color filters with high efficiency (>80%) based on broad resonance in a Fabry–Pérot cavity where asymmetric metal-dielectric-metal planar structures are employed. Broadband absorption properties allow the resonance in the visible range to remain nearly constant over a broad range of incident angles of up to 40° for both s- and p-polarizations. Effects of the angles of incidence and polarization state of incident light on the purity of the resulting colors are examined on the CIE 1931 chromaticity diagram. In addition, higher-order resonances of the proposed color filters and their electric field distributions are investigated for improved color purity. Lastly, the spectral properties of the proposed structures with different metallic layers are studied. The simple strategy described in this work could be adopted in a variety of research areas, such as color decoration devices, microscopy, and colorimetric sensors

Design of a Sensitive Balloon Sensor for Safe Human–Robot Interaction

As the safety of a human body is the main priority while interacting with robots, the field of tactile sensors has expanded for acquiring tactile information and ensuring safe human–robot interaction (HRI). Existing lightweight and thin tactile sensors exhibit high performance in detecting their surroundings. However, unexpected collisions caused by malfunctions or sudden external collisions can still cause injuries to rigid robots with thin tactile sensors. In this study, we present a sensitive balloon sensor for contact sensing and alleviating physical collisions over a large area of rigid robots. The balloon sensor is a pressure sensor composed of an inflatable body of low-density polyethylene (LDPE), and a highly sensitive and flexible strain sensor laminated onto it. The mechanical crack-based strain sensor with high sensitivity enables the detection of extremely small changes in the strain of the balloon. Adjusting the geometric parameters of the balloon allows for a large and easily customizable sensing area. The weight of the balloon sensor was approximately 2 g. The sensor is employed with a servo motor and detects a finger or a sheet of rolled paper gently touching it, without being damaged

Effect of Metal Thickness on the Sensitivity of Crack-Based Sensors

Among many attempts to make a decent human motion detector in various engineering fields, a mechanical crack-based sensor that deliberately generates and uses nano-scale cracks on a metal deposited thin film is gaining attention for its high sensitivity. While the metal layer of the sensor must be responsible for its high performance, its effects have not received much academic interest. In this paper, we studied the relationship between the thickness of the metal layer and the characteristics of the sensor by depositing a few nanometers of chromium (Cr) and gold (Au) on the PET film. We found that the sensitivity of the crack sensor improves/increases under the following conditions: (1) when Au is thin and Cr is thick; and (2) when the ratio of Au is lower than that of Cr, which also increases the transmittance of the sensor, along with its sensitivity. As we only need a small amount of Au to achieve high sensitivity of the sensor, we have suggested more efficient and economical fabrication methods. With this crack-based sensor, we were able to successfully detect finger motions and to distinguish various signs of American Sign Language (ASL)

Analysis of Preload-Dependent Reversible Mechanical Interlocking Using Beetle-Inspired Wing Locking Device

We report an analysis of preload-dependent reversible interlocking between regularly arrayed, high aspect ratio (AR) polymer micro- and nanofibers. Such a reversible interlocking is inspired from the wing-locking device of a beetle where densely populated microhairs (termed microtrichia) on the cuticular surface form numerous hair-to-hair contacts to maximize lateral shear adhesion. To mimic this, we fabricate various high AR, vertical micro- and nanopillars on a flexible substrate and investigate the shear locking force with different preloads (0.1–10 N/cm²). A simple theoretical model is developed based on the competition between van der Waals (VdW) attraction and deflection forces of pillars, which can explain the preload-dependent maximum deflection, tilting angle, and total shear adhesion force

Polyimide Encapsulation of Spider-Inspired Crack-Based Sensors for Durability Improvement

In mechanical sensory systems, encapsulation is one of the crucial issues to take care of when it comes to protection of the systems from external damage. Recently, a new type of a mechanical strain sensor inspired by spider’s slit organ has been reported, which has incredibly high sensitivity, flexibility, wearability, and multifunctional sensing abilities. In spite of many of these advantages, the sensor is still vulnerable in harsh environments of liquids and/or high temperature, because it has heat-vulnerable polyethylene terephthalate (PET) substrate without any encapsulation layer. Here, we present a mechanical crack-based strain sensor with heat, water and saline solution resistance by alternating the substrate from polyester film to polyimide film and encapsulating the sensor with polyimide. We have demonstrated the ability of the encapsulated crack-based sensor against heat, water, saline solution damage through experiments. Our sensor exhibited reproducibility and durability with high sensitivity to strain (gauge factor above 10,000 at strain of two percent). These results show a new potential of the crack-based sensory system to be used as a wearable voice/motion/pulse sensing device and a high-temperature strain sensor

Design of a Biologically Inspired Water-Walking Robot Powered by Artificial Muscle

The agile and power-efficient locomotion of a water strider has inspired many water-walking devices. These bioinspired water strider robots generally adopt a DC motor to create a sculling trajectory of the driving leg. These robots are, thus, inevitably heavy with many supporting legs decreasing the velocity of the robots. There have only been a few attempts to employ smart materials despite their advantages of being lightweight and having high power densities. This paper proposes an artificial muscle-based water-walking robot capable of moving forward and turning with four degrees of freedom. A compliant amplified shape memory alloy actuator (CASA) used to amplify the strain of a shape memory alloy wire enables a wide sculling motion of the actuation leg with only four supporting legs to support the entire weight of the robot. Design parameters to increase the actuation strain of the actuator and to achieve a desired swing angle (80°) are analyzed. Finally, experiments to measure the forward speed and angular velocities of the robot are carried out to compare with other robots. The robot weighs only 0.236 g and has a maximum and average speed of 1.56, 0.31 body length per second and a maximum and average angular velocity of 145.05°/s and 14.72°/s