Exploration Entropy for Reinforcement Learning

The training process analysis and termination condition of the training process of a Reinforcement Learning (RL) system have always been the key issues to train an RL agent. In this paper, a new approach based on State Entropy and Exploration Entropy is proposed to analyse the training process. The concept of State Entropy is used to denote the uncertainty for an RL agent to select the action at every state that the agent will traverse, while the Exploration Entropy denotes the action selection uncertainty of the whole system. Actually, the action selection uncertainty of a certain state or the whole system reflects the degree of exploration and the stage of the learning process for an agent. The Exploration Entropy is a new criterion to analyse and manage the training process of RL. The theoretical analysis and experiment results illustrate that the curve of Exploration Entropy contains more information than the existing analytical methods

Behavior-Based Formation Control of Swarm Robots

Swarm robotics is a specific research field of multirobotics where a large number of mobile robots are controlled in a coordinated way. Formation control is one of the most challenging goals for the coordination control of swarm robots. In this paper, a behavior-based control design approach is proposed for two kinds of important formation control problems: efficient initial formation and formation control while avoiding obstacles. In this approach, a classification-based searching method for generating large-scale robot formation is presented to reduce the computational complexity and speed up the initial formation process for any desired formation. The behavior-based method is applied for the formation control of swarm robot systems while navigating in an unknown environment with obstacles. Several groups of experimental results demonstrate the success of the proposed approach. These methods have potential applications for various swarm robot systems in both the simulation and the practical environments

Design of a Machine Vision-Based Automatic Digging Depth Control System for Garlic Combine Harvester

The digging depth is an important factor affecting the mechanized garlic harvesting quality. At present, the digging depth of the garlic combine harvester (GCH) is adjusted manually, which leads to disadvantages such as slow response, poor accuracy, and being very dependent on the operator’s experience. To solve this problem, this paper proposes a machine vision-based automatic digging depth control system for the original garlic digging device. The system uses the improved YOLOv5 algorithm to calculate the length of the garlic root at the front end of the clamping conveyor chain in real-time, and the calculation result is sent back to the system as feedback. Then, the STM32 microcontroller is used to control the digging depth by expanding and contracting the electric putter of the garlic digging device. The experimental results of the presented control system show that the detection time of the system is 30.4 ms, the average accuracy of detection is 99.1%, and the space occupied by the model deployment is 11.4 MB, which suits the design of the real-time detection of the system. Moreover, the length of the excavated garlic roots is shorter than that of the system before modification, which represents a lower energy consumption of the system and a lower rate of impurities in harvesting, and the modified system is automatically controlled, reducing the operator’s workload

Effects of ultrasonic and steam-cooking treatments on the physicochemical properties of bamboo shoots protein and the stability of O/W emulsion

In this study, the effects of ultrasonic and steam-cooking treatments on the physicochemical and emulsifying properties of bamboo shoots protein (BSP) were investigated. The particle size and the polydispersity index (PDI) of U-BSP (ultrasonic-BSP) both decreased. Fourier transform infrared spectroscopy (FTIR) showed that the secondary structure of U-BSP was more loose. Furthermore, X-ray diffraction (XRD) and thermogravimetric (TGA) analysis suggested that crystallinity amd thermal stability of U-BSP both deceased. The water and oil holding capacity (WHC/OHC) of U-BSP increased, while steam-cooking treatment had the reverse effect. We also investigated the effects of ultrasonic and steam-cooking treatments on BSP-stabilized emulsions. The viscosity of emulsion stabilized by U-BSP increased and the distribution of emulsion droplets was more uniform and smaller. The results showed that ultrasonic treatment significantly improved the stability of BSP-stabilized emulsions, while steam-cooking treatment had a significant negative impact on the stability of BSP-stabilized emulsions. The work indicated ultrasonication is an effective treatment to improve the emulsifying properties of BSP