An Intelligent Social Learning-based Optimization Strategy for Black-box Robotic Control with Reinforcement Learning

Implementing intelligent control of robots is a difficult task, especially when dealing with complex black-box systems, because of the lack of visibility and understanding of how these robots work internally. This paper proposes an Intelligent Social Learning (ISL) algorithm to enable intelligent control of black-box robotic systems. Inspired by mutual learning among individuals in human social groups, ISL includes learning, imitation, and self-study styles. Individuals in the learning style use the Levy flight search strategy to learn from the best performer and form the closest relationships. In the imitation style, individuals mimic the best performer with a second-level rapport by employing a random perturbation strategy. In the self-study style, individuals learn independently using a normal distribution sampling method while maintaining a distant relationship with the best performer. Individuals in the population are regarded as autonomous intelligent agents in each style. Neural networks perform strategic actions in three styles to interact with the environment and the robot and iteratively optimize the network policy. Overall, ISL builds on the principles of intelligent optimization, incorporating ideas from reinforcement learning, and possesses strong search capabilities, fast computation speed, fewer hyperparameters, and insensitivity to sparse rewards. The proposed ISL algorithm is compared with four state-of-the-art methods on six continuous control benchmark cases in MuJoCo to verify its effectiveness and advantages. Furthermore, ISL is adopted in the simulation and experimental grasping tasks of the UR3 robot for validations, and satisfactory solutions are yielded

Continual Learning For On-Device Environmental Sound Classification

Continuously learning new classes without catastrophic forgetting is a challenging problem for on-device environmental sound classification given the restrictions on computation resources (e.g., model size, running memory). To address this issue, we propose a simple and efficient continual learning method. Our method selects the historical data for the training by measuring the per-sample classification uncertainty. Specifically, we measure the uncertainty by observing how the classification probability of data fluctuates against the parallel perturbations added to the classifier embedding. In this way, the computation cost can be significantly reduced compared with adding perturbation to the raw data. Experimental results on the DCASE 2019 Task 1 and ESC-50 dataset show that our proposed method outperforms baseline continual learning methods on classification accuracy and computational efficiency, indicating our method can efficiently and incrementally learn new classes without the catastrophic forgetting problem for on-device environmental sound classification.Comment: The first two authors contributed equally, 5 pages one figure, submitted to DCASE2022 Worksho

Emotional Voice Puppetry

The paper presents emotional voice puppetry, an audio-based facial animation approach to portray characters with vivid emotional changes. The lips motion and the surrounding facial areas are controlled by the contents of the audio, and the facial dynamics are established by category of the emotion and the intensity. Our approach is exclusive because it takes account of perceptual validity and geometry instead of pure geometric processes. Another highlight of our approach is the generalizability to multiple characters. The findings showed that training new secondary characters when the rig parameters are categorized as eye, eyebrows, nose, mouth, and signature wrinkles is significant in achieving better generalization results compared to joint training. User studies demonstrate the effectiveness of our approach both qualitatively and quantitatively. Our approach can be applicable in AR/VR and 3DUI, namely, virtual reality avatars/self-avatars, teleconferencing and in-game dialogue

Maternal exposure to ambient air pollution and congenital heart defects in China

Background: Evidence of maternal exposure to ambient air pollution on congenital heart defects (CHD) has been mixed and are still relatively limited in developing countries. We aimed to investigate the association between maternal exposure to air pollution and CHD in China.Method: This longitudinal, population-based, case-control study consecutively recruited fetuses with CHD and healthy volunteers from 21 cities, Southern China, between January 2006 and December 2016. Residential address at delivery was linked to random forests models to estimate maternal exposure to particulate matter with an aerodynamic diameter of ≤1 µm (PM1), ≤2.5 µm, and ≤10 µm as well as nitrogen dioxides, in three trimesters. The CHD cases were evaluated by obstetrician, pediatrician, or cardiologist, and confirmed by cardia ultrasound. The CHD subtypes were coded using the International Classification Diseases. Adjusted logistic regression models were used to assess the associations between air pollutants and CHD and its subtypes.Results: A total of 7055 isolated CHD and 6423 controls were included in the current analysis. Maternal air pollution exposures were consistently higher among cases than those among controls. Logistic regression analyses showed that maternal exposure to all air pollutants during the first trimester was associated with an increased odds of CHD (e.g., an interquartile range [13.3 µg/m3] increase in PM1 was associated with 1.09-fold ([95% confidence interval, 1.01-1.18]) greater odds of CHD). No significant associations were observed for maternal air pollution exposures during the second trimester and the third trimester. The pattern of the associations between air pollutants and different CHD subtypes was mixed.Conclusions: Maternal exposure to greater levels of air pollutants during the pregnancy, especially the first trimester, is associated with higher odds of CHD in offspring. Further longitudinal well-designed studies are warranted to confirm our findings