Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learning

It is of significance for an agent to learn a widely applicable and general-purpose policy that can achieve diverse goals including images and text descriptions. Considering such perceptually-specific goals, the frontier of deep reinforcement learning research is to learn a goal-conditioned policy without hand-crafted rewards. To learn this kind of policy, recent works usually take as the reward the non-parametric distance to a given goal in an explicit embedding space. From a different viewpoint, we propose a novel unsupervised learning approach named goal-conditioned policy with intrinsic motivation (GPIM), which jointly learns both an abstract-level policy and a goal-conditioned policy. The abstract-level policy is conditioned on a latent variable to optimize a discriminator and discovers diverse states that are further rendered into perceptually-specific goals for the goal-conditioned policy. The learned discriminator serves as an intrinsic reward function for the goal-conditioned policy to imitate the trajectory induced by the abstract-level policy. Experiments on various robotic tasks demonstrate the effectiveness and efficiency of our proposed GPIM method which substantially outperforms prior techniques.Comment: Accepted by AAAI-2

Inferring Economic Condition Uncertainty from Electricity Big Data

Inferring the uncertainties in economic conditions are of significant importance for both decision makers as well as market players. In this paper, we propose a novel method based on Hidden Markov Model (HMM) to construct the Economic Condition Uncertainty (ECU) index that can be used to infer the economic condition uncertainties. The ECU index is a dimensionless index ranges between zero and one, this makes it to be comparable among sectors, regions and periods. We use the daily electricity consumption data of nearly 20 thousand firms in Shanghai from 2018 to 2020 to construct the ECU indexes. Results show that all ECU indexes, no matter at sectoral level or regional level, successfully captured the negative impacts of COVID-19 on Shanghai's economic conditions. Besides, the ECU indexes also presented the heterogeneities in different districts as well as in different sectors. This reflects the facts that changes in uncertainties of economic conditions are mainly related to regional economic structures and targeted regulation policies faced by sectors. The ECU index can also be easily extended to measure uncertainties of economic conditions in different fields which has great potentials in the future

Deep recurrent spiking neural networks capture both static and dynamic representations of the visual cortex under movie stimuli

In the real world, visual stimuli received by the biological visual system are predominantly dynamic rather than static. A better understanding of how the visual cortex represents movie stimuli could provide deeper insight into the information processing mechanisms of the visual system. Although some progress has been made in modeling neural responses to natural movies with deep neural networks, the visual representations of static and dynamic information under such time-series visual stimuli remain to be further explored. In this work, considering abundant recurrent connections in the mouse visual system, we design a recurrent module based on the hierarchy of the mouse cortex and add it into Deep Spiking Neural Networks, which have been demonstrated to be a more compelling computational model for the visual cortex. Using Time-Series Representational Similarity Analysis, we measure the representational similarity between networks and mouse cortical regions under natural movie stimuli. Subsequently, we conduct a comparison of the representational similarity across recurrent/feedforward networks and image/video training tasks. Trained on the video action recognition task, recurrent SNN achieves the highest representational similarity and significantly outperforms feedforward SNN trained on the same task by 15% and the recurrent SNN trained on the image classification task by 8%. We investigate how static and dynamic representations of SNNs influence the similarity, as a way to explain the importance of these two forms of representations in biological neural coding. Taken together, our work is the first to apply deep recurrent SNNs to model the mouse visual cortex under movie stimuli and we establish that these networks are competent to capture both static and dynamic representations and make contributions to understanding the movie information processing mechanisms of the visual cortex

A Review of Smart Materials in Tactile Actuators for Information Delivery

As the largest organ in the human body, the skin provides the important sensory channel for humans to receive external stimulations based on touch. By the information perceived through touch, people can feel and guess the properties of objects, like weight, temperature, textures, and motion, etc. In fact, those properties are nerve stimuli to our brain received by different kinds of receptors in the skin. Mechanical, electrical, and thermal stimuli can stimulate these receptors and cause different information to be conveyed through the nerves. Technologies for actuators to provide mechanical, electrical or thermal stimuli have been developed. These include static or vibrational actuation, electrostatic stimulation, focused ultrasound, and more. Smart materials, such as piezoelectric materials, carbon nanotubes, and shape memory alloys, play important roles in providing actuation for tactile sensation. This paper aims to review the background biological knowledge of human tactile sensing, to give an understanding of how we sense and interact with the world through the sense of touch, as well as the conventional and state-of-the-art technologies of tactile actuators for tactile feedback delivery

MSAT: Matrix stability analysis tool for shock-capturing schemes

The simulation of supersonic or hypersonic flows often suffers from numerical shock instabilities if the flow field contains strong shocks, limiting the further application of shock-capturing schemes. In this paper, we develop the unified matrix stability analysis method for schemes with three-point stencils and present MSAT, an open-source tool to quantitatively analyze the shock instability problem. Based on the finite-volume approach on the structured grid, MSAT can be employed to investigate the mechanism of the shock instability problem, evaluate the robustness of numerical schemes, and then help to develop robust schemes. Also, MSAT has the ability to analyze the practical simulation of supersonic or hypersonic flows, evaluate whether it will suffer from shock instabilities, and then assist in selecting appropriate numerical schemes accordingly. As a result, MSAT is a helpful tool that can investigate the shock instability problem and help to cure it.Comment: 18 pages, 6 figure

Deep Spiking Neural Networks with High Representation Similarity Model Visual Pathways of Macaque and Mouse

Deep artificial neural networks (ANNs) play a major role in modeling the visual pathways of primate and rodent. However, they highly simplify the computational properties of neurons compared to their biological counterparts. Instead, Spiking Neural Networks (SNNs) are more biologically plausible models since spiking neurons encode information with time sequences of spikes, just like biological neurons do. However, there is a lack of studies on visual pathways with deep SNNs models. In this study, we model the visual cortex with deep SNNs for the first time, and also with a wide range of state-of-the-art deep CNNs and ViTs for comparison. Using three similarity metrics, we conduct neural representation similarity experiments on three neural datasets collected from two species under three types of stimuli. Based on extensive similarity analyses, we further investigate the functional hierarchy and mechanisms across species. Almost all similarity scores of SNNs are higher than their counterparts of CNNs with an average of 6.6%. Depths of the layers with the highest similarity scores exhibit little differences across mouse cortical regions, but vary significantly across macaque regions, suggesting that the visual processing structure of mice is more regionally homogeneous than that of macaques. Besides, the multi-branch structures observed in some top mouse brain-like neural networks provide computational evidence of parallel processing streams in mice, and the different performance in fitting macaque neural representations under different stimuli exhibits the functional specialization of information processing in macaques. Taken together, our study demonstrates that SNNs could serve as promising candidates to better model and explain the functional hierarchy and mechanisms of the visual system.Comment: Accepted by Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI-23

Genome-wide analysis and identification of microRNAs in Medicago truncatula under aluminum stress

Numerous studies have shown that plant microRNAs (miRNAs) play key roles in plant growth and development, as well as in response to biotic and abiotic stresses; however, the role of miRNA in legumes under aluminum (Al) stress have rarely been reported. Therefore, here, we aimed to investigate the role of miRNAs in and their mechanism of Al tolerance in legumes. To this end, we sequenced a 12-strand-specific library of Medicago truncatula under Al stress. A total of 195.80 M clean reads were obtained, and 876 miRNAs were identified, of which, 673 were known miRNAs and 203 were unknown. A total of 55 miRNAs and their corresponding 2,502 target genes were differentially expressed at various time points during Al stress. Further analysis revealed that mtr-miR156g-3p was the only miRNA that was significantly upregulated at all time points under Al stress and could directly regulate the expression of genes associated with root cell growth. Three miRNAs, novel_miR_135, novel_miR_182, and novel_miR_36, simultaneously regulated the expression of four Al-tolerant transcription factors, GRAS, MYB, WRKY, and bHLH, at an early stage of Al stress, indicating a response to Al stress. In addition, legume-specific miR2119 and miR5213 were involved in the tolerance mechanism to Al stress by regulating F-box proteins that have protective effects against stress. Our results contribute to an improved understanding of the role of miRNAs in Al stress in legumes and provide a basis for studying the molecular mechanisms of Al stress regulation

Left and right ventricular myocardial deformation and late gadolinium enhancement:incremental prognostic value in amyloid light-chain amyloidosis

Background: Previous cardiac magnetic resonance (CMR) studies have shown that both late gadolinium enhancement (LGE) and left ventricular (LV) strain have prognostic value in amyloid light-chain (AL) amyloidosis, but the right ventricular (RV) strain has not yet been studied. We aim to determine the incremental prognostic value of LV and RV LGE and strain in AL amyloidosis. Methods: This prospective study recruited 87 patients (age, 56.9 +/- 9.1 years; M/F, 56/31) and 20 healthy subjects (age, 52.7 +/- 8.1 years; M/F, 11/9) who underwent CMR. The LV LGE was classified into no, patchy and global groups. The RV LGE was classified into negative and positive groups. Myocardial deformation was measured using a dedicated software. Follow-up was performed for all-cause mortality using Cox proportional hazards regression and Kaplan-Meier curves. Results: During a median follow-up of 21 months, 34 deaths occurred. Presence of LV LGE [HR 2.44, 95% confidence interval (CI), 1.10-5.45, P=0.029] and global longitudinal strain (GLS) (HR 1.13 per 1% absolute decrease, 95% CI, 1.02-1.25, P=0.025) were independent LV predictors. RV LGE (HR 4.07, 95% CI, 1.09-15.24, P=0.037) and GLS (HR 1.10 per 1% absolute decrease, 95% CI, 1.00-1.21, P=0.047) were independent RV predictors. Complementary to LV LGE, LV GLS impairment or RV LGE further reduced survival (both log rank P Conclusions: This study confirms the incremental prognostic value of LV GLS and RV LGE in AL amyloidosis, which refines the conventional risk evaluation based on LV LGE. GLS based on non-contrast-enhanced CMR are promising new predictors