Curriculum Reinforcement Learning via Morphology-Environment Co-Evolution

Throughout long history, natural species have learned to survive by evolving their physical structures adaptive to the environment changes. In contrast, current reinforcement learning (RL) studies mainly focus on training an agent with a fixed morphology (e.g., skeletal structure and joint attributes) in a fixed environment, which can hardly generalize to changing environments or new tasks. In this paper, we optimize an RL agent and its morphology through ``morphology-environment co-evolution (MECE)'', in which the morphology keeps being updated to adapt to the changing environment, while the environment is modified progressively to bring new challenges and stimulate the improvement of the morphology. This leads to a curriculum to train generalizable RL, whose morphology and policy are optimized for different environments. Instead of hand-crafting the curriculum, we train two policies to automatically change the morphology and the environment. To this end, (1) we develop two novel and effective rewards for the two policies, which are solely based on the learning dynamics of the RL agent; (2) we design a scheduler to automatically determine when to change the environment and the morphology. In experiments on two classes of tasks, the morphology and RL policies trained via MECE exhibit significantly better generalization performance in unseen test environments than SOTA morphology optimization methods. Our ablation studies on the two MECE policies further show that the co-evolution between the morphology and environment is the key to the success

Electric Vehicle Charging Recommendation and Enabling ICT Technologies: Recent Advances and Future Directions

The introduction of Electric Vehicles (EV) will have a significant impact on the sustainable economic development of urban city. However, compared with traditional gasoline-powered vehicles, EVs currently have limited range, which necessitates regular recharging. Considering the limited charging infrastructure currently available in most countries, infrastructure investments and Renewable Energy Sources (RES) are critical. Thus, service quality provisioning is necessary for realizing EV market. Unlike numerous previous works which investigate "charging scheduling" (referred to when/whether to charge) for EVs already been parked at home/Charging Stations (CSs), a few works focus on “charging recommendation” (refer to where/which CS to charge) for on-the-move EVs. The latter use case cannot be overlooked as it is the most important feature of EVs, especially for driving experience during journeys. On-the-move EVs will travel towards appropriate CSs for charging based on smart decision on where to charge, so as to experience a shorter waiting time for charging. The effort towards sustainable engagement of EVs has not attracted enough attention from both industrial and academia communities. Even if there have been many charging service providers available, the utilization of charging infrastructures is still in need of significant enhancement. Such a situation certainly requires the popularity of EVs towards the sustainable, green and economic market. Enabling the sustainability requires a joint contribution from each domain, e.g., how to guarantee accurate information involved in decision making, how to optimally guide EV drivers towards charging place with the least waiting time, how to schedule charging services for EVs being parked within grid capacity. Achieving this goal is of importance towards a positioning of efficient, scalable and smart ICT framework, makes it feasible to learn the whole picture of grid: - Necessary information needs to be disseminated between stakeholders CSs and EVs, e.g., expected queuing time at individual CSs. In this context, how accurate CSs condition information plays an important role on the optimality of charging recommendation. - Also, it is very time-consuming for the centralized Global Controller (GC) to achieve optimization, by seamlessly collecting data from all EVs and CSs, The complexity and computation load of this centralized solution, increases exponentially with the number of EVs. This paper summaries the recent interdisciplinary research works on EV charging recommendation along with novel ICT frameworks, with an original taxonomy on how Intelligent Transportation Systems (ITS) technologies support the EV charging use case. Future directions are also highlighted to promote the future research

Detection of novel viruses in porcine fecal samples from China

BACKGROUND: Pigs are well known source of human infectious disease. To better understand the spectrum of viruses present in pigs, we utilized the 454 Life Sciences GS-FLX high-throughput sequencing platform to sequence stool samples from healthy pigs. FINDINGS: Total nucleic acid was extracted from stool samples of healthy piglets and randomly amplified. The amplified materials were pooled and processed using a high-throughput pyrosequencing technique. The raw sequences were deconvoluted on the basis of the barcode and then processed through a standardized bioinformatics pipeline. The unique reads (348, 70 and 13) had limited similarity to known astroviruses, bocaviruses and parechoviruses. Specific primers were synthesized to assess the prevalence of the viruses in healthy piglets. Our results indicate extremely high rates of positivity. CONCLUSIONS: Several novel astroviruses, bocaviruses and Ljungan-like viruses were identified in stool samples from healthy pigs. The rates of isolation for the new viruses were high. The high detection rate, diverse sequences and categories indicate that pigs are well-established reservoirs for and likely sources of different enteric viruses

Modeling Multi-Targets Sentiment Classification via Graph Convolutional Networks and Auxiliary Relation

Existing solutions do not work well when multi-targets coexist in a sentence. The reason is that the existing solution is usually to separate multiple targets and process them separately. If the original sentence has N target, the original sentence will be repeated for N times, and only one target will be processed each time. To some extent, this approach degenerates the fine-grained sentiment classification task into the sentencelevel sentiment classification task, and the research method of processing the target separately ignores the internal relation and interaction between the targets. Based on the above considerations, we proposes to use Graph Convolutional Network (GCN) to model and process multi-targets appearing in sentences at the same time based on the positional relationship, and then to construct a graph of the sentiment relationship between targets based on the difference of the sentiment polarity between target words. In addition to the standard target-dependent sentiment classification task, an auxiliary node relation classification task is constructed. Experiments demonstrate that our model achieves new comparable performance on the benchmark datasets: SemEval-2014 Task 4, i.e., reviews for restaurants and laptops. Furthermore, the method of dividing the target words into isolated individuals has disadvantages, and the multi-task learning model is beneficial to enhance the feature extraction ability and expression ability of the model