Rearrangement with Nonprehensile Manipulation Using Deep Reinforcement Learning

Rearranging objects on a tabletop surface by means of nonprehensile manipulation is a task which requires skillful interaction with the physical world. Usually, this is achieved by precisely modeling physical properties of the objects, robot, and the environment for explicit planning. In contrast, as explicitly modeling the physical environment is not always feasible and involves various uncertainties, we learn a nonprehensile rearrangement strategy with deep reinforcement learning based on only visual feedback. For this, we model the task with rewards and train a deep Q-network. Our potential field-based heuristic exploration strategy reduces the amount of collisions which lead to suboptimal outcomes and we actively balance the training set to avoid bias towards poor examples. Our training process leads to quicker learning and better performance on the task as compared to uniform exploration and standard experience replay. We demonstrate empirical evidence from simulation that our method leads to a success rate of 85%, show that our system can cope with sudden changes of the environment, and compare our performance with human level performance.Comment: 2018 International Conference on Robotics and Automatio

Reversible K-Ion intercalation in CrSe2 cathodes for potassium-ion batteries: Combined operando PXRD and DFT studies

In the pursuit of more affordable battery technologies, potassium-ion batteries (KIBs) have emerged as a promising alternative to lithium-ion systems, owing to the abundance and wide distribution of potassium resources. While chalcogenides are uncommon as intercalation cathodes in KIBs, this study's electrochemical tests on CrSe2 revealed a reversible K+ ion intercalation/deintercalation process. The CrSe2 cathode achieved a KIB battery capacity of 125 mAh/g at a 0.1C rate within a practical 1- 3.5 V vs K+/K operation range, nearly matching the theoretical capacity of 127.7 mAh/g. Notably, the battery retained 85% of its initial capacity at a high 1C rate, suggesting that CrSe2 is competitive for high-power applications with many current state-of-the-art cathodes. In-operando PXRD studies uncovered the nature of the intercalation behavior, revealing an initial biphasic region followed by a solid-solution formation during the potassium intercalation process. DFT calculations helped with the possible assignment of intermediate phase structures across the entire CrSe2 – K1.0CrSe2 composition range, providing insights into the experimentally observed phase transformations. The results of this work underscore CrSe2's potential as a high-performance cathode material for KIBs, offering valuable insights into the intercalation mechanisms of layered transition metal chalcogenides and paving the way for future advancements in optimizing KIB cathodes

Arf6 controls beta-amyloid production by regulating macropinocytosis of the Amyloid Precursor Protein to lysosomes

Alzheimer’s disease (AD) is characterized by the deposition of Beta-Amyloid (Aβ) peptides in the brain. Aβ peptides are generated by cleavage of the Amyloid Precursor Protein (APP) by the β − and γ − secretase enzymes. Although this process is tightly linked to the internalization of cell surface APP, the compartments responsible are not well defined. We have found that APP can be rapidly internalized from the cell surface to lysosomes, bypassing early and late endosomes. Here we show by confocal microscopy and electron microscopy that this pathway is mediated by macropinocytosis. APP internalization is enhanced by antibody binding/crosslinking of APP suggesting that APP may function as a receptor. Furthermore, a dominant negative mutant of Arf6 blocks direct transport of APP to lysosomes, but does not affect classical endocytosis to endosomes. Arf6 expression increases through the hippocampus with the development of Alzheimer’s disease, being expressed mostly in the CA1 and CA2 regions in normal individuals but spreading through the CA3 and CA4 regions in individuals with pathologically diagnosed AD. Disruption of lysosomal transport of APP reduces both Aβ40 and Aβ42 production by more than 30 %. Our findings suggest that the lysosome is an important site for Aβ production and that altering APP trafficking represents a viable strategy to reduce Aβ production. ELECTRONIC SUPPLEMENTARY MATERIAL: The online version of this article (doi:10.1186/s13041-015-0129-7) contains supplementary material, which is available to authorized users

Novel Data-Driven decentralized coordination model for electric vehicle aggregator and energy hub entities in multi-energy system using an improved multi-agent DRL approach

Energy hub (EH) is an independent entity that benefits to the efficiency, flexibility, and reliability of integrated energy systems (IESs). On the other hand, the rapid emerging of electric vehicles (EVs) drives the EV aggregator (EVAGG) as another independent entity to facilitate the electricity exchange with the grid. However, due to privacy consideration for different owners, it is challenging to investigate the optimal coordinated strategies for such interconnected entities only by exchanging the information of electrical energy. Besides, the existence of parameter uncertainties (load demands, EVs’ charging behaviors, wind power and photovoltaic generation), continuous decision space, dynamic energy flows, and non-convex multi-objective function is difficult to solve. To this end, this paper proposes a novel model-free multi-agent deep reinforcement learning (MADRL) -based decentralized coordination model to minimize the energy costs of EH entities and maximize profits of EVAGGs. First, a long short-term memory (LSTM) module is used to capture the future trend of uncertainties. Then, the coordination problem is formulated as Markov games and solved by the attention enabled MADRL algorithm, where the EH or EVAGG entity is modeled as an adaptive agent. An attention mechanism makes each agent only focus on state information related to the reward. The proposed MADRL adopts the forms of offline centralized training to learn the optimal coordinated control strategy, and decentralized execution to enable agents’ online decisions to only require local measurements. A safety network is employed to cope with equality constraints (demand–supply balance). Simulation results illustrate that the proposed method achieves similar results compared to the traditional model-based method with perfect knowledge of system models, and the computation performance is at least two orders of magnitudes shorter than the traditional method. The testing results of the proposed method are better than those of the Concurrent and other MADRL method, with 10.79%/3.06% lower energy cost and 17.11%/6.82% higher profits of aggregator. Besides, the electric equality constraint of the proposed method is only 0.25 MW averaged per day, which is a small and acceptable violation

Quantifying wind-induced impacts on particulate Cu footprint in the Yangtze Estuary.

Under two wind conditions, a polar coordinated segmented quantification method (PCSQM) taking the easternmost point of Chongming Island (121°59'20″E, 31°29'38″N) as the origin of the coordinate was proposed to quantify wind-induced impacts on the heavy metal footprint emitted from four simulation sites on the main waterway of the Yangtze Estuary. One wind condition was that of a real wind field in 2019 called Case 1; the other one was a combination of monthly maximum wind speed selected from 1989 to 2019 called Case 2. In the comparison of these two conditions, the PCSQM was used to calculate the footprint excursion of four simulation sites mentioned, including three major urban sewage outlets and the upstream pollution source, represented by Xuliujing (XLJ) during the biological sensitive aggregation period of the Yangtze Estuary (BAPYE). The results showed that the Cu footprint was closer to Chongming Island and showed a trend of narrowing its coverage under Case2 compared with Case1. The Southeast section of the XLJ had the broadest width (83.46 km), while the Southwestern section of BLG had the narrowest width (3.52 km). Coincidentally, both the maximum (-29.99%) and the minimum excursion (-0.13%) were derived from XLJ, corresponding to its Southeast section and Southwest section