Learning Complicated Manipulation Skills via Deterministic Policy with Limited Demonstrations

Combined with demonstrations, deep reinforcement learning can efficiently develop policies for manipulators. However, it takes time to collect sufficient high-quality demonstrations in practice. And human demonstrations may be unsuitable for robots. The non-Markovian process and over-reliance on demonstrations are further challenges. For example, we found that RL agents are sensitive to demonstration quality in manipulation tasks and struggle to adapt to demonstrations directly from humans. Thus it is challenging to leverage low-quality and insufficient demonstrations to assist reinforcement learning in training better policies, and sometimes, limited demonstrations even lead to worse performance. We propose a new algorithm named TD3fG (TD3 learning from a generator) to solve these problems. It forms a smooth transition from learning from experts to learning from experience. This innovation can help agents extract prior knowledge while reducing the detrimental effects of the demonstrations. Our algorithm performs well in Adroit manipulator and MuJoCo tasks with limited demonstrations

A Versatile Agent for Fast Learning from Human Instructors

In recent years, a myriad of superlative works on intelligent robotics policies have been done, thanks to advances in machine learning. However, inefficiency and lack of transfer ability hindered algorithms from pragmatic applications, especially in human-robot collaboration, when few-shot fast learning and high flexibility become a wherewithal. To surmount this obstacle, we refer to a "Policy Pool", containing pre-trained skills that can be easily accessed and reused. An agent is employed to govern the "Policy Pool" by unfolding requisite skills in a flexible sequence, contingent on task specific predilection. This predilection can be automatically interpreted from one or few human expert demonstrations. Under this hierarchical setting, our algorithm is able to pick up a sparse-reward, multi-stage knack with only one demonstration in a Mini-Grid environment, showing the potential for instantly mastering complex robotics skills from human instructors. Additionally, the innate quality of our algorithm also allows for lifelong learning, making it a versatile agent

Performance analysis of high-speed railway communication systems subjected to co-channel interference and channel estimation errors

The performance of high-speed railway wireless communication systems is studied in the presence of co-channel interference and imperfect channel estimation in the uplink. The authors derive exact closed-form expressions for the outage probability and investigate the impact of fading severity. New explicit expressions are derived for both the level crossing rate and average outage duration for illustrating the impact of mobile speed and channel estimation errors on the achievable system performance. Our results are generalised and hence they subsume a range of previously reported results

Edge-aware Hard Clustering Graph Pooling for Brain Imaging Data

Graph Convolutional Networks (GCNs) can capture non-Euclidean spatial dependence between different brain regions, and the graph pooling operator in GCNs is key to enhancing the representation learning capability and acquiring abnormal brain maps. However, the majority of existing research designs graph pooling operators only from the perspective of nodes while disregarding the original edge features, in a way that not only confines graph pooling application scenarios, but also diminishes its ability to capture critical substructures. In this study, a clustering graph pooling method that first supports multidimensional edge features, called Edge-aware hard clustering graph pooling (EHCPool), is developed. EHCPool proposes the first 'Edge-to-node' score evaluation criterion based on edge features to assess node feature significance. To more effectively capture the critical subgraphs, a novel Iteration n-top strategy is further designed to adaptively learn sparse hard clustering assignments for graphs. Subsequently, an innovative N-E Aggregation strategy is presented to aggregate node and edge feature information in each independent subgraph. The proposed model was evaluated on multi-site brain imaging public datasets and yielded state-of-the-art performance. We believe this method is the first deep learning tool with the potential to probe different types of abnormal functional brain networks from data-driven perspective. Core code is at: https://github.com/swfen/EHCPool

Towards Fair Patient-Trial Matching via Patient-Criterion Level Fairness Constraint

Clinical trials are indispensable in developing new treatments, but they face obstacles in patient recruitment and retention, hindering the enrollment of necessary participants. To tackle these challenges, deep learning frameworks have been created to match patients to trials. These frameworks calculate the similarity between patients and clinical trial eligibility criteria, considering the discrepancy between inclusion and exclusion criteria. Recent studies have shown that these frameworks outperform earlier approaches. However, deep learning models may raise fairness issues in patient-trial matching when certain sensitive groups of individuals are underrepresented in clinical trials, leading to incomplete or inaccurate data and potential harm. To tackle the issue of fairness, this work proposes a fair patient-trial matching framework by generating a patient-criterion level fairness constraint. The proposed framework considers the inconsistency between the embedding of inclusion and exclusion criteria among patients of different sensitive groups. The experimental results on real-world patient-trial and patient-criterion matching tasks demonstrate that the proposed framework can successfully alleviate the predictions that tend to be biased

Robust Multimodal Failure Detection for Microservice Systems

Proactive failure detection of instances is vitally essential to microservice systems because an instance failure can propagate to the whole system and degrade the system's performance. Over the years, many single-modal (i.e., metrics, logs, or traces) data-based nomaly detection methods have been proposed. However, they tend to miss a large number of failures and generate numerous false alarms because they ignore the correlation of multimodal data. In this work, we propose AnoFusion, an unsupervised failure detection approach, to proactively detect instance failures through multimodal data for microservice systems. It applies a Graph Transformer Network (GTN) to learn the correlation of the heterogeneous multimodal data and integrates a Graph Attention Network (GAT) with Gated Recurrent Unit (GRU) to address the challenges introduced by dynamically changing multimodal data. We evaluate the performance of AnoFusion through two datasets, demonstrating that it achieves the F1-score of 0.857 and 0.922, respectively, outperforming the state-of-the-art failure detection approaches

Senescent renal tubular epithelial cells activate fibroblasts by secreting Shh to promote the progression of diabetic kidney disease

IntroductionDiabetic kidney disease (DKD) is one of the complications of diabetes; however, the pathogenesis is not yet clear. A recent study has shown that senescence is associated with the course of DKD. In the present study, we explored whether senescent renal tubular cells promote renal tubulointerstitial fibrosis by secreting Sonic hedgehog (Shh) which mediates fibroblast activation and proliferation in DKD.MethodsA 36-week-old db/db mice model and the renal tubular epithelial cells were cultured in high glucose (HG, 60 mmol/L) medium for in vivo and in vitro experiments.ResultsCompared to db/m mice, blood glucose, microalbuminuria, serum creatinine, urea nitrogen, and UACR (microalbuminuria/urine creatinine) were markedly increased in db/db mice. Collagen III, monocyte chemoattractant protein-1 (MCP-1), and tumor necrosis factor-alpha (TNF-α) were also increased in db/db mice kidneys, suggesting fibrosis and inflammation in the organ. Moreover, the detection of SA-β-galactosidase (SA-β-Gal) showed that the activity of SA-β-Gal in the cytoplasm of renal tubular epithelial cells increased, and the cell cycle inhibition of the expression of senescence-related gene cell cycle inhibitor p16INK4A protein and p21 protein increased, indicating that renal fibrosis in db/db mice was accompanied by cell senescence. Furthermore, Shh is highly expressed in the injured renal tubules and in the kidney tissue of db/db mice, as detected by enzyme-linked immunosorbent assay (ELISA). The results of immunofluorescence staining showed increased positive staining for Shh in renal tubular epithelial cells of db/db mice and decreased positive staining for Lamin B1, but increased positive staining for γH2A.X in cells with high Shh expression; similar results were obtained in vitro. In addition, HG stimulated renal tubular epithelial cells to secrete Shh in the supernatant of the medium. D-gal treatment of renal tubular epithelial cells increased the protein levels of Shh and p21. We also found enhanced activation and proliferation of fibroblasts cultured with the supernatant of renal tubular epithelial cells stimulated by HG medium but the proliferative effect was significantly diminished when co-cultured with cyclopamine (CPN), an inhibitor of the Shh pathway.DiscussionIn conclusion, HG induces renal tubular epithelial cell senescence, and the secretion of senescence-associated proteins and Shh mediates inflammatory responses and fibroblast activation and proliferation, ultimately leading to renal fibrosis

Low- and high-thermogenic brown adipocyte subpopulations coexist in murine adipose tissue

Brown adipose tissue (BAT), as the main site of adaptive thermogenesis, exerts beneficial metabolic effects on obesity and insulin resistance. BAT has been previously assumed to contain a homogeneous population of brown adipocytes. Utilizing multiple mouse models capable of genetically labeling different cellular populations, as well as single-cell RNA sequencing and 3D tissue profiling, we discovered a new brown adipocyte subpopulation with low thermogenic activity coexisting with the classical high-thermogenic brown adipocytes within the BAT. Compared with the high-thermogenic brown adipocytes, these low-thermogenic brown adipocytes had substantially lower Ucp1 and Adipoq expression, larger lipid droplets, and lower mitochondrial content. Functional analyses showed that, unlike the high-thermogenic brown adipocytes, the low-thermogenic brown adipocytes have markedly lower basal mitochondrial respiration, and they are specialized in fatty acid uptake. Upon changes in environmental temperature, the 2 brown adipocyte subpopulations underwent dynamic interconversions. Cold exposure converted low-thermogenic brown adipocytes into high-thermogenic cells. A thermoneutral environment had the opposite effect. The recruitment of high-thermogenic brown adipocytes by cold stimulation is not affected by high fat diet feeding, but it does substantially decline with age. Our results revealed a high degree of functional heterogeneity of brown adipocytes

Microgrid architectures for low voltage distributed generation

[EN] The high penetration of distributed generators, most of them based on renewable energy sources, is modifying the traditional structure of the electric distribution grid. If the power of distributed generators is high enough to feed the loads of a certain area, this area could be disconnected from the main grid and operate in islanded mode. Microgrids are composed by distributed generators, energy storage devices, intelligent circuit breakers and local loads. In this paper, a review of the main microgrid architectures proposed in the literature has been carried out. The microgrid architectures are first classified regarding their AC or DC distribution buses. Besides, more complex microgrid architectures are shown. Both advantages and disadvantages of each one of the microgrid families are discussed.This work is supported by the Spanish Ministry of Science and Innovation under Grant ENE2012-37667-C02-01.Patrao Herrero, I.; Figueres Amorós, E.; Garcerá Sanfeliú, G.; González Medina, R. (2015). Microgrid architectures for low voltage distributed generation. Renewable and Sustainable Energy Reviews. 43:415-424. https://doi.org/10.1016/j.rser.2014.11.054S4154244

Association of FTH1-expressing circulating tumor cells with efficacy of neoadjuvant chemotherapy for patients with breast cancer: a prospective cohort study

Background The association between different phenotypes and genotypes of circulating tumor cells (CTCs) and efficacy of neoadjuvant chemotherapy (NAC) remains uncertain. This study was conducted to evaluate the relationship of FTH1 gene-associated CTCs (F-CTC) with/without epithelial-mesenchymal transition (EMT) markers, or their dynamic changes with the efficacy of NAC in patients with non-metastatic breast cancer. Patients and Methods This study enrolled 120 patients with non-metastatic breast cancer who planned to undergo NAC. The FTH1 gene and EMT markers in CTCs were detected before NAC (T0), after 2 cycles of chemotherapy (T1), and before surgery (T2). The associations of these different types of CTCs with rates of pathological complete response (pCR) and breast-conserving surgery (BCS) were evaluated using the binary logistic regression analysis. Results F-CTC in peripheral blood ≥1 at T0 was an independent factor for pCR rate in patients with HER2-positive (odds ratio [OR]=0.08, 95% confidence interval [CI], 0.01-0.98, P = .048). The reduction in the number of F-CTC at T2 was an independent factor for BCS rate (OR = 4.54, 95% CI, 1.14-18.08, P = .03). Conclusions The number of F-CTC prior to NAC was related to poor response to NAC. Monitoring of F-CTC may help clinicians formulate personalized NAC regimens and implement BCS for patients with non-metastatic breast cancer