Deep reinforcement learning for predictive aircraft maintenance using probabilistic Remaining-Useful-Life prognostics

The increasing availability of sensor monitoring data has stimulated the development of Remaining-Useful-Life (RUL) prognostics and maintenance planning models. However, existing studies focus either on RUL prognostics only, or propose maintenance planning based on simple assumptions about degradation trends. We propose a framework to integrate data-driven probabilistic RUL prognostics into predictive maintenance planning. We estimate the distribution of RUL using Convolutional Neural Networks with Monte Carlo dropout. These prognostics are updated over time, as more measurements become available. We further pose the maintenance planning problem as a Deep Reinforcement Learning (DRL) problem where maintenance actions are triggered based on the estimates of the RUL distribution. We illustrate our framework for the maintenance of aircraft turbofan engines. Using our DRL approach, the total maintenance cost is reduced by 29.3% compared to the case when engines are replaced at the mean-estimated-RUL. In addition, 95.6% of unscheduled maintenance is prevented, and the wasted life of the engines is limited to only 12.81 cycles. Overall, we propose a roadmap for predictive maintenance from sensor measurements to data-driven probabilistic RUL prognostics, to maintenance planning

An investigation of the role of mutations in the spliceosome machinery in the myelodysplastic syndromes

The myelodysplastic syndromes (MDS) are common myeloid malignancies. Mutations in splicing factor genes (including SF3B1, SRSF2 and U2AF1) occur in over half of MDS patients and result in aberrant pre-mRNA splicing of many target genes, indicating that aberrant spliceosome function plays a key role in the pathogenesis of MDS. However, the molecular mechanisms through which the splicing factor mutations drive the MDS phenotype are not fully understood. A previous study from our group has shown that the U2AF1S34F mutations induces aberrant splicing of STRAP in cells of the erythroid lineage. We have identified a splicing event of STRAP, identical to the one caused by the U2AF1S34F mutation, in the erythroid precursors of SRSF2 mutant MDS cases. Functional studies demonstrated that knockdown of STRAP leads to inactivation of p38 MAPK and downregulation of CSDE1-bound transcripts, suggesting that underlying mechanism of ineffective erythropoiesis in MDS with low expression of STRAP is caused by SRSF2 mutations. We have also performed an analysis on global splicing alteration using transcriptomic data of splicing factor mutant MDS patients, which revealed that splicing factor mutations (SF3B1 and SRSF2) reshape the mRNA splicing landscape in MDS. This analysis identified that alternative splicing is cell-type dependent and splicing factor mutations alter the pattern of whole mRNA splicing within each bone marrow subpopulation. SRSF2 is the most frequently mutated splicing factor gene with adverse prognosis in MDS. SRSF2 mutations commonly co-occur with mutations of other specific genes, most frequently TET2 and ASXL1. We performed single-cell transcriptomic analysis using the 10X Genomics platform on haematopoietic stem and progenitor cells of MDS patients harbouring SRSF2 mutations and co-mutations of TET2 and/or ASXL1. Cell population composition analysis revealed differences in population abundance across the genotype groups. Differential gene expression analysis unveiled genotype-specific gene expression signatures and dysregulated pathways

Deep Recurrent Q-Network Methods for mmWave Beam Tracking systems

This article studies a reinforcement learning (RL) approach for beam tracking problems in millimeter-wave massive multiple-input multiple-output (MIMO) systems. Entire beam sweeping in traditional beam training problems is intractable due to prohibitive search overheads. To solve this issue, a partially observable Markov decision process (POMDP) formulation can be applied where decisions are made with partial beam sweeping. However, the POMDP cannot be straightforwardly addressed by existing RL approaches which are intended for fully observable environments. In this paper, we propose a deep recurrent Q-learning (DRQN) method which provides an efficient beam decision policy only with partial observations. Numerical results validate the superiority of the proposed method over conventional schemes

Decentralized Computation Offloading with Cooperative UAVs: Multi-Agent Deep Reinforcement Learning Perspective

Limited computing resources of internet-of-things (IoT) nodes incur prohibitive latency in processing input data. This triggers new research opportunities toward task offloading systems where edge servers handle intensive computations of IoT devices. Deploying the computing servers at existing base stations may not be sufficient to support IoT nodes operating in a harsh environment. This requests mobile edge servers to be mounted on unmanned aerial vehicles (UAVs) that provide on-demand mobile edge computing (MEC) services. Time-varying offloading demands and mobility of UAVs need a joint design of the optimization variables for all time instances. Therefore, an online decision mechanism is essential for UAV-aided MEC networks. This article presents an overview of recent deep reinforcement learning (DRL) approaches where decisions about UAVs and IoT nodes are taken in an online manner. Specifically, joint optimization over task offloading, resource allocation, and UAV mobility is addressed from the DRL perspective. For the decentralized implementation, a multi-agent DRL method is proposed where multiple intelligent UAVs cooperatively determine their computations and communication policies without central coordination. Numerical results demonstrate that the proposed decentralized learning strategy is superior to existing DRL solutions. The proposed framework sheds light on the viability of the decentralized DRL techniques in designing self-organizing IoT networks

Deep reinforcement learning for predictive aircraft maintenance using probabilistic Remaining-Useful-Life prognostics

The increasing availability of sensor monitoring data has stimulated the development of Remaining-Useful-Life (RUL) prognostics and maintenance planning models. However, existing studies focus either on RUL prognostics only, or propose maintenance planning based on simple assumptions about degradation trends. We propose a framework to integrate data-driven probabilistic RUL prognostics into predictive maintenance planning. We estimate the distribution of RUL using Convolutional Neural Networks with Monte Carlo dropout. These prognostics are updated over time, as more measurements become available. We further pose the maintenance planning problem as a Deep Reinforcement Learning (DRL) problem where maintenance actions are triggered based on the estimates of the RUL distribution. We illustrate our framework for the maintenance of aircraft turbofan engines. Using our DRL approach, the total maintenance cost is reduced by 29.3% compared to the case when engines are replaced at the mean-estimated-RUL. In addition, 95.6% of unscheduled maintenance is prevented, and the wasted life of the engines is limited to only 12.81 cycles. Overall, we propose a roadmap for predictive maintenance from sensor measurements to data-driven probabilistic RUL prognostics, to maintenance planning

Modified Dynamic Physical Model of Valence Change Mechanism Memristors

© 2022 American Chemical Society.Valence change-type resistance switching behaviors in oxides can be understood by well-established physical models describing the field-driven oxygen vacancy distribution change. In those models, electroformed residual oxygen vacancy filaments are crucial as they work as an electric field concentrator and limit the oxygen vacancy movement along the vertical direction. Therefore, their movement outward by diffusion is negligible. However, this situation may not be applicable in the electroforming-free system, where the field-driven movement is less prominent, and the isotropic oxygen vacancy diffusion by concentration gradient is more significant, which has not been given much consideration in the conventional model. Here, we propose a modified physical model that considers the change in the oxygen vacancies' charged state depending on their concentrations and the resulting change in diffusivity during switching to interpret the electroforming-free device behaviors. The model suggests formation of an hourglass-shaped filament constituting a lower concentration of oxygen vacancies due to the fluid oxygen diffusion in the thin oxide. Consequently, the proposed model can explain the electroforming-free device behaviors, including the retention failure mechanism, and suggest an optimized filament configuration for improved retention characteristics. The proposed model can plausibly explain both the electroformed and the electroforming-free devices. Therefore, it can be a standard model for valence change memristors.N