Fast Unsupervised Deep Outlier Model Selection with Hypernetworks

Outlier detection (OD) finds many applications with a rich literature of numerous techniques. Deep neural network based OD (DOD) has seen a recent surge of attention thanks to the many advances in deep learning. In this paper, we consider a critical-yet-understudied challenge with unsupervised DOD, that is, effective hyperparameter (HP) tuning/model selection. While several prior work report the sensitivity of OD models to HPs, it becomes ever so critical for the modern DOD models that exhibit a long list of HPs. We introduce HYPER for tuning DOD models, tackling two fundamental challenges: (1) validation without supervision (due to lack of labeled anomalies), and (2) efficient search of the HP/model space (due to exponential growth in the number of HPs). A key idea is to design and train a novel hypernetwork (HN) that maps HPs onto optimal weights of the main DOD model. In turn, HYPER capitalizes on a single HN that can dynamically generate weights for many DOD models (corresponding to varying HPs), which offers significant speed-up. In addition, it employs meta-learning on historical OD tasks with labels to train a proxy validation function, likewise trained with our proposed HN efficiently. Extensive experiments on 35 OD tasks show that HYPER achieves high performance against 8 baselines with significant efficiency gains.Comment: 10 pages, 6 figure

PINNsFormer: A Transformer-Based Framework For Physics-Informed Neural Networks

Physics-Informed Neural Networks (PINNs) have emerged as a promising deep learning framework for approximating numerical solutions for partial differential equations (PDEs). While conventional PINNs and most related studies adopt fully-connected multilayer perceptrons (MLP) as the backbone structure, they have neglected the temporal relations in PDEs and failed to approximate the true solution. In this paper, we propose a novel Transformer-based framework, namely PINNsFormer, that accurately approximates PDEs' solutions by capturing the temporal dependencies with multi-head attention mechanisms in Transformer-based models. Instead of approximating point predictions, PINNsFormer adapts input vectors to pseudo sequences and point-wise PINNs loss to a sequential PINNs loss. In addition, PINNsFormer is equipped with a novel activation function, namely Wavelet, which anticipates the Fourier decomposition through deep neural networks. We empirically demonstrate PINNsFormer's ability to capture the PDE solutions for various scenarios, in which conventional PINNs have failed to learn. We also show that PINNsFormer achieves superior approximation accuracy on such problems than conventional PINNs with non-sensitive hyperparameters, in trade of marginal computational and memory costs, with extensive experiments.Comment: 15 pages (including 9 pages of main text, 3 pages of references, and 3 pages of appendix), 4 figures, 5 table

Combining Machine Learning Models using combo Library

Model combination, often regarded as a key sub-field of ensemble learning, has been widely used in both academic research and industry applications. To facilitate this process, we propose and implement an easy-to-use Python toolkit, combo, to aggregate models and scores under various scenarios, including classification, clustering, and anomaly detection. In a nutshell, combo provides a unified and consistent way to combine both raw and pretrained models from popular machine learning libraries, e.g., scikit-learn, XGBoost, and LightGBM. With accessibility and robustness in mind, combo is designed with detailed documentation, interactive examples, continuous integration, code coverage, and maintainability check; it can be installed easily through Python Package Index (PyPI) or https://github.com/yzhao062/combo.Comment: In Proceedings of Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020

Downregulation of Organic Anion Transporting Polypeptide (OATP) 1B1 Transport Function by Lysosomotropic Drug Chloroquine: Implication in OATP-Mediated Drug-Drug Interactions

Organic anion transporting polypeptide (OATP) 1B1 mediates the hepatic uptake of many drugs including lipid-lowering statins. Decreased OATP1B1 transport activity is often associated with increased systemic exposure of statins and statin-induced myopathy. Antimalarial drug chloroquine (CQ) is also used for long-term treatment of rheumatoid arthritis and systemic lupus erythematosus. CQ is lysosomotropic and inhibits protein degradation in lysosomes. The current studies were designed to determine the effects of CQ on OATP1B1 protein degradation, OATP1B1-mediated transport in OATP1B1-overexpressing cell line, and statin uptake in human sandwich-cultured hepatocytes (SCH). Treatment with lysosome inhibitor CQ increased OATP1B1 total protein levels in HEK293-OATP1B1 cells and in human SCH as determined by OATP1B1 immunoblot. In HEK293-FLAG-tagged OATP1B1 stable cell line, co-immunofluorescence staining indicated that intracellular FLAG-OATP1B1 is colocalized with lysosomal associated membrane glycoprotein (LAMP)-2, a marker protein of late endosome/lysosome. Enlarged LAMP-2-positive vacuoles with FLAG-OATP1B1 protein retained inside were readily detected in CQ-treated cells, consistent with blocking lysosomal degradation of OATP1B1 by CQ. In HEK293-OATP1B1 cells, without pre-incubation, CQ concentrations up to 100 μM did not affect OATP1B1-mediated [3H]E217G accumulation. However, pre-incubation with CQ at clinically relevant concentration(s) significantly decreased [3H]E217G and [3H]pitavastatin accumulation in HEK293-OATP1B1 cells and [3H]pitavastatin accumulation in human SCH. CQ pretreatment (25 μM, 2 h) resulted in ∼1.9-fold decrease in Vmax without affecting Km of OATP1B1-mediated [3H]E217G transport in HEK293-OATP1B1 cells. Pretreatment with monensin and bafilomycin A1, which also have lysosome inhibition activity, significantly decreased OATP1B1-mediated transport in HEK293-OATP1B1 cells. Pharmacoepidemiologic studies using data from the U.S. Food and Drug Administration Adverse Event Reporting System indicated that CQ plus pitavastatin, rosuvastatin, and pravastatin, which are minimally metabolized by the cytochrome P450 enzymes, led to higher myopathy risk than these statins alone. In summary, the present studies report novel findings that lysosome is involved in degradation of OATP1B1 protein and that pre-incubation with lysosomotropic drug CQ downregulates OATP1B1 transport activity. Our in vitro data in combination with pharmacoepidemiologic studies support that CQ has potential to cause OATP-mediated drug–drug interactions

Degradable gene delivery systems based on Pluronics-modified low-molecular-weight polyethylenimine: preparation, characterization, intracellular trafficking, and cellular distribution

Wei Fan1,2,*, Xin Wu1,*, Baoyue Ding3,*, Jing Gao4, Zhen Cai1, Wei Zhang1, Dongfeng Yin1, Xiang Wang1, Quangang Zhu1, Jiyong Liu1, Xueying Ding4, Shen Gao1 1Department of Pharmaceutics, Changhai Hospital, Second Military Medical University, Shanghai, 2Department of Pharmaceutics, The 425th Hospital of PLA, Sanya, 3Department of Pharmaceutics, Medical College of Jiaxing University, Jiaxing, 4Department of Pharmaceutics, School of Pharmacy, Second Military Medical University, Shanghai, People's Republic of China*These authors contributed equally to this workBackground: Cationic copolymers consisting of polycations linked to nonionic amphiphilic block polymers have been evaluated as nonviral gene delivery systems, and a large number of different polymers and copolymers of linear, branched, and dendrimeric architectures have been tested in terms of their suitability and efficacy for in vitro and in vivo transfection. However, the discovery of new potent materials still largely relies on empiric approaches rather than a rational design. The authors investigated the relationship between the polymers' structures and their biological performance, including DNA compaction, toxicity, transfection efficiency, and the effect of cellular uptake.Methods: This article reports the synthesis and characterization of a series of cationic copolymers obtained by grafting polyethyleneimine with nonionic amphiphilic surfactant polyether-Pluronic® consisting of hydrophilic ethylene oxide and hydrophobic propylene oxide blocks. Transgene expression, cytotoxicity, localization of plasmids, and cellular uptake of these copolymers were evaluated following in vitro transfection of HeLa cell lines with various individual components of the copolymers.Results: Pluronics can exhibit biological activity including effects on enhancing DNA cellular uptake, nuclear translocation, and gene expression. The Pluronics with a higher hydrophilic-lipophilic balance value lead to homogeneous distribution in the cytoplasm; those with a lower hydrophilic-lipophilic balance value prefer to localize in the nucleus.Conclusion: This Pluronic-polyethyleneimine system may be worth exploring as components in the cationic copolymers as the DNA or small interfering RNA/microRNA delivery system in the near future.Keywords: Pluronics, gene transfer, nonviral vectors, transfection efficiency, cellular uptak

Diffusion-weighted imaging lesions after endovascular treatment of cerebral aneurysms: A network meta-analysis

BackgroundThromboembolism is one of the common complications in endovascular treatments including coiling alone, stent-assisted coiling (SAC), balloon-assisted coiling (BAC), and flow-diverting (FD) stents. Such treatments are widely used in intracranial aneurysms (IAs), which usually present as positive lesions in diffusion-weighted imaging (DWI). Whether these adjunctive techniques increase postprocedural DWI-positive lesions after endovascular treatment remains unclear.MethodsA thorough electronic search for the literature published in English between January 2000 and October 2022 was conducted on PubMed, Medline, and EMBASE. Eighteen studies (3 cohort studies and 15 case–control studies) involving 1,843 patients with unruptured IAs (UIAs) were included. We performed a frequentist framework network meta-analysis (NMA) to compare the rank risks of cerebral thromboembolism of the above four endovascular treatments. The incoherence test was used to analyze the statistical disagreement between direct and indirect evidence. Funnel plots were used to analyze publication bias.ResultsThe incidences of DWI lesions in patients who received FD stents, SAC, BAC, and coiling alone were 66.1% (109/165), 37.6% (299/795), 31.1% (236/759), and 25.6% (236/921). The incidence of DWI lesions in patients who received FD stents was higher than that in patients who received SAC [OR: 2.40; 95% CI (1.15, 5.00), P < 0.05], BAC [OR: 2.62; 95% CI (1.19, 5.77), P < 0.05], or coiling alone [OR: 2.77; 95% CI (1.26, 6.07), P < 0.05]. The incoherence test showed preferable consistency in this NMA. No obvious publication bias was found in the funnel plot.ConclusionFD stent placement brings more ischemic lesions identified by DWI than any other procedures for patients with UIA. The characteristics of FD stents may result in a high incidence of DWI lesions