Stochastic Gradient Descent in the Viewpoint of Graduated Optimization

Stochastic gradient descent (SGD) method is popular for solving non-convex optimization problems in machine learning. This work investigates SGD from a viewpoint of graduated optimization, which is a widely applied approach for non-convex optimization problems. Instead of the actual optimization problem, a series of smoothed optimization problems that can be achieved in various ways are solved in the graduated optimization approach. In this work, a formal formulation of the graduated optimization is provided based on the nonnegative approximate identity, which generalizes the idea of Gaussian smoothing. Also, an asymptotic convergence result is achieved with the techniques in variational analysis. Then, we show that the traditional SGD method can be applied to solve the smoothed optimization problem. The Monte Carlo integration is used to achieve the gradient in the smoothed problem, which may be consistent with distributed computing schemes in real-life applications. From the assumptions on the actual optimization problem, the convergence results of SGD for the smoothed problem can be derived straightforwardly. Numerical examples show evidence that the graduated optimization approach may provide more accurate training results in certain cases.Comment: 23 pages, 4 figure

Bi-level Programming Based Optimal Strategy to LSEs with Demand Response Bids

With the increasing demand-side participation in electricity market, as a profit-seeking market participant, load-serving entities (LSEs) have been trying to apply demand response (DR) programs to induce the demand elasticity to further their profit. However, due to the different preference of DRs, it is difficult for LSEs to generate the optimal strategic bidding strategy considering DR in the ISO/RTO’s market. Therefore, this paper proposed a bi-level optimization model with the consideration of demand response bidding to maximize the total profit of LSEs: 1) conceptually, different from previous related works, the consumers participate DR through setting their bidding prices to LSEs with respect to their own preference and LSEs should determine the optimal reward value of DR as well as the amount of demanded electricity; and 2) technically, an original method has been implemented to solve the bi-level optimization model. The closed form of shadow price function with respect to the total load demand is derived to reduce the complexity of the proposed bi-level model. Hence, the proposed model is converted to a mixed integer second order cone programming and able to achieve the global optimality. It needs to be note that the closed form of shadow price introduced in this paper can also be applied to other bi-level programming models. Moreover, case studies have been performed to demonstrate the validity of the proposed method: 1) the proposed method to obtain the closed form of real-time price is verified on a 9-bus system; 2) 118-bus test system with three demand response participants is tested to show that by the proposed method, LSE can benefit from the DRs under various circumstance

The Structural Characterization and Antigenicity of the S Protein of SARS-CoV

The corona-like spikes or peplomers on the surface of the virion under electronic microscope are the most striking features of coronaviruses. The S (spike) protein is the largest structural protein, with 1,255 amino acids, in the viral genome. Its structure can be divided into three regions: a long N-terminal region in the exterior, a characteristic transmembrane (TM) region, and a short C-terminus in the interior of a virion. We detected fifteen substitutions of nucleotides by comparisons with the seventeen published SARS-CoV genome sequences, eight (53.3%) of which are non-synonymous mutations leading to amino acid alternations with predicted physiochemical changes. The possible antigenic determinants of the S protein are predicted, and the result is confirmed by ELISA (enzyme-linked immunosorbent assay) with synthesized peptides. Another profound finding is that three disulfide bonds are defined at the C-terminus with the N-terminus of the E (envelope) protein, based on the typical sequence and positions, thus establishing the structural connection with these two important structural proteins, if confirmed. Phylogenetic analysis reveals several conserved regions that might be potent drug targets

PoolHap: Inferring Haplotype Frequencies from Pooled Samples by Next Generation Sequencing

With the advance of next-generation sequencing (NGS) technologies, increasingly ambitious applications are becoming feasible. A particularly powerful one is the sequencing of polymorphic, pooled samples. The pool can be naturally occurring, as in the case of multiple pathogen strains in a blood sample, multiple types of cells in a cancerous tissue sample, or multiple isoforms of mRNA in a cell. In these cases, it's difficult or impossible to partition the subtypes experimentally before sequencing, and those subtype frequencies must hence be inferred. In addition, investigators may occasionally want to artificially pool the sample of a large number of individuals for reasons of cost-efficiency, e. g., when carrying out genetic mapping using bulked segregant analysis. Here we describe PoolHap, a computational tool for inferring haplotype frequencies from pooled samples when haplotypes are known. The key insight into why PoolHap works is that the large number of SNPs that come with genome-wide coverage can compensate for the uneven coverage across the genome. The performance of PoolHap is illustrated and discussed using simulated and real data. We show that PoolHap is able to accurately estimate the proportions of haplotypes with less than 2% error for 34-strain mixtures with 2X total coverage Arabidopsis thaliana whole genome polymorphism data. This method should facilitate greater biological insight into heterogeneous samples that are difficult or impossible to isolate experimentally. Software and users manual are freely available at http://arabidopsis.gmi.oeaw.ac.at/quan/poolhap/

XA4C: eXplainable representation learning via Autoencoders revealing Critical genes.

Machine Learning models have been frequently used in transcriptome analyses. Particularly, Representation Learning (RL), e.g., autoencoders, are effective in learning critical representations in noisy data. However, learned representations, e.g., the "latent variables" in an autoencoder, are difficult to interpret, not to mention prioritizing essential genes for functional follow-up. In contrast, in traditional analyses, one may identify important genes such as Differentially Expressed (DiffEx), Differentially Co-Expressed (DiffCoEx), and Hub genes. Intuitively, the complex gene-gene interactions may be beyond the capture of marginal effects (DiffEx) or correlations (DiffCoEx and Hub), indicating the need of powerful RL models. However, the lack of interpretability and individual target genes is an obstacle for RL's broad use in practice. To facilitate interpretable analysis and gene-identification using RL, we propose "Critical genes", defined as genes that contribute highly to learned representations (e.g., latent variables in an autoencoder). As a proof-of-concept, supported by eXplainable Artificial Intelligence (XAI), we implemented eXplainable Autoencoder for Critical genes (XA4C) that quantifies each gene's contribution to latent variables, based on which Critical genes are prioritized. Applying XA4C to gene expression data in six cancers showed that Critical genes capture essential pathways underlying cancers. Remarkably, Critical genes has little overlap with Hub or DiffEx genes, however, has a higher enrichment in a comprehensive disease gene database (DisGeNET) and a cancer-specific database (COSMIC), evidencing its potential to disclose massive unknown biology. As an example, we discovered five Critical genes sitting in the center of Lysine degradation (hsa00310) pathway, displaying distinct interaction patterns in tumor and normal tissues. In conclusion, XA4C facilitates explainable analysis using RL and Critical genes discovered by explainable RL empowers the study of complex interactions

CT-based radiomics models predict spontaneous intracerebral hemorrhage expansion and are comparable with CT angiography spot sign

Background and purposeThis study aimed to investigate the efficacy of radiomics, based on non-contrast computed tomography (NCCT) and computed tomography angiography (CTA) images, in predicting early hematoma expansion (HE) in patients with spontaneous intracerebral hemorrhage (SICH). Additionally, the predictive performance of these models was compared with that of the established CTA spot sign.Materials and methodsA retrospective analysis was conducted using CT images from 182 patients with SICH. Data from the patients were divided into a training set (145 cases) and a testing set (37 cases) using random stratified sampling. Two radiomics models were constructed by combining quantitative features extracted from NCCT images (the NCCT model) and CTA images (the CTA model) using a logistic regression (LR) classifier. Additionally, a univariate LR model based on the CTA spot sign (the spot sign model) was established. The predictive performance of the two radiomics models and the spot sign model was compared according to the area under the receiver operating characteristic (ROC) curve (AUC).ResultsFor the training set, the AUCs of the NCCT, CTA, and spot sign models were 0.938, 0.904, and 0.726, respectively. Both the NCCT and CTA models demonstrated superior predictive performance compared to the spot sign model (all P < 0.001), with the performance of the two radiomics models being comparable (P = 0.068). For the testing set, the AUCs of the NCCT, CTA, and spot sign models were 0.925, 0.873, and 0.720, respectively, with only the NCCT model exhibiting significantly greater predictive value than the spot sign model (P = 0.041).ConclusionRadiomics models based on NCCT and CTA images effectively predicted HE in patients with SICH. The predictive performances of the NCCT and CTA models were similar, with the NCCT model outperforming the spot sign model. These findings suggest that this approach has the potential to reduce the need for CTA examinations, thereby reducing radiation exposure and the use of contrast agents in future practice for the purpose of predicting hematoma expansion

The Application of SILAC Mouse in Human Body Fluid Proteomics Analysis Reveals Protein Patterns Associated with IgA Nephropathy

Body fluid proteome is the most informative proteome from a medical viewpoint. But the lack of accurate quantitation method for complicated body fluid limited its application in disease research and biomarker discovery. To address this problem, we introduced a novel strategy, in which SILAC-labeled mouse serum was used as internal standard for human serum and urine proteome analysis. The SILAC-labeled mouse serum was mixed with human serum and urine, and multidimensional separation coupled with tandem mass spectrometry (IEF-LC-MS/MS) analysis was performed. The shared peptides between two species were quantified by their SILAC pairs, and the human-only peptides were quantified by mouse peptides with coelution. The comparison for the results from two replicate experiments indicated the high repeatability of our strategy. Then the urine from Immunoglobulin A nephropathy patients treated and untreated was compared by this quantitation strategy. Fifty-three peptides were found to be significantly changed between two groups, including both known diagnostic markers for IgAN and novel candidates, such as Complement C3, Albumin, VDBP, ApoA,1 and IGFBP7. In conclusion, we have developed a practical and accurate quantitation strategy for comparison of complicated human body fluid proteome. The results from such strategy could provide potential disease-related biomarkers for evaluation of treatment