Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 2009, p. 509-512Paper no. 2203Network component analysis (NCA) has been established as a promising tool for reconstructing gene regulatory networks from microarray data. NCA is a method that can resolve the problem of blind source separation when the mixing matrix instead has a known sparse structure despite the correlation among the source signals. The original NCA algorithm relies on alternating least squares (ALS) and suffers from local convergence as well as slow convergence. In this paper, we develop new and more robust NCA algorithms by incorporating additional signal constraints. In particular, we introduce the biologically sound constraints that all nonzero entries in the connectivity network are positive. Our new approach formulates a convex optimization problem which can be solved efficiently and effectively by fast convex programming algorithms. We verify the effectiveness and robustness of our new approach using simulations and gene regulatory network reconstruction from experimental yeast cell cycle microarray data. ©2009 IEEE.published_or_final_versio

Chang, C

Ding, Z

Yeung, SH

HKU Scholars Hub

Title A new optimization algorithm for network component analysisbased on convex programmingAuthor(s) Chang, C; Yeung, SH; Ding, ZCitation Icassp, Ieee International Conference On Acoustics, Speech AndSignal Processing - Proceedings, 2009, p. 509-512Issued Date 2009URL http://hdl.handle.net/10722/62038Rights IEEE International Conference on Acoustics, Speech and SignalProcessing Proceedings. Copyright © IEEE.A NEW OPTIMIZATION ALGORITHM FOR NETWORK COMPONENT ANALYSIS BASEDON CONVEX PROGRAMMINGChunqi Chang1, Yeung Sam Hung1Electrical and Electronic EngineeringThe University of Hong KongPokfulam Road, Hong KongZhi DingElectrical and Computer EngineeringUniversity of CaliforniaDavis, CA95616, USAABSTRACTNetwork component analysis (NCA) has been establishedas a promising tool for reconstructing gene regulatory net-works from microarray data. NCA is a method that can re-solve the problem of blind source separation when the mix-ing matrix instead has a known sparse structure despite thecorrelation among the source signals. The original NCA al-gorithm relies on alternating least squares (ALS) and suffersfrom local convergence as well as slow convergence. In thispaper, we develop new and more robust NCA algorithms byincorporating additional signal constraints. In particular, weintroduce the biologically sound constraints that all nonzeroentries in the connectivity network are positive. Our new ap-proach formulates a convex optimization problem which canbe solved efficiently and effectively by fast convex program-ming algorithms. We verify the effectiveness and robustnessof our new approach using simulations and gene regulatorynetwork reconstruction from experimental yeast cell cycle mi-croarray data.Index Terms— Gene regulatory networks, microarray,network component analysis, convex programming1. INTRODUCTIONRecent advances in genome sequencing and high-throughputmicroarray technologies make it possible to quantitatively in-vestigate the underlying mechanisms that define gene inter-action. More specifically, we can now consider the under-lying biological model as an information processing system.With this signal processing framework, the challenge is tomodel gene interaction from experimental microarray data asa gene regulatory network. A key focus is on developingtractable system identification techniques capable of recon-structing gene regulatory networks of a large size from smallmicroarray datasets.Earlier approaches to modeling gene regulatory networkinclude dynamic Bayesian networks (DBN) [1], probabilis-tic Boolean network (PBN) [2], and differential/difference1Supported by The University of Hong Kong CRCG grants.equations [3]. It has also been shown that the network canbe approximated by a linear instantaneous signal system [4].Among various algorithms assuming linear instantaneous sig-nal models, such as principal component analysis (PCA) [5]and independent component analysis (ICA)[6], the networkcomponent analysis (NCA)[4] is a very effective approachsince it incorporates useful and biologically sound assump-tions.More specifically, a true gene regulatory network alwayshas a sparse structure, and sometimes the non-zero entries ofits (sparse) connectivity matrix can be obtained from ChIP-chip experiments [7]. Even when experimental data on thestructure of the network are unavailable, the structural infor-mation sometimes can be extracted partially from the litera-ture or predicted by bioinformatics methods [8]. The NCAapproach makes use of this (sparse) structure information. Ifsome mild conditions, called NCA criteria, can be satisfied,the NCA algorithm can fit the gene expression data (measuredby microarray) to the linear network model under the con-straints represented by the NCA criteria. The NCA algorithmcan then fully reconstruct the network (including the connec-tivity matrix and the regulatory signals) if the measurement isnoise-free.The original NCA algorithm applies alternating leastsquares (ALS) to solve a not very well posed optimizationproblem. Though very effective, it suffers from three compu-tational drawbacks as pointed out in [9]:(1) it may be unstable due to ill-conditioned matrices;(2) it may converge to local minima;(3) it is inefficient and time consuming for relatively largenetworks.Tikhov regularization has been proposed to overcome theconvergence problem [9]. Most recently, authors of thismanuscript proposed a new algorithm, FastNCA, that over-come the 3 drawbacks [10]. FastNCA provides a closed-formsolution to NCA through matrix factorizations.One common feature of the existing NCA approaches isthat it exploit only the knowledge on the zero location of the(sparse) connectivity matrix. Since microarray data are very509978-1-4244-2354-5/09/$25.00 ©2009 IEEE ICASSP 2009Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on July 13,2010 at 04:01:00 UTC from IEEE Xplore.  Restrictions apply. noisy, we seek to incorporate additional constraints to reducenoise sensitivity. From biological information, we find that,without loss of generality, all nonzero entries in the connec-tivity matrix can be positive. We hence develop a new NCAalgorithm to take advantage of such constraints against noise.We further formulate an optimization problem that can besolved by efficient convex programming algorithms.Our paper is organized as follows. In Section 2 we intro-duce the model of gene regulatory network and the NCA for-mulation. We formulate, in Section 3, a convex optimizationproblem to tackle the NCA problem. In Section 4 we imposenew solution constraints onto the NCA problem, and incor-porate efficiently in the convex programming framework. Weprovide simulation and experimental results in Section 5, fol-lowed by discussions and conclusions in Section 6.2. GENE REGULATION MODEL AND NETWORKCOMPONENT ANALYSISx1 x2 x3 x4 x5 x6 x7TF1: s1 TF2:s2 TF3: s3Fig. 1. Transcriptional regulatory model.We consider gene expression with a transcriptional regu-latory model, as illustrated by Figure 1. Gene expressions areregulated by transcription factors. The upper layer in the fig-ure represents the expression level of activated transcriptionfactors (TF) or transcription factor activities (TFA), and thelower layer represents the microarray gene expression data.Consider a network of N genes regulated by M TFs with ob-served measurements over K time points. This network canbe modeled as an instantaneous linear mixing systemX = AS + Γ, (1)where the mixtures X (with dimension N ×K) are gene ex-pression levels which can be measured by microarray, themixing matrix A (N × M ) is called connectivity matrix ofthe network, the sources S (M ×K) are the unknown TFAs,and Γ is the measurement noise.We only have access to the gene expression data X , i.e.,the microarray data. The aim is to estimate the connectivitymatrix A and the TFAs S from the microarray data X. Ourstrategy is to estimate the connectivity matrix first. With theestimation of A, denoted by Aˆ, from which the TFAs can beestimated by algorithms such as minimum mean square error(MMSE) or zeroforcing inverse fromSˆ = Aˆ+X, (2)where Aˆ+ is the pseudo inverse of Aˆ.The method developed in [4] to solve such inverse prob-lem is called network component analysis (NCA), which isbased on the following three assumptions referred to as NCAcriteria: (i) connectivity matrix A must be full-column rank;(ii) when an element in the regulatory domain is removedalong with all the output elements connected to it, the con-nectivity matrix of the resulting network is still of full-columnrank; and (iii) S must have full row rank.3. A CONVEX PROGRAMMING NCAFRAMEWORKHere we develop a new approach to network component anal-ysis, which is based on the same three NCA criteria but, likeour FastNCA approach, does not suffer the drawbacks of theoriginal NCA algorithm using ALS and its improved versionwith regularization. In addition, this approach, as will beshown later, can easily incorporate other kinds of prior in-formation to achieve improved performance.First we can find the range of X: X¯ = range{X}, and itsnull space C = X¯⊥ such that CT X¯ = 0. When the measure-ment is noiseless, i.e., X = AS, then X¯ and C are also therange and null space of A. When there is noise, we can use arobust estimation of C through singular value decomposition(SVD) [11]:X = UΣVT , (3)in which the diagonal matrix Σ consists of singular valuessorted in descending order. From which we can estimate Cas the last N −M columns of U.Because A is orthogonal to C, this condition can be usedto estimate A by minimizing the Frobenius norm ‖CTA‖Fsubject to the constraints imposed by the prior information onnetwork connectivity. In other words, we propose to estimateAˆ as the solution of the following optimization problem.minAˆ||CT Aˆ||F subject to Aˆ(I) = 0, (4)where I contains the indices where the entries of A are zeros.This optimization problem is globally convergent. First,the cost function is a convex function of the unknown ma-trix Aˆ. Second, the constraints Aˆ(I) = 0 are linear (henceconvex). Thus, the optimization problem 4 is a convex pro-gramming problem and contains no local minima. It can alsobe solved with very fast interior-point algorithms [12].4. POSITIVITY CONSTRAINTSThough the three NCA criteria are enough to estimate the con-nectivity matrix when there is no noise in the model, addi-tional constraints, if can be used in the algorithm, will cer-tainly yield a more robust estimate in practice when noise isinevitable. The original approach in [9] is not easily amenable510Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on July 13,2010 at 04:01:00 UTC from IEEE Xplore.  Restrictions apply. to the addition of additional constraints and prior knowledges.Our convex optimization approach, on the other hand, is verysimple for integration with new (convex) constraints and canoften converge faster. This property makes it perfectly suitedto integrate other system knowledge to combat measurementnoises and modeling errors.In this paper, we assume the prior knowledge that allnonzero entries of the connectivity matrix are of the samesign (either all positive or all negative). As long as any spe-cific transcription factor has the same effect to all genes,either all positive or all negative, this assumption has soundbiological support [13]. This is because if a transcriptionfactor regulates the genes negatively, then we can simplymultiply its transcription factor activity (TFA) by −1 to meetthe requirement that the new TFA will have positive effect onthe genes.Note that the positivity constraints are linear inequalitiesand are convex, they are easily incorporated into a modifiedconvex programming for NCA (named PosNCA) asminAˆ||CT Aˆ||F subject to Aˆ(I) = 0, Aˆ(J) ≥ c, (5)where J contains the indices where entries of A are nonzero,and c is a constant small positive value. The convex problem 5can be solved by effective algorithms [12].5. RESULTS5.1. Simulation resultsWithout positivity constraints, simulations (not shown here)demonstrated that the convex programming of (4) matcheswell with the results of FastNCA.The new PosNCA with positivity constraints is testedon the chemical simulation data identical to [4]. This is ahemoglobin spectroscopy data set with a 7× 3 mixing matrixand a 7× 321 measurement.Without noise, both the source spectra and the mixing ma-trix can be perfectly estimated using convex programmingwith positivity constraints. Here we study the proposed al-gorithm under noisy case. The FastNCA method [10] is alsoused as a comparison. To this end we add noise to the spectralmeasurement to get a signal to noise ratio (SNR) of 9dB. Thetrue (original) and the estimated source spectra are shown inin Figure 2, while the (normalized) true and estimated mixingmatrices are compared in Table 1.The estimated entries of A by FastNCA can be both pos-itive and negative. Such estimate is unfavorable since all trueentries in A are non-negative. On the other hand, the esti-mated connectivity matrix by the PosNCA algorithm containsmuch closer results as expected. The root mean square er-ror of the A estimate from PosNCA is 0.3114, much smallerthan that from FastNCA (0.7328). The estimated source spec-tra shown in Figure 2 also demonstrate that PosNCA is muchmore robust.Table 1. True connectivity matrix and its estimation by Fast-NCA and the proposed PosNCA algorithmTrue FastNCA PosNCA0.417 1.18 0 -1.78 1.03 0 0.985 1.03 02.08 0 1.25 -0.20 0 2.06 1.51 0 1.240 1.18 0.25 0 1.24 -0.47 0 1.24 0.6700.417 1.05 0.25 -0.946 0.966 0.096 1.03 0.969 0.6701.25 0.921 0 -0.279 1.09 0 0.741 1.09 00.833 0 2 1.79 0 1.78 0.741 0 1.750 0.658 1.25 0 0.675 0.589 0 0.677 0.6700 200 400−0.200.20.40.6True0 200 400−0.2−0.100.10.2FastNCA0 200 400−0.500.5PosNCA0 200 400−0.200.20.40.60 200 400−0.500.510 200 400−0.200.20.40.60 200 400−0.200.20.40.60 200 400−0.500.510 200 400−0.500.51Fig. 2. True sources and their estimates using FastNCA andPosNCA.5.2. Experimental ResultsThe time-course Yeast cell cycle microarray data from [14]are analyzed by our PosNCA with positivity constraints. Thenetwork topology data are from [7]. After trimming, the net-work has 441 genes and 33 transcription factors in which 11TFs are cell cycle related. The estimated TFAs for these 11cell cycle related transcription factors are shown in Figure 3for the new PosNCA algorithm and the previous FastNCA al-gorithm, respectively. Only results from the experiment usingthe synchronization method by arrest of cdc15 temperature-sensitive mutant are presented here. It can be seen that thetwo methods produce similar results in general. However,it is clear that the estimated TFAs by the new PosNCA ex-hibit more cyclic behaviors. Cyclic behavior is expected inthis case since all these transcription factors are cell cycle re-lated. Therefore, it leads to the reasonable speculation thatthe PosNCA results are more reliable than those from otherNCA methods without incorporating these biologically soundconstraints. Still, we stress that such conclusion is to be con-firmed by further biological studies.511Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on July 13,2010 at 04:01:00 UTC from IEEE Xplore.  Restrictions apply. 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.200.20 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.200.20 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.1(a) PosNCA0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.200.20 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.10 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.200.20 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5−0.100.1(b) FastNCAFig. 3. Estimated yeast cell cycle related TFAs by (a) theproposed PosNCA method; and (b) FastNCA6. DISCUSSION AND CONCLUSIONAccording to the biological literatures, the microarray dataare in fact considered very noisy. Thus, the developmentof PosNCA with positivity constraints is expected to providepractical advantage against measurement noise by incorporat-ing biologically sound constraints. The constraints are seam-lessly integrated into our convex optimization algorithm. Asshown by the yeast cell cycle data analysis, the new PosNCAalgorithm with positivity constraints presents better resultsthan FastNCA without such constraints.7. REFERENCES[1] N. Friedman, M. Linial, Nachman I., and Peer D.,“Using bayesian networks to analyze expression data,”Journal of Computational Biology, vol. 7, no. 3-4, pp.601–620, 2000.[2] I. Shmulevich, E. R. Dougherty, S. Kim, and W. Zhang,“Probabilistic boolean networks: a rule-based uncer-tainty model for gene regulatory networks,” Bioinfor-matics, vol. 18, no. 2, pp. 261–274, 2002.[3] E. R. Dougherty, I Shmulevich, J Chen, and Z. J. Wang,Genomic Signal Processing and Statistics, HindawiPublishing Corporation, 2005.[4] J C Liao, R Boscolo, Y L Yang, L M Tran, C Sabatti,and V P Roychowdhury, “Network component analy-sis: Reconstruction of regulatory signals in biologicalsystems,” Proceedings of the National Academy of Sci-ences of the United States of America, vol. 100, no. 26,pp. 15522–15527, 2003.[5] O Alter, P O Brown, and D Botstein, “Singular valuedecomposition for genome-wide expression data pro-cessing and modeling,” Proceedings of the NationalAcademy of Sciences of the United States of America,vol. 97, no. 18, pp. 10101–10106, 2000.[6] S I Lee and S Batzoglou, “Application of independentcomponent analysis to microarrays,” Genome Biology,vol. 4, no. 11, pp. R76, 2003.[7] T. I. Lee, N. J. Rinaldi, F. Robert, D. T. Odom, Z. Bar-Joseph, G. K. Gerber, N. M. Hannett, C. T. Harbison,C. M. Thompson, I. Simon, J. Zeitlinger, E. G. Jennings,H. L. Murray, D. B. Gordon, B. Ren, J. J. Wyrick, J. B.Tagne, T. L. Volkert, E. Fraenkel, D. K. Gifford, andR. A. Young, “Transcriptional regulatory networks insaccharomyces cerevisiae,” Science, vol. 298, no. 5594,pp. 799–804, 2002.[8] C. Wang, J. Xuan, L. Chen, P. Zhao, Y. Wang, R. Clarke,and E. Hoffman, “Motif-directed network componentanalysis for regulatory network inference,” BMC Bioin-formatics, vol. 9, no. Suppl 1, pp. S21, 2008.[9] L M Tran, M P Brynildsen, K C Kao, J K Suen, andJ C Liao, “gnca: A framework for determining tran-scription factor activity based on transcriptome: identi-fiability and numerical implementation,” Metabolic En-gineering, vol. 7, no. 2, pp. 128–141, 2005.[10] C. Q. Chang, Z. Ding, Y. S. Hung, and P. C. W.Fung, “Fast network component analysis (fastnca) forgene regulatory network reconstruction from microarraydata,” Bioinformatics, vol. 24, pp. 1349–1358, 2008.[11] G. H. Golub and C. F. van Loan, Matrix Computation,The Johns Hopkins University Press, 3rd edition, 1996.[12] S. Boyd and L. Vandenberghe, Convex optimization,Cambridge University Press, Cambridge, UK, 2004.[13] U. Alon, An Introduction to Systems Biology: De-sign Principles of Biological Circuits, Chapman &Hall/CRC, 2007.[14] P. T. Spellman, G. Sherlock, M. Q. Zhang, V. R. Iyer,K. Anders, M. B. Eisen, P. O. Brown, Botsein D., andB. Futcher, “Comprehensive identification of cell cycle-regulated genes of the yeast saccharomyces cerevisiaeby microarray hybridization,” Molecular Biology of theCell, vol. 9, no. 12, pp. 3273–3297, 1998.512Authorized licensed use limited to: The University of Hong Kong Libraries. Downloaded on July 13,2010 at 04:01:00 UTC from IEEE Xplore.  Restrictions apply. 

A new optimization algorithm for network component analysis based on convex programming

Abstract

Similar works

Full text

Available Versions

HKU Scholars Hub

Crossref