The secondary structure of RNA is essential for its biological role. Recently, Do, Woods, Batzoglou, (ISMB 2006) proposed a probabilistic approach that generalizes SCFGs using conditional maximum likelihood to estimate the model parameters. We propose an alternative approach to parameter estimation which is based on an SVM-like large margin method

De Bona, F.

Ong, C.

Rätsch, G.

Zien, A.

English

MPG.PuRe

RNA Secondary Structure Prediction Using Large Margin MethodsF. De Bona1, C. S. Ong1,2, A. Zien1,2,, G. Ra¨tsch,11 Friedrich Miescher Laboratory, Max Planck Society, Spemannstr. 39, Tu¨bingen, Germany;2 Max Planck Institute for Biological Cybernetics, Spemannstr. 38, Tu¨bingen, Germanyfabio@tuebingen.mpg.deSummaryThe majority of traditional computational approaches to RNA sec-ondary structure prediction consist of two components:•Dynamic programming algorithm for decoding the structure, and• a parameter inference method.While the dynamic programming algorithm is quite similar for all thesemethods parameter inference is usually either than in a probabilisticway or using experimental thermodynamical data. A recently, (1, )proposed probabilistic approach used a conditional maximum likeli-hood scheme for parameter inference. This model could outperformexisting thermodynamic models. We will use a large margin methodrelated to Support Vector Machines. The central idea is to find a pa-rameter vector that assigns highest score to correct and lower scoreto incorrect structures.Problem SettingAn RNA Secondary Structure for a nucleotide sequence of length Ncan be seen as a set of ordered pairs (i, j) denoting that nucleotideat position i is paired with nucleotide at position j.If (i, j) and (i′, j′) are two pairs (w.l.o.g i ≤ i′), then either• i = i′ and j = j′,• i < j < i′ < j′, or• i < i′ < j′ < j.So our input domain is X = Σ∗ where Σ = {A,C,G, U} and our outputdomain is Y = {{(i, j) : i, j,∈ N, i < j}}. Based on these pairs onecan identify substructures such as stems, hairpins etc..The Dynamic Programming Component• In thermodynamic models: Total free energy of y is the sum theenergies of the substructures• In a probabilistic setting: Log probability of a structure is the sum ofthe log probabilities.These values can be expressed as a dot product between parametersand feature (substructure) counts:〈w,ψ(x, y)〉 =whairpin length[1]whairpin length[2]whairpin length[3]··nr. of hairpins of length 1nr. of hairpins of length 2nr. of hairpins of length 3··(ψ : X × Y → Rk is a feature mapping function). As we are interestedin the highest probable structure we calculate:y∗ = argmaxyˆP (yˆ|x) = argmaxyˆ〈w,ψ(xi, yˆ)〉which calculates the structure yˆ ∈ Y whose probability of being ex-actly equal to the correct one y is optimal. The result of the abovemaximization is usually calculated using Dynamic Programming andis called Viterbi decoding.Informally the dynamic programming algorithm has to check for eachposition i against all remaining positions j ∈ {1, .., n} \ {i} whether apair occurs between these positions. When looking at two particularpositions i and j of the nucleotide sequence we can identify severalpossibilities:SVM Learning for Structured Output SpacesThe following method was proposed by (2, ) as a framework for infer-ring structured output variables such as graphs.Given the space of input sequences X and the space of secondarystructures Y we define the prediction function f : X → Y to be:f (x) = argmaxy∈YF (x, y;w) = 〈w,ψ(x, y)〉,which is exactly the output of the Viterbi decoding.This means we have to find a w which maximizes the score (Note thatour parameters wi can not be interpreted as probabilities anymore!)of the correct structure.In optimization lingo this means that we want constraints to hold suchthat:F (xi, yi;w) ≥ F (xi, y;w) + 1 ∀y ∈ Y \ yi,which means informally: For each example (xi, yi)i∈I F should as-sign highest score to the correct structure yi and a lower score to allincorrect structures y. The +1 term is a so-called margin we enforce.•Generating a constraint for each possible incorrect structure leadsto a number of constraints exponential in the size of the structure.•We use a technique called column generation to approximate thesolutionThis is done by using a modified viterbi which incorporates some lossterms:maxy∈Y\yi{∆(y, yi) + 〈w,ψ(xi, y)〉}• Above maximization can be efficiently computed via dynamic pro-gramming•Resembles “standard” secondary structure calculation except theadditional loss term ∆(y, yi)•Modification of textbook Viterbi needed and possibleNew Viterbi for structure calculation with loss:OUTER(i, j) = max`pair + PAIR(i, j)`res(i) +OUTER(i + 1, j)`res(j) +OUTER(i, j − 1)maxi≤k<j OUTER(i, k) +OUTER(k + 1, j)Why bother ?• Large-margin methods (SVMs et al) outperform probabilistic mod-els in lots of applications.•What about RNA secondary structure prediction ?• Is RNA secondary structure prediction a solved problem ?• Are probabilistic methods “superior” ?• Is there a single label in RNA secondary structure prediction ?The ` term are contributions of the Hamming loss to the Viterbi score.Removing these terms would lead to a “standard” Viterbi algorithmas it is implemented in mfold and other packages. Using these ideasand formulations we can state a convex mathematical programmwhich summarizes our requirements:minw,ξ12Ω(w) +Cn∑i ξis.t. ξi ≥ maxy∈Y \yi{1− (F (xi, yi)− F (xi, y))} ∀i ∈ Iξi ≥ 0 ∀i ∈ I.where Ω is a regularization function taking care of the complexity ofour solution w. ξi.• Problem: All structures are penalized equally.• Better: Penalize incorrect structures according to their distance tothe real solution.• Solution: Structural (Hamming) loss ⇒ count number of incorrectpositions.To take the structural loss into account we can multiply the marginwith the Hamming loss ∆:minw,ξ12||w||2 + Cn∑ni=1 ξis.t. maxy∈Y\yi〈w,ψ(xi, yi)− ψ(xi, y)〉 ≥ ∆(y, yi)− ξi ∀i ∈ Iξi ≥ 0 ∀i ∈ I.This approach where the margin is multiplied with the Hamming lossis known as margin rescaling.It can be argued whether this is the right thing to do. Alternative:Slack-rescaling. Formal difference between these methods: ∀i ∀y ∈Y\yi :〈ψ(xi, yi)− ψ(xi, y)〉 ≥ ∆(yi, y)− ξi (margin-rescaling)〈ψ(xi, yi)− ψ(xi, y)〉 ≥ 1−ξi∆(yi, y)(slack-rescaling)Parameters and Algorithm DetailsWe want to solve:minw,ξ12||w||2 + Cnn∑i=1max∀y∈Y\yi(`(〈w,ψ(xi, yi)− ψ(xi, y)〉))the algorithm for solving this problem can be sketched as follows:Tuning of our method involves:• Selection of adequate features,• choice of regularization term, and• choice of an adequate loss function.Choice of Features and Regularization• The total number of features: 1216.• Features include hairpin, stem sizes,• ocurrence of certain motifs, etc.• Feature dependencies/relations modeled via regularization termOur quadratic programm has a regularization term wTPw.•Naive regularization could be P = I (identity matrix)•We would like to include biological prior knowledge in P , such as:– Loop size parameters should be “smooth”,– no difference between AU and UA.•Coupling of the parameters: (wi − wj)2⇒ Proper regularization affects sensitivity / specificity by 2% (in total)each.Structural LossWhat if there is no best structure ?•Maybe the highest scoring structure is not the correct one but thesecond highest scoring.⇒ Achieving structural loss of zero is not possible at all.• It might make sense to allow for certain number of positions to beincorrect during training.⇒ ∆ could be insensitive to a certain loss range.We investigated several forms of this -insensitive loss, namely• Loss insensitive for 3 positions,• Loss insensitive for 0.1 · length(x) positions, and• Loss insensitive for 0.02 · length(x) at least 2 positions.The 10% insensitive loss performed best for these 3 cases. Howeveronly marginal better than a standard Hamming loss.Results•Data set proposed in (1, ).•Data set is a subset of the Rfam database (3, )•Consists of 151 secondary structuresOur method is denoted with RFP .Method Sensitivity SpecificityContrafold 0.73 0.66CG 0.73 0.65mfold 0.69 0.60RFP 0.66 0.67Results were obtained by five-fold cross validation.Discussion pAs we have seen there is still a question to be adressed: Are proba-bilistic models superior to large-margin methods in structure predic-tion ?The conditional likelihood approach and our large-margin methodhave a lot in common:• Feature set seems pretty fixed and very similar•Regularization offers a lot of tuning possibilities but is also verysimilar and reported to contribute only 2% in total.•Objective is convex.•Results reported with using plain Viterbi are very close.⇒ However there is still a gap in terms of sensitivity and specificity.Possible explanantions:• It was reported that maximum expected accuracy outperformsViterbi (1, ).•Rfam consists of consensus structures we instead assume thatthere is one correct labelAmong the things we tried are:•Different regularization terms max ±2% sensitivity / specificity.• Slack / Margin rescaling workarounds showed no benefit.• -insensitive structural loss functionReferencesDo C.B. and Woods D.A. and Batzoglou S., CONTRAfold: RNA Secondary Structure Predictionwithout Energy-Based Models. Bioinformatics 22(14), 2006.Ioannis Tsochantaridis and Thomas Hofmann and Thorsten Joachims and Yasemin Altun SupportVector Machine Learning for Interdependent and Sturcutured Output Spaces Proceedingsof the 16th International Conference on Machine Learning, 2004SamGriffiths-Jones and Simon Moxon and Mhairi Marshall and Ajay Khanna and Sean R. Eddy and AlexBateman. Rfam: annotating non-coding RNAs in complete genomes. Nucleic Acids Res.,2005.R. Durbin and S. Eddy and A. Krogh and G. Mitchison Biological Sequence Analysis CambridgeUniversity Press 1998Michael Zuker Lectures on RNA Secondary Structure Prediction, 2003http://www.bioinfo.rpi.edu/ zukerm/lectures/RNAfold-html/Joachim Dahl and Lieven Vandenberghe CVXOPT: A Python Package for Convex Optimizationhttp://www.ee.ucla.edu/ vandenbe/cvxopt/ Current release: 0.81. 2006

RNA secondary structure prediction using large margin methods

http://hdl.handle.net/21.11116/0000-0004-01B8-B

RNA secondary structure prediction using large margin methods

Abstract

Similar works

Full text

Available Versions

MPG.PuRe