Typical problems in the analysis of data sets like time-series or images crucially rely on the extraction of primitive features based on segmentation. Variational approaches are a popular and convenient framework in which such problems can be studied. We focus on Potts models as simple nontrivial instances. The discussion proceeds along two data sets from brain mapping and functional genomics

Kempe, A.

Liebscher, V.

Winkler, Gerhard

Wittich, O.

English

Open Access LMU

Winkler, Kempe, Liebscher, Wittich:Parsimonious Segmentation of Time Series’ by PottsModelsSonderforschungsbereich 386, Paper 348 (2003)Online unter: http://epub.ub.uni-muenchen.de/ProjektpartnerParsimonious Segmentation of Time Series’by Potts ModelsGerhard Winkler1, Angela Kempe2, Volkmar Liebscher1, and Olaf Wittich11 Institute of Biomathematics and BiometryGSF - National Research Center for Environment and Health,D-85758 Neuherberg/Mu¨nchen, Germany2 Graduate Programme Applied Algorithmic MathematicsCenter for Mathematical Sciences, Munich University of TechnologyD-85747 GarchingAbstract. Typical problems in the analysis of data sets like time-series or imagescrucially rely on the extraction of primitive features based on segmentation. Varia-tional approaches are a popular and convenient framework in which such problemscan be studied. We focus on Potts models as simple nontrivial instances. The dis-cussion proceeds along two data sets from brain mapping and functional genomics.1 IntroductionThe purpose of the present note is twofold: We want to give an elementaryintroduction to variational approaches to the analysis of one- and multi-dimensional data, and further to illustrate by way of simple data sets andstatistical models what we mean by parsimonious statistics.We will briefly discuss a particularly simple parsimonious approach to thestatistical analysis of real-world data sets from life-sciences. Frequently, thereis little or no ground truth, and the stochastic mechanism generating (noisy)data is essentially unknown. The only way to associate data to some hiddenreal event is to verify or falsify rough and basic criteria which characterizethe event in question. Such criteria frequently are based on primitive signalfeatures. In images these may be boundaries between regions of differentintensity or texture, in time series they may be morphological features likemodes or ‘ups and downs’, domains of monotony, or plateaus where the signalis constant. We start the discussion with two one-dimensional data sets, onefrom brain mapping and one from functional genomics. We expect that inthese examples the observation period can be partitioned into intervals wherethe underlying signal can reasonably be represented by a constant. This is aprimitive morphological feature, and the resulting step functions allow soundbiological interpretations.To extract piecewise constant ‘regressions’ from data, we adopt the sim-plest variational approach based on the Potts model. It is well known tophysicists as the straightforward generalization of the Ising model for binaryspins to multiple states. For a detailed discussion see Winkler (2003).2 Winkler and al.2 Two Data Sets from Life SciencesIn order to introduce and illustrate the concept, we present two sets of data.The first one consists of time series from functional magnetic resonance imag-ing (fMRI) of the human brain, and the second one of melting or fractionationcurves for spots on a cDNA microchip.Example 1 (fMRI Brain Data: Identification of Response Regions). The finalaim is to identify regions of increased activity in the human brain in responseto outer stimuli. Typically such stimuli are boxcar shaped as indicated inFig. 1. They may represent ‘light or sound on and off’, i.e. visual or acousticstimuli, or tactile ones like finger tipping on a desk. Functional magnetic res-onance imaging (fMRI) exploits the BOLD effect which basically is a changeof paramagnetic properties caused by an increase of blood flow in response tothe demand of activated neurons for more oxygen. The degradation mecha-Fig. 1. A box car shaped signal representing ‘on-off’ stimuli in fMRI brain mapping.nism along the path ‘(complex) eye - (highly complex) brain - (complicated)measuring device’ is only partially known. Moreover, measurement is indi-rect, since the recorded BOLD effect is a physiological quantity related toincrease of blood flow and not a direct function of cortical activation. Hencea parsimonious approach based on significant plateaus should be appropriate.Example 2 (Fractionation Curves from Gene Expression). The aim of thisexperiment is to explore the structure of unknown genes. To this end, singlestranded sections of known cDNA are put on spots of microchips, whichtypically consist of about 20.000 spots. Each section is a finite sequence of fournucleic acids, which are coded by the letters A(denin), C(ytosin), G(uanin),and T(hymin). If further nucleic acids are added then they tend to bind tothe known nucleic acids where T binds to A, and G binds to C.Hence sections of single stranded unknown cDNA tend to pair with DNA ofsimilar sequence. The binding energy is maximal for perfect matches likeA C T A C A G T A C C C AT G A T G T C A T G G G Tand such a perfect match means high stability. With perfect match theunknown sequence could be identified perfectly. A main problem is cross-hybridisation, which means that DNA sections pair with DNA of similar -but not precisely equal - sequence, for exampleSegmentation by Potts Models 3A C T A C A G T A C C C AT G A T T T C A T G A G TPerfect match and mismatch are illustrated in Figure 2. A new and innova-tive experiment provides data which hopefully will allow to identify mismatchdissociation at low stringency. It is called ‘Specificity Assessment From Frac-Fig. 2. Specific and unspecific hybridizationtionation Experiments’ or in short-hand notation ‘SAFE’, see Drobyshev etal.(2003). It is plausible that ‘the melting temperature’ of double strandedDNA depends on length and contents of specific sequences. It is also plausiblethat increasing temperature has similar effects as increasing washing strin-gencies with formamide solutions. Both decrease the binding energies andthus cross-hybridisation. This is the basis for the measurement of specificand cross-hybridisation. In the experiment, the chips are washed repeatedly(29 times) with formamide solutions of increasing concentration, and fraction-ation curves like in Fig. 3 are recorded. The aim of the statistical analysis isFig. 3. Typical fractionation curves of single spots: lousy, fairly good, intermediate.(Data from Drobyshev et al. (2003))to identify locations and heights of abrupt decreases, since they indicate thata certain type of cross-hybridizing cDNA was washed away.In view of such data, one may doubt about too ‘specific’ methods or toodetailed models for their analysis, and in fact we do so. A way out of thismisery is to try a parsimonious approach as indicated above, see Davies(1995). There are attempts like in Davies and Kovac (2001), who adopt thetaut string algorithm and its relatives for certain types of data. We tried aparsimonious variational approach called the Potts Model for our data.4 Winkler and al.3 The Potts Model: Rigorous ResultsThe relevant features in Example 1 are successions of high and low plateaus,and in Example 2 the positions of rapid decreases and their height. Thereforewe try to fit piecewise constant functions to our data.The Potts functional is defined byx = (x1, . . . , xn) 7−→ Pγ,y(x) = γ|J(x)|+n∑k=1(xk − yk)2, (1)where y = (y1, . . . , yn) denotes real (fixed) data, and J(x) is the set of timepoints k where xk 6= xk+1, k = 1, . . . , n− 1. |J(x)| denotes the cardinality ofJ(x). The second term rates fidelity of the signal x to data y, and the firstone penalizes undesired properties of x. Thus the functional is a penalizedlikelihood function.A minute of contemplation reveals that there are three elementary con-cepts combined in this model:(i) A notion of a ‘jump’ or ‘break’: In the Potts model such a jump is present,where the values of the signal x in two subsequent time points differ fromeach other.(ii) A notion of smoothness: this concerns the behaviour of the signal betweentwo subsequent jumps. It is a consequence of (i) that in the Potts modela signal is constant there.(iii) A notion of fidelity to data, i.e. some measure of distance between datay and the signal x.Note that(ii) is a rather strict notion of smoothness: the signal on a discreteinterval is ‘smooth’ only if it is constant. The first term penalizes the numberof jumps irrespective of their size and the parameter γ > 0 controls the degreeof smoothness.Given data y, a ‘filter output’ Tγ(y) is defined as a signal which minimizesPγ,y. In general, it is not unique, but fortunately the following result fromthe forthcoming thesis Kempe (2003) guarantees uniqueness almost surely:Theorem 1. Suppose that the law of data y admits a Lebesgue density. Thenfor almost all y the functional Pγ,y in (1) has one and only one minimizer.If the hypotheses hold a filter is defined uniquely for almost all y by the signalTγ(y) = argminxPγ,y(x).We are going now to report essential properties of the filter. It is crucial thatthe range of hyperparameters γ can be partitioned into intervals, on whichthe estimate does not change as shown in Kempe (2003). Dependence onhyperparameters is illustrated in Fig. 4.Segmentation by Potts Models 5Theorem 2. For almost all data y the following is true: There are an integerk and hyperparameters ∞ = γ0 > γ1 > · · · > γk > γk+1 = 0 such that Tγ(y)is unique for all γ ∈ (γj+1, γj). Moreover, it is the same time-series for allγ ∈ (γj+1, γj). Tγ(y) is a constant signal for each γ > γ1, and Tγ(y) = y forγ < γk. The number of jumps of Tγ(y) on the intervals (γj+1, γj) increases inj. For each 0 < i ≤ k, the functional Pγi,y has precisely the two minimizersbelonging to the γ-intervals adjacent to γi.γrγ1rγ2rγ3rγkr0 data constantFig. 4. Intervals on which estimates do not change.Significant simulations can only be carried out with an exact algorithmfor the computation of the minimizer. This rules out stochastic algorithmslike simulated annealing. For one dimension, an algorithm based on ideasfrom dynamic programming was presented in Winkler and Liebscher (2002).Theorem 3. There is an algorithm to compute a minimizer of Pγ,y in timecomplexity O(n3) for all γ ∈ R simultaneously.The filter has some more pleasant properties. In particular, the iterationof the filter stops after one step More precisely, a repeated application returnsthe same signal as a single one, or in other words the filter is idempotent inthe sense that Tγ ◦ Tγ = Tγ . This implies that Tγ is a morphological filter inthe sense of Serra (1982, 1988), see Winkler and Liebscher (2002):Theorem 4. The Potts filter is idempotent.The filter has continuity or consistency properties like the following one:Theorem 5 (V. Liebscher, O. Wittich, unpublished). Let γ ∈ R andy∞ ∈ Rn. Suppose that Tγ(y∞) is unique. If y∞ is degraded by random noiseεn according to Y n = y∞+εn, and noise εn tends to zero in probability, thenTγ(Y n) tends to Tγ(y∞) in probability.We are not interested in a ‘restoration’ but in feature extraction. This isreflected by the theorem since we recover Tγ(y∞) - and not y∞ - in the limit.4 Back to DataScanning the filter outputs Tγ(y) along the hyperparameter γ in view ofTheorem 2, is illustrated in the Figures 5 and 6 for the brain and gene data.Visual inspection of the plots reveals clearly what the desired outputs of themethod are. On the other hand, we did not yet find an overall satisfying un-supervised method for the identification of an appropriate hyperparameter γ.6 Winkler and al.Fig. 5. Some steps of a scan through Tγ(y) along decreasing hyperparameters γ forfMRI brain data. Dots indicate data y. Upper right is desired. (Data from D. Auer)This problem is crucial for such and many similar approaches. For example,there is a host of model selection criteria like the classical ones from Akaike(1974) and Schwarz (1978). Cavanaugh (1997) develops exact criteria whichcan be adapted to our case, see Kempe (2003). Unfortunately, estimatorsbased on these methods basically return data, as shown in Fig. 7. Thereforewe watched out for a criterion which is stable under changes of the hyperpa-rameter γ and of data y. Our first naive idea was to choose Tγ∗(y) with γ∗from the longest interval of γ-values according to Theorem 2. For the braindata, this simple method outruled the classical criteria. Its results for braindata are contrasted to the classical criteria in Fig. 7.Fig. 6. Scanning Tγ(y) along decreas-ing hyperparameters γ for gene data;steps 1, 2, 3, 4, 7, and 11. Dots indi-cate data y. Third frame desired, cf.stimulus in Fig. 1.Segmentation by Potts Models 7Fig. 7. Stimulus, data, Tγ(y) for hyperparameter from Akaike’s and Schwarz’ infor-mation criterion and longest interval criterion: brain data. The latter gives a decentestimate whereas the former basically return data.For the gene data, we have the additional restriction that the ‘true’ signalshould be decreasing. Therefore, we modified our strategy to choose γ fromthe leftmost γ-interval on which Tγ(y) decreases. Fig. 8 displays some of theseestimates.Fig. 8. Estimate for leftmost γ-interval with decreasing Tγ(y): gene data.We obtained partial results about such ‘estimators’ of intervals, but asatisfying rigorous justification is still missing. This is work in progress.5 Summary and OutlookThe Potts functional discussed above is a simple instance of a family of similarfunctionals. There are modified and more complicated penalties, or other dataterms, for example with the sum of squares replaced by absolute deviations.The functionals may live on signals with discrete or continuous time.For example, the Blake-Zisserman functionalx 7−→ BZ(x) =n−1∑k=1min{(xk+1 − xk)2/µ2, ν}+n∑k=1(xk − yk)2considers a deviation δ = |xi+1−xi| as jump if δ > ν1/2µ. Between subsequentjumps it returns a signal which is smooth in the L2-sense. A comprehensivetreatment is Blake and Zisserman (1987). This functional was constructed asa discrete approximation of the Mumford-Shah functionalEµ,ν,g(fS) := ν|S|+n+1∑k=1∫Jk(S)(fk(x)− g(x))2 + 1µ2|f ′k(x)|2 dx, (2)8 Winkler and al.where time varies over a continuous time interval and the functions f arecombined of pieces from functions in Sobolev spaces; it was introduced inMumford and Shah (1989). A discussion of such functionals in the spirit ofthe above considerations is work in progress.Acknowledgement. All simulations were performed by means of thesoftware package Ants In Fields developed by Friedrich (2003a) in cooper-ation with the University of Heidelberg. The CD-ROM is attached to Winkler(2003). For a free download see Friedrich (2003b). We are also indebted to J.Beckers, A. Drobyshev, and D. Auer for providing data and introducing usto their subjects.ReferencesAKAIKE, H. (1974):A new look at the statistical model idenfication. IEEE Trans-actions on Automatic Control, 19(6):716–723.BLAKE, A. and ZISSERMAN, A. (1987): Visual Reconstruction. The MIT PressSeries in Artificial Intelligence, MIT Press, Massachusetts, USA.CAVANAUGH, J.E. (1997): Unifying the derivations for the Akaike and correctedAkaike information criteria. Statistics & Probability Letters, 33:201–208.DAVIES, P.L. (1995): Data features. J. of the Netherlands Society for Statisticsand Operations Research, 49:2,185–245.DAVIES, P.L. and KOVAC, A. (2001): Local extremes, runs, strings and multires-olution. Ann. Stat., 29, 1–65.DROBYSHEV, A. L., MACHKA, CHR., HORSCH, M., SELTMANN, M., LIEB-SCHER, V., HRABE´ DE ANGELIS, V. and BECKERS, J. (2003): Specificityassessment from fractionation experiments, (SAFE): a novel method to eval-uate microarray probe specificity based on hybridization stringencies. NucleicAcids Res. 31:2, 1-10.FRIEDRICH, F. (2003a): Stochastic Simulation and Bayesian Inference for Gibbsfields. CD-ROM, Springer Verlag, Heidelberg, New York.FRIEDRICH, F. (2003b): AntsInFields: Stochastic simulation and Bayesian in-ference for Gibbs fields, URL: http://www.AntsInFields.de.KEMPE, A. (2003), Statistical analysis of the Potts model and applications inbiomedical imaging, Thesis, Institute of Biomathematics and Biometry, Na-tional Research Center for Environment and Health Munich, Germany.MUMFORD, D. and SHAH, J. (1989): Optimal approximation by piecewise smoothfunctions and associated variational problems. Comm. Pure Appl. Math. 42,577-685.SCHWARZ, G. (1978): Estimating the dimension of a model. The Annals of Statis-tics, 6(2):461–464.SERRA, J. (1982, 1988): Image analysis and mathematical morphology. Vol. I, II.Acad. Press, London.WINKLER, G. (2003):. Image Analysis, Random Fields and Markov Chain MonteCarlo Methods. A Mathematical Introduction. volume 27 of Applications ofMathematics, Springer Verlag, Berlin, Heidelberg, New York, second edition.WINKLER, G. and LIEBSCHER, V. (2002): Smoothers for Discontinuous Signals.J. Nonpar. Statist. 14:1-2, 203–222.

(2003a): Stochastic Simulation and Bayesian Inference for Gibbs ﬁelds.

AntsInFields: Stochastic simulation and Bayesian inference for Gibbs ﬁelds,

Data features.

Estimating the dimension of a model.

Image analysis and mathematical

Image Analysis, Random Fields and Markov Chain Monte Carlo Methods. A Mathematical Introduction.

Local extremes, runs, strings and multiresolution.

new look at the statistical model idenﬁcation.

Optimal approximation by piecewise smooth functions and associated variational problems.

Smoothers for Discontinuous Signals.

Speciﬁcity assessment from fractionation experiments, (SAFE): a novel method to evaluate microarray probe speciﬁcity based on hybridization stringencies.

Statistical analysis of the Potts model and applications in biomedical imaging, Thesis,

Unifying the derivations for the Akaike and corrected Akaike information criteria.

Visual Reconstruction.

Parsimonious Segmentation of Time Series' by Potts Models

Abstract. Typical problems in the analysis of data sets like time-series or images crucially rely on the extraction of primitive features based on segmentation. Varia-tional approaches are a popular and convenient framework in which such problems can be studied. We focus on Potts models as simple nontrivial instances. The dis-cussion proceeds along two data sets from brain mapping and functional genomics. 

Gerhard Winkler

Angela Kempe

Volkmar Liebscher

Olaf Wittich

CiteSeerX

Parsimonious segmentation of time series by potts models

Winkler, G

Kempe, A

Liebscher, V

Wittich, O Olaf

Repository TU/e

Parsimonious segmentation of time series by Potts models

Winkler, G.

Pure OAI Repository

Universität München: Elektronischen Publikationen

Winkler, Kempe, Liebscher, Wittich:
Parsimonious Segmentation of Time Series’ by Potts
Models
Sonderforschungsbereich 386, Paper 348 (2003)
Online unter: http://epub.ub.uni-muenchen.de/
Projektpartner
Parsimonious Segmentation of Time Series’
by Potts Models
Gerhard Winkler1, Angela Kempe2, Volkmar Liebscher1, and Olaf Wittich1
1 Institute of Biomathematics and Biometry
GSF - National Research Center for Environment and Health,
D-85758 Neuherberg/München, Germany
2 Graduate Programme Applied Algorithmic Mathematics
Center for Mathematical Sciences, Munich University of Technology
D-85747 Garching
Abstract. Typical problems in the analysis of data sets like time-series or images
crucially rely on the extraction of primitive features based on segmentation. Varia-
tional approaches are a popular and convenient framework in which such problems
can be studied. We focus on Potts models as simple nontrivial instances. The dis-
cussion proceeds along two data sets from brain mapping and functional genomics.
1 Introduction
The purpose of the present note is twofold: We want to give an elementary
introduction to variational approaches to the analysis of one- and multi-
dimensional data, and further to illustrate by way of simple data sets and
statistical models what we mean by parsimonious statistics.
We will briefly discuss a particularly simple parsimonious approach to the
statistical analysis of real-world data sets from life-sciences. Frequently, there
is little or no ground truth, and the stochastic mechanism generating (noisy)
data is essentially unknown. The only way to associate data to some hidden
real event is to verify or falsify rough and basic criteria which characterize
the event in question. Such criteria frequently are based on primitive signal
features. In images these may be boundaries between regions of different
intensity or texture, in time series they may be morphological features like
modes or ‘ups and downs’, domains of monotony, or plateaus where the signal
is constant. We start the discussion with two one-dimensional data sets, one
from brain mapping and one from functional genomics. We expect that in
these examples the observation period can be partitioned into intervals where
the underlying signal can reasonably be represented by a constant. This is a
primitive morphological feature, and the resulting step functions allow sound
biological interpretations.
To extract piecewise constant ‘regressions’ from data, we adopt the sim-
plest variational approach based on the Potts model. It is well known to
physicists as the straightforward generalization of the Ising model for binary
spins to multiple states. For a detailed discussion see Winkler (2003).
2 Winkler and al.
2 Two Data Sets from Life Sciences
In order to introduce and illustrate the concept, we present two sets of data.
The first one consists of time series from functional magnetic resonance imag-
ing (fMRI) of the human brain, and the second one of melting or fractionation
curves for spots on a cDNA microchip.
Example 1 (fMRI Brain Data: Identification of Response Regions). The final
aim is to identify regions of increased activity in the human brain in response
to outer stimuli. Typically such stimuli are boxcar shaped as indicated in
Fig. 1. They may represent ‘light or sound on and off’, i.e. visual or acoustic
stimuli, or tactile ones like finger tipping on a desk. Functional magnetic res-
onance imaging (fMRI) exploits the BOLD effect which basically is a change
of paramagnetic properties caused by an increase of blood flow in response to
the demand of activated neurons for more oxygen. The degradation mecha-
Fig. 1. A box car shaped signal representing ‘on-
off’ stimuli in fMRI brain mapping.
nism along the path ‘(complex) eye - (highly complex) brain - (complicated)
measuring device’ is only partially known. Moreover, measurement is indi-
rect, since the recorded BOLD effect is a physiological quantity related to
increase of blood flow and not a direct function of cortical activation. Hence
a parsimonious approach based on significant plateaus should be appropriate.
Example 2 (Fractionation Curves from Gene Expression). The aim of this
experiment is to explore the structure of unknown genes. To this end, single
stranded sections of known cDNA are put on spots of microchips, which
typically consist of about 20.000 spots. Each section is a finite sequence of four
nucleic acids, which are coded by the letters A(denin), C(ytosin), G(uanin),
and T(hymin). If further nucleic acids are added then they tend to bind to
the known nucleic acids where T binds to A, and G binds to C.
Hence sections of single stranded unknown cDNA tend to pair with DNA of
similar sequence. The binding energy is maximal for perfect matches like
A C T A C A G T A C C C A
T G A T G T C A T G G G T
and such a perfect match means high stability. With perfect match the
unknown sequence could be identified perfectly. A main problem is cross-
hybridisation, which means that DNA sections pair with DNA of similar -
but not precisely equal - sequence, for example
Segmentation by Potts Models 3
A C T A C A G T A C C C A
T G A T T T C A T G A G T
Perfect match and mismatch are illustrated in Figure 2. A new and innova-
tive experiment provides data which hopefully will allow to identify mismatch
dissociation at low stringency. It is called ‘Specificity Assessment From Frac-
Fig. 2. Specific and unspecific hybridization
tionation Experiments’ or in short-hand notation ‘SAFE’, see Drobyshev et
al.(2003). It is plausible that ‘the melting temperature’ of double stranded
DNA depends on length and contents of specific sequences. It is also plausible
that increasing temperature has similar effects as increasing washing strin-
gencies with formamide solutions. Both decrease the binding energies and
thus cross-hybridisation. This is the basis for the measurement of specific
and cross-hybridisation. In the experiment, the chips are washed repeatedly
(29 times) with formamide solutions of increasing concentration, and fraction-
ation curves like in Fig. 3 are recorded. The aim of the statistical analysis is
Fig. 3. Typical fractionation curves of single spots: lousy, fairly good, intermediate.
(Data from Drobyshev et al. (2003))
to identify locations and heights of abrupt decreases, since they indicate that
a certain type of cross-hybridizing cDNA was washed away.
In view of such data, one may doubt about too ‘specific’ methods or too
detailed models for their analysis, and in fact we do so. A way out of this
misery is to try a parsimonious approach as indicated above, see Davies
(1995). There are attempts like in Davies and Kovac (2001), who adopt the
taut string algorithm and its relatives for certain types of data. We tried a
parsimonious variational approach called the Potts Model for our data.
4 Winkler and al.
3 The Potts Model: Rigorous Results
The relevant features in Example 1 are successions of high and low plateaus,
and in Example 2 the positions of rapid decreases and their height. Therefore
we try to fit piecewise constant functions to our data.
The Potts functional is defined by
x = (x1, . . . , xn) 7−→ Pγ,y(x) = γ|J(x)|+
n∑
k=1
(
xk − yk
)2
, (1)
where y = (y1, . . . , yn) denotes real (fixed) data, and J(x) is the set of time
points k where xk 6= xk+1, k = 1, . . . , n− 1. |J(x)| denotes the cardinality of
J(x). The second term rates fidelity of the signal x to data y, and the first
one penalizes undesired properties of x. Thus the functional is a penalized
likelihood function.
A minute of contemplation reveals that there are three elementary con-
cepts combined in this model:
(i) A notion of a ‘jump’ or ‘break’: In the Potts model such a jump is present,
where the values of the signal x in two subsequent time points differ from
each other.
(ii) A notion of smoothness: this concerns the behaviour of the signal between
two subsequent jumps. It is a consequence of (i) that in the Potts model
a signal is constant there.
(iii) A notion of fidelity to data, i.e. some measure of distance between data
y and the signal x.
Note that(ii) is a rather strict notion of smoothness: the signal on a discrete
interval is ‘smooth’ only if it is constant. The first term penalizes the number
of jumps irrespective of their size and the parameter γ > 0 controls the degree
of smoothness.
Given data y, a ‘filter output’ Tγ(y) is defined as a signal which minimizes
Pγ,y. In general, it is not unique, but fortunately the following result from
the forthcoming thesis Kempe (2003) guarantees uniqueness almost surely:
Theorem 1. Suppose that the law of data y admits a Lebesgue density. Then
for almost all y the functional Pγ,y in (1) has one and only one minimizer.
If the hypotheses hold a filter is defined uniquely for almost all y by the signal
Tγ(y) = argmin
x
Pγ,y(x).
We are going now to report essential properties of the filter. It is crucial that
the range of hyperparameters γ can be partitioned into intervals, on which
the estimate does not change as shown in Kempe (2003). Dependence on
hyperparameters is illustrated in Fig. 4.
Segmentation by Potts Models 5
Theorem 2. For almost all data y the following is true: There are an integer
k and hyperparameters ∞ = γ0 > γ1 > · · · > γk > γk+1 = 0 such that Tγ(y)
is unique for all γ ∈ (γj+1, γj). Moreover, it is the same time-series for all
γ ∈ (γj+1, γj). Tγ(y) is a constant signal for each γ > γ1, and Tγ(y) = y for
γ < γk. The number of jumps of Tγ(y) on the intervals (γj+1, γj) increases in
j. For each 0 < i ≤ k, the functional Pγi,y has precisely the two minimizers
belonging to the γ-intervals adjacent to γi.
γrγ1rγ2rγ3rγkr0 data constant
Fig. 4. Intervals on which estimates do not change.
Significant simulations can only be carried out with an exact algorithm
for the computation of the minimizer. This rules out stochastic algorithms
like simulated annealing. For one dimension, an algorithm based on ideas
from dynamic programming was presented in Winkler and Liebscher (2002).
Theorem 3. There is an algorithm to compute a minimizer of Pγ,y in time
complexity O(n3) for all γ ∈ R simultaneously.
The filter has some more pleasant properties. In particular, the iteration
of the filter stops after one step More precisely, a repeated application returns
the same signal as a single one, or in other words the filter is idempotent in
the sense that Tγ ◦ Tγ = Tγ . This implies that Tγ is a morphological filter in
the sense of Serra (1982, 1988), see Winkler and Liebscher (2002):
Theorem 4. The Potts filter is idempotent.
The filter has continuity or consistency properties like the following one:
Theorem 5 (V. Liebscher, O. Wittich, unpublished). Let γ ∈ R and
y∞ ∈ Rn. Suppose that Tγ(y∞) is unique. If y∞ is degraded by random noise
εn according to Y n = y∞+εn, and noise εn tends to zero in probability, then
Tγ(Y n) tends to Tγ(y∞) in probability.
We are not interested in a ‘restoration’ but in feature extraction. This is
reflected by the theorem since we recover Tγ(y∞) - and not y∞ - in the limit.
4 Back to Data
Scanning the filter outputs Tγ(y) along the hyperparameter γ in view of
Theorem 2, is illustrated in the Figures 5 and 6 for the brain and gene data.
Visual inspection of the plots reveals clearly what the desired outputs of the
method are. On the other hand, we did not yet find an overall satisfying un-
supervised method for the identification of an appropriate hyperparameter γ.
6 Winkler and al.
Fig. 5. Some steps of a scan through Tγ(y) along decreasing hyperparameters γ for
fMRI brain data. Dots indicate data y. Upper right is desired. (Data from D. Auer)
This problem is crucial for such and many similar approaches. For example,
there is a host of model selection criteria like the classical ones from Akaike
(1974) and Schwarz (1978). Cavanaugh (1997) develops exact criteria which
can be adapted to our case, see Kempe (2003). Unfortunately, estimators
based on these methods basically return data, as shown in Fig. 7. Therefore
we watched out for a criterion which is stable under changes of the hyperpa-
rameter γ and of data y. Our first naive idea was to choose Tγ∗(y) with γ∗
from the longest interval of γ-values according to Theorem 2. For the brain
data, this simple method outruled the classical criteria. Its results for brain
data are contrasted to the classical criteria in Fig. 7.
Fig. 6. Scanning Tγ(y) along decreas-
ing hyperparameters γ for gene data;
steps 1, 2, 3, 4, 7, and 11. Dots indi-
cate data y. Third frame desired, cf.
stimulus in Fig. 1.
Segmentation by Potts Models 7
Fig. 7. Stimulus, data, Tγ(y) for hyperparameter from Akaike’s and Schwarz’ infor-
mation criterion and longest interval criterion: brain data. The latter gives a decent
estimate whereas the former basically return data.
For the gene data, we have the additional restriction that the ‘true’ signal
should be decreasing. Therefore, we modified our strategy to choose γ from
the leftmost γ-interval on which Tγ(y) decreases. Fig. 8 displays some of these
estimates.
Fig. 8. Estimate for leftmost γ-interval with decreasing Tγ(y): gene data.
We obtained partial results about such ‘estimators’ of intervals, but a
satisfying rigorous justification is still missing. This is work in progress.
5 Summary and Outlook
The Potts functional discussed above is a simple instance of a family of similar
functionals. There are modified and more complicated penalties, or other data
terms, for example with the sum of squares replaced by absolute deviations.
The functionals may live on signals with discrete or continuous time.
For example, the Blake-Zisserman functional
x 7−→ BZ(x) =
n−1∑
k=1
min{(xk+1 − xk)2/µ2, ν}+
n∑
k=1
(
xk − yk
)2
considers a deviation δ = |xi+1−xi| as jump if δ > ν1/2µ. Between subsequent
jumps it returns a signal which is smooth in the L2-sense. A comprehensive
treatment is Blake and Zisserman (1987). This functional was constructed as
a discrete approximation of the Mumford-Shah functional
Eµ,ν,g(fS) := ν|S|+
n+1∑
k=1
∫
Jk(S)
(
fk(x)− g(x)
)2 + 1
µ2
|f ′k(x)|2 dx, (2)
8 Winkler and al.
where time varies over a continuous time interval and the functions f are
combined of pieces from functions in Sobolev spaces; it was introduced in
Mumford and Shah (1989). A discussion of such functionals in the spirit of
the above considerations is work in progress.
Acknowledgement. All simulations were performed by means of the
software package Ants In Fields developed by Friedrich (2003a) in cooper-
ation with the University of Heidelberg. The CD-ROM is attached to Winkler
(2003). For a free download see Friedrich (2003b). We are also indebted to J.
Beckers, A. Drobyshev, and D. Auer for providing data and introducing us
to their subjects.
References
AKAIKE, H. (1974):A new look at the statistical model idenfication. IEEE Trans-
actions on Automatic Control, 19(6):716–723.
BLAKE, A. and ZISSERMAN, A. (1987): Visual Reconstruction. The MIT Press
Series in Artificial Intelligence, MIT Press, Massachusetts, USA.
CAVANAUGH, J.E. (1997): Unifying the derivations for the Akaike and corrected
Akaike information criteria. Statistics & Probability Letters, 33:201–208.
DAVIES, P.L. (1995): Data features. J. of the Netherlands Society for Statistics
and Operations Research, 49:2,185–245.
DAVIES, P.L. and KOVAC, A. (2001): Local extremes, runs, strings and multires-
olution. Ann. Stat., 29, 1–65.
DROBYSHEV, A. L., MACHKA, CHR., HORSCH, M., SELTMANN, M., LIEB-
SCHER, V., HRABÉ DE ANGELIS, V. and BECKERS, J. (2003): Specificity
assessment from fractionation experiments, (SAFE): a novel method to eval-
uate microarray probe specificity based on hybridization stringencies. Nucleic
Acids Res. 31:2, 1-10.
FRIEDRICH, F. (2003a): Stochastic Simulation and Bayesian Inference for Gibbs
fields. CD-ROM, Springer Verlag, Heidelberg, New York.
FRIEDRICH, F. (2003b): AntsInFields: Stochastic simulation and Bayesian in-
ference for Gibbs fields, URL: http://www.AntsInFields.de.
KEMPE, A. (2003), Statistical analysis of the Potts model and applications in
biomedical imaging, Thesis, Institute of Biomathematics and Biometry, Na-
tional Research Center for Environment and Health Munich, Germany.
MUMFORD, D. and SHAH, J. (1989): Optimal approximation by piecewise smooth
functions and associated variational problems. Comm. Pure Appl. Math. 42,
577-685.
SCHWARZ, G. (1978): Estimating the dimension of a model. The Annals of Statis-
tics, 6(2):461–464.
SERRA, J. (1982, 1988): Image analysis and mathematical morphology. Vol. I, II.
Acad. Press, London.
WINKLER, G. (2003):. Image Analysis, Random Fields and Markov Chain Monte
Carlo Methods. A Mathematical Introduction. volume 27 of Applications of
Mathematics, Springer Verlag, Berlin, Heidelberg, New York, second edition.
WINKLER, G. and LIEBSCHER, V. (2002): Smoothers for Discontinuous Signals.
J. Nonpar. Statist. 14:1-2, 203–222.


Abstract. Typical problems in the analysis of data sets like time-series or images crucially rely on the extraction of primitive features based on segmentation. Variational approaches are a popular and convenient framework in which such problems can be studied. We focus on Potts models as simple nontrivial instances. The discussion proceeds along two data sets from brain mapping and functional genomics. 

Parsimonious segmentation of time series’ by Potts models

PuSH

Parsimonious segmentation of time series by potts models.

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.73.6633

Parsimonious Segmentation of Time Series' by Potts Models

Abstract

Similar works

Full text

Available Versions

Open Access LMU

CiteSeerX

Repository TU/e

Pure OAI Repository

Universität München: Elektronischen Publikationen

CiteSeerX

CiteSeerX

PuSH

Pure OAI Repository