Measuring the transmitted information in metric-based clustering has become
something of a standard test for the performance of a spike train metric. In
this comment, the recently proposed L_p Victor-Purpura metric is used to
cluster spiking responses to zebra finch songs, recorded from field L of
anesthetized zebra finch. It is found that for these data the L_p metrics with
p>1 modestly outperform the standard, p=1, Victor-Purpura metric. It is argued
that this is because for larger values of p, the metric comes closer to
performing windowed coincidence detection.Comment: 9 pages, 3 figures included as late

Houghton, Conor

English

arXiv

non-peer-reviewedMeasuring the transmitted information in metric-based clustering has become something of a standard test for the performance of a spike train metric. In this comment, the recently proposed L_p Victor-Purpura metric is used to cluster spiking responses to zebra finch songs, recorded from field L of anesthetized zebra finch. It is found that for these data the L_p metrics with p>1 modestly outperform the standard, p=1, Victor-Purpura metric. It is argued that this is because for larger values of p, the metric comes closer to performing windowed coincidence detection

HOUGHTON, CONOR JAMES

Irish Universities

A comment on "A fast L_p spike alignment metric" by A. J. Dubbs, B. A. Seiler and M. O. Magnasco [arXiv:0907.3137]

Measuring the transmitted information in metric-based clustering has become something of a standard test for the performance of a spike train metric. In this comment, the recently proposed L_p Victor-Purpura metric is used to cluster spiking responses to zebra finch songs, recorded from field L of anesthetized zebra finch. It is found that for these data the L_p metrics with p&gt;1 modestly outperform the standard, p=1, Victor-Purpura metric. It is argued that this is because for larger values of p, the metric comes closer to performing windowed coincidence detection

Explore Bristol Research

                          Houghton, C. (2009). A comment on "A fast L_p spike alignmentmetric" by A. J. Dubbs, B. A. Seiler and M. O. Magnasco[arXiv:0907.3137].Early version, also known as pre-printLink to publication record in Explore Bristol ResearchPDF-documentUniversity of Bristol - Explore Bristol ResearchGeneral rightsThis document is made available in accordance with publisher policies. Please cite only thepublished version using the reference above. Full terms of use are available:http://www.bristol.ac.uk/red/research-policy/pure/user-guides/ebr-terms/arXiv:0908.1260v1  [q-bio.NC]  10 Aug 2009A comment on ‘A fast Lp spikealignment metric’ by A. J. Dubbs,B. A. Seiler and M. O. Magnasco[arXiv:0907.3137]Conor Houghton∗School of Mathematics, Trinity College Dublin, Ireland27 July 2009AbstractMeasuring the transmitted information in metric-based clusteringhas become something of a standard test for the performance of aspike train metric. In this comment, the recently proposed Lp Victor-Purpura metric is used to cluster spiking responses to zebra finchsongs, recorded from field L of anesthetized zebra finch. It is foundthat for these data the Lp metrics with p > 1 modestly outperformthe standard, p = 1, Victor-Purpura metric. It is argued that this isbecause for larger values of p, the metric comes closer to performingwindowed coincidence detection.1 IntroductionThere are a number of reasons why metrics are believed to be useful forstudying spike trains. It can be argued that it is the most general, useful,mathematical framework for spike trains [Victor and Purpura 1997], it maybe possible to find a manifold of spike trains using local linear methods and,∗houghton@maths.tcd.ie1of most relevance to the present discussion, it is possible that studying spiketrain metrics is a useful approach to understanding how content is coded inspike trains.The Victor-Purpura metric is an edit distance metric on the space ofspike trains. The distance between two spike trains is, in effect, calculatedas the cost of changing one spike train into the other by adding, deleting, ormoving spikes, with an individual cost for each type of elementary moves. Inparticular, a cost of one is associated with adding or deleting a spike and acost of qδt with moving a spike a time δt: the distance isd(u,v; q) = minγcγ;q(u,v) (1)where cγ(u,v) is the cost of a sequence of elementary moves γ and is calcu-lated by adding the cost of all the elementary moves in the sequence. Theminimum is taken over all sequences γ changing u to v. The parameter 2/qgives an important timescale for the metric: it is never worthwhile to movea spike more than 2/q since it would be cheaper to delete the spike from onetemporal location and to add it to the other. This gives a timescale that,roughly speaking, separates jitter from unreliability in comparing spikes inthe spike trains. However, the Victor-Purpura metric does more than this;it explicitly pairs up those spikes that can be thought of as being related byjitter.The Victor-Purpura metric has an l1 character: the cost cγ is a simplelinear sum of the individual costs of the individual moves. The general-ization proposed in [Dubbs et al. 2009] changes this to an lp-like sum. Fornotational convenience, from now on the set of sequences will be restricted:the minimum sequence will never involve moving a spike after it has beenadded, it would always be cheaper to add the spike in the correct location,similarly, spikes are never moved before they are deleted. It is also specifiedthat spikes can only be moved once. With these restrictions, the sequenceγ can be considered to be an unordered set made up of deletions, additionsand moves. Letγ = α ∪ µ (2)where α is the set of additions and deletions and µ is the set of moves. Theelements of µ are pairs of spikes (u, v), with u ∈ u and v ∈ v, that are relatedby jitter.Now,cγ;q(u,v) = |α| +∑(u,v)∈µq|u − v| (3)2This is generalized in [Dubbs et al. 2009] tocγ;q,p(u,v) =|α| +∑(u,v)∈µqp|u − v|p1/p(4)with p ≥ 1, to give the Lp Victor-Purpura metricd(u,v; q, p) = minγcγ;q,p(u,v) (5)where the minimum is taken over sequences γ changing u to v and satisfy-ing the restrictions specified above. An algorithm for calculating this quan-tity is given in [Dubbs et al. 2009]; the existing algorithms used to calculatethe Victor-Purpura metric, [Victor et al. 2009], for example, could also beadapted to p > 1.2 Evaluating the metric using clusteringIn the data that will be considered here, spike trains were recorded from fieldL of the auditory fore-brain of anesthetized zebra finch during playback of20 con-specific songs with each song repeated ten times, to give a total of200 spike trains. These spike trains, and the experimental conditions usedto produce them, are described in [Narayan et al. 2006, Wang et al. 2007]and they have previously been used to compare metrics in [Houghton 2009,Houghton and Victor 2009]. The key point, in this context, is that a goodmetric, one which depends on the content encoded in the spike train, shouldmeasure a smaller distance between spiking responses to the same song thanbetween responses to different songs. One motivation for looking at spiketrain metrics is that scoring metrics in this way, and examining how goodmetrics measure distance, might reveal details about how content is encoded.The usual method for evaluating how well a metric succeeds in this way isto calculate the transmitted information [Victor and Purpura 1996]. Thereare two ways to cluster the spike trains: a true clustering, based on theidentity of the song that elicited the response and an estimated clustering,based on the metric distances between the responses. Roughly speaking, thetransmitted information quantifies the amount of information about the trueclustering given by the estimated clustering.Calculating the transmitted information relies on the calculation of aconfusion matrix N . This is described in detail in [Victor and Purpura 1996],3described specifically for the data considered here, N is a 20 × 20 matrix ofintegers where the entry Nij counts how many of the spike trains from theith true cluster is closest, on average, to the spike trains in the jth truecluster. As in [Victor and Purpura 1996] this averaging is carried out as aroot-mean-square averaging to under-weigh outliers. If the metric clusteringis close to the true clustering, N will be nearly diagonal. The transmittedinformation gives a measure of this, it ish̃ =1n ln 20∑ijNij(ln Nij − ln∑kNkj − ln∑kNik + ln n). (6)where n =∑i∑j Nij . All sums are from one to 20: the number of clusters.The factor of ln 20 normalizes h̃ and so that it takes values between zeroand one; a low value corresponds to poor clustering and a value near onecorresponds to a near-diagonal confusion matrix.The main purpose of this comment is to evaluate the Lp Victor-Purpurametric by calculating h̃ for the zebra finch data previous used to evaluateother spike train metrics [Houghton 2009, Houghton and Victor 2009]. Thedata set contains 24 sets of spike trains, corresponding to 24 recording sitesfrom multiple birds. Although the songs themselves are of different lengths,all of the songs are at least a second long and, in each case, the spike trainis truncated to one second, starting at song onset.The performance of the Lp Victor-Purpura metric is illustrated in Fig. 1.A remarkable feature is that h̃ rarely decreases as p is increased. Other multi-parameter metrics show a mixed effect; although changing a parameter mightincrease the average h̃ across the sites, there will typically be many individualsites where h̃ decreases. In the case of p, while the average improvement as pincreases is modest and a small number of sites show no change; there is nosite where the clustering is negatively affected when p is changed from oneto two, three do show a tiny decrease between p = 1 and p = 1.5. h̃ increasesfor 19 sites between p = 1 and p = 2 and for 15 between p = 2 and p = 3,the improvement gets less and less and appears to have plateaued by p = 10.The average performance is plotted against p in Fig. p. The optimal valueof q does not vary significantly as p changes; however, the range of q thatproduces the optimal performance does get wider.40h̃1FEDCBAFigure 1: Performance as p changes. A, B and C correspond to p = 1, p = 2and p = 10. Each horizontal line runs from zero to one. For clarity, there aretiny gaps at 0.25, 0.5 and 0.75. Each small vertical dash marks the h̃ value forone site where for each site the optimal value of q is used. The large verticaldash marks the average, 0.732, 0.751 and 0.761 respectively. To indicate therange over which h̃ values vary, two other metrics are plotted for comparison.D corresponds to the van Rossum metric [van Rossum 2001], with each siteusing its optimal value of τ . E corresponds to the synapse metric, with eachsite using its optimal value of τ and µ [Houghton 2009]. Finally, F corre-sponds to the L1 van Rossum metric, with a boxcar filter; this metric mea-sures a very similar distance to the p = 1 metric [Houghton and Sen 2008].50.740.750.761 2 3 4 5 6 7 8 9 10Figure 2: The average performance plotted against p. h̃ has been calculatedfor a range of p values using the optimal value of q for each site and theaverage value of h̃ is plotted against p.3 DiscussionIt is interesting to compare this metric to another generalization of the Victor-Purpura metric [Victor and Purpura 1996, Victor and Purpura 1997]. Thelinear distance function qδt can be replaced by any convex function, f(|δt|),of distance:cγ;f(·)(u,v) = |α| +∑(u,v)∈µf(|u − v|). (7)The distance function f(|δt|) must be convex to ensure that the triangularinequality is satisfied. Now, the function qpδtp in cγ;q,p above is concave forp > 1. However, because of the 1/p exponent in d(u,v; q, p), this does notcause the same difficulty with the triangular inequality: an interesting prop-erty of the Lp metrics is that they satisfy the triangular inequality despitehaving a concave distance function.As p is increased, qp|δt|p becomes increasingly concave and spikes locatedat similar times contribute less and less to the total cost and this cost ispredominately made up of the cost of adding and deleting spikes that arenot related by jitter. This means that the metric performs more and moreas a windowed coincidence detector as p increases. The fact that it is alwaysbeneficial to increase p appears to imply that, for these data, coincidence andcoincidence detection has a role in the encoding and decoding of content inspike trains.60.250.50.7510.25 0.5 0.75 1               Figure 3: Comparing the L1 boxcar van Rossum metric with the p = 1and p = 10 Lp metrics. Here h̃ values for the p = 1 and p = 10 metricsare graphed against the values for the L1 van Rossum metric with a boxcarfilter with optimal values of the q or τ parameter has been used for eachsite. The p = 1 and p = 10 values are marked by a horizontal line, the twovalues for a given site are joined by a vertical line. Since increasing p neverdecreases h̃ between p = 1 and p = 10, the topmost of the two horizontallines corresponds to p = 10. The ‘x=y’ line is also plotted for clarity.7The pairing of near coincident spikes is the key distinction between theVictor-Purpura metrics and the van Rossum metrics, a family of spike trainmetrics which are calculated by comparing reconstructed rate functions. Infact, computationally, an L1 van Rossum metric with a boxcar filter measuresa very similar distance to the Victor-Purpura metric [Houghton and Sen 2008]and, as seen in Fig. 3, has a very similar performance. The way in which LpVictor-Purpura metric performance increased with p seems to indicate thatthe significant temporal structure of spikes is not fully accounted for by thetemporal structure of a spike rate.AcknowledgmentsC.H. is supported by Science Foundation Ireland grant 08/RFP/MTH1280.He thanks Kamal Sen for the use of the zebra finch data analyzed here andJonathan Victor for comments on an early draft of this comment.References[Dubbs et al. 2009] A. J. Dubbs, B. A. Seiler, and M. O. Magnasco. A fastLp spike alignment metric. (2009, preprint, [arXiv:0907.3137])[Houghton 2009] C. Houghton. Studying spike trains using a van Rossummetric with a synapses-like filter. Journal of Computational Neuroscience,26:149–155, 2009.[Houghton and Sen 2008] C. Houghton and K. Sen. A new multi-neuronspike-train metric. Neural Computation 20:1495–1511, 2008.[Houghton and Victor 2009] C. Houghton and J. D. Victor. Spike codes andspike metrics. (2009, book chapter, under review).[Narayan et al. 2006] R. Narayan, G. Graña, and K. Sen. Distinct time scalesin cortical discrimination of natural sounds in songbirds. Journal of Neu-rophysiology, 96:252–258, 2006.[van Rossum 2001] M. van Rossum. A novel spike distance. Neural Compu-tation, 13:751–763, 2001.8[Victor et al. 2009] J. D. Victor, D. Goldberg, and D. Gardner. Dynamicprogramming algorithms for comparing multineuronal spike trains via cost-based metrics and alignments. Journal of Neuroscience Methods, 161:351–360, 2007.[Victor and Purpura 1996] J. D. Victor and K. P. Purpura. Nature and pre-cision of temporal coding in visual cortex: a metric-space analysis. Journalof Neurophysiology, 76:1310–1326, 1996.[Victor and Purpura 1997] J. D. Victor and K. P. Purpura. Metric-spaceanalysis of spike trains: theory, algorithms and application. Network:Computation in Neural Systems, 8:127–164, 1997.[Wang et al. 2007] L. Wang, R. Narayan, G. Graña, M. Shamir, and K. Sen.Cortical discrimination of complex natural stimuli: can single neuronsmatch behavior? Journal of Neuroscience, 27:582–9, 2007.9

A comment on "A fast L_p spike alignment metric" by A. J. Dubbs, B. A. Seiler and M. O. Magnasco [arXiv:0907.3137]

Abstract

Similar works

Full text

Available Versions

Irish Universities

Explore Bristol Research