The latent block model (LBM) is a flexible probabilistic tool to describe
interactions between node sets in bipartite networks, but it does not account
for interactions of time varying intensity between nodes in unknown classes. In
this paper we propose a non stationary temporal extension of the LBM that
clusters simultaneously the two node sets of a bipartite network and constructs
classes of time intervals on which interactions are stationary. The number of
clusters as well as the membership to classes are obtained by maximizing the
exact complete-data integrated likelihood relying on a greedy search approach.
Experiments on simulated and real data are carried out in order to assess the
proposed methodology.Comment: European Symposium on Artificial Neural Networks, Computational
  Intelligence and Machine Learning (ESANN), Apr 2015, Bruges, Belgium.
  pp.225-230, 2015, Proceedings of the 23-th European Symposium on Artificial
  Neural Networks, Computational Intelligence and Machine Learning (ESANN 2015

Corneli, Marco

Latouche, Pierre

Rossi, Fabrice

English

arXiv

International audienceThe latent block model (LBM) is a flexible probabilistic tool to describe interactions between node sets in bipartite networks, but it does not account for interactions of time varying intensity between nodes in unknown classes. In this paper we propose a non stationary temporal extension of the LBM that clusters simultaneously the two node sets of a bipartite network and constructs classes of time intervals on which interactions are stationary. The number of clusters as well as the membership to classes are obtained by maximizing the exact complete-data integrated likelihood relying on a greedy search approach. Experiments on simulated and real data are carried out in order to assess the proposed methodology

Archive Ouverte en Sciences de l'Information et de la Communication

HAL Id: hal-01163367
https://hal.archives-ouvertes.fr/hal-01163367
Submitted on 12 Jun 2015
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
Distributed under a Creative Commons Attribution| 4.0 International License
Exact ICL maximization in a non-stationary time
extension of the latent block model for dynamic
networks
Marco Corneli, Pierre Latouche, Fabrice Rossi
To cite this version:
Marco Corneli, Pierre Latouche, Fabrice Rossi. Exact ICL maximization in a non-stationary time
extension of the latent block model for dynamic networks. European Symposium on Artificial Neural
Networks, Computational Intelligence and Machine Learning (ESANN), Apr 2015, Bruges, Belgium.
pp.225-230. ￿hal-01163367￿
Exact ICL maximization in a non-stationary
time extension of the latent block model for
dynamic networks
Marco Corneli, Pierre Latouche and Fabrice Rossi
Université Paris 1 Panthéon-Sorbonne - Laboratoire SAMM
90 rue de Tolbiac, F-75634 Paris Cedex 13 - France
Abstract. The latent block model (LBM) is a flexible probabilistic tool
to describe interactions between node sets in bipartite networks, but it
does not account for interactions of time varying intensity between nodes
in unknown classes. In this paper we propose a non stationary temporal
extension of the LBM that clusters simultaneously the two node sets of a
bipartite network and constructs classes of time intervals on which inter-
actions are stationary. The number of clusters as well as the membership
to classes are obtained by maximizing the exact complete-data integrated
likelihood relying on a greedy search approach. Experiments on simulated
and real data are carried out in order to assess the proposed methodology.
1 Introduction
Since the interactions between nodes of a network generally have a time varying
intensity, the network has a non trivial time structure that we aim at inferring.
The approach we follow to introduce a temporal dimension, consists in partition-
ing the entire time horizon, during which we observe interactions, in disjoint time
intervals, having an arbitrary fixed length. Then, we simultaneously cluster the
nodes of the bipartite network and these time intervals, assuming interactions
are generated by a latent block model. A similar view is adopted by Randria-
manamihaga, Côme and Govaert [1], nonetheless with a substantial difference:
they consider time intervals whose membership is not hidden but known in ad-
vance and hence exogenous, whereas in this paper we infer the membership of
each interval by maximizing a likelihood criterion. A similar task is accom-
plished by Guigourès, Boullé and Rossi [3] in a different point of view: they
do not consider fixed-length time intervals, but associate a time stamp to each
interaction “in order to build time segments and clusters of nodes whose edge
distributions are similar and evolve in the same way over the time segments”.
In order to obtain the optimal number of time and nodes clusters, we maximize
the integrated complete-data likelihood (ICL) using a greedy search in a very
similar fashion as in Wyse, Frial and Latouche [4]. This paper is structured as
follows: in Section 2 we present the classical LBM and detail the time extension
proposed. In Section 3 we derive the ICL for this model and in Section 4 we
discuss the experiments we conducted with both simulated and real data. The
Section 5 concludes the paper.
2 A non stationary latent block model
We present here the LBM (Holland et al. [2]), as described in Wyse, Friel,
Latouche (2014). Two sets of nodes are considered: A = {a1, . . . , aN} and
B = {b1, . . . , bM}. Undirected links between node i from A and node j from
B, are counted by the observed variable Xij , being the component (i, j) of the
N ×M adjacency matrix X = {Xij}i≤N,j≤M . Nodes in A and B are clustered
in K and G disjointed subgroups respectively:
A = ∪k≤KAk, Ai ∩Aj = ∅, ∀i 6= j
and similarly for B. Nodes in the same cluster in A have linking attributes of the
same nature to clusters of B. We introduce two hidden vectors c = {c1, . . . , cN}
and w = {w1, . . . , wM} labeling each node’s membership:
ci = k iff ai ∈ Ak, ∀k ≤ K and wj = g iff bj ∈ Bg, ∀g ≤ G.
In order to introduce the temporal dimension, consider now a sequence of equally
spaced, adjacent time steps {∆u := tu − tu−1}u≤U over the interval [0, T ] and a
partition C1, . . . , CD of the same interval
1. We introduce furthermore a random
vector y = {yu}u≤U , such that yu = d if and only if Iu :=]tu−1, tu] ∈ Cd,∀d ≤ D.
We attach to y a multinomial distribution:
p(y|β, D) =
∏
d≤D
β
|Cd|
d ,
where |Cd| = #{Iu : Iu ∈ Cd}. Now we define N Iuij as the number of observed
connections between ai and bj , in the time interval Iu and we make the following
crucial assumption:
p(N Iuij |ci = k,wj = g, yu = d) follows a Poisson(∆uλkgd), (1)
hence the number of interactions is conditionally distributed like a Poisson ran-
dom variable with parameter depending on k, g, d (∆u is constant).
Notation: In the following, for seek of simplicity, we will note:∏
k,g,d
:=
∏
k≤K
∏
g≤G
∏
d≤D
and
∏
ci
:=
∏
i:ci=k
and similarly for
∏
wj
and
∏
yu
.
The adjacency matrix, noted N∆, has three dimensions (N ×M × U) and its
observed likelihood can be computed explicitly:
p(N∆|Λ, c,w,y,K,G,D) =
∏
k,g,d
∆Skgd∏
ci
∏
wj
∏
yu
N Iuij !
e−∆λkgdRkgdλ
Skgd
kgd , (2)
1T and U are linked by the following relation: T = ∆uU .
where we noted Skgd :=
∑
ci
∑
wj
∑
yu
N Iuij and Rkgd := |Ak||Bg||Cd| and the
subscript u was removed from ∆u to emphasize that time steps are equally
spaced for every u.
Since c,w and y are not known, a multinomial factorizing probability density
p(c,w,y|Φ,K,G,D), depending on hyperparameter Φ, is introduced. The joint
distribution of labels looks finally as follows:
p(c,w,y|Φ,K,G,D) =
∏
k≤K
ω
|Ak|
k
∏
g≤G
ρ|Bg|g
∏
d≤D
β
|Cd|
d
 , (3)
where Φ = {ω,ρ,β}.
3 Exact ICL for non stationary LBM
3.1 Exact ICL derivation
The integrated classification criterion (ICL) was introduced as a model selection
criterion in the context of Gaussian mixture models by Biernacky et al. [5].
Côme and Latouche [6] proposed an exact version of the ICL based on a Bayesian
approach for the stochastic block model and Wyse, Friel and Latouche [4] applied
the exact ICL to select the number of clusters in a bipartite network using an
LBM model. This is the approach we follow here. The quantity we focus on is the
complete data log-likelihood, integrated with respect to the model parameters Φ
and Λ = {λkgd}k≤K,g≤G,d≤D:
ICL = log
(∫
p(N∆, c,w,y,Λ,Φ|K,G,D)dΛdΦ
)
. (4)
Introducing a prior distribution ν(Λ,Φ|K,G,D) over the pair Φ,Λ and thanks
to ad hoc independence assumptions, the ICL can be rewritten as follows:
ICL = log
(
ν(N∆|c,w,y,K,G,D)
)
+ log (ν(c,w,y|K,G,D)) . (5)
The choice of prior distributions over the model parameters is crucial to have an
explicit form of the ICL.
3.2 A priori distributions
We consider the conjugate prior distributions. Thus we impose a Gamma a
priori over Λ:
ν(λkgd|akgd, bkgd) =
b
akgd
kgd
Γ(akgd)
λ
akgd−1
kgd e
−bkgdλkgd
and a factorizing Dirichlet a priori distribution to Φ:
ν(Φ|K,G,D) = DirK(ω;α, . . . , α)×DirG(ρ; δ, . . . , δ)×DirD(β; γ, . . . , γ).
It can be proven that the two terms in (5), reduce to:
ν(N∆|c,w,y,K,G,D) =
∏
k,g,d
b
akgd
kgd ∆
Skgd
Γ(akgd)
∏
ci
∏
wj
∏
yu
N Iuij !
(6)
Γ(Skgd + akgd)
[∆Rkgd + bkgd]Skgd+akgd
and:
ν(c,w,y|K,G,D) =Γ(αK)
Γ(α)K
∏
k≤K Γ(|Ak|+ α)
Γ(N + αK)
× Γ(δG)
Γ(δ)G
∏
g≤G Γ(|Bg|+ δ)
Γ(M + δG)
×Γ(γD)
Γ(γ)D
∏
d≤D Γ(|Cd|+ γ)
Γ(U + γD)
. (7)
3.3 ICL Maximization
In order to maximize the integrated complete likelihood (ICL) in equation (5)
with respect to the six unknowns c,w,y,K,G,D, we rely on a greedy search over
labels and the number of nodes and time clusters. This approach is described in
Wyse, Frial and Latouche [4] for a stationary latent block model.
4 Experiments
4.1 Simulated data
Some experiments on simulated data have initially been conducted. Based on
the model described in Section 2, we simulated interactions between 50 source
nodes and 50 destination nodes, both clustered in three groups (K,G = 3).
Interactions take place into 24 time intervals of unitary length (ideally one hour),
clustered into three groups too (D = 3). Nodes and time intervals labels are
sampled from multinomial distributions, whose hyperparameters (ω,ρ, δ) have
all been set equal to {1/3, 1/3, 1/3}. With these settings, we consider 27 different
Poisson parameters (λs) generating connections between nodes. The generative
model used to produce them is described by:
λkgl = s1[k] + s2[g] + s3[l], k, g, l ∈ {1, 2, 3}
where:
s1 = [0, 2, 4] s2 = [0.5, 1, 1.5] s3 = [0.5, 1, 1.5]
ans s1[k] denotes the k-th component of s1. Similarly for s2 and s3. The greedy
search algorithm we coded was able to exactly recover these initial settings, con-
verging to the true ICL of −122410. Other experiments were run with different
values inside vectors s1, s2, s3. Not surprisingly the more nuanced differences
between λs are, the more difficult it is for the algorithm to converge to the true
value of the ICL2.
2Greedy search algorithms are path dependent and they could converge to local maxima.
4.2 Real Data
The dataset we used was collected during the ACM Hypertext conference
held in Turin, June 29th - July 1st, 2009. Conference attendees volunteered to
to wear radio badges that monitored their face-to-face proximity. The dataset
represents the dynamical network of face-to-face proximity of 113 conference
attendees over about 2.5 days3. Further details can be found in Isella, Stehlé,
Barrat, Cattuto, Pinton, Van den Broeck [7]. We focused on the first conference
day, namely the twenty four hours going from 8am of June 29th to 7.59am of
June 30th. The day was partitioned in small time intervals of 20 seconds in the
original data frame. We considered 15 minutes time aggregations, thus leading
to a partition of the day made of 96 consecutive quarter-hours (U = 96 with
previous notation). A typical row of the adjacency matrix we analyzed, looks
like:
Person 1 Person 2 Time Interval (15m) Number of interactions
52 26 5 16
It means that conference attendees 52 and 26, between 9am and 9.15am have
spoken for 16× 20s ≈ 5m30s.
The greedy search algorithm converged to a final ICL of -53217.4, corresponding
to 23 clusters for nodes (people) and 3 time clusters. In Figure (1a) we show how
Time
C
lu
s
te
r
8h 10h 12h 14h 16h 18h 20h 22h
1
2
3
Time Clusters
(a) Clustered time intervals.
0
200
400
600
800
1000
1200
Time
In
te
a
c
ti
o
n
s
8h 10h 12h 14h 16h 18h 20h 22h
Aggregated number of Interactions
(b) Connections for every time interval.
Fig. 1: The aggregated connections for every time interval (1b) and time clusters found
by our model (1a) are compared.
daily quarter-hours are assigned to each cluster: the class C1 contains intervals
marked by a weaker intensity of interactions (on average), whereas intervals in-
side C3 are characterized by the highest intensity of interactions. This can either
be verified analytically by averaging estimated Poisson intensities for each one
of the three clusters or graphically by looking at Figure (1b). In this Figure we
3More informations can be found at:
http://www.sociopatterns.org/datasets/hypertext-2009-dynamic-contact-network/.
computed the total number of interactions between conference attendees for each
quarter-hour and it can clearly be seen how time intervals corresponding to the
higher number of interactions have been placed in cluster C3, those correspond-
ing to an intermediate interaction intensity, in C2 and so on. It is interesting
to remark how the model can quite closely recover times of social gathering like
the lunch break (13.00-15.00) or the “wine and cheese reception” (18.00-19.00).
A complete program of the day can be found at:
http://www.ht2009.org/program.php.
5 Conclusions
We proposed a non-stationary evolution of the latent block model (LBM) al-
lowing us to simultaneously infer the time structure of a bipartite network and
cluster the two node sets. The approach we chose consists in partitioning the
entire time horizon in fixed-length time intervals to be clustered on the basis of
the intensity of connections in each interval. We derived the complete ICL for
such a model and maximized it numerically, by means of a greedy search, for two
different networks: a simulated and a real one. The results of these two tests
highlight the capacity of the model to capture non-stationary time structures.
References
[1] A.N. Randriamanamihaga, E. Côme, L. Oukhellou and G. Govaert, Clustering the Vélib’
dynamic Origin/Destination flows using a family of Poisson mixture models, Neurocom-
puting, vol. 141, pp. 124-138, 2014.
[2] P.W.Holland, K.B. Laskey and S. Leinhardt, Stochastic blockmodels: first steps, Social
networks, vol.5, pp.109-137, 1983.
[3] R. Guigourès, M. Boullé and F. Rossi, A Triclustering Approach for Time Evolving
Graphs, in Co-clustering and Applications, IEEE 12th International Conference on Data
Mining Workshops (ICDMW 2012), pages 115-122, Brussels, Belgium, December 2012.
[4] J. Wyse, N. Friel and P. Latouche, Inferring structure in bipartite networks using the
latent block model and exact ICL, arXiv pre-print arXiv: 1404.2911, 2014.
[5] C. Biernacky, G. Celeux and G. Govaert, Assessing a mixture model for clustering with
the integrated completed likelihood, IEEE Trans. Pattern Anal. Machine Intel, vol.7, pp.
719-725, 2000.
[6] E. Côme and P. Latouche, Model selection and clustering in stochastic block models with
the exact integrated complete data likelihood, arXiv pre-print, arXiv:1303.2962, 2013.
[7] L. Isella, J. Sthelé, A. Barrat, C.Cattuto, J.F. Pinton, W. Van den Broeck, What’s in a
crowd? Analysis of face-to-face behavioral networks, Journal of Theoretical Biology, vol.
271, pp. 166-180, 2011.


Exact ICL maximization in a non-stationary time extension of the latent block model for dynamic networks

HAL-Paris1

Exact ICL maximization in a non-stationary timeextension of the latent block model for dynamicnetworksMarco Corneli, Pierre Latouche, Fabrice RossiTo cite this version:Marco Corneli, Pierre Latouche, Fabrice Rossi. Exact ICL maximization in a non-stationarytime extension of the latent block model for dynamic networks. European Symposium onArtificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Apr2015, Bruges, Belgium. pp.225-230, 2015, Proceedings of the 23-th European Symposium onArtificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2015).<hal-01163367>HAL Id: hal-01163367https://hal.archives-ouvertes.fr/hal-01163367Submitted on 12 Jun 2015HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestine´e au de´poˆt et a` la diffusion de documentsscientifiques de niveau recherche, publie´s ou non,e´manant des e´tablissements d’enseignement et derecherche franc¸ais ou e´trangers, des laboratoirespublics ou prive´s.Distributed under a Creative Commons Attribution 4.0 International LicenseExact ICL maximization in a non-stationarytime extension of the latent block model fordynamic networksMarco Corneli, Pierre Latouche and Fabrice RossiUniversite´ Paris 1 Panthe´on-Sorbonne - Laboratoire SAMM90 rue de Tolbiac, F-75634 Paris Cedex 13 - FranceAbstract. The latent block model (LBM) is a flexible probabilistic toolto describe interactions between node sets in bipartite networks, but itdoes not account for interactions of time varying intensity between nodesin unknown classes. In this paper we propose a non stationary temporalextension of the LBM that clusters simultaneously the two node sets of abipartite network and constructs classes of time intervals on which inter-actions are stationary. The number of clusters as well as the membershipto classes are obtained by maximizing the exact complete-data integratedlikelihood relying on a greedy search approach. Experiments on simulatedand real data are carried out in order to assess the proposed methodology.1 IntroductionSince the interactions between nodes of a network generally have a time varyingintensity, the network has a non trivial time structure that we aim at inferring.The approach we follow to introduce a temporal dimension, consists in partition-ing the entire time horizon, during which we observe interactions, in disjoint timeintervals, having an arbitrary fixed length. Then, we simultaneously cluster thenodes of the bipartite network and these time intervals, assuming interactionsare generated by a latent block model. A similar view is adopted by Randria-manamihaga, Coˆme and Govaert [1], nonetheless with a substantial difference:they consider time intervals whose membership is not hidden but known in ad-vance and hence exogenous, whereas in this paper we infer the membership ofeach interval by maximizing a likelihood criterion. A similar task is accom-plished by Guigoure`s, Boulle´ and Rossi [3] in a different point of view: theydo not consider fixed-length time intervals, but associate a time stamp to eachinteraction “in order to build time segments and clusters of nodes whose edgedistributions are similar and evolve in the same way over the time segments”.In order to obtain the optimal number of time and nodes clusters, we maximizethe integrated complete-data likelihood (ICL) using a greedy search in a verysimilar fashion as in Wyse, Frial and Latouche [4]. This paper is structured asfollows: in Section 2 we present the classical LBM and detail the time extensionproposed. In Section 3 we derive the ICL for this model and in Section 4 wediscuss the experiments we conducted with both simulated and real data. TheSection 5 concludes the paper.2 A non stationary latent block modelWe present here the LBM (Holland et al. [2]), as described in Wyse, Friel,Latouche (2014). Two sets of nodes are considered: A = {a1, . . . , aN} andB = {b1, . . . , bM}. Undirected links between node i from A and node j fromB, are counted by the observed variable Xij , being the component (i, j) of theN ×M adjacency matrix X = {Xij}i≤N,j≤M . Nodes in A and B are clusteredin K and G disjointed subgroups respectively:A = ∪k≤KAk, Ai ∩Aj = ∅, ∀i 6= jand similarly for B. Nodes in the same cluster in A have linking attributes of thesame nature to clusters of B. We introduce two hidden vectors c = {c1, . . . , cN}and w = {w1, . . . , wM} labeling each node’s membership:ci = k iff ai ∈ Ak, ∀k ≤ K and wj = g iff bj ∈ Bg, ∀g ≤ G.In order to introduce the temporal dimension, consider now a sequence of equallyspaced, adjacent time steps {∆u := tu − tu−1}u≤U over the interval [0, T ] and apartition C1, . . . , CD of the same interval1. We introduce furthermore a randomvector y = {yu}u≤U , such that yu = d if and only if Iu :=]tu−1, tu] ∈ Cd,∀d ≤ D.We attach to y a multinomial distribution:p(y|β, D) =∏d≤Dβ|Cd|d ,where |Cd| = #{Iu : Iu ∈ Cd}. Now we define N Iuij as the number of observedconnections between ai and bj , in the time interval Iu and we make the followingcrucial assumption:p(N Iuij |ci = k,wj = g, yu = d) follows a Poisson(∆uλkgd), (1)hence the number of interactions is conditionally distributed like a Poisson ran-dom variable with parameter depending on k, g, d (∆u is constant).Notation: In the following, for seek of simplicity, we will note:∏k,g,d:=∏k≤K∏g≤G∏d≤Dand∏ci:=∏i:ci=kand similarly for∏wjand∏yu.The adjacency matrix, noted N∆, has three dimensions (N ×M × U) and itsobserved likelihood can be computed explicitly:p(N∆|Λ, c,w,y,K,G,D) =∏k,g,d∆Skgd∏ci∏wj∏yuN Iuij !e−∆λkgdRkgdλSkgdkgd , (2)1T and U are linked by the following relation: T = ∆uU .where we noted Skgd :=∑ci∑wj∑yuN Iuij and Rkgd := |Ak||Bg||Cd| and thesubscript u was removed from ∆u to emphasize that time steps are equallyspaced for every u.Since c,w and y are not known, a multinomial factorizing probability densityp(c,w,y|Φ,K,G,D), depending on hyperparameter Φ, is introduced. The jointdistribution of labels looks finally as follows:p(c,w,y|Φ,K,G,D) =∏k≤Kω|Ak|k∏g≤Gρ|Bg|g∏d≤Dβ|Cd|d , (3)where Φ = {ω,ρ,β}.3 Exact ICL for non stationary LBM3.1 Exact ICL derivationThe integrated classification criterion (ICL) was introduced as a model selectioncriterion in the context of Gaussian mixture models by Biernacky et al. [5].Coˆme and Latouche [6] proposed an exact version of the ICL based on a Bayesianapproach for the stochastic block model and Wyse, Friel and Latouche [4] appliedthe exact ICL to select the number of clusters in a bipartite network using anLBM model. This is the approach we follow here. The quantity we focus on is thecomplete data log-likelihood, integrated with respect to the model parameters Φand Λ = {λkgd}k≤K,g≤G,d≤D:ICL = log(∫p(N∆, c,w,y,Λ,Φ|K,G,D)dΛdΦ). (4)Introducing a prior distribution ν(Λ,Φ|K,G,D) over the pair Φ,Λ and thanksto ad hoc independence assumptions, the ICL can be rewritten as follows:ICL = log (ν(N∆|c,w,y,K,G,D))+ log (ν(c,w,y|K,G,D)) . (5)The choice of prior distributions over the model parameters is crucial to have anexplicit form of the ICL.3.2 A priori distributionsWe consider the conjugate prior distributions. Thus we impose a Gamma apriori over Λ:ν(λkgd|akgd, bkgd) =bakgdkgdΓ(akgd)λakgd−1kgd e−bkgdλkgdand a factorizing Dirichlet a priori distribution to Φ:ν(Φ|K,G,D) = DirK(ω;α, . . . , α)×DirG(ρ; δ, . . . , δ)×DirD(β; γ, . . . , γ).It can be proven that the two terms in (5), reduce to:ν(N∆|c,w,y,K,G,D) =∏k,g,dbakgdkgd ∆SkgdΓ(akgd)∏ci∏wj∏yuN Iuij !(6)Γ(Skgd + akgd)[∆Rkgd + bkgd]Skgd+akgdand:ν(c,w,y|K,G,D) =Γ(αK)Γ(α)K∏k≤K Γ(|Ak|+ α)Γ(N + αK)× Γ(δG)Γ(δ)G∏g≤G Γ(|Bg|+ δ)Γ(M + δG)×Γ(γD)Γ(γ)D∏d≤D Γ(|Cd|+ γ)Γ(U + γD). (7)3.3 ICL MaximizationIn order to maximize the integrated complete likelihood (ICL) in equation (5)with respect to the six unknowns c,w,y,K,G,D, we rely on a greedy search overlabels and the number of nodes and time clusters. This approach is described inWyse, Frial and Latouche [4] for a stationary latent block model.4 Experiments4.1 Simulated dataSome experiments on simulated data have initially been conducted. Based onthe model described in Section 2, we simulated interactions between 50 sourcenodes and 50 destination nodes, both clustered in three groups (K,G = 3).Interactions take place into 24 time intervals of unitary length (ideally one hour),clustered into three groups too (D = 3). Nodes and time intervals labels aresampled from multinomial distributions, whose hyperparameters (ω,ρ, δ) haveall been set equal to {1/3, 1/3, 1/3}. With these settings, we consider 27 differentPoisson parameters (λs) generating connections between nodes. The generativemodel used to produce them is described by:λkgl = s1[k] + s2[g] + s3[l], k, g, l ∈ {1, 2, 3}where:s1 = [0, 2, 4] s2 = [0.5, 1, 1.5] s3 = [0.5, 1, 1.5]ans s1[k] denotes the k-th component of s1. Similarly for s2 and s3. The greedysearch algorithm we coded was able to exactly recover these initial settings, con-verging to the true ICL of −122410. Other experiments were run with differentvalues inside vectors s1, s2, s3. Not surprisingly the more nuanced differencesbetween λs are, the more difficult it is for the algorithm to converge to the truevalue of the ICL2.2Greedy search algorithms are path dependent and they could converge to local maxima.4.2 Real DataThe dataset we used was collected during the ACM Hypertext conferenceheld in Turin, June 29th - July 1st, 2009. Conference attendees volunteered toto wear radio badges that monitored their face-to-face proximity. The datasetrepresents the dynamical network of face-to-face proximity of 113 conferenceattendees over about 2.5 days3. Further details can be found in Isella, Stehle´,Barrat, Cattuto, Pinton, Van den Broeck [7]. We focused on the first conferenceday, namely the twenty four hours going from 8am of June 29th to 7.59am ofJune 30th. The day was partitioned in small time intervals of 20 seconds in theoriginal data frame. We considered 15 minutes time aggregations, thus leadingto a partition of the day made of 96 consecutive quarter-hours (U = 96 withprevious notation). A typical row of the adjacency matrix we analyzed, lookslike:Person 1 Person 2 Time Interval (15m) Number of interactions52 26 5 16It means that conference attendees 52 and 26, between 9am and 9.15am havespoken for 16× 20s ≈ 5m30s.The greedy search algorithm converged to a final ICL of -53217.4, correspondingto 23 clusters for nodes (people) and 3 time clusters. In Figure (1a) we show howTimeCluster8h 10h 12h 14h 16h 18h 20h 22h123Time Clusters(a) Clustered time intervals.020040060080010001200TimeInteactions8h 10h 12h 14h 16h 18h 20h 22hAggregated number of Interactions(b) Connections for every time interval.Fig. 1: The aggregated connections for every time interval (1b) and time clusters foundby our model (1a) are compared.daily quarter-hours are assigned to each cluster: the class C1 contains intervalsmarked by a weaker intensity of interactions (on average), whereas intervals in-side C3 are characterized by the highest intensity of interactions. This can eitherbe verified analytically by averaging estimated Poisson intensities for each oneof the three clusters or graphically by looking at Figure (1b). In this Figure we3More informations can be found at:http://www.sociopatterns.org/datasets/hypertext-2009-dynamic-contact-network/.computed the total number of interactions between conference attendees for eachquarter-hour and it can clearly be seen how time intervals corresponding to thehigher number of interactions have been placed in cluster C3, those correspond-ing to an intermediate interaction intensity, in C2 and so on. It is interestingto remark how the model can quite closely recover times of social gathering likethe lunch break (13.00-15.00) or the “wine and cheese reception” (18.00-19.00).A complete program of the day can be found at:http://www.ht2009.org/program.php.5 ConclusionsWe proposed a non-stationary evolution of the latent block model (LBM) al-lowing us to simultaneously infer the time structure of a bipartite network andcluster the two node sets. The approach we chose consists in partitioning theentire time horizon in fixed-length time intervals to be clustered on the basis ofthe intensity of connections in each interval. We derived the complete ICL forsuch a model and maximized it numerically, by means of a greedy search, for twodifferent networks: a simulated and a real one. The results of these two testshighlight the capacity of the model to capture non-stationary time structures.References[1] A.N. Randriamanamihaga, E. Coˆme, L. Oukhellou and G. Govaert, Clustering the Ve´lib’dynamic Origin/Destination flows using a family of Poisson mixture models, Neurocom-puting, vol. 141, pp. 124-138, 2014.[2] P.W.Holland, K.B. Laskey and S. Leinhardt, Stochastic blockmodels: first steps, Socialnetworks, vol.5, pp.109-137, 1983.[3] R. Guigoure`s, M. Boulle´ and F. Rossi, A Triclustering Approach for Time EvolvingGraphs, in Co-clustering and Applications, IEEE 12th International Conference on DataMining Workshops (ICDMW 2012), pages 115-122, Brussels, Belgium, December 2012.[4] J. Wyse, N. Friel and P. Latouche, Inferring structure in bipartite networks using thelatent block model and exact ICL, arXiv pre-print arXiv: 1404.2911, 2014.[5] C. Biernacky, G. Celeux and G. Govaert, Assessing a mixture model for clustering withthe integrated completed likelihood, IEEE Trans. Pattern Anal. Machine Intel, vol.7, pp.719-725, 2000.[6] E. Coˆme and P. Latouche, Model selection and clustering in stochastic block models withthe exact integrated complete data likelihood, arXiv pre-print, arXiv:1303.2962, 2013.[7] L. Isella, J. Sthele´, A. Barrat, C.Cattuto, J.F. Pinton, W. Van den Broeck, What’s in acrowd? Analysis of face-to-face behavioral networks, Journal of Theoretical Biology, vol.271, pp. 166-180, 2011.

Exact ICL maximization in a non-stationary time extension of the latent block model for dynamic networks

Abstract

Similar works

Full text

Available Versions

Archive Ouverte en Sciences de l'Information et de la Communication

HAL-Paris1