Bipartite data is common in data engineering and brings unique challenges, particularly when it comes to clustering tasks that impose strong structural assumptions. This work presents an unsupervised method for assessing similarity in bipartite data. The method is based on regular equivalence in graphs and uses spectral properties of a bipartite adjacency matrix to estimate similarity in both dimensions. The method is reflexive in that similarity in one dimension informs similarity in the other. The method also uses local graph transitivities, a contribution governed by its only free parameter. Reflexive regular equivalence can be used to validate assumptions of co-similarity, which are required but often untested in co-clustering analyses. The method is robust to noise and asymmetric data, making it particularly suited for cluster analysis and recommendation in data of unknown structure

Gerow, Aaron

Matwin, Stan

Shi, Feng

Zhou, Mingyang

Goldsmiths Research Online

Reflexive Regular Equivalence for Bipartite DataAaron Gerow1∗, Mingyang Zhou2, Stan Matwin1, and Feng Shi31 Faculty of Computer Science, Dalhousie University, Halifax, NS, Canada2 Department of Computer Science, University of Chicago, Chicago, USA3 University of North Carolina, Chapel Hill, NS, USA*Corresponding author: gerow@dal.caAbstract. Bipartite data is common in data engineering and bringsunique challenges, particularly when it comes to clustering tasks that im-pose strong structural assumptions. This work presents an unsupervisedmethod for assessing similarity in bipartite data. The method is based onregular equivalence in graphs and uses spectral properties of a bipartiteadjacency matrix to estimate similarity in both dimensions. The methodis reflexive in that similarity in one dimension informs similarity in theother. The method also uses local graph transitivities, a contributiongoverned by its only free parameter. Reflexive regular equivalence canbe used to validate assumptions of co-similarity, which are required butoften untested in co-clustering analyses. The method is robust to noiseand asymmetric data, making it particularly suited for cluster analysisand recommendation in data of unknown structure.4In bipartite data, co-similarity is the notion that similarity in one dimensionis matched by similarity in some other dimension. Such data occurs in a manyareas: text mining, gene expression networks, consumer co-purchasing data andsocial affiliation. In bipartite analyses, co-clustering is an increasingly promi-nent technique in a range of applications [11], but has strong co-similarity as-sumptions. One example of co-similar structure is in text analysis where similarwords appear in similar documents, where there is assumed to be a permutationof the word-document co-occurrence matrix that exposes co-similarity amongwords and documents [4]. The work here describes a way to assess co-similarityusing regular equivalence [8] with a reflexive conception of similarity that ac-commodates nodes’ (data-points) local structures. This assessment is a kind ofpre-condition for co-clustering: if there is little co-similarity, co-clustering willyield a poor clustering solution. Assessing co-similarity will produce similarityin one dimension to expose potential clustering without requiring it across di-mensions. This is particularly useful for asymmetric data when a non-clustereddimension informs, but does not reciprocate clustering in the other. Whereasas co-clustering finds clusters across dimensions, our method provides a decou-pled solution in each mode. Reflexive regular equivalence is able to quantify howmuch one dimension informs similarity in the other. Additionally, our resultsshow that by incorporating local structures, it can better overcome noise andaccommodate asymmetry.4 An extended preprint of this paper is available at arxiv.org/abs/1702.04956.2Background Measuring co-similarity amounts to calculating similarity in two di-mensions. In a graph setting, this can be done with bipartite generalizationsof regular equivalence [1,5], which measure similarity based on the similar-ity of neighboring nodes. Bipartite regular equivalence makes sense of within-dimension regularity and between-dimension structure. As we will see, allowingthe inter-dimensional structure to be “reflexive”, as defined below, greatly helpsassess co-similarity. Specifically, we conceive of bipartite data as a two-modenetwork. In simple networks, vertex similarity can be measured using pairwisemetrics like the Jaccard index, the cosine of a pair’s connectivity patterns, theircorrelation or simply the overlap of neighbors. Recent efforts have sought toaccount for structure beyond nodes’ immediate neighbors. Regular equivalenceproposes that nodes are similar to the extent their neighbors are similar [8], amethod common in social network analysis [7].ABCDE12345ABCDE12345Fig. 1. With the same network (left; a) a biclustering solution yields partitions thatgroup vertices in both modes. On the right (b) is a hypothetical grouping based on sim-ilarities produced by reflexive regular equivalence; there may not be a mapping betweenclusters in each mode and groupings are exposed by similarity, not by partitioning.In a social setting, co-similarity was formalized by Ronald Breiger in 1974where similar people were found to participate in similar groups [2]. With the ad-jacency structure, A, of people×groups, similarity can be measured as Speople =AAT and Sgroups = ATA. Such multi-modal conceptions of social structures arenow common in models of social search and recommendation [6,10]. Our methodextends regular equivalence and Breiger’s notion of co-similarity. Crucially, itconsiders structural equivalence by adding contributions from local transitivity– two nodes are similar if their neighbors are similar within and across dimen-sions. Our goal is not to find partitions that optimally group nodes across di-mensions, but to measure similarity in each mode of the data using informationfrom the other. Only by extension, may any clustered structure be revealed inthe data. Figure 1 depicts a bipartite network and hypothetical co-clusteringsolution (a) and a potential solution based on reflexive regular equivalence (b).Though different from our method, co-clustering algorithms have much in com-mon with reflexive regular equivalence. Co-clustering algorithms yield partitionsacross modes that group self-similar rows with self-similar columns, and assumeco-similarity just as clustering assumes similarity. Reflexive equivalence assess3co-similarity, which may justify a co-clustering strategy. For examples of co-clustering algorithms that implement related notions of similarity see [3,9,12].Method Our reflexive equivalence method is an unsupervised approach to mea-suring vertex similarity in a bipartite network. This notion is equivalent to apairwise similarity metric that operates on row- and column-vectors of the adja-cency structure in bimodal data. Bipartite networks are equivalent to what arenormally treated as rows and columns in a matrix. Our method iterates betweeneach mode incorporating information from the other. There is no restriction tobinary-valued data: edges may be weighted.Let G be a bipartite graph with two sets of nodes, V and V ′ and A be theadjacency matrix in which Aij > 0 represents how strongly i ∈ V is connectedto j ∈ V ′. Let S be a square |V | matrix where entry Sij is the similarity betweeni and j in V , and S′ be a square |V ′| matrix where S′ij is the similarity betweeni and j in V ′. Assuming the similarity of i and j in one mode is informed bysimilarity between neighbors of i and any neighbor of j in the other mode, thenSij =∑k∑lAikAjlS′kl, (1)S′ij =∑k∑lAkiAljSkl, (2)This provides a procedure for inferring similar nodes in the adjacency matrix.Starting from randomly initialized S and S′, Eqs. 1 and 2 can be applied itera-tively until convergence of ‖S‖F and ‖S′‖F. This formulation, however, treats allneighbors equally, regardless of their structural importance to the pair of nodesin question. Thus, we weight similarity between common neighbors more thannon-common neighbors. Denoting neighbors of i and j by Γi and Γj respectively,we define this similarity as follows:Sij = (1− α)[∑k∈(Γi−Γi∩Γj)∑l∈ΓjS′kl +∑k∈Γi∩Γj∑l∈(Γj−Γi∩Γj)S′kl] +∑k∈Γi∩Γj∑l∈Γi∩ΓjS′kl,S′ij = (1− α)[∑k∈(Γi−Γi∩Γj)∑l∈ΓjSkl +∑k∈Γi∩Γj∑l∈(Γj−Γi∩Γj)Skl] +∑k∈Γi∩Γj∑l∈Γi∩ΓjSkl.This combines structural equivalence, regular equivalence and reflexivity intoa single model. A parameter α balances the contribution of non-common andcommon neighbors. Rearranging the terms, the equations can be rewritten asSij = (1− α)∑k∑lAikAjlS′kl + α∑k∑lAikAjkAilAjlS′kl, (3)S′ij = (1− α)∑k∑lAkiAljSkl + α∑k∑lAkiAkjAliAljSkl, (4)or in matrix form:S = (1− α)AS′AT + α(A⊗AT) · S′ · (A⊗AT)T, (5)S′ = (1− α)ATSA+ α(AT ⊗A) · S · (AT ⊗A)T, (6)4where (·) is the conventional inner product defined in tensor algebra. Eqs. 5and 6 compute reflexivity between the two dimensions and the effect of localstructure is controlled by α. When α = 0, the method is a bipartite form ofregular equivalence. As α increases, similarity between common neighbors playsa larger role. A is normalized prior to applying the algorithm, and S and S′ arenormalized after every iteration by their L1-, L2- or L∞-vector norm.Results We evaluated the method on data with known structure. A set of semi-random n×m versions of A were generated with diagonal blocks of random sizes:the resulting similarity matrices should have diagonal blocks proportional to therow- and column-wise block sizes in A. The test is to run the algorithm onrandomly permuted data, Aˆ, after which if we apply the original ordering of Ato Sˆ and Sˆ′. The solution is then assessed asµ =12‖S − Sˆ‖F 1|S| +12‖S′ − Sˆ′‖F 1|S′| . (7)Here, µ is simply the mean difference between results on A and Aˆ, and in theperfect case will be 0. We compare reflexive similarity to three pairwise metricsoperating on the rows and columns of A. The results show that reflexive variantsperform marginally better (Figure 1b); α had no effect, which is expected givenall node interactions are purely local.Ref. maxRef. L1Ref L2CosineJaccardCorrelation0.0000.0020.0040.0060.008µPerformance on AˆFig. 2. (a; left) Similarity of Aˆ should produce results similar to those computed forA. (b; right) Performance (µ; lower is better) on Aˆ using variants of reflexive similaritymethod. Error-bars are ±2 s.e. of the mean over ten versions of A.Robustness to Noise Because real-world data is often noisy, we evaluated themethod with noise added to A of the form N (0, σ) to produce A˜. Note thatadding noise breaks the element-wise symmetry of A but the underlying block-structure remains symmetric. Figure 3 shows results with respect to noise, nor-malization variant and α. With little noise, the L∞-norm variant outperformsother methods when α > 0.5. As noise is increased, pairwise metrics performworse, as do the reflexive methods with α = 0. This confirms that our methodis well-suited to finding co-similarity structure in both dimensions of noisy datawhere the L∞ variant with α = 1 performs best.50.0000.0020.0040.006µσ=0.1Ref. MaxRef. L1Ref. L2CosineJaccardCorrelationσ=0.2 σ=0.40.0 0.2 0.4 0.6 0.8 1.0α0.0000.0020.0040.006µσ=0.60.0 0.2 0.4 0.6 0.8 1.0ασ=0.80.0 0.2 0.4 0.6 0.8 1.0ασ=1.0Performance on A˜Fig. 3. Performance (µ; lower is better) with respect to noise, σ, in A˜ for differentvalues of α. Error-bands are ±2 s.e. of the mean over ten versions of A.Unbalanced Co-similarity Because clusters exposed by similarity in S and S′are not coupled as they are in co-clustering tasks, reflexive similarity is ableto find unbalanced co-similarity structure. With this kind of data, the reflex-ive method should produce similarity matrices with different structure. For thistask, another set of semi-random versions of A were generated with random un-balanced block structure, which were then randomly permuted to get Aˆ. Resultson Aˆ were compared to results from A. Figure 4a shows the results on variantsof reflexive regular equivalence and the pairwise metrics. Results show that allvariants of reflexive similarity outperform the pairwise metrics, but that neithernormalization choice nor α make significant difference.0.0 0.2 0.4 0.6 0.8 1.0α0.0010.0030.005µRef. MaxRef. L1Ref. L2CosineJaccardCorrelationPerformance on Aˆ0.0000.0020.004µσ=0.1Ref. MaxRef. L1Ref. L2CosineJaccardCorrelationσ=0.2 σ=0.40.0 0.2 0.4 0.6 0.8 1.0α0.0000.0020.004µσ=0.60.0 0.2 0.4 0.6 0.8 1.0ασ=0.80.0 0.2 0.4 0.6 0.8 1.0ασ=1.0Performance on A˜Fig. 4. (a; left) Performance on ten versions of Aˆ with unbalanced structure and, (b;left) with varying amounts of noise.6The parameter α is crucial to our method’s ability to overcome noise insymmetric data. The same task was run on asymmetric data with added noise.Figure 4b shows performance with different noise levels. The results show thatafter adding a small amount of noise, the reflexive variants significantly outper-form pairwise metrics. In particular, for low levels of noise, the L∞-norm variantperforms best, but for higher levels (σ > 0.2) the L1- and L2-norms are better.In all variants, higher values of α yield better performance. This suggests localstructure is useful even when co-similarity is asymmetric. Overall, the evalua-tions show that when the structure of A is known, reflexive similarity is able toleverage inter-dimensional similarity in noisy and asymmetric data to providebetter results than methods restricted to one dimension.Discussion Co-clustering analysis often employs trial-and-error when assessingsimilarity. There are two problems with this: a priori estimates of the numberof clusters can be hard to make, and clusters may not be coupled across dimen-sion. Reflexive regular equivalence offers a way to attenuate inter-dimensionalstructure and local transitivities in similarity calculations. The results show thisis particularly important in noisy data. The method also offers a way to vali-date co-similarity assumptions: if the same permutation for S and S′ exposesblock structure in each, then A is co-similar. By varying α, one can enhance orreduce the contribution of local equivalence, essentially backing off to spectralsimilarity. In this way, our method offers a way to measure co-similarity acrossdimensions, helping confirm assumptions of co-similarity.References1. S. P. Borgatti. 2-mode concepts in social network analysis. Encyclopedia of com-plexity and system science, 6, 2009.2. R. L. Breiger. The duality of persons and groups. Social forces, 53:181–190, 1974.3. V. Codocedo and A. Napoli. Lattice-based biclustering using partition patternstructures. In ECAI 14’, pages 213–218, 2014.4. I. S. Dhillon. Co-clustering documents and words using bipartite spectral graphpartitioning. In Proc. SIGKDD 01’, pages 269–274, 2001.5. P. Doreian, V. Batagelj, and A. Ferligoj. Generalized blockmodeling of two-modenetwork data. Social networks, 26(1):29–53, 2004.6. A. Gerow, B. Lou, E. Duede, and J. Evans. Proposing ties in a dense hypergraphof academics. In Proc. of Social Informatics 2015, pages 209–226. Springer, 2015.7. D. Gnatyshak, D. I. Ignatov, A. Semenov, and J. Poelmans. Gaining insight insocial networks with biclustering and triclustering. LNBIP, 128:162–171, 2012.8. E. A. Leicht, P. Holme, and M. E. Newman. Vertex similarity in networks. PhysicalReview E, 73(2):026120, 2006.9. R. G. Pensa, D. Ienco, and R. Meo. Hierarchical co-clustering: off-line and incre-mental approaches. Data Mining and Knowledge Discovery, 28(1):31–64, 2014.10. F. Shi, J. G. Foster, and J. A. Evans. Weaving the fabric of science: Dynamicnetwork models of science’s unfolding structure. Social Networks, 43:73–85, 2015.11. A. Tanay, R. Sharan, and R. Shamir. Biclustering algorithms: A survey. Handbookof computational molecular biology, 9(1-20):122–124, 2005.12. L. Teng and K. Tan. Finding combinatorial histone code by semi-supervised bi-clustering. BMC Genomics, 13(1):301, 2012.

Reflexive Regular Equivalence in Bipartite Data

https://research.gold.ac.uk/22720/1/gerow-cai17-manuscript.pdf

Reflexive Regular Equivalence in Bipartite Data

Abstract

Similar works

Full text

Available Versions

Goldsmiths Research Online