The focus of this paper is on multitask learning over adaptive networks where different clusters of nodes have different objectives. We propose an adaptive regularized diffusion strategy using Gaussian kernel regularization to enable the agents to learn about the objectives of their neighbors and to ignore misleading information. In this way, the nodes will be able to meet their objectives more accurately and improve the performance of the network. Simulation results are provided to illustrate the performance of the proposed adaptive regularization procedure in comparison with other implementations

Monajemi, Sadaf

Ong, Sim-Heng

Sanei, Saeid

Sayed, Ali H.

Infoscience - École polytechnique fédérale de Lausanne

2015 IEEE INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING, SEPT. 17–20, 2015, BOSTON, USAADAPTIVE REGULARIZED DIFFUSION ADAPTATION OVER MULTITASK NETWORKSSadaf Monajemi† Saeid Sanei∗ Sim-Heng Ong‡ Ali H. Sayed§† NUS Graduate School for Integrative Sciences and Engineering, NUS, Singapore∗ Faculty of Engineering and Physical Sciences, University of Surrey, Guildford, UK‡ Department of Electrical and Computer Engineering, NUS, Singapore§ Department of Electrical Engineering, University of California, Los AngelesABSTRACTThe focus of this paper is on multitask learning over adaptivenetworks where different clusters of nodes have different ob-jectives. We propose an adaptive regularized diffusion strat-egy using Gaussian kernel regularization to enable the agentsto learn about the objectives of their neighbors and to ignoremisleading information. In this way, the nodes will be ableto meet their objectives more accurately and improve the per-formance of the network. Simulation results are provided toillustrate the performance of the proposed adaptive regular-ization procedure in comparison with other implementations.Index Terms— Distributed optimization, adaptive com-bination and regularization, diffusion LMS.1. INTRODUCTIONDistributed optimization and learning over networks is an at-tractive research area with several applications from signalprocessing and optimization to modeling of biological and so-cial networks [1–4]. Several strategies have been proposed inthe literature for distributed processing over networks such asincremental strategies [5], consensus strategies [6, 7] and dif-fusion strategies [1, 2, 8]. It has been shown that among thesestrategies, the diffusion algorithm is robust, scalable, and ca-pable of real-time adaptation and learning. Diffusion strate-gies also have superior performance and stability compared toconsensus methods [1,2,9]. We therefore focus on the imple-mentation of diffusion strategies in this article. In particular,we examine networks where different clusters of agents maybe interested in different objectives [10–15]. In this case, it isimportant to develop algorithms that enable the agents to con-tinuously learn which of their neighbors belong to the samecluster and which ones are from different clusters.References [13, 15] study the important case of two ob-jectives where agents receive data from one of two possiblemodels. Reference [16] considers multiple clusters under theassumption that the objectives of adjacent clusters are relatedThe work of A. H. Sayed was supported in part by NSF grants CCF-1011918 and ECCS-1407712.to each other and that agents are aware of their clusters. Thereare also variations of multi-task networks where agents dealwith the estimation of different types of parameters [17]; oneof the parameters is common to all agents and the second pa-rameter can vary across agents.In this paper we consider a multitask network consistingof several connected clusters with different objectives. We donot assume that the agents have access to any prior cluster-ing information. In particular, the agents do not know whichclusters they belong to. They also do not know the clustersof their neighbors. Moreover, we do not assume prior knowl-edge about how the objectives of the clusters are related toeach other. Therefore, the proposed method is able to handlesituations where there are different objectives in the networkwithout interference among the clusters. For this purpose, wepropose a multitask learning method that employs Gaussiankernel regularization. In this method, both the combinationweights and the regularization coefﬁcients are learned adap-tively and continuously. In this way, the agents are able tocooperate only with neighbors that share the same objective.2. MODELING OF MULTITASK NETWORKSIn this paper, we use plain letters to denote scalars, boldfacelowercase letters to denote vectors, and boldface uppercaseletters to denote matrices. Furthermore, Nk represents neigh-bors of node k, including k.2.1. Network modelWe adopt a multitask problem formulation similar to the onestudied in [11,16] with some variation in the cost formulation,as explained further ahead. We consider a connected networkconsisting ofN nodes. Each node k wants to estimate anM×1 unknown vector ωok from collected measurements. Eachnode k has access to a scalar measurement dk(i) and an M ×1 regression vector xk(i) at every time instant i ≥ 0. Thedata at each node is assumed to be related to the unknown978-1-4673-7454-5/15/$31.00 c©2015 IEEEFig. 1. An example of a multitask network consisting of N =15 nodes and Q = 3 clusters. The solid lines are the linksbetween the nodes of the same cluster and the dashed linesrepresent connections between nodes in different clusters.parameter vector ωok via a linear regression model:dk(i) = xTk (i)ωok + nk(i), (1)where nk(i) is the measurement noise at node k and time in-stant i.We assume that the network consists of Q different clus-ters and we write Cq to represent the set of nodes in cluster q.Each cluster is a collection of nodes that are interested in thesame parameter vector:ωok = ωoCq , for all k ∈ Cq. (2)Nodes of different clusters can be connected to each other butnodes do not have prior information about the clusters thattheir neighbors belong to. This means that during the initialstages of adaptation, nodes do not know whether their neigh-bors are following the same objective as them. An exampleof such a network is shown in Figure 1. In this network, thetotal number of agents is N = 15 and there are Q = 3 clus-ters. The solid lines represent links between nodes of the samecluster while the dashed lines represent connections betweennodes in different clusters. In general, processing data fromneighbors without considering their clusters can lead to ad-verse performance. In the next section we formulate an opti-mization problem for such a network and propose an adaptivemethod to solve this problem.3. PROBLEM FORMULATIONIn order to solve the estimation problem over the multitasknetwork, we assign a local cost function, Jk(ωC(k)), to eachnode k where C(k) represents the cluster that node k belongsto:Jk(ωC(k)) = E {|dk(i)− xTk (i)ωC(k)|2}. (3)The nodes that belong to the same cluster have a mutual in-terest in estimating the same parameter vector. Therefore, inorder to encourage this type of cooperation, we can use an ap-propriate regularization to promote similarities among agentswith similar objectives. There are several possible regulariza-tion terms that can be used for this particular purpose. Here,following [11, 16], we consider the squared Euclidean dis-tance as a similarity regularizer:Δ(ωC(k) − ωC())  ‖ωC(k) − ωC()‖2, (4)where ‖ωC(k) − ωC()‖ is the Euclidean distance between theparameter vectors for nodes  and k. Combining equations (3)and (4) results in the regularized global cost function:Jglob(ωC(1), ...,ωC(Q)) =N∑k=1E {|dk(i)− xTk (i)ωC(k)|2}+ βN∑k=1∑∈Nkγk ‖ωC(k) − ωC()‖2, (5)where β ≥ 0 is a strength parameter. Moreover, the weightsγk ≥ 0 adjust the role of the regularization term between thetwo nodes k and  and they satisfy the conditions:N∑=1γk = 1, and γk = 0 if  /∈ Nk (6)It is important to note that similar to the case studied in[16], the role of adding the second term on the right hand sideof (5) is to promote similarity among nodes with similar ob-jectives. However, unlike [16], here the nodes have no priorinformation about the objectives of their neighbors. There-fore, the internal summation in this term is over all the neigh-bors of node k. Consequently, the regularization weights γkmust be adjusted such that they allocate more weight to neigh-bors with similar objectives while giving less weight to theneighbors from different clusters.In the following section, we propose an adaptive diffusionstrategy that enables agents to learn the regularization coefﬁ-cients, as well as the combination weights, in such a way thatthey end up assigning relatively larger weights to the neigh-bors with similar objectives.4. DIFFUSION ADAPTATION STRATEGYWithout loss of generality, we employ the adapt-then-combine(ATC) diffusion algorithm to solve the optimization problemin equation (5) due to its superior performance even in com-parison to consensus strategies [1]. The result of applyingATC to equation (5) leads to the following distributed strat-egy:ψk(i) = ωk(i− 1) + μkxk(i)[dk(i)− xTk (i)ωk(i− 1)]+μkβ∑∈Nkγk(i) (ω(i− 1)− ωk(i− 1)),(7)ωk(i) =∑∈Nkak(i)ψ(i), (8)where μk > 0 is the step-size parameter used by node k, andthe coefﬁcients ak(i) are non-negative entries of an N × Ncombination matrix Ai at time instant i. It is important tonote that since Ai has to be left-stochastic, we have:ATi 1 = 1, ak(i) = 0 if  /∈ Nk, (9)where 1 is an N × 1 vector with all entries equal to one. Ob-serve in (7) that we are allowing the regularization coefﬁcientsγk(i) to vary with time because they will be adapted as well.4.1. Selection of Regularization WeightsAs indicated previously, selection of the regularization coef-ﬁcients γk in equation (5) has a signiﬁcant impact on theperformance of the network. These coefﬁcients must be esti-mated in an adaptive manner so that agents can be clusteredmore accurately. Now since the nodes do not have prior infor-mation about the clusters of their neighbors, the weights mustbe estimated in such a way that they allocate higher weightto neighbors sharing similar objectives. In other words, theregularization penalty term in equation (5) must be omittedwhen the objectives of nodes k and  are not similar. There-fore, the regularization weights γk(i) must be inversely, butnot necessarily linearly, proportional to the distance betweenthe objectives of two nodes, i.e., ‖ωok − ωo‖2.Among several possible adaptive regularization terms thathave been used in the literature, we select γk(i) proportionalto exp(−‖ωok − ωo‖2/h) [18, 19]. It can be seen that thismethod is analogous to Gaussian kernel regularization withsufﬁcient ﬂexibility when h > 0:γk(i) =exp(−‖ωok − ωo‖2/h)∑n∈Nk exp(−‖ωok − ωon‖2/h)(10)It is still not feasible to evaluate the regularization weightsbased on equation (10). This is because the nodes do not knowthe true objectives ωok and ωo . Therefore, we replace theseobjectives by the best available estimates at each time instantand reformulate the weights as:γk(i) ≈{ exp(−‖ωk(i−1)−ω(i−1)‖2/h)∑n∈Nk exp(−‖ωk(i−1)−ωn(i−1)‖2/h),  ∈ Nk0, otherwise(11)Algorithm 1 Regularized ATC Diffusion LMS for clusteredmultitask networks with adaptive regularization weights.Require: ωk,0 = ψk,0 = 0, γk(0) = 0 for all kfor i ≥ 1 doγk(i) ≈{ exp(−‖ωk(i−1)−ω(i−1)‖2/h)∑n∈Nk exp(−‖ωk(i−1)−ωn(i−1)‖2/h),  ∈ Nk0, otherwiseψk(i) = ωk(i− 1) + μkxk(i)[dk(i)− xTk (i)ωk(i− 1)]+μkβ∑∈Nkγk(i) (ω(i− 1)− ωk(i− 1))ak(i) ≈{ ‖ωk(i−1)−ψ(i)‖−2∑n∈Nk ‖ωk(i−1)−ψn(i)‖−2,  ∈ Nk0, otherwiseωk(i) =∑∈Nk ak(i)ψ(i)end for4.2. Estimation of Combination WeightsIn order to estimate the combination weights, we follow thesame procedure developed in [11, 13] and introduce the in-stantaneous MSD of the network at time i:MSD(i)  1NN∑k=1E‖ω˜k(i)‖2, (12)where ω˜k(i)  ωok − ωk(i) is the error vector at node k attime instant i. Then, the combination coefﬁcients ak(i) canbe obtained by solving the optimization problem:minAiMSD(i) (13)subject to (9). It was shown in [13] that the optimal solutioncan be approximated by:ak(i) ≈{ ‖ωk(i−1)−ψ(i)‖−2∑n∈Nk ‖ωk(i−1)−ψn(i)‖−2,  ∈ Nk0, otherwise(14)One important conclusion derived from equation (14) is thatthe combination coefﬁcient ak(i) is inversely proportional tothe distance between the estimate of node k and the intermedi-ate estimateψ(i) of node . In other words, this combinationapproach enables the agents to continuously learn about theobjective of their neighbors so that they can distinguish be-tween useful and misleading information. This method helpsthe nodes of multitask networks to acquire an effective coop-erative strategy. Estimating both combination and regulariza-tion weights in this manner results in an adaptive multitaskdiffusion algorithm, which helps the agents beneﬁt from co-operation by ignoring misleading information.Fig. 2. Performance of the network for different learningstrategies. The MSD curves are shown for different diffusionstrategies.5. SIMULATION RESULTSWe present a numerical example in order to illustrate the be-havior of the proposed algorithm over a multitask network.In this example, we consider the network of 15 nodes, i.e.,N = 15, shown in Figure 1. As can be seen in the ﬁgure,the network is divided into 3 clusters: C1 = {1, 2, 3, 4, 5},C2 = {6, 7, 8, 9, 10, 11, 12}, and C3 = {13, 14, 15}. The pa-rameter vector for each cluster is two-dimensional and cho-sen as ωoC1 = [0.5,−0.4]T , ωoC2 = [−1, 3]T , and ωoC3 =[5.19, 2.81]T . Moreover, the regression input signals xk(i)are 2 × 1 zero-mean Guassian random vectors with covari-ance matricesRx,k = σ2x,kIM where σ2x,k is shown in Figure3. Additionally, the measurement noises nk(i) are zero-meanrandom variables with a Guassian distribution and their vari-ances σ2n,k are also shown in Figure 3.The regularization weights γk(i) and the combinationweights ak(i) are estimated using (11) and (14), respectively.The results of different strategies have been averaged over 100Monte-Carlo runs for μ = 0.1, as shown in Figure 2. Theperformance of the proposed method is compared to otherlearning methods: (a) the non-cooperative algorithm, whereeach node of the network attempts to estimate the requiredparameter vector without using any information from othernodes. In this case, a cluster can be assigned to each nodeand we set: Ai = IN , β = 0; (b) the regularized multi-task algorithm with uniform regularization weights, where thenodes employ a diffusion strategy with uniform regularizationweights: γk = |Nk|−1; (c) the regularized multitask algo-rithm with uniform combination and regularization weights,where both the regularization weights γk(i) and combina-tion weights ak(i) are set equal to |Nk|−1; (d) the regular-ized multitask algorithm with β = 0, where the nodes of thenetwork solve the multitask problem cooperatively and withadaptive weights, but the regularization term is omitted.Comparing the ﬁve learning strategies, we can see that theproposed regularized method with adaptive weights enablesFig. 3. Variances of the network input and noise for each nodeof the network.the nodes of the network to achieve superior performance. Onthe other hand, in cases where the regularization and combi-nation weights are uniform and have not been estimated adap-tively, the nodes are unable to distinguish misleading informa-tion from beneﬁcial data. Therefore, most of the nodes are notsuccessful in estimating their own objectives and their perfor-mance is even worse than the case where there is no coopera-tion. In fact, this result clearly reveals the challenge of coop-eration in multitask networks since it shows that in the caseswhere the nodes allocate equal weights to all of their neigh-bors without considering their objectives, the performance ofthe network may be even worse than the case where the nodesdo not cooperate at all. Finally, comparing the performance ofthe proposed method for two regularization factors of β = 0and β = 1 shows that adding the regularization term can bebeneﬁcial.6. CONCLUSIONIn this paper, we have considered the multitask estimationproblem where the nodes of the network have different objec-tives. We developed a regularized adaptive learning algorithminvolving Gaussian kernels, which guides the nodes to learnthe beneﬁcial information and ignore the misleading data re-ceived over time.7. REFERENCES[1] A. H. Sayed, “Adaptive networks,” Proceedings of theIEEE, vol. 102, no. 4, pp. 460–497, April 2014.[2] A. H. Sayed, “Adaptation, learning, and optimizationover networks,” Foundations and Trends in MachineLearning, vol. 7, no. 4-5, pp. 311–801, 2014.[3] S. Barbarossa, S. Sardellitti, and P. Di Lorenzo, “Dis-tributed detection and estimation in wireless sensor net-works,” inAcademic Press Library in Signal Processing,R. Chellapa and S. Theodoridis, Eds., Elsevier, 2014,vol. 2, pp. 329–408.[4] S. Monajemi, S. Sanei, and S.H. Ong, “Advances inbacteria motility modelling via diffusion adaptation,” inProc. EUSIPCO, pp. 2335–2339.[5] A. Nedic and D. P. Bertsekas, “Incremental subgradi-ent methods for nondifferentiable optimization,” SIAMJournal on Optimization, vol. 12, no. 1, pp. 109–138,2001.[6] L. Xiao and S. Boyd, “Fast linear iterations for dis-tributed averaging,” Systems & Control Letters, vol. 53,no. 1, pp. 65–78, 2004.[7] A. Nedic and A. Ozdaglar, “Distributed subgradientmethods for multi-agent optimization,” IEEE Transac-tions on Automatic Control, vol. 54, no. 1, pp. 48–61,2009.[8] J. Chen and A. H. Sayed, “Distributed Pareto optimiza-tion via diffusion strategies,” IEEE Journal of SelectedTopics in Signal Processing, vol. 7, no. 2, pp. 205–220,April 2013.[9] S-Y. Tu and A. H. Sayed, “Diffusion strategies outper-form consensus strategies for distributed estimation overadaptive networks,” IEEE Trans. Signal Process., vol.60, no. 12, pp. 6217–6234, Dec. 2012.[10] C-K. Yu, M. van der Schaar, and A. H. Sayed, “Clusterformation over adaptive networks with selﬁsh agents,”in Proc. EUSIPCO, Marrakech, Morocco, September2013, pp. 1–5.[11] J. Chen, C. Richard, and A. H. Sayed, “Diffusion LMSover multitask networks,” IEEE Trans. Signal Process.,vol. 63, no. 11, pp. 2733–2748, June 2015.[12] J. Chen and C. Richard, “Performance analysis of dif-fusion LMS in multitask networks,” in Proc. IEEEInternational Workshop on Computational Advancesin Multi-Sensor Adaptive Processing (CAMSAP), SaintMartin, December 2013, pp. 137–140.[13] X. Zhao and A. H. Sayed, “Clustering via diffu-sion adaptation over networks,” in Proc. InternationalWorkshop on Cognitive Information Processing (CIP),Parador de Baiona, Spain, May 2012, pp. 1–6.[14] S-Y. Tu and A. H. Sayed, “Distributed decision-makingover adaptive networks,” IEEE Trans. Signal Process.,vol. 62, no. 5, pp. 1054–1069, March 2014.[15] X. Zhao and A. H. Sayed, “Distributed clustering andlearning over networks,” IEEE Trans. Signal Process.,vol. 63, no. 13, pp. 3285–3300, July 2015.[16] J. Chen, C. Richard, and A. H. Sayed, “Multitask dif-fusion adaptation over networks,” IEEE Trans. SignalProcess., vol. 62, no. 16, pp. 4129–4144, August 2014.[17] N. Bogdanovic, J. Plata-Chaves, and K. Berberidis,“Distributed incremental-based LMS for node-speciﬁcadaptive parameter estimation,” IEEE Trans. SignalProcess., vol. 62, no. 20, pp. 5382–5397, 2014.[18] B. Scho¨lkopf and A. J. Smola, Learning with Kernels:Support Vector Machines, Regularization, Optimization,and Beyond, MIT Press, 2002.[19] A. J. Smola and R. Kondor, “Kernels and regularizationon graphs,” in Learning Theory and Kernel Machines,pp. 144–158. Springer, 2003.

Adaptive regularized diffusion adaptation over multitask networks

http://infoscience.epfl.ch/record/233507/files/mlsp_2015.pdf

Adaptive regularized diffusion adaptation over multitask networks

Abstract

Similar works

Full text

Available Versions

Infoscience - École polytechnique fédérale de Lausanne