Machinery, Assoc Comp

Oliehoek, Frans A

Satsangi, Yash

Whiteson, Shimon

Submodular function maximization finds application in a variety of real-world decision-making problems. However, most existing methods, based on greedy maximization, assume it is computation- ally feasible to evaluate F, the function being maximized. Unfortunately, in many realistic settings F is too expensive to evaluate exactly even once. We present probably approximately correct greedy maximization, which requires access only to cheap anytime confidence bounds on F and uses them to prune elements. We show that, with high probability, our method returns an approximately optimal set. We propose novel, cheap confidence bounds for conditional entropy, which appears in many common choices of F and for which it is difficult to find unbiased or bounded estimates. Finally, results on a real-world dataset from a multi-camera tracking system in a shopping mall demonstrate that our approach performs comparably to existing methods, but at a fraction of the computational cost

Satsangi, Y.

Whiteson, S.

Oliehoek, F.A.

International Migration, Integration and Social Cohesion online publications

UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl)UvA-DARE (Digital Academic Repository)Probably Approximately Correct Greedy MaximizationSatsangi, Y.; Whiteson, S.; Oliehoek, F.A.Publication date2016Document VersionFinal published versionPublished inAAMAS'16Link to publicationCitation for published version (APA):Satsangi, Y., Whiteson, S., & Oliehoek, F. A. (2016). Probably Approximately Correct GreedyMaximization. In J. Thangarajah, K. Tuyls, C. Jonker, & S. Marsella (Eds.), AAMAS'16:proceedings of the 2016 International Conference on Autonomous Agents & MultiagentSystems : May, 9-13, 2016, Singapore, Singapore (Vol. 2, pp. 1387-1388). InternationalFoundation for Autonomous Agents and Multiagent Systems. http://www.aamas-conference.org/Proceedings/aamas2016/pdfs/p1387.pdfGeneral rightsIt is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s)and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an opencontent license (like Creative Commons).Disclaimer/Complaints regulationsIf you believe that digital publication of certain material infringes any of your rights or (privacy) interests, pleaselet the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the materialinaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letterto: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. Youwill be contacted as soon as possible.Download date:11 Nov 2022Probably Approximately Correct Greedy Maximization(Extended Abstract)Yash SatsangiUniversity of Amsterdamy.satsangi@uva.nlShimon WhitesonUniversity of Oxfordshimon.whiteson@cs.ox.ac.ukFrans A. OliehoekUniversity of LiverpoolUniversity of Amsterdamfrans.oliehoek@liverpool.ac.ukABSTRACTSubmodular function maximization finds application in a varietyof real-world decision-making problems. However, most existingmethods, based on greedy maximization, assume it is computation-ally feasible to evaluate F , the function being maximized. Unfortu-nately, in many realistic settings F is too expensive to evaluate ex-actly even once. We present probably approximately correct greedymaximization, which requires access only to cheap anytime confi-dence bounds on F and uses them to prune elements. We showthat, with high probability, our method returns an approximatelyoptimal set. We propose novel, cheap confidence bounds for con-ditional entropy, which appears in many common choices of F andfor which it is difficult to find unbiased or bounded estimates. Fi-nally, results on a real-world dataset from a multi-camera trackingsystem in a shopping mall demonstrate that our approach performscomparably to existing methods, but at a fraction of the computa-tional cost.KeywordsSubmodularity; Greedy maximization; Sensor scheduling; Plan-ning.1. INTRODUCTIONSubmodularity is a property of set functions that formalizes thenotion of diminishing returns. Many real-world problems involvemaximizing submodular functions, e.g., summarizing text, imagecollection summarization, or selecting sensors to minimize uncer-tainty about a hidden variable. Formally, given a ground set X ={1, 2 . . . n}, a set function F : 2X → R, is submodular if for everyAM ⊆ AN ⊆ X and i ∈ X \ AN , ∆F (i|AM ) ≥ ∆F (i|AN ),where ∆F (i|A) = F (A∪i)−F (A) is the marginal gain of addingi toA. Typically, the aim is to find anA∗ that maximizes F subjectto certain constraints. Here, we consider a constraint on A∗’s size:A∗ = arg maxA∈A+ F (A), where A+ = {A ⊆ X : |A| ≤ k}.As n increases, the(nk)possibilities forA∗ grow rapidly, render-ing naive maximization intractable. Instead, greedy maximizationfinds an approximate solution AG faster by iteratively adding toa partial solution the element that maximizes the marginal gain.Nemhauser et. al. [4] showed that the value obtained by greedymaximization is close to that of full maximization, i.e., F (AG) ≥(1− e−1)F (A∗), if F is submodular, non-negative and monotone.Appears in: Proceedings of the 15th International Conference on Au-tonomous Agents and Multiagent Systems (AAMAS 2016),J. Thangarajah, K. Tuyls, C. Jonker, S. Marsella (eds.),May 9–13, 2016, Singapore.Copyright c© 2016, International Foundation for Autonomous Agents andMultiagent Systems (www.ifaamas.org). All rights reserved.Most of the existing methods [3] assume it is computationallyfeasible to exactly compute F , and thus the marginal gain. In manysettings, this is not the case. For example, consider a surveillancetask [6] in which an agent aims to minimise uncertainty about a hid-den state by selecting a subset of sensors that maximise informationgain. Computing information gain is computationally expensive,especially when the hidden variable can take many values, as it in-volves an expectation over the entropy of posterior beliefs about thehidden variable. When surveilling large areas like shopping malls,exactly computing the entropy of a single posterior belief becomesinfeasible, let alone an expectation over them.In this paper, we present a new algorithm called probably ap-proximately correct greedy maximization. Rather than assumingaccess to F itself, we assume access only to confidence bounds onF that are cheaper to compute than F and are anytime, i.e., we cantighten them by spending more computation time, e.g., by gener-ating additional samples. Our method uses confidence bounds toprune elements, thereby avoiding the need to further tighten theirbounds. We provide a PAC analysis that shows that, with high prob-ability, our method returns an approximately optimal set.Given an unbiased estimator of F , it is possible to use concen-tration inequalities like Hoeffding’s inequality to obtain the con-fidence bounds needed by PAC greedy maximization. Unfortu-nately, many applications, such as sensor placement and decisiontree induction require information-theoretic definitions of F suchas information gain that depend on computing entropy over pos-terior beliefs, which are impossible to estimate in unbiased way[5]. The absence of an unbiased estimator makes it hard to obtaincomputationally cheap confidence bounds on conditional entropy[2]. Therefore, in this paper, we propose novel, cheap confidencebounds on conditional entropy.Finally, we apply PAC greedy maximization with these new con-fidence bounds to a real-life dataset collected a multi-camera track-ing system employed in a shopping mall. Our empirical resultsdemonstrate that our approach performs comparably to greedy andlazier greedy maximization, but at a fraction of the computationalcost, leading to much better scalability.2. METHOD & ANALYSISIn this section, we propose probably approximately correct greedymaximization. We assume U and L be set functions such that forA ∈ A+, with probability 1 − δu, U(A) ≥ F (A) and for eachA ∈ A+, with probability 1 − δl, F (A) ≥ L(A). The main ideais to use U and L to prune elements that with high probability donot maximize marginal gain.PAC greedy maximization works by initializing a partial solu-tion AP as ∅ and adding at each iteration the element iP , whichmaximizes the marginal gain for that iteration to AP until |AP |1387equals k. Algorithm 1 shows the main subroutine for finding iP .Algorithm 1 maintains a queue of unpruned elements prioritized bytheir upper bound. In each iteration of the outer while loop, the al-gorithm examines each of these elements and prunes it if its upperbound is not at least ε1 greater than the max lower bound foundso far. In addition, the element with the max lower bound is neverpruned. If an element is not pruned, then its bounds are tightened.Algorithm 1 pac-max(U,L,AP , ε1, t)iP ← 0 % element with max lower boundfor i ∈ X \ AP doρ.enqueue(i, U(AP ∪ i)) % initialize priority queueiP ← arg maxj∈{i,iP } L(AP ∪ j)}end forwhile ρ.length() > 1 ∨ change in U & L is > t doρ′ ← empty queuewhile ¬ρ.empty() doi← ρ.dequeue()if i = iP ∨ U(AP ∪ i) >= L(AP ∪ iP ) + ε1 thentighten(AP ∪ i)iP ← arg maxj∈{i,iP } L(AP ∪ j)ρ′.enqueue(i, U(AP ∪ i))end ifend whileρ← ρ′end whilereturn iP .Assumption 1. pac-max always terminates with ρ.length() = 1.In other words, we assume that U and L can be tightened enoughto disambiguate iP . Using this assumption we show that PACgreedy maximization with high probability achieves the same guar-antees as regular greedy maximization minus a constant term.Theorem 1. If F is non-negative, monotone and submodular in Xand if Assumption 1 holds then, with probability 1− δ,F (AP ) ≥ (1− e−1)F (A∗)− ε, (1)where AP is the solution returned by PAC greedy maximization,A∗ = arg maxA∈A+ F (A), δ = k(δu + δl), and ε = kε1.Proof. See [7].Our main focus is on settings where F has information-theoreticdefinitions like information gain, entropy or conditional entropy,for which computationally cheap upper and lower bounds are noteasy to find in absence of an unbiased estimator. In the sensor se-lection task, given a set of n sensors X , the aim is to find A∗ =arg maxA∈A+ [−HAb (s|z)]. Here s denotes a hidden variable, b(s)denotes the prior probability distribution over s, z = 〈z1, z2 . . . zn〉denotes the observation vector, zi stands for the observation abouts from sensor i and HAb (s|z) = Ez|b,A[HbAz (s)] is the conditionalentropy [5] of s given z, which is the expected value for the entropyHbAz (s) [5] of the posterior probability distribution bAz (s).The upper bound on F (A) = −HAb (s|z) can be obtained bysimply approximating the entropy of bAz ,HbAz (s) with its maximumlikelihood estimatesHb̂Az (s), that is, U(A) = −Ez|b,A[Hb̂Az (s)]+η [7]. The lower bound, which is rather difficult to find, is obtainedby using r = 〈r1, r2 . . . , rn〉 in place of z, which is a crude ap-proximation of z, e.g., obtained by clustering zi into d clusters de-terministically and ri denotes the cluster zi belongs to. L(A) isdefined as L(A) = −[HAb̂(s|r) + η + log(1 + 1M(ψb(s)− 1))],where ψb(s) is the support of b(s). [7] provides the proofs anddetails that show, with probability 1 − δu, U(A) ≥ F (A) andwith probability 1 − δl, L(A) ≤ F (A) with exact definitions ofδu, δl, η and M . The following theorem ties our results togetherto show that for F (A) = −HAb (s|z), PAC greedy maximizationwith above definitions of U and L, computesAP such that F (AP )has bounded error with respect to F (A∗), and achieves the sameguarantees as greedy maximization minus a constant term.Theorem 2. For the above described definitions of F , U and L, ifAssumption 1 holds and if z is conditionally independent given s,then with probability 1− δ, (1) is true.Proof. See [7].3. EXPERIMENTS & RESULTSWe evaluated PAC greedy maximization on the problem of track-ing multiple people using a multi-camera system. The problem wasextracted from a real-world dataset [1] collected in a shopping mallusing 13 CCTV cameras for over 4 hours. To evaluate a given al-gorithm, a trajectory was sampled randomly. The goal is to select asubset of cameras at each time step that induce a low-entropy beliefabout the person’s location that can be used to predict those loca-tions. As a baseline, we tested against greedy maximization whichsimply uses an approximation based on the MLE estimates of pos-terior beliefs and lazier greedy maximization [3] which, in eachiteration, samples a subset of size R from X and selects i from Rthat maximizes marginal gain.0 0.5 1 1.5 2 2.5x 104050100150200250Time taken (in seconds)Total correct predictions  Greedy maximization, M=30Lazier greedy max., M=50,R=12Lazier greedy max., M=30,R=15PAC greedy max, eps=0.37PAC greedy max, eps=0.47Figure 1: Multi-persontracking for n = 20 andk = 3.Figure 1 shows the numberof correct predictions (y-axis)against the runtime (x-axis) ofeach method. Thus, the topleft is the most desirable re-gion. PAC greedy maximizationperforms nearly as well as thegreedy maximization and laziergreedy maximization but doesso at lower computational cost leading to much better scalability.AcknowledgmentsThis research is supported by the Dutch Technology Foundation STW(project #12622) and NWO Innovational Research Incentives Scheme Veni#639.021.336. We thank Henri Bouma and TNO for providing us with thedataset used in our experiments and the STW User Committee for its usefulinput.REFERENCES[1] H. Bouma, J. Baan, S. Landsmeer, C. Kruszynski, G. van Antwerpen,and J. Dijk. Real-time tracking and fast retrieval of persons in multiplesurveillance cameras of a shopping mall. 2013.[2] P.-L. Loh and S. Nowozin. Faster hoeffding racing: Bernstein races viajackknife estimates. In Algorithmic Learning Theory, 2013.[3] B. Mirzasoleiman, A. Badanidiyuru, A. Karbasi, J. Vondrák, andA. Krause. Lazier than lazy greedy. In AAAI, 2015.[4] G. Nemhauser, L. Wolsey, and M. Fisher. An analysis ofapproximations for maximizing submodular set functions.Mathematical Programming, 1978.[5] L. Paninski. Estimation of entropy and mutual information. Neuralcomputation, 2003.[6] Y. Satsangi, S. Whiteson, and F. Oliehoek. Exploiting submodularvalue functions for faster dynamic sensor selection. In AAAI, 2015.[7] Y. Satsangi, S. Whiteson, and F. Oliehoek. Probably approximatelycorrect greedy maximization. ArXiv e-prints, Feb. 2016.http://arxiv.org/abs/1602.07860.1388

English

Probably Approximately Correct Greedy Maximization

University of Liverpool Repository

Probably Approximately Correct Greedy Maximization(Extended Abstract)Yash SatsangiUniversity of Amsterdamy.satsangi@uva.nlShimon WhitesonUniversity of Oxfordshimon.whiteson@cs.ox.ac.ukFrans A. OliehoekUniversity of LiverpoolUniversity of Amsterdamfrans.oliehoek@liverpool.ac.ukABSTRACTSubmodular function maximization finds application in a varietyof real-world decision-making problems. However, most existingmethods, based on greedy maximization, assume it is computation-ally feasible to evaluate F , the function being maximized. Unfortu-nately, in many realistic settings F is too expensive to evaluate ex-actly even once. We present probably approximately correct greedymaximization, which requires access only to cheap anytime confi-dence bounds on F and uses them to prune elements. We showthat, with high probability, our method returns an approximatelyoptimal set. We propose novel, cheap confidence bounds for con-ditional entropy, which appears in many common choices of F andfor which it is difficult to find unbiased or bounded estimates. Fi-nally, results on a real-world dataset from a multi-camera trackingsystem in a shopping mall demonstrate that our approach performscomparably to existing methods, but at a fraction of the computa-tional cost.KeywordsSubmodularity; Greedy maximization; Sensor scheduling; Plan-ning.1. INTRODUCTIONSubmodularity is a property of set functions that formalizes thenotion of diminishing returns. Many real-world problems involvemaximizing submodular functions, e.g., summarizing text, imagecollection summarization, or selecting sensors to minimize uncer-tainty about a hidden variable. Formally, given a ground set X ={1, 2 . . . n}, a set function F : 2X → R, is submodular if for everyAM ⊆ AN ⊆ X and i ∈ X \ AN , ∆F (i|AM ) ≥ ∆F (i|AN ),where ∆F (i|A) = F (A∪i)−F (A) is the marginal gain of addingi toA. Typically, the aim is to find anA∗ that maximizes F subjectto certain constraints. Here, we consider a constraint on A∗’s size:A∗ = arg maxA∈A+ F (A), where A+ = {A ⊆ X : |A| ≤ k}.As n increases, the(nk)possibilities forA∗ grow rapidly, render-ing naive maximization intractable. Instead, greedy maximizationfinds an approximate solution AG faster by iteratively adding toa partial solution the element that maximizes the marginal gain.Nemhauser et. al. [4] showed that the value obtained by greedymaximization is close to that of full maximization, i.e., F (AG) ≥(1− e−1)F (A∗), if F is submodular, non-negative and monotone.Appears in: Proceedings of the 15th International Conference on Au-tonomous Agents and Multiagent Systems (AAMAS 2016),J. Thangarajah, K. Tuyls, C. Jonker, S. Marsella (eds.),May 9–13, 2016, Singapore.Copyright c© 2016, International Foundation for Autonomous Agents andMultiagent Systems (www.ifaamas.org). All rights reserved.Most of the existing methods [3] assume it is computationallyfeasible to exactly compute F , and thus the marginal gain. In manysettings, this is not the case. For example, consider a surveillancetask [6] in which an agent aims to minimise uncertainty about a hid-den state by selecting a subset of sensors that maximise informationgain. Computing information gain is computationally expensive,especially when the hidden variable can take many values, as it in-volves an expectation over the entropy of posterior beliefs about thehidden variable. When surveilling large areas like shopping malls,exactly computing the entropy of a single posterior belief becomesinfeasible, let alone an expectation over them.In this paper, we present a new algorithm called probably ap-proximately correct greedy maximization. Rather than assumingaccess to F itself, we assume access only to confidence bounds onF that are cheaper to compute than F and are anytime, i.e., we cantighten them by spending more computation time, e.g., by gener-ating additional samples. Our method uses confidence bounds toprune elements, thereby avoiding the need to further tighten theirbounds. We provide a PAC analysis that shows that, with high prob-ability, our method returns an approximately optimal set.Given an unbiased estimator of F , it is possible to use concen-tration inequalities like Hoeffding’s inequality to obtain the con-fidence bounds needed by PAC greedy maximization. Unfortu-nately, many applications, such as sensor placement and decisiontree induction require information-theoretic definitions of F suchas information gain that depend on computing entropy over pos-terior beliefs, which are impossible to estimate in unbiased way[5]. The absence of an unbiased estimator makes it hard to obtaincomputationally cheap confidence bounds on conditional entropy[2]. Therefore, in this paper, we propose novel, cheap confidencebounds on conditional entropy.Finally, we apply PAC greedy maximization with these new con-fidence bounds to a real-life dataset collected a multi-camera track-ing system employed in a shopping mall. Our empirical resultsdemonstrate that our approach performs comparably to greedy andlazier greedy maximization, but at a fraction of the computationalcost, leading to much better scalability.2. METHOD & ANALYSISIn this section, we propose probably approximately correct greedymaximization. We assume U and L be set functions such that forA ∈ A+, with probability 1 − δu, U(A) ≥ F (A) and for eachA ∈ A+, with probability 1 − δl, F (A) ≥ L(A). The main ideais to use U and L to prune elements that with high probability donot maximize marginal gain.PAC greedy maximization works by initializing a partial solu-tion AP as ∅ and adding at each iteration the element iP , whichmaximizes the marginal gain for that iteration to AP until |AP |equals k. Algorithm 1 shows the main subroutine for finding iP .Algorithm 1 maintains a queue of unpruned elements prioritized bytheir upper bound. In each iteration of the outer while loop, the al-gorithm examines each of these elements and prunes it if its upperbound is not at least 1 greater than the max lower bound foundso far. In addition, the element with the max lower bound is neverpruned. If an element is not pruned, then its bounds are tightened.Algorithm 1 pac-max(U,L,AP , 1, t)iP ← 0 % element with max lower boundfor i ∈ X \ AP doρ.enqueue(i, U(AP ∪ i)) % initialize priority queueiP ← arg maxj∈{i,iP } L(AP ∪ j)}end forwhile ρ.length() > 1 ∨ change in U & L is > t doρ′ ← empty queuewhile ¬ρ.empty() doi← ρ.dequeue()if i = iP ∨ U(AP ∪ i) >= L(AP ∪ iP ) + 1 thentighten(AP ∪ i)iP ← arg maxj∈{i,iP } L(AP ∪ j)ρ′.enqueue(i, U(AP ∪ i))end ifend whileρ← ρ′end whilereturn iP .Assumption 1. pac-max always terminates with ρ.length() = 1.In other words, we assume that U and L can be tightened enoughto disambiguate iP . Using this assumption we show that PACgreedy maximization with high probability achieves the same guar-antees as regular greedy maximization minus a constant term.Theorem 1. If F is non-negative, monotone and submodular in Xand if Assumption 1 holds then, with probability 1− δ,F (AP ) ≥ (1− e−1)F (A∗)− , (1)where AP is the solution returned by PAC greedy maximization,A∗ = arg maxA∈A+ F (A), δ = k(δu + δl), and  = k1.Proof. See [7].Our main focus is on settings where F has information-theoreticdefinitions like information gain, entropy or conditional entropy,for which computationally cheap upper and lower bounds are noteasy to find in absence of an unbiased estimator. In the sensor se-lection task, given a set of n sensors X , the aim is to find A∗ =arg maxA∈A+ [−HAb (s|z)]. Here s denotes a hidden variable, b(s)denotes the prior probability distribution over s, z = 〈z1, z2 . . . zn〉denotes the observation vector, zi stands for the observation abouts from sensor i and HAb (s|z) = Ez|b,A[HbAz (s)] is the conditionalentropy [5] of s given z, which is the expected value for the entropyHbAz (s) [5] of the posterior probability distribution bAz (s).The upper bound on F (A) = −HAb (s|z) can be obtained bysimply approximating the entropy of bAz ,HbAz (s) with its maximumlikelihood estimatesHbˆAz (s), that is, U(A) = −Ez|b,A[HbˆAz (s)]+η [7]. The lower bound, which is rather difficult to find, is obtainedby using r = 〈r1, r2 . . . , rn〉 in place of z, which is a crude ap-proximation of z, e.g., obtained by clustering zi into d clusters de-terministically and ri denotes the cluster zi belongs to. L(A) isdefined as L(A) = −[HAbˆ(s|r) + η + log(1 + 1M(ψb(s)− 1))],where ψb(s) is the support of b(s). [7] provides the proofs anddetails that show, with probability 1 − δu, U(A) ≥ F (A) andwith probability 1 − δl, L(A) ≤ F (A) with exact definitions ofδu, δl, η and M . The following theorem ties our results togetherto show that for F (A) = −HAb (s|z), PAC greedy maximizationwith above definitions of U and L, computesAP such that F (AP )has bounded error with respect to F (A∗), and achieves the sameguarantees as greedy maximization minus a constant term.Theorem 2. For the above described definitions of F , U and L, ifAssumption 1 holds and if z is conditionally independent given s,then with probability 1− δ, (1) is true.Proof. See [7].3. EXPERIMENTS & RESULTSWe evaluated PAC greedy maximization on the problem of track-ing multiple people using a multi-camera system. The problem wasextracted from a real-world dataset [1] collected in a shopping mallusing 13 CCTV cameras for over 4 hours. To evaluate a given al-gorithm, a trajectory was sampled randomly. The goal is to select asubset of cameras at each time step that induce a low-entropy beliefabout the person’s location that can be used to predict those loca-tions. As a baseline, we tested against greedy maximization whichsimply uses an approximation based on the MLE estimates of pos-terior beliefs and lazier greedy maximization [3] which, in eachiteration, samples a subset of size R from X and selects i from Rthat maximizes marginal gain.0 0.5 1 1.5 2 2.5x 104050100150200250Time taken (in seconds)Total correct predictions  Greedy maximization, M=30Lazier greedy max., M=50,R=12Lazier greedy max., M=30,R=15PAC greedy max, eps=0.37PAC greedy max, eps=0.47Figure 1: Multi-persontracking for n = 20 andk = 3.Figure 1 shows the numberof correct predictions (y-axis)against the runtime (x-axis) ofeach method. Thus, the topleft is the most desirable re-gion. PAC greedy maximizationperforms nearly as well as thegreedy maximization and laziergreedy maximization but doesso at lower computational cost leading to much better scalability.AcknowledgmentsThis research is supported by the Dutch Technology Foundation STW(project #12622) and NWO Innovational Research Incentives Scheme Veni#639.021.336. We thank Henri Bouma and TNO for providing us with thedataset used in our experiments and the STW User Committee for its usefulinput.REFERENCES[1] H. Bouma, J. Baan, S. Landsmeer, C. Kruszynski, G. van Antwerpen,and J. Dijk. Real-time tracking and fast retrieval of persons in multiplesurveillance cameras of a shopping mall. 2013.[2] P.-L. Loh and S. Nowozin. Faster hoeffding racing: Bernstein races viajackknife estimates. In Algorithmic Learning Theory, 2013.[3] B. Mirzasoleiman, A. Badanidiyuru, A. Karbasi, J. Vondrák, andA. Krause. Lazier than lazy greedy. In AAAI, 2015.[4] G. Nemhauser, L. Wolsey, and M. Fisher. An analysis ofapproximations for maximizing submodular set functions.Mathematical Programming, 1978.[5] L. Paninski. Estimation of entropy and mutual information. Neuralcomputation, 2003.[6] Y. Satsangi, S. Whiteson, and F. Oliehoek. Exploiting submodularvalue functions for faster dynamic sensor selection. In AAAI, 2015.[7] Y. Satsangi, S. Whiteson, and F. Oliehoek. Probably approximatelycorrect greedy maximization. ArXiv e-prints, Feb. 2016.http://arxiv.org/abs/1602.07860.

Probably Approximately Correct Greedy Maximization: (Extended Abstract).

Search4Dev

UvA-DARE is a service provided by the library of the University of Amsterdam (http://dare.uva.nl)
UvA-DARE (Digital Academic Repository)
Probably Approximately Correct Greedy Maximization
Satsangi, Y.; Whiteson, S.; Oliehoek, F.A.
Published in:
AAMAS'16
Link to publication
Citation for published version (APA):
Satsangi, Y., Whiteson, S., & Oliehoek, F. A. (2016). Probably Approximately Correct Greedy Maximization. In J.
Thangarajah, K. Tuyls, C. Jonker, & S. Marsella (Eds.), AAMAS'16: proceedings of the 2016 International
Conference on Autonomous Agents & Multiagent Systems : May, 9-13, 2016, Singapore, Singapore (Vol. 2, pp.
1387-1388). International Foundation for Autonomous Agents and Multiagent Systems. http://www.aamas-
conference.org/Proceedings/aamas2016/pdfs/p1387.pdf
General rights
It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s),
other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons).
Disclaimer/Complaints regulations
If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating
your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask
the Library: https://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam,
The Netherlands. You will be contacted as soon as possible.
Download date: 04 Feb 2021
Probably Approximately Correct Greedy Maximization
(Extended Abstract)
Yash Satsangi
University of Amsterdam
y.satsangi@uva.nl
Shimon Whiteson
University of Oxford
shimon.whiteson@cs.ox.ac.uk
Frans A. Oliehoek
University of Liverpool
University of Amsterdam
frans.oliehoek@liverpool.ac.uk
ABSTRACT
Submodular function maximization finds application in a variety
of real-world decision-making problems. However, most existing
methods, based on greedy maximization, assume it is computation-
ally feasible to evaluate F , the function being maximized. Unfortu-
nately, in many realistic settings F is too expensive to evaluate ex-
actly even once. We present probably approximately correct greedy
maximization, which requires access only to cheap anytime confi-
dence bounds on F and uses them to prune elements. We show
that, with high probability, our method returns an approximately
optimal set. We propose novel, cheap confidence bounds for con-
ditional entropy, which appears in many common choices of F and
for which it is difficult to find unbiased or bounded estimates. Fi-
nally, results on a real-world dataset from a multi-camera tracking
system in a shopping mall demonstrate that our approach performs
comparably to existing methods, but at a fraction of the computa-
tional cost.
Keywords
Submodularity; Greedy maximization; Sensor scheduling; Plan-
ning.
1. INTRODUCTION
Submodularity is a property of set functions that formalizes the
notion of diminishing returns. Many real-world problems involve
maximizing submodular functions, e.g., summarizing text, image
collection summarization, or selecting sensors to minimize uncer-
tainty about a hidden variable. Formally, given a ground set X =
{1, 2 . . . n}, a set function F : 2X → R, is submodular if for every
AM ⊆ AN ⊆ X and i ∈ X \ AN , ∆F (i|AM ) ≥ ∆F (i|AN ),
where ∆F (i|A) = F (A∪i)−F (A) is the marginal gain of adding
i toA. Typically, the aim is to find anA∗ that maximizes F subject
to certain constraints. Here, we consider a constraint on A∗’s size:
A∗ = arg maxA∈A+ F (A), where A+ = {A ⊆ X : |A| ≤ k}.
As n increases, the
(
n
k
)
possibilities forA∗ grow rapidly, render-
ing naive maximization intractable. Instead, greedy maximization
finds an approximate solution AG faster by iteratively adding to
a partial solution the element that maximizes the marginal gain.
Nemhauser et. al. [4] showed that the value obtained by greedy
maximization is close to that of full maximization, i.e., F (AG) ≥
(1− e−1)F (A∗), if F is submodular, non-negative and monotone.
Appears in: Proceedings of the 15th International Conference on Au-
tonomous Agents and Multiagent Systems (AAMAS 2016),
J. Thangarajah, K. Tuyls, C. Jonker, S. Marsella (eds.),
May 9–13, 2016, Singapore.
Copyright c© 2016, International Foundation for Autonomous Agents and
Multiagent Systems (www.ifaamas.org). All rights reserved.
Most of the existing methods [3] assume it is computationally
feasible to exactly compute F , and thus the marginal gain. In many
settings, this is not the case. For example, consider a surveillance
task [6] in which an agent aims to minimise uncertainty about a hid-
den state by selecting a subset of sensors that maximise information
gain. Computing information gain is computationally expensive,
especially when the hidden variable can take many values, as it in-
volves an expectation over the entropy of posterior beliefs about the
hidden variable. When surveilling large areas like shopping malls,
exactly computing the entropy of a single posterior belief becomes
infeasible, let alone an expectation over them.
In this paper, we present a new algorithm called probably ap-
proximately correct greedy maximization. Rather than assuming
access to F itself, we assume access only to confidence bounds on
F that are cheaper to compute than F and are anytime, i.e., we can
tighten them by spending more computation time, e.g., by gener-
ating additional samples. Our method uses confidence bounds to
prune elements, thereby avoiding the need to further tighten their
bounds. We provide a PAC analysis that shows that, with high prob-
ability, our method returns an approximately optimal set.
Given an unbiased estimator of F , it is possible to use concen-
tration inequalities like Hoeffding’s inequality to obtain the con-
fidence bounds needed by PAC greedy maximization. Unfortu-
nately, many applications, such as sensor placement and decision
tree induction require information-theoretic definitions of F such
as information gain that depend on computing entropy over pos-
terior beliefs, which are impossible to estimate in unbiased way
[5]. The absence of an unbiased estimator makes it hard to obtain
computationally cheap confidence bounds on conditional entropy
[2]. Therefore, in this paper, we propose novel, cheap confidence
bounds on conditional entropy.
Finally, we apply PAC greedy maximization with these new con-
fidence bounds to a real-life dataset collected a multi-camera track-
ing system employed in a shopping mall. Our empirical results
demonstrate that our approach performs comparably to greedy and
lazier greedy maximization, but at a fraction of the computational
cost, leading to much better scalability.
2. METHOD & ANALYSIS
In this section, we propose probably approximately correct greedy
maximization. We assume U and L be set functions such that for
A ∈ A+, with probability 1 − δu, U(A) ≥ F (A) and for each
A ∈ A+, with probability 1 − δl, F (A) ≥ L(A). The main idea
is to use U and L to prune elements that with high probability do
not maximize marginal gain.
PAC greedy maximization works by initializing a partial solu-
tion AP as ∅ and adding at each iteration the element iP , which
maximizes the marginal gain for that iteration to AP until |AP |
1387
equals k. Algorithm 1 shows the main subroutine for finding iP .
Algorithm 1 maintains a queue of unpruned elements prioritized by
their upper bound. In each iteration of the outer while loop, the al-
gorithm examines each of these elements and prunes it if its upper
bound is not at least ε1 greater than the max lower bound found
so far. In addition, the element with the max lower bound is never
pruned. If an element is not pruned, then its bounds are tightened.
Algorithm 1 pac-max(U,L,AP , ε1, t)
iP ← 0 % element with max lower bound
for i ∈ X \ AP do
ρ.enqueue(i, U(AP ∪ i)) % initialize priority queue
iP ← arg maxj∈{i,iP } L(AP ∪ j)}
end for
while ρ.length() > 1 ∨ change in U & L is > t do
ρ′ ← empty queue
while ¬ρ.empty() do
i← ρ.dequeue()
if i = iP ∨ U(AP ∪ i) >= L(AP ∪ iP ) + ε1 then
tighten(AP ∪ i)
iP ← arg maxj∈{i,iP } L(AP ∪ j)
ρ′.enqueue(i, U(AP ∪ i))
end if
end while
ρ← ρ′
end while
return iP .
Assumption 1. pac-max always terminates with ρ.length() = 1.
In other words, we assume that U and L can be tightened enough
to disambiguate iP . Using this assumption we show that PAC
greedy maximization with high probability achieves the same guar-
antees as regular greedy maximization minus a constant term.
Theorem 1. If F is non-negative, monotone and submodular in X
and if Assumption 1 holds then, with probability 1− δ,
F (AP ) ≥ (1− e−1)F (A∗)− ε, (1)
where AP is the solution returned by PAC greedy maximization,
A∗ = arg maxA∈A+ F (A), δ = k(δu + δl), and ε = kε1.
Proof. See [7].
Our main focus is on settings where F has information-theoretic
definitions like information gain, entropy or conditional entropy,
for which computationally cheap upper and lower bounds are not
easy to find in absence of an unbiased estimator. In the sensor se-
lection task, given a set of n sensors X , the aim is to find A∗ =
arg maxA∈A+ [−HAb (s|z)]. Here s denotes a hidden variable, b(s)
denotes the prior probability distribution over s, z = 〈z1, z2 . . . zn〉
denotes the observation vector, zi stands for the observation about
s from sensor i and HAb (s|z) = Ez|b,A[HbAz (s)] is the conditional
entropy [5] of s given z, which is the expected value for the entropy
HbAz (s) [5] of the posterior probability distribution b
A
z (s).
The upper bound on F (A) = −HAb (s|z) can be obtained by
simply approximating the entropy of bAz ,HbAz (s) with its maximum
likelihood estimatesHb̂Az (s), that is, U(A) = −Ez|b,A[Hb̂Az (s)]+
η [7]. The lower bound, which is rather difficult to find, is obtained
by using r = 〈r1, r2 . . . , rn〉 in place of z, which is a crude ap-
proximation of z, e.g., obtained by clustering zi into d clusters de-
terministically and ri denotes the cluster zi belongs to. L(A) is
defined as L(A) = −[HA
b̂
(s|r) + η + log(1 + 1
M
(ψb(s)− 1))],
where ψb(s) is the support of b(s). [7] provides the proofs and
details that show, with probability 1 − δu, U(A) ≥ F (A) and
with probability 1 − δl, L(A) ≤ F (A) with exact definitions of
δu, δl, η and M . The following theorem ties our results together
to show that for F (A) = −HAb (s|z), PAC greedy maximization
with above definitions of U and L, computesAP such that F (AP )
has bounded error with respect to F (A∗), and achieves the same
guarantees as greedy maximization minus a constant term.
Theorem 2. For the above described definitions of F , U and L, if
Assumption 1 holds and if z is conditionally independent given s,
then with probability 1− δ, (1) is true.
Proof. See [7].
3. EXPERIMENTS & RESULTS
We evaluated PAC greedy maximization on the problem of track-
ing multiple people using a multi-camera system. The problem was
extracted from a real-world dataset [1] collected in a shopping mall
using 13 CCTV cameras for over 4 hours. To evaluate a given al-
gorithm, a trajectory was sampled randomly. The goal is to select a
subset of cameras at each time step that induce a low-entropy belief
about the person’s location that can be used to predict those loca-
tions. As a baseline, we tested against greedy maximization which
simply uses an approximation based on the MLE estimates of pos-
terior beliefs and lazier greedy maximization [3] which, in each
iteration, samples a subset of size R from X and selects i from R
that maximizes marginal gain.
0 0.5 1 1.5 2 2.5
x 10
4
0
50
100
150
200
250
Time taken (in seconds)
T
o
ta
l 
c
o
rr
e
c
t 
p
re
d
ic
ti
o
n
s
 
 
Greedy maximization, M=30
Lazier greedy max., M=50,R=12
Lazier greedy max., M=30,R=15
PAC greedy max, eps=0.37
PAC greedy max, eps=0.47
Figure 1: Multi-person
tracking for n = 20 and
k = 3.
Figure 1 shows the number
of correct predictions (y-axis)
against the runtime (x-axis) of
each method. Thus, the top
left is the most desirable re-
gion. PAC greedy maximization
performs nearly as well as the
greedy maximization and lazier
greedy maximization but does
so at lower computational cost leading to much better scalability.
Acknowledgments
This research is supported by the Dutch Technology Foundation STW
(project #12622) and NWO Innovational Research Incentives Scheme Veni
#639.021.336. We thank Henri Bouma and TNO for providing us with the
dataset used in our experiments and the STW User Committee for its useful
input.
REFERENCES
[1] H. Bouma, J. Baan, S. Landsmeer, C. Kruszynski, G. van Antwerpen,
and J. Dijk. Real-time tracking and fast retrieval of persons in multiple
surveillance cameras of a shopping mall. 2013.
[2] P.-L. Loh and S. Nowozin. Faster hoeffding racing: Bernstein races via
jackknife estimates. In Algorithmic Learning Theory, 2013.
[3] B. Mirzasoleiman, A. Badanidiyuru, A. Karbasi, J. Vondrák, and
A. Krause. Lazier than lazy greedy. In AAAI, 2015.
[4] G. Nemhauser, L. Wolsey, and M. Fisher. An analysis of
approximations for maximizing submodular set functions.
Mathematical Programming, 1978.
[5] L. Paninski. Estimation of entropy and mutual information. Neural
computation, 2003.
[6] Y. Satsangi, S. Whiteson, and F. Oliehoek. Exploiting submodular
value functions for faster dynamic sensor selection. In AAAI, 2015.
[7] Y. Satsangi, S. Whiteson, and F. Oliehoek. Probably approximately
correct greedy maximization. ArXiv e-prints, Feb. 2016.
http://arxiv.org/abs/1602.07860.
1388


Submodular function maximization finds application in a variety of real-world
decision-making problems. However, most existing methods, based on greedy
maximization, assume it is computationally feasible to evaluate F, the function
being maximized. Unfortunately, in many realistic settings F is too expensive
to evaluate exactly even once. We present probably approximately correct greedy
maximization, which requires access only to cheap anytime confidence bounds on
F and uses them to prune elements. We show that, with high probability, our
method returns an approximately optimal set. We propose novel, cheap confidence
bounds for conditional entropy, which appears in many common choices of F and
for which it is difficult to find unbiased or bounded estimates. Finally,
results on a real-world dataset from a multi-camera tracking system in a
shopping mall demonstrate that our approach performs comparably to existing
methods, but at a fraction of the computational cost

UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl)UvA-DARE (Digital Academic Repository)Probably Approximately Correct Greedy MaximizationSatsangi, Y.; Whiteson, S.; Oliehoek, F.A.Publication date2016Document VersionFinal published versionPublished inAAMAS'16Link to publicationCitation for published version (APA):Satsangi, Y., Whiteson, S., & Oliehoek, F. A. (2016). Probably Approximately Correct GreedyMaximization. In J. Thangarajah, K. Tuyls, C. Jonker, & S. Marsella (Eds.), AAMAS'16:proceedings of the 2016 International Conference on Autonomous Agents & MultiagentSystems : May, 9-13, 2016, Singapore, Singapore (Vol. 2, pp. 1387-1388). InternationalFoundation for Autonomous Agents and Multiagent Systems. http://www.aamas-conference.org/Proceedings/aamas2016/pdfs/p1387.pdfGeneral rightsIt is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s)and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an opencontent license (like Creative Commons).Disclaimer/Complaints regulationsIf you believe that digital publication of certain material infringes any of your rights or (privacy) interests, pleaselet the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the materialinaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letterto: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. Youwill be contacted as soon as possible.Download date:09 Mar 2023Probably Approximately Correct Greedy Maximization(Extended Abstract)Yash SatsangiUniversity of Amsterdamy.satsangi@uva.nlShimon WhitesonUniversity of Oxfordshimon.whiteson@cs.ox.ac.ukFrans A. OliehoekUniversity of LiverpoolUniversity of Amsterdamfrans.oliehoek@liverpool.ac.ukABSTRACTSubmodular function maximization finds application in a varietyof real-world decision-making problems. However, most existingmethods, based on greedy maximization, assume it is computation-ally feasible to evaluate F , the function being maximized. Unfortu-nately, in many realistic settings F is too expensive to evaluate ex-actly even once. We present probably approximately correct greedymaximization, which requires access only to cheap anytime confi-dence bounds on F and uses them to prune elements. We showthat, with high probability, our method returns an approximatelyoptimal set. We propose novel, cheap confidence bounds for con-ditional entropy, which appears in many common choices of F andfor which it is difficult to find unbiased or bounded estimates. Fi-nally, results on a real-world dataset from a multi-camera trackingsystem in a shopping mall demonstrate that our approach performscomparably to existing methods, but at a fraction of the computa-tional cost.KeywordsSubmodularity; Greedy maximization; Sensor scheduling; Plan-ning.1. INTRODUCTIONSubmodularity is a property of set functions that formalizes thenotion of diminishing returns. Many real-world problems involvemaximizing submodular functions, e.g., summarizing text, imagecollection summarization, or selecting sensors to minimize uncer-tainty about a hidden variable. Formally, given a ground set X ={1, 2 . . . n}, a set function F : 2X → R, is submodular if for everyAM ⊆ AN ⊆ X and i ∈ X \ AN , ∆F (i|AM ) ≥ ∆F (i|AN ),where ∆F (i|A) = F (A∪i)−F (A) is the marginal gain of addingi toA. Typically, the aim is to find anA∗ that maximizes F subjectto certain constraints. Here, we consider a constraint on A∗’s size:A∗ = arg maxA∈A+ F (A), where A+ = {A ⊆ X : |A| ≤ k}.As n increases, the(nk)possibilities forA∗ grow rapidly, render-ing naive maximization intractable. Instead, greedy maximizationfinds an approximate solution AG faster by iteratively adding toa partial solution the element that maximizes the marginal gain.Nemhauser et. al. [4] showed that the value obtained by greedymaximization is close to that of full maximization, i.e., F (AG) ≥(1− e−1)F (A∗), if F is submodular, non-negative and monotone.Appears in: Proceedings of the 15th International Conference on Au-tonomous Agents and Multiagent Systems (AAMAS 2016),J. Thangarajah, K. Tuyls, C. Jonker, S. Marsella (eds.),May 9–13, 2016, Singapore.Copyright c© 2016, International Foundation for Autonomous Agents andMultiagent Systems (www.ifaamas.org). All rights reserved.Most of the existing methods [3] assume it is computationallyfeasible to exactly compute F , and thus the marginal gain. In manysettings, this is not the case. For example, consider a surveillancetask [6] in which an agent aims to minimise uncertainty about a hid-den state by selecting a subset of sensors that maximise informationgain. Computing information gain is computationally expensive,especially when the hidden variable can take many values, as it in-volves an expectation over the entropy of posterior beliefs about thehidden variable. When surveilling large areas like shopping malls,exactly computing the entropy of a single posterior belief becomesinfeasible, let alone an expectation over them.In this paper, we present a new algorithm called probably ap-proximately correct greedy maximization. Rather than assumingaccess to F itself, we assume access only to confidence bounds onF that are cheaper to compute than F and are anytime, i.e., we cantighten them by spending more computation time, e.g., by gener-ating additional samples. Our method uses confidence bounds toprune elements, thereby avoiding the need to further tighten theirbounds. We provide a PAC analysis that shows that, with high prob-ability, our method returns an approximately optimal set.Given an unbiased estimator of F , it is possible to use concen-tration inequalities like Hoeffding’s inequality to obtain the con-fidence bounds needed by PAC greedy maximization. Unfortu-nately, many applications, such as sensor placement and decisiontree induction require information-theoretic definitions of F suchas information gain that depend on computing entropy over pos-terior beliefs, which are impossible to estimate in unbiased way[5]. The absence of an unbiased estimator makes it hard to obtaincomputationally cheap confidence bounds on conditional entropy[2]. Therefore, in this paper, we propose novel, cheap confidencebounds on conditional entropy.Finally, we apply PAC greedy maximization with these new con-fidence bounds to a real-life dataset collected a multi-camera track-ing system employed in a shopping mall. Our empirical resultsdemonstrate that our approach performs comparably to greedy andlazier greedy maximization, but at a fraction of the computationalcost, leading to much better scalability.2. METHOD & ANALYSISIn this section, we propose probably approximately correct greedymaximization. We assume U and L be set functions such that forA ∈ A+, with probability 1 − δu, U(A) ≥ F (A) and for eachA ∈ A+, with probability 1 − δl, F (A) ≥ L(A). The main ideais to use U and L to prune elements that with high probability donot maximize marginal gain.PAC greedy maximization works by initializing a partial solu-tion AP as ∅ and adding at each iteration the element iP , whichmaximizes the marginal gain for that iteration to AP until |AP |1387equals k. Algorithm 1 shows the main subroutine for finding iP .Algorithm 1 maintains a queue of unpruned elements prioritized bytheir upper bound. In each iteration of the outer while loop, the al-gorithm examines each of these elements and prunes it if its upperbound is not at least ε1 greater than the max lower bound foundso far. In addition, the element with the max lower bound is neverpruned. If an element is not pruned, then its bounds are tightened.Algorithm 1 pac-max(U,L,AP , ε1, t)iP ← 0 % element with max lower boundfor i ∈ X \ AP doρ.enqueue(i, U(AP ∪ i)) % initialize priority queueiP ← arg maxj∈{i,iP } L(AP ∪ j)}end forwhile ρ.length() > 1 ∨ change in U & L is > t doρ′ ← empty queuewhile ¬ρ.empty() doi← ρ.dequeue()if i = iP ∨ U(AP ∪ i) >= L(AP ∪ iP ) + ε1 thentighten(AP ∪ i)iP ← arg maxj∈{i,iP } L(AP ∪ j)ρ′.enqueue(i, U(AP ∪ i))end ifend whileρ← ρ′end whilereturn iP .Assumption 1. pac-max always terminates with ρ.length() = 1.In other words, we assume that U and L can be tightened enoughto disambiguate iP . Using this assumption we show that PACgreedy maximization with high probability achieves the same guar-antees as regular greedy maximization minus a constant term.Theorem 1. If F is non-negative, monotone and submodular in Xand if Assumption 1 holds then, with probability 1− δ,F (AP ) ≥ (1− e−1)F (A∗)− ε, (1)where AP is the solution returned by PAC greedy maximization,A∗ = arg maxA∈A+ F (A), δ = k(δu + δl), and ε = kε1.Proof. See [7].Our main focus is on settings where F has information-theoreticdefinitions like information gain, entropy or conditional entropy,for which computationally cheap upper and lower bounds are noteasy to find in absence of an unbiased estimator. In the sensor se-lection task, given a set of n sensors X , the aim is to find A∗ =arg maxA∈A+ [−HAb (s|z)]. Here s denotes a hidden variable, b(s)denotes the prior probability distribution over s, z = 〈z1, z2 . . . zn〉denotes the observation vector, zi stands for the observation abouts from sensor i and HAb (s|z) = Ez|b,A[HbAz (s)] is the conditionalentropy [5] of s given z, which is the expected value for the entropyHbAz (s) [5] of the posterior probability distribution bAz (s).The upper bound on F (A) = −HAb (s|z) can be obtained bysimply approximating the entropy of bAz ,HbAz (s) with its maximumlikelihood estimatesHb̂Az (s), that is, U(A) = −Ez|b,A[Hb̂Az (s)]+η [7]. The lower bound, which is rather difficult to find, is obtainedby using r = 〈r1, r2 . . . , rn〉 in place of z, which is a crude ap-proximation of z, e.g., obtained by clustering zi into d clusters de-terministically and ri denotes the cluster zi belongs to. L(A) isdefined as L(A) = −[HAb̂(s|r) + η + log(1 + 1M(ψb(s)− 1))],where ψb(s) is the support of b(s). [7] provides the proofs anddetails that show, with probability 1 − δu, U(A) ≥ F (A) andwith probability 1 − δl, L(A) ≤ F (A) with exact definitions ofδu, δl, η and M . The following theorem ties our results togetherto show that for F (A) = −HAb (s|z), PAC greedy maximizationwith above definitions of U and L, computesAP such that F (AP )has bounded error with respect to F (A∗), and achieves the sameguarantees as greedy maximization minus a constant term.Theorem 2. For the above described definitions of F , U and L, ifAssumption 1 holds and if z is conditionally independent given s,then with probability 1− δ, (1) is true.Proof. See [7].3. EXPERIMENTS & RESULTSWe evaluated PAC greedy maximization on the problem of track-ing multiple people using a multi-camera system. The problem wasextracted from a real-world dataset [1] collected in a shopping mallusing 13 CCTV cameras for over 4 hours. To evaluate a given al-gorithm, a trajectory was sampled randomly. The goal is to select asubset of cameras at each time step that induce a low-entropy beliefabout the person’s location that can be used to predict those loca-tions. As a baseline, we tested against greedy maximization whichsimply uses an approximation based on the MLE estimates of pos-terior beliefs and lazier greedy maximization [3] which, in eachiteration, samples a subset of size R from X and selects i from Rthat maximizes marginal gain.0 0.5 1 1.5 2 2.5x 104050100150200250Time taken (in seconds)Total correct predictions  Greedy maximization, M=30Lazier greedy max., M=50,R=12Lazier greedy max., M=30,R=15PAC greedy max, eps=0.37PAC greedy max, eps=0.47Figure 1: Multi-persontracking for n = 20 andk = 3.Figure 1 shows the numberof correct predictions (y-axis)against the runtime (x-axis) ofeach method. Thus, the topleft is the most desirable re-gion. PAC greedy maximizationperforms nearly as well as thegreedy maximization and laziergreedy maximization but doesso at lower computational cost leading to much better scalability.AcknowledgmentsThis research is supported by the Dutch Technology Foundation STW(project #12622) and NWO Innovational Research Incentives Scheme Veni#639.021.336. We thank Henri Bouma and TNO for providing us with thedataset used in our experiments and the STW User Committee for its usefulinput.REFERENCES[1] H. Bouma, J. Baan, S. Landsmeer, C. Kruszynski, G. van Antwerpen,and J. Dijk. Real-time tracking and fast retrieval of persons in multiplesurveillance cameras of a shopping mall. 2013.[2] P.-L. Loh and S. Nowozin. Faster hoeffding racing: Bernstein races viajackknife estimates. In Algorithmic Learning Theory, 2013.[3] B. Mirzasoleiman, A. Badanidiyuru, A. Karbasi, J. Vondrák, andA. Krause. Lazier than lazy greedy. In AAAI, 2015.[4] G. Nemhauser, L. Wolsey, and M. Fisher. An analysis ofapproximations for maximizing submodular set functions.Mathematical Programming, 1978.[5] L. Paninski. Estimation of entropy and mutual information. Neuralcomputation, 2003.[6] Y. Satsangi, S. Whiteson, and F. Oliehoek. Exploiting submodularvalue functions for faster dynamic sensor selection. In AAAI, 2015.[7] Y. Satsangi, S. Whiteson, and F. Oliehoek. Probably approximatelycorrect greedy maximization. ArXiv e-prints, Feb. 2016.http://arxiv.org/abs/1602.07860.1388

Satsangi, Y

Whiteson, S

Oliehoek, FA

UvA-DARE is a service provided by the library of the University of Amsterdam (http://dare.uva.nl)
UvA-DARE (Digital Academic Repository)
Probably Approximately Correct Greedy Maximization
Satsangi, Y.; Whiteson, S.; Oliehoek, F.A.
Published in:
AAMAS'16
Link to publication
Citation for published version (APA):
Satsangi, Y., Whiteson, S., & Oliehoek, F. A. (2016). Probably Approximately Correct Greedy Maximization. In J.
Thangarajah, K. Tuyls, C. Jonker, & S. Marsella (Eds.), AAMAS'16: proceedings of the 2016 International
Conference on Autonomous Agents & Multiagent Systems : May, 9-13, 2016, Singapore, Singapore (Vol. 2, pp.
1387-1388). International Foundation for Autonomous Agents and Multiagent Systems. http://www.aamas-
conference.org/Proceedings/aamas2016/pdfs/p1387.pdf
General rights
It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s),
other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons).
Disclaimer/Complaints regulations
If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating
your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask
the Library: https://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam,
The Netherlands. You will be contacted as soon as possible.
Download date: 09 Feb 2021
Probably Approximately Correct Greedy Maximization
(Extended Abstract)
Yash Satsangi
University of Amsterdam
y.satsangi@uva.nl
Shimon Whiteson
University of Oxford
shimon.whiteson@cs.ox.ac.uk
Frans A. Oliehoek
University of Liverpool
University of Amsterdam
frans.oliehoek@liverpool.ac.uk
ABSTRACT
Submodular function maximization finds application in a variety
of real-world decision-making problems. However, most existing
methods, based on greedy maximization, assume it is computation-
ally feasible to evaluate F , the function being maximized. Unfortu-
nately, in many realistic settings F is too expensive to evaluate ex-
actly even once. We present probably approximately correct greedy
maximization, which requires access only to cheap anytime confi-
dence bounds on F and uses them to prune elements. We show
that, with high probability, our method returns an approximately
optimal set. We propose novel, cheap confidence bounds for con-
ditional entropy, which appears in many common choices of F and
for which it is difficult to find unbiased or bounded estimates. Fi-
nally, results on a real-world dataset from a multi-camera tracking
system in a shopping mall demonstrate that our approach performs
comparably to existing methods, but at a fraction of the computa-
tional cost.
Keywords
Submodularity; Greedy maximization; Sensor scheduling; Plan-
ning.
1. INTRODUCTION
Submodularity is a property of set functions that formalizes the
notion of diminishing returns. Many real-world problems involve
maximizing submodular functions, e.g., summarizing text, image
collection summarization, or selecting sensors to minimize uncer-
tainty about a hidden variable. Formally, given a ground set X =
{1, 2 . . . n}, a set function F : 2X → R, is submodular if for every
AM ⊆ AN ⊆ X and i ∈ X \ AN , ∆F (i|AM ) ≥ ∆F (i|AN ),
where ∆F (i|A) = F (A∪i)−F (A) is the marginal gain of adding
i toA. Typically, the aim is to find anA∗ that maximizes F subject
to certain constraints. Here, we consider a constraint on A∗’s size:
A∗ = arg maxA∈A+ F (A), where A+ = {A ⊆ X : |A| ≤ k}.
As n increases, the
(
n
k
)
possibilities forA∗ grow rapidly, render-
ing naive maximization intractable. Instead, greedy maximization
finds an approximate solution AG faster by iteratively adding to
a partial solution the element that maximizes the marginal gain.
Nemhauser et. al. [4] showed that the value obtained by greedy
maximization is close to that of full maximization, i.e., F (AG) ≥
(1− e−1)F (A∗), if F is submodular, non-negative and monotone.
Appears in: Proceedings of the 15th International Conference on Au-
tonomous Agents and Multiagent Systems (AAMAS 2016),
J. Thangarajah, K. Tuyls, C. Jonker, S. Marsella (eds.),
May 9–13, 2016, Singapore.
Copyright c© 2016, International Foundation for Autonomous Agents and
Multiagent Systems (www.ifaamas.org). All rights reserved.
Most of the existing methods [3] assume it is computationally
feasible to exactly compute F , and thus the marginal gain. In many
settings, this is not the case. For example, consider a surveillance
task [6] in which an agent aims to minimise uncertainty about a hid-
den state by selecting a subset of sensors that maximise information
gain. Computing information gain is computationally expensive,
especially when the hidden variable can take many values, as it in-
volves an expectation over the entropy of posterior beliefs about the
hidden variable. When surveilling large areas like shopping malls,
exactly computing the entropy of a single posterior belief becomes
infeasible, let alone an expectation over them.
In this paper, we present a new algorithm called probably ap-
proximately correct greedy maximization. Rather than assuming
access to F itself, we assume access only to confidence bounds on
F that are cheaper to compute than F and are anytime, i.e., we can
tighten them by spending more computation time, e.g., by gener-
ating additional samples. Our method uses confidence bounds to
prune elements, thereby avoiding the need to further tighten their
bounds. We provide a PAC analysis that shows that, with high prob-
ability, our method returns an approximately optimal set.
Given an unbiased estimator of F , it is possible to use concen-
tration inequalities like Hoeffding’s inequality to obtain the con-
fidence bounds needed by PAC greedy maximization. Unfortu-
nately, many applications, such as sensor placement and decision
tree induction require information-theoretic definitions of F such
as information gain that depend on computing entropy over pos-
terior beliefs, which are impossible to estimate in unbiased way
[5]. The absence of an unbiased estimator makes it hard to obtain
computationally cheap confidence bounds on conditional entropy
[2]. Therefore, in this paper, we propose novel, cheap confidence
bounds on conditional entropy.
Finally, we apply PAC greedy maximization with these new con-
fidence bounds to a real-life dataset collected a multi-camera track-
ing system employed in a shopping mall. Our empirical results
demonstrate that our approach performs comparably to greedy and
lazier greedy maximization, but at a fraction of the computational
cost, leading to much better scalability.
2. METHOD & ANALYSIS
In this section, we propose probably approximately correct greedy
maximization. We assume U and L be set functions such that for
A ∈ A+, with probability 1 − δu, U(A) ≥ F (A) and for each
A ∈ A+, with probability 1 − δl, F (A) ≥ L(A). The main idea
is to use U and L to prune elements that with high probability do
not maximize marginal gain.
PAC greedy maximization works by initializing a partial solu-
tion AP as ∅ and adding at each iteration the element iP , which
maximizes the marginal gain for that iteration to AP until |AP |
1387
equals k. Algorithm 1 shows the main subroutine for finding iP .
Algorithm 1 maintains a queue of unpruned elements prioritized by
their upper bound. In each iteration of the outer while loop, the al-
gorithm examines each of these elements and prunes it if its upper
bound is not at least ε1 greater than the max lower bound found
so far. In addition, the element with the max lower bound is never
pruned. If an element is not pruned, then its bounds are tightened.
Algorithm 1 pac-max(U,L,AP , ε1, t)
iP ← 0 % element with max lower bound
for i ∈ X \ AP do
ρ.enqueue(i, U(AP ∪ i)) % initialize priority queue
iP ← arg maxj∈{i,iP } L(AP ∪ j)}
end for
while ρ.length() > 1 ∨ change in U & L is > t do
ρ′ ← empty queue
while ¬ρ.empty() do
i← ρ.dequeue()
if i = iP ∨ U(AP ∪ i) >= L(AP ∪ iP ) + ε1 then
tighten(AP ∪ i)
iP ← arg maxj∈{i,iP } L(AP ∪ j)
ρ′.enqueue(i, U(AP ∪ i))
end if
end while
ρ← ρ′
end while
return iP .
Assumption 1. pac-max always terminates with ρ.length() = 1.
In other words, we assume that U and L can be tightened enough
to disambiguate iP . Using this assumption we show that PAC
greedy maximization with high probability achieves the same guar-
antees as regular greedy maximization minus a constant term.
Theorem 1. If F is non-negative, monotone and submodular in X
and if Assumption 1 holds then, with probability 1− δ,
F (AP ) ≥ (1− e−1)F (A∗)− ε, (1)
where AP is the solution returned by PAC greedy maximization,
A∗ = arg maxA∈A+ F (A), δ = k(δu + δl), and ε = kε1.
Proof. See [7].
Our main focus is on settings where F has information-theoretic
definitions like information gain, entropy or conditional entropy,
for which computationally cheap upper and lower bounds are not
easy to find in absence of an unbiased estimator. In the sensor se-
lection task, given a set of n sensors X , the aim is to find A∗ =
arg maxA∈A+ [−HAb (s|z)]. Here s denotes a hidden variable, b(s)
denotes the prior probability distribution over s, z = 〈z1, z2 . . . zn〉
denotes the observation vector, zi stands for the observation about
s from sensor i and HAb (s|z) = Ez|b,A[HbAz (s)] is the conditional
entropy [5] of s given z, which is the expected value for the entropy
HbAz (s) [5] of the posterior probability distribution b
A
z (s).
The upper bound on F (A) = −HAb (s|z) can be obtained by
simply approximating the entropy of bAz ,HbAz (s) with its maximum
likelihood estimatesHb̂Az (s), that is, U(A) = −Ez|b,A[Hb̂Az (s)]+
η [7]. The lower bound, which is rather difficult to find, is obtained
by using r = 〈r1, r2 . . . , rn〉 in place of z, which is a crude ap-
proximation of z, e.g., obtained by clustering zi into d clusters de-
terministically and ri denotes the cluster zi belongs to. L(A) is
defined as L(A) = −[HA
b̂
(s|r) + η + log(1 + 1
M
(ψb(s)− 1))],
where ψb(s) is the support of b(s). [7] provides the proofs and
details that show, with probability 1 − δu, U(A) ≥ F (A) and
with probability 1 − δl, L(A) ≤ F (A) with exact definitions of
δu, δl, η and M . The following theorem ties our results together
to show that for F (A) = −HAb (s|z), PAC greedy maximization
with above definitions of U and L, computesAP such that F (AP )
has bounded error with respect to F (A∗), and achieves the same
guarantees as greedy maximization minus a constant term.
Theorem 2. For the above described definitions of F , U and L, if
Assumption 1 holds and if z is conditionally independent given s,
then with probability 1− δ, (1) is true.
Proof. See [7].
3. EXPERIMENTS & RESULTS
We evaluated PAC greedy maximization on the problem of track-
ing multiple people using a multi-camera system. The problem was
extracted from a real-world dataset [1] collected in a shopping mall
using 13 CCTV cameras for over 4 hours. To evaluate a given al-
gorithm, a trajectory was sampled randomly. The goal is to select a
subset of cameras at each time step that induce a low-entropy belief
about the person’s location that can be used to predict those loca-
tions. As a baseline, we tested against greedy maximization which
simply uses an approximation based on the MLE estimates of pos-
terior beliefs and lazier greedy maximization [3] which, in each
iteration, samples a subset of size R from X and selects i from R
that maximizes marginal gain.
0 0.5 1 1.5 2 2.5
x 10
4
0
50
100
150
200
250
Time taken (in seconds)
T
o
ta
l 
c
o
rr
e
c
t 
p
re
d
ic
ti
o
n
s
 
 
Greedy maximization, M=30
Lazier greedy max., M=50,R=12
Lazier greedy max., M=30,R=15
PAC greedy max, eps=0.37
PAC greedy max, eps=0.47
Figure 1: Multi-person
tracking for n = 20 and
k = 3.
Figure 1 shows the number
of correct predictions (y-axis)
against the runtime (x-axis) of
each method. Thus, the top
left is the most desirable re-
gion. PAC greedy maximization
performs nearly as well as the
greedy maximization and lazier
greedy maximization but does
so at lower computational cost leading to much better scalability.
Acknowledgments
This research is supported by the Dutch Technology Foundation STW
(project #12622) and NWO Innovational Research Incentives Scheme Veni
#639.021.336. We thank Henri Bouma and TNO for providing us with the
dataset used in our experiments and the STW User Committee for its useful
input.
REFERENCES
[1] H. Bouma, J. Baan, S. Landsmeer, C. Kruszynski, G. van Antwerpen,
and J. Dijk. Real-time tracking and fast retrieval of persons in multiple
surveillance cameras of a shopping mall. 2013.
[2] P.-L. Loh and S. Nowozin. Faster hoeffding racing: Bernstein races via
jackknife estimates. In Algorithmic Learning Theory, 2013.
[3] B. Mirzasoleiman, A. Badanidiyuru, A. Karbasi, J. Vondrák, and
A. Krause. Lazier than lazy greedy. In AAAI, 2015.
[4] G. Nemhauser, L. Wolsey, and M. Fisher. An analysis of
approximations for maximizing submodular set functions.
Mathematical Programming, 1978.
[5] L. Paninski. Estimation of entropy and mutual information. Neural
computation, 2003.
[6] Y. Satsangi, S. Whiteson, and F. Oliehoek. Exploiting submodular
value functions for faster dynamic sensor selection. In AAAI, 2015.
[7] Y. Satsangi, S. Whiteson, and F. Oliehoek. Probably approximately
correct greedy maximization. ArXiv e-prints, Feb. 2016.
http://arxiv.org/abs/1602.07860.
1388


UvA-DARE

https://pure.uva.nl/ws/files/42910840/p1387_satsangi.pdf

Probably Approximately Correct Greedy Maximization

Abstract

Similar works

Full text

Available Versions

International Migration, Integration and Social Cohesion online publications

University of Liverpool Repository

Search4Dev

University of Liverpool Repository

International Migration, Integration and Social Cohesion online publications

University of Liverpool Repository

Search4Dev

UvA-DARE