Gretton, A

Poczos, B

Sriperumbudur, B

Szabo, Z

UCL Discovery

Optimal Regression on SetsZolta´n Szabo´1, Bharath K. Sriperumbudur2, Barnaba´s Po´czos3, Arthur Gretton1 (1GatsbyUnit, UCL; 2Department of Statistics, PSU; 3Machine Learning Department, CMU).I Example: aerosol prediction ⇒ regression on labelled bags.I Bag-of-objects representation: experimental design, computer vision, NLP, networks.I Problem formulation:◦ Labelled training bags: zˆ = {(Pˆi , yi)}`i=1. Test bag: Pˆ . N := |Pˆi |.◦ Estimator, prediction:f λzˆ = arg minf ∈H(K )1`∑`i=1[f(µPˆi︸︷︷︸feature of Pˆi)− yi]2 + λ ‖f ‖2H ,yˆ(Pˆ)= gT (G + `λI)−1y, g =[K(µPˆ , µPˆi)], G =[K(µPˆi , µPˆj)], y = [yi ].◦ Similarity of distributions: K (P ,Q) = Ea∼P,b∼Qk(a, b) =〈Eaϕ(a)︸ ︷︷ ︸feature of distribution P=:µP,Ebϕ(b)〉.◦ Quality of estimator: R(f ) = E(µP ,y)∼ρ[f (µP)− y ]2, fρ = best regressor.I Known: assuming fρ ∈ H◦ best/achieved rate: R(f λz )−R(fρ) = O(`−bcbc+1),◦ b – size of the input space, c – smoothness of fρ.I Our result:◦ With N = O˜(`a) and 2 ≤ a, f λzˆ has the best achievable rate; a = b(c+1)bc+1 < 2 : X◦ ⇒ Regression with set kernel is consistent (17-year-old open problem).◦ Practical: state-of-the-art aerosol prediction.I Details (JMLR submission), code (in ITE):http://arxiv.org/abs/1411.2066, https://bitbucket.org/szzoli/ite/I Acknowledgements: This work was supported by the Gatsby Charitable Foundation, andby NSF grants IIS1247658 and IIS1250350. The work was carried out while Bharath K.Sriperumbudur was a research fellow in the Statistical Laboratory, Department of PureMathematics and Mathematical Statistics at the University of Cambridge, UK.

Optimal Regression on Sets

http://discovery.ucl.ac.uk/1497169/1/Zoltan_Szabo_invited_poster_eResearch_Domain_Launch_event_29_06_2016.pdf

Optimal Regression on Sets

Abstract

Similar works

Full text

Available Versions

UCL Discovery