This paper wants to investigate the deepest meaning of the word class that is often used in machine learning and classification as a well-defined concept. This adventure will lead the reader to the fundamentals of Mathematics like set theory from Zermelo-Fraenkel. This will be our start, like is all Mathematics, to understand how well defined is the class concept. A broader theory will be outlined with the courageous attempt to give an homogenous framework to deal with machine learning problems

Chiancone, Alessandro

Hal - Université Grenoble Alpes

What’s wrong with classes? The theory of KnowledgeAlessandro ChianconeTo cite this version:Alessandro Chiancone. What’s wrong with classes? The theory of Knowledge. [ResearchReport] GIPSA-lab. 2015. <hal-01113112v2>HAL Id: hal-01113112https://hal.archives-ouvertes.fr/hal-01113112v2Submitted on 30 Mar 2016HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestine´e au de´poˆt et a` la diffusion de documentsscientifiques de niveau recherche, publie´s ou non,e´manant des e´tablissements d’enseignement et derecherche franc¸ais ou e´trangers, des laboratoirespublics ou prive´s.What’s wrong with classes? The theory of KnowledgeAlessandro Chiancone,1,2,3,4∗1Rome,Italy2Laboratoire Jean Kuntzmann & INRIA Rhone-Alpes, team Mistis,Inovallee, 655, av. de l’Europe, Montbonnot, 38334 Saint-Ismier cedex, France3GIPSA-Lab, Grenoble INP,Saint Martin d’Heres, France4Institute of Statistics Graz University of TechnologyKopernikusgasse 24/III A-8010 Graz, Austria∗Alessandro Chiancone; E-mail: al.chiancone@gmail.com.12 February 2016AbstractThis paper wants to investigate the deepest meaning of the word class that is often usedin machine learning and classification as a well-defined concept. This adventure will leadthe reader to the fundamentals of Mathematics like set theory from Zermelo-Fraenkel. Thiswill be our start, like is all Mathematics, to understand how well defined is the class con-cept. A broader theory will be outlined with the courageous attempt to give an homogenousframework to deal with machine learning problems.IntroductionYes, I need to cite all of sudden the paper that inspired the title [Blaschke and Strobl, 2001],this was the first time that I started thinking about classification. Thanks to this paper I tried to1clear my mind on some problems affecting image processing. But, what’s wrong with classes?I would say nothing is wrong by default, we just need to be sure to agree on the same definition.Machine learning has been recently applied to face complex problems and I strongly believethat image processing (remote sensing, medical imaging,...) will be in the near future, muchmore than now, a powerful tool to improve our quality of life. To make future the present, Ithink, we need to deeply analyze what classification is, in general. I would say that a lot ofeffort of human kind is represented by classification results. For example, DNA could be seen(also) as a very good feature to overcome the incredible work of the Swedish Linnaeus. But whydo we need to classify things? Humans developed a powerful tool, the language (spoken andwritten), that is a way to give speed to communication and something much more important.The main difference between animal language and human language, I think, is the possibilitythat gives to refer to facts that are not happening in the very exact moment of communication.Past and Future. This is a beautiful idea (dangerous to control sometime, not being anymorecapable of enjoying moments) that gave us the possibility to build what we call Knowledge.This paper wants to underline, explore and try to catch which are the axioms of our language,that are coming with strengths and limitations.1 A good example to start: ColorsThanks to internet some years ago I had the luck to watch several videos talking about colors.What is a color? It is clearly a class, since spectra of light is continuos (and oriented), colorscan be seen as a regression problem with the function f : R → A ( N. Being, e.g., A =1, 2, 3, ..., 230 like the HDMI specification 1.073 billion colors. A good question is: do we have1.073 billion words for colors in our languages? No. As Raymond Queneau reported in thepreface of Cent Mille Milliards des Poemes citing Alan Turing: Seul une machine peut aprecierun sonnet ecrit par une autre machine (more on this can be found in [Jefferson, 1949]). This2means that we are really not capable, or just do not want because of the small benefit, to inventsuch an enormous number of word for just colors. Our memory is limited after all. In thisexample we can see how language is the attempt to describe a continuos variable (which in ourexample is the spectra of light) with a discrete number of classes. A good question is: is itpossible, in theory, to give a word to each point in the spectra? This is a very good questionand we should be thankful to Cantor if we can give the answer. Which is No. The problemis that even if we suppose our language to have an infinity of words (crazy assumption, but Ihave a degree in Mathematics, forgive me) there is no way to make correspond points in the linewith the infinite words created. The cardinality of the two infinities is different and the one oflanguage is ℵ0 Aleph-null, the smallest infinite cardinal number (thank you Claudio Bernardifor the beautiful course of fundamentals of Mathematics). I now dare to define social sciences,in the common meaning, as the brave attempt to describe an infinity of higher cardinality withthe one of language that, even at his full-capability, will be ℵ0. To sum up we need to be careful,the world we want to describe is much wider that we may think, and our tool, which is language,is limited. Sometime is hard to fully agree on colors. If we imagine colors as intervals on thespectra the closer we are to the limits the more uncertainty is the interval to which we will assignthe color: is a shade of Blue or Green?2 Empathy for the MachineWhen I read this paper [Blaschke and Strobl, 2001] I started working on object-based classifi-cation. We proposed a hierarchy of segmentation to insert spatial information in classification[Chini et al., 2014b]. What I was trying to do at that time was an honest attempt to improve thestate of the art which was already full of interesting papers [Dalla Mura et al., 2010; Aytekinand Ulusoy, 2011; Tarabalka et al., 2012; Chini et al., 2014a,b]. In remote sensing, almost everytime, the ground truth is plotted by hand on the image and then a percentage of pixels in the3different classes are used to train the classifier, the rest is used for validation. The question is:is a per-pixel classifier watching the same image we are watching? Well, of course yes, but fora per-pixel classifier the following two images (Fig.1, Fig 2), once the training set is given, areexactly the same. When we look at the result we are usually disappointed, the so called saltand pepper effect is affecting our solution. But, if we start from the Fig. 2, the same problembecomes more challenging. If we empathize with the machine, and remember what Touringsaid, we should understand that the only problem is language. The solution of the per-pixelclassifier is not expressed in our language. What are we missing then? Knowledge3 Calvino and the Theory of KnowledgeAt the beginning of my Master’s thesis, back in Rome, I cited a beautiful passage of an in-credible book which is Le citta` Invisibili, probably the best book I have ever read so far onclassification and much more. The book is organized in cities, and the city which opened mymind was Zoe. What is really Knowledge? Knowledge, in my opinion, is what we agreed on,as a society. It has not general value but only relative, there are many different kinds of Knowl-edge and, among all societies, our personal ideas developed on what we agreed on is Culture.Different cultures are arising from what I will call Knowledgediversity. Now I need the helpof Mathematics, the most self-aware of the languages, to define Knowledge.DefinitionKnowledge K is the set of all functions from X to Y . Let X be the Information set and Y theset of Representations (the fruit of Knowledge).RemarksWe can look atX as the set of Information we filter from real World through our senses (sensors4if machines), Y will be the set of notions, ideas, theories... The fruit of Knowledge. Historytaught us that if we fix y ∈ Y : ⋂f∈Kf−1(y) = ∅ (1)i.e. the same notions (ideas, theories, etc..) can arise from different sets of information. Agri-culture was developed independently by at least three civilizations. It is in the end reasonableto think that there is no surjective f ∈ K.Communication is the first reason to develop a language, Bernard Shaw once said: The singlebiggest problem of communication is the illusion that is has taken place. Suppose that we havetwo entities A and B and let us denote with XA and XB as the information that they filter fromreal World. And let us suppose that YA = YB, in other words the set of Representations arethe same (i.e. they speak the same language). If we fix y ∈ Y and denote XyA = f−1A (y) andXyB = f−1B (y) as the set of information that are sent in the same representation y:XyA⋂XyB 6= ∅ and XyA 6= XyB. (2)To make it clear suppose that y = chair, it is sure that the intersection of XyA and XyB when theword is pronounced is non zero but it is also reasonable to think that there will be somethingwhich is considered chair by only one of the entities and viceversa. What Shaw was tryingto say, I believe, is that often we exchange only representations and rearely try to estimatefunctions (f−1B , f−1A ) capable of revealing XyB to the entity A and XyA to B. I like to define loveas the deepest way to Knowledge, what I believe we do when we are in love is that we try toovercome the set of representations and connect directly to the set of information that the otheris filtering from reality, having indeed access to an infinity of cardinality higher that ℵ0.I want to thank professor Paolo Piccinni, the most generous of the professors I had at uni-versity. He introduced us to the wildlife of Topology. In my opinion the most imaginative partof Mathematics, rarely studied. The pearls of this theory are classification results (e.g. Classi-5fication theorem of closed surfaces), going deep to the fundamental properties of space. Howto remove uninteresting properties of the space ending up selecting the best features to distin-guish them? Functions. Homeomorphisms in particular. In this theory, that can be seen as thehighest point of human classification, features are called topological invariants. Thanks to thisdefinition and the work of many Mathematicians we can see that what is important, most dis-criminating, are things that even if something changes (also dramatically) do not change. Thoseare the features we are looking for to solve a specific classification problem.4 Knowledge vs InformationIn machine learning, classifiers can be seen as specific functions, as topology teaches we ratherneed a set of functions (e.g. Homeomorphism) to properly solve a classification problem. Themassive information we have from a series of sensors that nowadays are much better than hu-mans in collecting data is amazing. What we should do, I think, is to be aware that Informationis potentially Knowledge, it will be actual Knowledge once we agree, once we classify it. InGrenoble, where I am doing my PhD, I had the chance to attend lectures by E. Cande`s, hisbeautiful theory of Compressive Sensing is, in my opinion, so impressive because he is tryingto solve a problem equipped with the correct Knowledge (i.e the set of norm functions). Ma-chine learning community should be inspired by this work and realize that once we are properlyequipped with the right Knowledge we can solve many problems in an elegant way, like Cande`sdid. Once we are aware of what we are asking to the machine, and once we fully give the correctset of functions, then we can be sure that the result will reflect our language. Features should bedefined as the invariant under the set of functions we are considering, those are then related tothe choice of Knowledge and not the other way around (trying many different kinds of featureswill confuse more a poor classifier).65 ConclusionsEach language is coming with strength and limitations, axioms on which we base our theoriescan be seen as an orientation of the language. Before solving problems one should be surethat the starting direction will develop a language capable to describe and hopefully solve thoseproblems. I strongly believe that Knowledgediversity is the most important thing to fight forin this world. The set K is not ordered (i.e there is NO element in this set better than others).Machine learning will soon help us to have more time to think. If for a moment we assume thereal world being described with R, then let us consider as Q the set of our language. We knowthat if we want to select a point r ∈ R \Q in our line then we can go really close i.e. ∀∃q ∈ Qs.t. r = q+ . Even if we can come arbitrarily close to point r we will never get there, I believethat the role of science is to give humans time to fill this infinitely small, but always presentgap. This research I call it happiness. Happiness is my axiom.AcknowledgmentsI would like, once again, to abuse of Mathematics for my personal benefit. Let B be the set ofall beautiful things in the universe and let S be the set of Stupidity then:A.Chiancone = B ∪ S (3)I only hope that card(S) = ℵ0!Among this beauty there are people who helped me in my life, the ones I call my family.Thank you each and all of you, I will never forget that.7Academic AcknowledgmentsMichele Emmer was for me the ambassador of beauty in the Department of Mathematics GuidoCastelnuovo, Rome. I will always be grateful for his passion. I would like to say thank you firstto Marco Chini (who is now part of my family), he believed in me even in the darkest momentsgiving me the possibility to be happy. I would like to say thank you to Jocelyn Chanussot thatgave me the possibility to win the scholarship that gifted me the time to think. Dulcis in fundo,I would like to thank Mistis Team (P. Mesejo, first reader of this paper, proof-reader and strongbeliever, A. Arnaud, F. De Coninck, J-B. Durand, F. Forbes, C. Loquet, G. Mazo, T. Perret, A.Studeny, N. Sylla, S. Torres) and Stephane Girard, my supervisor. He is just the best possibleguide I can ever image in the wildlife of Statistics. He helped me to manage my, sometime,too strong emotions. I deeply thank professor Enric Trillas for the interesting conversations andconsideration.This work has been partially supported by the LabEx PERSYVAL-Lab (ANR-11-LABX-0025).IN VARIETATE CONCORDIAReferencesO¨rsan Aytekin and I˙lkay Ulusoy. Automatic segmentation of vhr images using type informationof local structures acquired by mathematical morphology. Pattern Recognition Letters, 32(13):1618–1625, 2011.Thomas Blaschke and Josef Strobl. What’s wrong with pixels? some recent developmentsinterfacing remote sensing and gis. GeoBIT/GIS, 6(01):12–17, 2001.Marco Chini, Christian Bignami, Alessandro Chiancone, and Salvatore Stramondo. Classifica-tion of vhr optical data for land use change analysis by scale object seletion (sos) algorithm.8In Geoscience and Remote Sensing Symposium (IGARSS), 2014 IEEE International, pages2834–2837. IEEE, 2014a.Marco Chini, Alessandro Chiancone, and Salvatore Stramondo. Scale object selection (sos)through a hierarchical segmentation by a multi-spectral per-pixel classification. PatternRecognition Letters, 49:214–223, 2014b.Mauro Dalla Mura, Jo´n Atli Benediktsson, Bjo¨rn Waske, and Lorenzo Bruzzone. Morphologi-cal attribute profiles for the analysis of very high resolution images. Geoscience and RemoteSensing, IEEE Transactions on, 48(10):3747–3762, 2010.Geoffrey Jefferson. The mind of mechanical man. British Medical Journal, 1(4616):1105,1949.Yuliya Tarabalka, James C Tilton, Jo´n Atli Benediktsson, and Jocelyn Chanussot. A marker-based approach for the automated selection of a single segmentation from a hierarchical setof image segmentations. Selected Topics in Applied Earth Observations and Remote Sensing,IEEE Journal of, 5(1):262–272, 2012.9Fig. 1. From the right to the left my mom Aurora, my dad Marco and Camillo.Fig. 2. Permuted version of Fig.110Figure 1: From the left to the right my mom Aurora, my dad Marco and Camillo11Figure 2: Permuted version of Fig.112

What's wrong with classes? The theory of Knowledge

Abstract

Similar works

Full text

Available Versions

Hal - Université Grenoble Alpes