© ACM 2015. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM SIGKDD Explorations Newsletter http://dx.doi.org/10.1145/2830544.2830551The first international workshop on Learning over Multiple Contexts,
devoted to generalization and reuse of machine learning models
over multiple contexts, was held on September 19th, 2014, as
part of the 7th European machine learning and data mining conference
(ECML-PKDD 2014) in Nancy, France. This short report
summarizes the presentations and discussions held during the
LMCE 2014 workshop, as well as the workshop conclusions and
the future agenda.Ferri Ramírez, C.; Flach, P.; Lachiche, N. (2015). Report of the First International Workshop on Learning over Multiple Contexts (LMCE 2014). ACM SIGKDD Explorations Newsletter. 17(1):48-50. doi:10.1145/2830544.2830551S485017

Ferri Ramírez, César

Flach, Peter

Lachiche, Nicolas

Spanish

The first international workshop on Learning over Multiple Contexts, devoted to generalization and reuse of machine learning models over multiple contexts, was held on September 19th, 2014, as part of the 7th European machine learning and data mining conference (ECML-PKDD 2014) in Nancy, France. This short report summarizes the presentations and discussions held during the LMCE 2014 workshop, as well as the workshop conclusions and nthe future agenda.</jats:p

Cèsar Ferri

Peter Flach

Nicolas Lachiche

Crossref

ACM SIGKDD Explorations Newsletter

Report of the First International Workshop on Learning over Multiple Contexts (LMCE 2014)

© ACM 2015. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM SIGKDD Explorations Newsletter http://dx.doi.org/10.1145/2830544.2830551The first international workshop on Learning over Multiple Contexts,
devoted to generalization and reuse of machine learning models
over multiple contexts, was held on September 19th, 2014, as
part of the 7th European machine learning and data mining conference
(ECML-PKDD 2014) in Nancy, France. This short report
summarizes the presentations and discussions held during the
LMCE 2014 workshop, as well as the workshop conclusions and
the future agenda.Ferri Ramírez, C.; Flach, P.; Lachiche, N. (2015). Report of the First International Workshop on Learning over Multiple Contexts (LMCE 2014). ACM SIGKDD Explorations Newsletter. 17(1):48-50. doi:10.1145/2830544.2830551485017

RiuNet

 Document downloaded from:  This paper must be cited as:                  The final publication is available at   Copyright  Additional Information  © ACM 2015. This is the author's version of the work. It is posted here for your personal use.Not for redistribution. The definitive Version of Record was published in ACM SIGKDDExplorations Newsletter http://dx.doi.org/10.1145/2830544.2830551http://dx.doi.org/10.1145/2830544.2830551http://hdl.handle.net/10251/67661ACMFerri Ramírez, C.; Flach, P.; Lachiche, N. (2015). Report of the First International Workshopon Learning over Multiple Contexts (LMCE 2014). ACM SIGKDD Explorations Newsletter.17(1):48-50. doi:10.1145/2830544.2830551.Report of the First International Workshop onLearning over Multiple Contexts (LMCE 2014)Cèsar FerriUniv. Politèc. de ValènciaCami de Vera, S/N 46022València, Spaincferri@dsic.upv.esPeter FlachUniversity of BristolWoodland RoadBristol BS8 1UB, U.K.peter.flach@bristol.ac.ukNicolas LachicheICube, Univ. Strasbourg300 bd Brant67400 Illkirch, Francenicolas.lachiche@unistra.frABSTRACTThe first international workshop on Learning over Multiple Con-texts, devoted to generalization and reuse of machine learning mod-els over multiple contexts, was held on September 19th, 2014, aspart of the 7th European machine learning and data mining con-ference (ECML-PKDD 2014) in Nancy, France. This short re-port summarizes the presentations and discussions held during theLMCE 2014 workshop, as well as the workshop conclusions andthe future agenda.1. INTRODUCTION AND MOTIVATIONAdaptive reuse of learnt knowledge is of critical importance inthe majority of knowledge-intensive application areas, particularlywhen the context in which the learnt model operates can be ex-pected to vary from training to deployment. In data mining, theextracted knowledge will generally be used in a new deploymentcontext with different characteristics. Additionally the model mightneed adaptation, for example if there are manifestations of datashift or concept drift. This has been studied also in machine learn-ing, for example, in relation to variations in class and cost skew in(binary) classification, leading to the development of tools such asROC analysis for adjusting decision thresholds to operating condi-tions concerning class and cost skew. More recently, considerableeffort has been devoted to research on transfer learning, domainadaptation, and related approaches.Given that the main business of predictive models is to generalisefrom the training context to the deployment context, there is clearlyscope for developing a general notion of operating context. With-out such a notion, a model predicting ‘sales in Prague for this week’may perform poorly in Nancy for next Wednesday. The operat-ing context has changed in terms of location as well as resolution.While a given predictive model may be sufficient and highly spe-cialised for one particular operating context, it may not performwell in other contexts. If sufficient training data for the new con-text is available it might be feasible to retrain a new model; how-ever, this is generally not a good use of resources, and one wouldexpect it to be more cost-effective to learn one general, versatilemodel that effectively generalizes over multiple contexts.The aim of this workshop was to bring together people workingin areas related to versatile models and model reuse over multiplecontexts. Given the advances made in recent years on specific ap-proaches such as transfer learning, an attempt to start developingan overarching theory is now feasible and timely, and could be ex-pected to generate considerable interest from the machine learningand data mining communities. Papers were solicited in all areasrelating to model reuse and model generalization including the fol-lowing areas:• Transfer learning• Data shift and concept drift• Domain adaptation• Transductive learning• Multi-task learning• ROC analysis and cost-sensitive learning• Background knowledge, relational learning• Context-aware applications• Incomplete information, abduction• Meta-learningThe program committee was selected to cover all those areas, with18 members from 8 countries: Brazil, China, France, Japan, Portu-gal, Spain, UK and USA.1The workshop was held at the European machine learning and datamining conference in order to gather specialists from all relevantdisciplines. We received submissions from 10 countries: Austria,Bangladesh, Belgium, Brazil, France, Germany, The Netherlands,Romania, Spain and UK. Each paper was reviewed by at least 2program committee members. 13 regular papers and 3 work-in-progress papers were accepted. Around 40 people participated inthe day-long workshop.1Chowdhury Farhan Ahmed, Univ Strasbourg, FranceCharles Elkan, Univ California - San Diego, USACèsar Ferri, Univ Politècnica de València, SpainPeter Flach, Univ Bristol, UKAmaury Habrard, Univ Jean Monnet, Saint-Etienne, FranceFrancisco Herrera, Univ Granada, SpainMeelis Kull, Univ Bristol, UKNicolas Lachiche, Univ Strasbourg, FranceDragos Margineantu, Boeing Research, USAAdolfo Martı́nez-Usó, Univ Politècnica de València, SpainWeike Pan, Shenzhen U, ChinaRicardo Prudencio, Univ Pernambuco, BrazilJoaquin Quiñonero, Facebook, USAMarı́a José Ramı́rez-Quintana, Univ Politècnica de València, SpainCarlos Soares, Univ Porto, PortugalMasashi Sugiyama, Tokyo Institute of Technology, JapanBianca Zadrozny, Univ Fluminense, BrazilHuimin Zhao, Univ Wisconsin - Milwaukee, USA2. ORGANISATION AND SESSIONSThe technical program of LMCE 2014 consisted of two invitedtalks and five sessions devoted to a total of 16 contributed papers.2.1 Invited talksThe workshop started with an invited talk by José Hernández-Orallo(Universitat Politècnica de València, Spain) on Context Change andVersatile Models in Machine Learning. Noting that models are of-ten deployed in a different context than training, the speaker listedmany types of contexts and proposed an explicitation of contextswith the corresponding adaptation procedures. His presentationemphasized the importance and usefulness of versatile models thatcan be adapted to different contexts, in an approach called refram-ing, with several examples.Antonio M. Lopez (Universitat Autònoma de Barcelona, Spain)gave the second invited talk to start the afternoon sessions. His pre-sentation entitled Domain Adaptation of Virtual and Real Worldsfor Pedestrian Detection concerned autonomous driving, and fo-cused on pedestrian (object) detection. Since labelling examplesin images is time-consuming and error-prone, he advocated the useof image synthesis to generate examples. Since going from thetraining data to deployment data requires domain adaptation, hesuggested to use active learning to collect a few examples real ex-amples to add to the many virtual examples. The speaker presentedmany results on real datasets.2.2 Cost-sensitive learningThe first regular session dealt with cost-sensitive learning.Ricardo Prudencio presented the first paper, entitled Cost-sensitivemeasures of instance hardness, by Carlos Melo and himself. Heconsidered the cost of misclassifying an instance, averaged overseveral learning algorithms, with respect to the cost proportion, un-der three decision threshold choice methods: score-fixed, score-driven, and rate driven.The second paper of the session was Classification in Context:Adapting to changes in class and cost distribution by Peter A. Flach.The talk presented known as well as new results on ROC curves andcalibration, in particular with respect to F-measure.Wouter Duivesteijn presented the third paper ROCsearch in a WiderContext – A ROC-Guided Search Strategy for Subgroup Discov-ery and Beyond by Wouter Duivesteijn, Marvin Meeng and ArnoKnobbe. Based on previous work on subgroup discovery publishedat the 2014 SIAM International Conference on Data Mining, heintroduced perspectives for continuous-target subgroup discoveryand for exceptional model mining.2.3 Dataset shiftMeelis Kull presented the first paper, Patterns of Dataset Shift byMeelis Kull and Peter Flach. He described a graph notation formodelling different types of dataset shifts, covering known patternsand introducing new patterns as well.The second paper of the session was Dataset Shift in a Real-LifeDataset by Chowdhury Farhan Ahmed, Nicolas Lachiche, ClémentCharnay and Agnès Braud. Chowdhury Farhan Ahmed demon-strated that the UCI dataset ‘Washington Bike Sharing’ exhibits arange of shifts on input as well as output attributes, depending onthe seasons, and suggested to use such dataset to compare existingapproaches.Chowdhury Farhan Ahmed also presented the third paper of thissession, Efficient Graph Classification in Shifted Datasets UsingWeighted Correlated Feature Selection by Md. Samiullah, Chowd-hury Farhan Ahmed, Anna Fariha and Akiz Uddin Ahmed. Theauthors proposed a correlation-based graph classification approachas well as a new diversity capturing measure.2.4 Multi-instance, multi-dimensional, multi-task and multi-label learningThe third session gathered four papers that generalise the standardclassification setting in some way.The first speaker was Gitte Vanwinckelen, presenting her work withHendrik Blockeel, A meta-learning system for multi-instance clas-sification. They evaluated 14 multiple-instance learning techniqueson three dataset domains in order to learn a meta-model. Some in-teresting conclusions were presented, although they need a largermeta-dataset to generalise.Adolfo Martı́nez-Usó presented Predictive models for multidimen-sional data when the resolution context changes by José Hernández-Orallo, Nicolas Lachiche and himself. They considered differentaggregation levels in multidimensional data cubes and comparedtwo learning approaches on three datasets.The third paper of the session was SymReg-MT: Iterative Multi-task Feature Learning Through Weighted Symbolic Regression byMichael Zwick, Holger Schoner and Ehsan Rezaie. The authorspresented an approach to multi-task feature learning that uses weight-ing functions based on the frequency of occurrences when lookingat the models across different tasks.The last paper of this session, Multi-label Classification: A Com-parative Study on Threshold Selection Methods by Reem Al-Otaibi,Peter Flach and Meelis Kull was presented by the first author. Sheevaluated the performance on six datasets of three different meth-ods to set the thresholds: global, label-wise and instance-wise.2.5 Transfer learningThe session started with the paper Multi-System Identification forEfficient Knowledge Transfer with Factored Tensor Recurrent Neu-ral Networks by Sigurd Spieckermann, Siegmund Dull, Steffen Ud-luft and Thomas Runkler. They proposed a transfer learning ap-proach through parameter transfer to deal with similar dynamicalsystems and evaluated it on two problems.Christian Politz presented his paper Subset based Hilbert spaceprojections for transfer learning. He proposed two greedy ap-proaches for selecting data samples from the source in order to im-prove the transfer learning (using a kernel PCA) when target dataand source data follow different distributions.The third paper was Sequential Decision-Making under Non-statio-nary Environments via Sequential Change-point Detection by Em-manuel Hadoux, Aurelie Beynier and Paul Weng. The authors con-sidered reinforcement learning using context detection, defined bya drop of quality. They proposed a new algorithm and evaluated iton three domains.2.6 ApplicationsThe final session of the day concerned applications.Niall Twomey presented the first paper of this session, about a workwith Peter Flach on Context Modulation of Sensor Data Applied toActivity Recognition in Smart Homes. They evaluated whether dif-ferent subsets of sensors lead to different performances and foundthat light/door sensors are less valuable than motion sensors on thedataset they considered.The second paper, Identifying dominant models when the noisecontext is known by Cèsar Ferri, José Hernández-Orallo, AdolfoMartı́nez-Usó and Marı́a José Ramı́rez-Quintana was presented bythe last author. This talk was focussed on an experimental studyof several regression algorithms according to the noise level. As-suming that the noise level is known would enable to select the bestmodel.The last paper was A L2-Norm Regularized Pseudo-Code for ChangeAnalysis in Satellite Image Time by Anamaria Radoi and MihaiDatcu. The authors proposed and evaluated a new clustering ap-proach to detect changes in a flow of satellite images.3. CONCLUSION AND DISCUSSIONThe workshop concluded with the general impression that it suc-ceeded as a first workshop on learning models over multiple con-texts. First, it was a success mainly because of the number andquality of submissions. Secondly, contributions and participantscovered a broad range of related areas. Thirdly, there was a broadconsensus that those areas are related and that further efforts to clar-ify their relationships, similarities and differences would be pro-ductive.The open discussion centred around three issues: 1) domain speci-ficities, 2) support for comparing approaches, and 3) continuation.First, participants noticed that some approaches are domain-specific.For example, domain adaptation often deals with images. It wouldbe worth investigating some cross-over, evaluating approaches ofone domain on another application domain. It would need pre-processing data, eg. images, to get a common representation, eg.attribute-value.Support for sharing data, algorithms and results would help collab-orations. Web platforms such as OpenML.org or ClowdFlows.org were suggested. Another way to promote the comparison ofapproaches would be to set up a challenge for the next workshop.Finally, there was a consensus that it would be worthwhile to con-tinue with the workshop. Most people also thought that it shouldstill be associated to a big data mining or machine learning con-ference: KDD or ECML-PKDD 2015. The possibility of a journalspecial issue devoted to the topic of model reuse and context changewas also considered, possibly after more authors have had a chanceto present their work at the second workshop.AcknowledgementsThe workshop was sponsored by the REFRAME project (http://reframe-d2k.org) and the data mining team of the ICubelaboratory (http://icube-bfo.unistra.fr/en/). Fur-ther information about the workshop, contributed full papers andpresentations can be found at http://users.dsic.upv.es/˜flip/LMCE2014/.

https://riunet.upv.es/bitstream/10251/67661/2/reportLMCE2014final.pdf

Report of the First International Workshop on Learning over Multiple Contexts (LMCE 2014)

Abstract

Similar works

Full text

Available Versions

Crossref

RiuNet