Atkinson, Malcolm

Brezany, Peter

Corcho, Oscar

Han, Liangxiu

Hluchy, Ladislav

Hume, Ally

Janciak, Ivan

Krause, Amrey

Snelling, David

van Hemert, Jano

Wohrer, Alex

This is an extended abstract, so please see the document attache

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      Advanced Data Mining and Integration Research for EuropeCitation for published version:Atkinson, M, Brezany, P, Corcho, O, Han, L, van Hemert, J, Hluchy, L, Hume, A, Janciak, I, Krause, A,Snelling, D & Wohrer, A 2009, Advanced Data Mining and Integration Research for Europe. in UK e-Science All Hands Meeting: December 2009, Oxford.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Peer reviewed versionPublished In:UK e-Science All Hands MeetingGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 05. Apr. 2019Advanced Data Mining and Integration Research for EuropeMalcolm Atkinson1, Peter Brezany2, Oscar Corcho3, Liangxiu Han1, Jano van Hemert1,∗,Ladislav Hluchy´4, Ally Hume5, Ivan Janciak2, Amy Krause5, Dave Snelling6, Alex Wo¨hrer21National e-Science Centre, School of Informatics, University of Edinburgh, United Kingdom2Institute of Scientific Computing, Faculty of Computer Science, University of Vienna, Austria3Departamento de Inteligencia Artificial, Facultad de Informa´tica, Universidad Polite´cnica de Madrid, Spain4Institute of Informatics, Slovak Academy of Sciences (IISAS), Bratislava, Slovakia5EPCC, University of Edinburgh, United Kingdom6Fujitsu Laboratories Europe Limited, United Kingdom∗Corresponding author: j.vanhemert@ed.ac.ukMotivation: There is a rapidly growing wealth of data [1]. The number of sources of data is increasing,while, at the same time, the diversity, complexity and scale of these data resources are also increasingdramatically. This cornucopia of data offers much potential; a combinatorial explosion of opportunitiesfor knowledge discovery, improved decisions and better policies. Today, most of these opportunities arenot realised because composing data from multiple sources and extracting information is too difficult.Every business, organisation and government faces problems that can only be addressed successfully ifwe improve our techniques for exploiting the data we gather.Strategy: We report the rationale for a new architecture, DMI architecture, for combined data inte-gration and data mining under development in the Advanced Data Mining and Integration Research forEurope (ADMIRE) project. The DMI architecture is intended to enable society to make better use ofthe rapidly expanding wealth of data. The proposed DMI architecture must make all of the stages ofDMI process development and enactment as identified in [2] easier and more economic.The ADMIRE project aims to significantly improve the exploitation of data by delivering three cate-gories of output: a framework, an architecture and a set of use cases that illustrate how they can be usedto improve DMI. These will be built on a consistent set of principles:• The scale and complexity of each data source grows. The DMI architecture addresses this with data-flow technology to reduce data handling and to move data reduction and transformation operationscloser to data sources.• The number and variety of independent data sources increases; warehousing and virtualisation becomeinfeasible at the envisaged scale, which we address by proposing dynamic composition of processes.• The computational complexity of extracting information grows as a result of the above and of increas-ingly sophisticated application requirements. The DMI architecture addresses this by enabling the workof data-aware distributed computing (DADC) engineers and by supporting the incremental definitionand revision of libraries and patterns.• The number of application domains using DMI grows, becomes more diverse and engages more users.The DMI architecture addresses this by recognising communities of users, by supporting them withtheir own environments and by delivering packaged production versions of DMI processes.• The number of experts involved in developing new DMI processes and supporting their applicationgrows. The DMI architecture addresses this by separating support for DMI experts from that forDADC engineers and application-domain users. Support for communities with aligned DMI interests isachieved by sharing between DMI-developers’ workbenches via a common registry for their community.• The number of providers of data and DMI services grows. The DMI architecture separates the organ-isation of environments for DMI-process development from the complexities of DMI-service provisionby interposing DMI gateways using a canonical language.• The growing sophistication of information extraction from large bodies of data requires ever morecomplex and refined workflows. The DMI architecture addresses this by structuring collections ofcomponents into libraries that correspond to a conceptual structure captured in DMI ontologies andby supporting the incremental refinement of libraries and the DMI processes that use them.• The providers of data and services autonomously change their offered services and schema at a ratewhich defies manual adaptation when many data resources are in use. The DMI architecture proposesto exploit type systems, semantic description, community effort and light-weight composition to semi-automatically adapt to change and to pool the intelligence of human interventions.ADMIRE Page 1 of 3Overview: The consequences of the above architectural decisions are summarised in this section. Thethree layers of abstraction lead to a communication via a restricted canonical form between a user-orientedtools layer and a system and provider-oriented platform layer, as is illustrated diagrammatically below.User and application diversitySystem diversity and complexityIterative DMIprocessdevelopmentMappingoptimisationandenactmentAccommodatingMany application domainsMany tool setsMany process representationsMany working practicesDMI canonical representation and abstract machineComposing or hidingMany autonomous resources & servicesMultiple enactment mechanismsMultiple platform implementationsGateway interfaceone modelTool levelEnactmentlevelFigure 1: Separating DMI levels of diversityFigure 1 shows how the complexities of matching the diversity of user requirements at the toolslevel can be separated from the complexity of the enactment level, accommodating the diversity ofdata resources and services, by interposing the single canonical domain of discourse represented by anappropriate DMI language. The ADMIRE hypothesis is that, by enforcing this logical decoupling, boththe tools development and the platform engineering will proceed rapidly and independently. Of course,this depends on the quality of the abstract machine and the language operating at the gateway. Developingthat quality is one of ADMIRE’s research goals.To allow providers of DMI services to amortise and smooth their costs over many communities ofusers and to allow selection of DMI services from multiple providers, the DMI architecture provides amany-to-many relationship between workbenches and portals and the DMI gateways.A community of developers will use a number of workbenches, e.g. A, B & C, and one registry 1 thatholds descriptions of the DMI components they are either developing or have obtained from gateways,e.g., a and b. Some of these descriptions will refer to representations of implementations in a repositoryN. Each gateway has its own registry, which describes all of the resources, services and components it isable to work with. Requests to a gateway can interrogate this information, can update it and can causeDMI processes that use the gateway’s capabilities to be enacted. A gateway may present services thatare obtained by delegating work to other gateways; shown as a delegating to c in Figure 2.Each gateway has the need for its services to evolve independently. We envisage a large, distributedpopulation of DMI gateways with a variety of owners and purposes, and with varying policies on theadoption of new versions. Hence, a unique registry is associated with each gateway. Workbench C maybe connected to both a and b so that it can draw on different services provided by these, or to deliver itsdefinitions of DMI components and processes to both a and c, or to mediate the transfer of descriptionsbetween a and c.ADMIRE Page 2 of 3workbench workbench workbenchGatewayAregistryB CGatewaya bregistry1registry2 4RepositoryNGatewaycregistry3Figure 2: The relationships between workbenches, gateways, registries and repositoriesImpact: ADMIRE will develop a model, language and implementation framework that has the followingsignificant practical benefits:• By providing DMI-experts with a well equipped workbench, a powerful kit of DMI components andan enactment technology that takes responsibility for many distributed system and data heterogeneityissues, ADMIRE will allow those experts to focus on building methods and tools for domain experts.This will improve the quality of systems used, accelerate experiments and exploration of methods anddata, and increase the DMI-experts’ productivity. Ingenuity and insight will remain critical; it shouldbe more effectively deployed as distractions are eliminated by consistency and automation.• By providing more rapid provision of tailored tool sets and by taking care of operational and perfor-mance issues, the ADMIRE strategy will accelerate the pace with which domain experts explore andmine compositions of heterogeneous data. The improved framework should allow them to focus on hy-potheses and questions in their domain and improve their productivity. Knowledge of the domain andexpertise in formulating hypotheses and deploying methods will remain crucially important; accelerationof enactment and reduction of extraneous detail should allow these to be better exploited.• By providing a computational framework supporting standard computational patterns, multi-scale dataflow and composition, parallelisation, distribution, recovery, adaptation, dynamic code placement andoptimisation, ADMIRE will present a context that stimulates innovation of data mining and integrationmethods. This stimulation should subsequently extend to other communities developing algorithms inany application domain that exploits distributed data and that designs and conducts complex analyses.• Through automation driven by component descriptions ADMIRE will allow abstract descriptions ofa very wide range of DMI process requests to be submitted to a DMI service. This will enable theprovision of utility services that enact those DMI processes.The principal innovations of the DMI architecture are: (1) a de-coupling of the enactment technologyfrom the tools used to prepare DMI processes, which in turn enables (2) multiple independent DMIenactment services, some of which may be tightly coupled with curated data collections, and (3) theenactment of each DMI process by distributing it over these services, c.f. distributed queries.The prototype of the architecture is being evaluated with applications from: customer relationshipmanagement, flood modelling and disaster mitigation, gene expression map analysis and data-centrefailure reduction. (Experience with the prototype and this evaluation will be reported in the full paper.)AcknowledgementsThe EU Framework Programme 7 FP7-ICT-215024 funding of the ADMIRE project is key to bringingthe partners together and to undertaking the research (www.admire-project.eu).References[1] Gordon Bell, Tony Hey, and Alex Szalay. COMPUTER SCIENCE: Beyond the Data Deluge. Science,323(5919):1297–1298, 2009.[2] P. Chapman, J. Clinton, R. Kerber, T. Khabaza, T. Reinartz, C. Shearer, and R. Wirth. The CRISP-DM reference model. Technical report, The CRISP-DM Consortium, August 2000.ADMIRE Page 3 of 3

Advanced Data Mining and Integration Research for Europe

https://www.pure.ed.ac.uk/ws/files/19426613/Atkinson_et_al_2009_Advanced_Data_Mining_and_INtegration_Research_for_Europe.pdf

Advanced Data Mining and Integration Research for Europe

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer