作者单位：中国科学院上海生命科学信息中

陈涛

Xiamen University Institutional Repository

关联数据系统开发实例与 平台详解 陈  涛 中国科学院上海生命科学信息中心 2015-05-05 QQ群：三人行（语义有你） 150461365 Agenda Concept LOD Status Apache Jena Triple Store RDFizer Ontology Semantic Search Virtuoso Data Visual Linked Data & Semantic Web Linked Data & Semantic Web Things Everything The Internet of It’s estimated that by the year 2020 more than fifty billion “things” will be connected to the internet. 50,000,000,000 Valuation of IoE 19,000,000,000,000 How much is the Internet of everything worth? Cisco says $19 trillion. Everything Linked Data & Semantic Web manufacturing (15%) ,healthcare (15%) ,insurance (11%) Linked Data & Semantic Web Semantic Web  Semantic web is a Web of data.   The term “Semantic Web” refers to W3C’s vision of the Web of linked data.   It is about common formats for integration and combination of data drawn from diverse sources.   It is also about language for recording how the data relates to real world objects. That allows a person, or a machine, to start off in one database, and then move through an unending set of databases which are connected not by wires but by being about the same thing. A term used to describe a recommended best practice for exposing, sharing, and connecting pieces of data, information, and knowledge on the Semantic Web using URIs and RDF. Linked Data & Semantic Web Linked Data Connect Distributed Data across the Web Linked Data & Semantic Web Linked Open Data Cloud 01002003004005006001LOD Datasets 2007/5/1 2007/10/8 2007/11/7 2008/2/28 2008/3/31 2008/9/182009/3/5 2009/3/27 2009/7/14 2010/9/22 2011/9/19 2014/8/30Social web 51% Government 18% Publications 10% Life sciences 8% User-generated content 5% Cross-domain 4% Media 2% Geographic 2% Social webGovernmentPublicationsLife sciencesUser-generated contentCross-domainMediaGeographicLinked Data & Semantic Web LOD Cloud Status Linked Data & Semantic Web Linked Life Data 17,720,129,260 Linked Data & Semantic Web Use Cases for SW & LD Linked Data & Semantic Web Semantic Web Technologies RDF RDFs SPARQL SKOS OWL RIF Entity Extraction … … Linked Data & Semantic Web Core Components VISUALS ALGORITHM WORK FLOW INDEXING TRIPLE  STORE DATA FUSION SHARING SECURITY Query and merge data from many different sources, both structured and unstructured Rapid on free text, complex phrases, semantic search Apply filter, extractors, transformation algorithms and inference rules Business or services logic control and work flow Rich perspective on displaying ontology and data RDF is the core data model;  stores triples: Subject, Predicate, Object The auditing and adjudication of data as it is accessed and transformed Discovery of web-services, and user workspaces Linked Data & Semantic Web Framework for SW & LD Apache Jena (or Jena in short) is a free and open source Java framework for building semantic web and Linked Data applications. OpenRDF Sesame is a powerful Java framework for processing and handling RDF data. This includes creating, parsing, storing, inferencing and querying over such data.  Linked Data & Semantic Web Apache Jena Store API In-memory SDB TDB custom SQL Database Native Tuple Store Triple Store Inference API none Build-in rule reasoner External reasoner Application Code Ontology API SPARQL API Parsers and  Writers RDF/XML Turtle N-Triples RDFa RDF API Linked Data & Semantic Web Data Fusion / Integration Points Linked Data & Semantic Web Bio2RDF Process • Dataset: 35      Triples: 11,900,533,153      Entities: 1,108,204,952 RDFization URI Normalization Triple Store http://hgnc.bio2rdf.org/sparql http://omim.bio2rdf.org/sparql SERVICES: Virtuoso Linked Data & Semantic Web Fusion Process Download the original data Transform the data into RDF Keeping a link to the original data in the new RDF document Store the converted database in a triple store Give services for these using a RESTful interface 5 1 2 3 4 Linked Data on the Web RDB-2-RDF Wrapper (e.g. D2R) CMS with  RDFa output (e.g. Drupal) Relational Database Data Source with API RDF Store RDF Files Web Server (e.g. Apache) Custom Linked Data Wrapper Structured Data Text Linked Data Interface (e.g. Pubby) RDFizers for CVS, XML, Excel,… Entity Extractor (e.g. Calais) Linked Data & Semantic Web Transformation / RDFization Linked Data & Semantic Web Convert to RDF Tools Format Tools BibTex • BibBase • bibtex2rdf • Java BibTex-To-RDF Converter SQL • D2RQ • OpenLink Virtuoso • R2RML Excel • XLWrap • Excel2rdf • Sheet2RDF MARC • easyM2R • marcmods2rdf XML • GRDDL  • SPARQL2XQuery Microformats • Any23 Linked Data & Semantic Web RDF is only the First step, not End Semantic Normalization Syntactic Normalization Open Data Linked Data Linked Knowledge Raw Linked Data & Semantic Web Ontology Linked Data & Semantic Web Ontology for the representations of diverse biomedical knowledge Linked Data & Semantic Web Jena Ontology API • Jena OntModel • RDF Graph - SPARQL Ontology model compound document structure for imports Linked Data & Semantic Web Jena Inference API • Transitive reasoner: Provides support for storing and traversing class and property lattices. This implements just the transitive and reflexive properties of rdfs:subPropertyOf and rdfs:subClassOf. • RDFs rule reasoner: Implements a configurable subset of the RDFs entailments. • OWL, OWL Mini, OWL Micro Reasoners: A set of useful but incomplete implementation of the OWL/Lite subset of the OWL/Full language.  • Generic rule reasoner: A rule based reasoner that supports user defined rules. Forward chaining, tabled backward chaining and hybrid execution strategies are supported.  Linked Data & Semantic Web Jena Inference API [rule1: (?a fa:hasHusband ?b)(?a fa:isMotherOf ?c)->(?b fa:isFatherOf ?c)]   [rule2: (?a fa:hasHusband ?b)(?a fa:hasDaughter ?c)->(?b fa:isFartherOf ?c)]    Lucy fa:hasHusband Jackie ; fa:isMotherOf Nick ; fa:hasDaughter Linda. Jackie Lucy Nick Linda fa:hasHusband Lucy fa:hasHusband Jackie fa:isFatherOf fa:isFatherOf Linda Nick InfModel Model Reasoner reasoner = new GenericRuleReasoner(Rule.parseRules(rules)); InfModel inf = ModelFactory.createInfModel(reasoned, rawData); Linked Data & Semantic Web A NoSQL Concept Tree Linked Data & Semantic Web RDF Databases / Triple Store • Schema flexibility - it's possible to do the equivalent of a schema change to an RDF store live, and without any downtime, or redesign . • More modern - RDF stores are typically queried over HTTP, it's very easy to fit them into Service Architectures. Also they handle internationalized content better than typical SQL databases. • Standardization - the level of standardization of implementations using RDF and SPARQL is much higher than SQL. Moving data between stores is easy, as they all speak the same language. • Expressivity - it's much easier to model complex data in RDF than in SQL, and the query language makes it easier to do things like LEFT JOINs (called OPTIONAL in SPARQL). • Provenance - SPARQL lets you track where each piece of information came from, and you can store metadata about it, letting you easily do sophisticated queries. Linked Data & Semantic Web RDF Databases / Triple Store Databases Customers OpenLink Virtuoso AllegroGraph Stardog Oracle NoSQL Others: Jena TDB, BigData, Sesame, 4store, MarcLogic, YARS2 Linked Data & Semantic Web How to access Linked Data Sets? Downloads Web interface (lookup + services) SPARQL endpoint Flint Sparql editor Virtuoso Faceted Browser Linked Data & Semantic Web RDF Serialization Linked Data & Semantic Web Semantic Search One of the primary goals of semantic web: • Not only keyword full-text search • Query includes relations between resources • Connecting data: mash-up from different sources Semantic search provides more meaningful search results by evaluating and understanding the search phrase and finding the most relevant results in a website, database or any other data repository. Linked Data & Semantic Web Get the phenotypes of knock-out mouse models for the targets of a selected drug Drug: Imatinib  "hemorrhage [mp:0001914]"@en "intracranial hemorrhage [mp:0001915]"@en "perinatal lethality [mp:0002081]"@en "hydrops fetalis [mp:0002192]"@en "kidney hemorrhage [mp:0000533]"@en "skin hemorrhage [mp:0011514]"@en "anemia [mp:0001577]"@en "出血" "颅内出血" "围产期的杀伤力" "胎儿水肿" "肾出血" "皮肤出血" "贫血" Results: Linked Data & Semantic Web Semantic Search TripleStore • enriched content • increased wealth      of data • improved precision      and recall 慢性阻塞性肺疾病 支气管疾病 呼吸疾病 Linked Data & Semantic Web Precision & Recall Linked Data & Semantic Web SIREn {Semantic Information Retrieval Engine} SIREn is a highly scalable open-source full-text search engine especially suited for nested and schemaless data.  Lucence’s BlockJoin: 100*100 = 10,000 docs SIREn: 100 = 100 docs Linked Data & Semantic Web Visual Tools – RDF Graph Linked Data & Semantic Web RDF/OWL Ontology Visualization RDF Gravity is a tool for visualizing RDF/OWL Graphs/ontologies Welkin is a graph-based RDF visualizer HP RDF Graph Visualization describes an experimental node-centric approach to RDF RelFinder extracts and visualizes relationships between given objects in RDF data and makes these relationships interactively explorable Linked Data & Semantic Web HP RDF Graph Visualization Linked Data & Semantic Web Welkin Linked Data & Semantic Web RelFinder Linked Data & Semantic Web RDF Data Visualization RDF-spark is a graph-based RDF visualizer Sgvizler provides a javascript based SPARQL query form that passes the SPARQL result set to a powerful visualizer Visual Browser is a Java application that can visualize the data in RDF scheme Map4rdf provides your users with a nice map-based visualization of your data Linked Data & Semantic Web Sgvizler  Sgvizler is a javascript which renders the result of SPARQL SELECT queries into charts or html elements. SELECT * WHERE {     [] w:hasCountryCode ?ID ;          w:hasGNP ?GNP;          w:hasCountryPopulation ?Population;         w:isCountryInContinent [ rdfs:label ?Continent ];          w:hasLifeExpectancy ?LifeExpectancy .  } Linked Data & Semantic Web Sgvizler Result Linked Data & Semantic Web Map4Rdf geo.linkeddata.es Linked Data & Semantic Web Map4Rdf Linked Data & Semantic Web Visual Tools Simile Exhibit Echarts Google Visualization API D3 Data Publica GeoAPI Cytoscape An open source software platform for visualizing complex networks and integrating these with any type of attribute data. Linked Data & Semantic Web Cytoscape Linked Data & Semantic Web BIO-Linked Scientific Data Linked Data & Semantic Web VISUalization Playground Linked Data & Semantic Web Linked Data & Semantic Web R script R provides a wide variety of statistical (linear and nonlinear modelling, classical statistical tests, time-series analysis, classification, clustering, …) and graphical techniques. PEDIATRIC BLOOD & CANCER ( 41 )BIOMATERIALS ( 19 )CELL RESEARCH ( 15 )BIOLOGY OF BLOOD AND MARROW TRANSPLANTATION ( 15 )TISSUE ENGINEERING PART A ( 14 )JOURNAL OF CELLULAR PHYSIOLOGY ( 12 ) JOURNAL OF CONTROLLED RELEASE ( 11 )STEM CELLS AND DEVELOPMENT ( 10 )CYTOTHERAPY ( 10 )JOURNAL OF BIOLOGICAL CHEMISTRY ( 9 )Pie Chart of JournalLinked Data & Semantic Web R - Pie Chart Linked Data & Semantic Web R - Brazilian Amazon Rainforest relative deforestation per pixel during 20020.000.020.040.060.080.100.120.140.16Linked Data & Semantic Web R - Brazilian Amazon Rainforest (2002) relative deforestation per pixelDEFOR_2002 DEFOR_2003 DEFOR_2004DEFOR_2005 DEFOR_2006 DEFOR_2007DEFOR_20080.000.050.100.150.200.25Linked Data & Semantic Web Linked Data & Semantic Web OpenLink Virtuoso Open-Source Edition Use it for linked data Install VAD plugin packages Visualization interface (Demo datasets: Map, Article) Install  Sparql endpoint  iSparql  Facet view  Pivot viewer Linked Data & Semantic Web 

关联数据系统开发实例与平台详解

http://dspace.xmu.edu.cn/bitstream/handle/2288/85046/%e9%99%88%e6%b6%9b%ef%bc%8d%e5%85%b3%e8%81%94%e6%95%b0%e6%8d%ae%e7%b3%bb%e7%bb%9f%e5%bc%80%e5%8f%91%e5%ae%9e%e4%be%8b%e4%b8%8e%e5%b9%b3%e5%8f%b0%e8%af%a6%e8%a7%a3.pdf?sequence=1&isAllowed=y

关联数据系统开发实例与平台详解

Abstract

Similar works

Full text

Available Versions

Xiamen University Institutional Repository