Lynnes, Christopher

Newman, Doug

English

NASA Technical Reports Server

Connecting Data with Data Usage: 
A Graph Approach
Winter ESIP, 10:15am, Thursday, January 9th 2020
This work was supported by NAS A/GS FC  under R aytheon C o. contract number NNG15HZ39C .
This document does not contain technology or Technical Data controlled under either the U.S. International Traffic in 
Arms Regulations or the U.S. Export Administration Regulations.
Doug Newman
NASA EED-2 Data Use Architect
douglas.j.newman@nasa.gov
Dr. C hristopher Lynnes
NASA EOSDIS System Architect
christopher.s.lynnes@nasa.gov
https://ntrs.nasa.gov/search.jsp?R=20200000256 2020-03-11T13:34:19+00:00Z
INTRODUCTION
3The rationale
‘Connect together the main elements of 
Earth Observation knowledge AND context 
in a way that is: machine-readable, human-
usable and curatable’
Chris Lynnes, WGISS-47
4Or...
‘Scientists are swimming in a sea of 
datasets and want to know which ones to 
use.’
Chris Lynnes, numerous
5Quick overview
● Elements of knowledge: Missions, instruments, 
datasets, measurements, articles
● Context of knowledge: this instrument belongs to this 
mission, this dataset contains this measurement, this 
article cites this dataset
● All this can be modeled efficiently and intuitively using 
graph concepts
6
7CBERS-2
NASA Earthdata
9Overview
● Effective technologies
● Sources of knowledge
○ NASA EOSDIS Common Metadata 
Repository
○ World meteorological organization
○ The CEOS database
10
Graph Technologies
Implementations
● AWS Neptune
● Neo4J
APIs
● SPARQL
● Gremlin
● Cypher - now GQL
Amazon Web Services and the “Powered by AWS” logo are trademarks of 
Amazon.com, Inc. or its affiliates in the United States and/or other countries
Neo4j is a registered trademarks
of Neo4j
Apache Gremlin are either registered trademarks or trademarks of the Apache 
Software Foundation
11
AWS Neptune + Gremlin
Amazon Web Services and the “Powered by AWS” logo are 
trademarks of Amazon.com, Inc. or its affiliates in the United 
States and/or other countries
Apache Gremlin are either registered trademarks or 
trademarks of the Apache Software Foundation
12
Within NASA Earthdata
The common metadata repository contains dataset metadata records and 
references to their respective documentation. We used this inventory to create 
a graph of datasets and documentation vertices and ‘documents’ edges, 
originally to serve as a stepping stone towards scientific literature.
13
Side trip to recommendations
g.V().hasLabel('dataset')
.has('conceptid',conceptId)
.inE('documents')
.outV()
.hasLabel('documentation')
.outE('documents')
.inV()
.hasLabel('dataset')
AIRS3STM
14
Essential Climate Variables (1)
● A good starting point for discovery: ‘I want 
total ozone data’
● Total Ozone is one of 496 well-defined 
products, spread across 30 variables in 3 
domains
15
Essential Climate Variables (2)
425 datasets have been mapped to 
those products in our graph 
implementation
16
Essential Climate Variables (2)
17
Missions and instruments
The CEOS database contains extensive 
information about missions and instruments 
and their linkage.
CMR contains linkage information between 
datasets and mission/instruments.
18
Missions and instruments
19
Keywords
The hierarchy of science keywords and their 
associations with datasets
● Science keywords - GCMD
● Datasets - CMR
20
Roadmap
● Graph implementation in EED by Q2 2020
○ CMR concept associations
● ECV mappings for GES_DISC by Q2 
2020
● ECV graph implementation by Q3 2020
CONCLUSIONS
22
NASA: If you build it...
● 10 minutes looking at a visualization of two CMR 
concepts gave me the inspiration for the 
recommendation engine. Imagine what an expert could 
do?
● Providing a read-only traversal API over a knowledge 
base containing more concepts could open the 
floodgates for novel discovery techniques
NEXT STEPS
24
Link all the things
● With help from each other we would like 
to scale these data connections to 
facilitate data discovery
● Graph traversal APIs will enable novel 
discovery techniques
QUESTIONS?
Approved for Public Release
26
This work was supported by NASA/GSFC under 
R aytheon C o. contract number NNG15HZ39C .
in partnership 
with


Connecting Data with Data Usage: A Graph Approach

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server