The objective of this presentation is to provide IDN updates on features and activities to the Committee on Earth Observation Satellites (CEOS) Working Group on Information Systems and Services (WGISS) and provider community. The following topics will be will be discussed during the presentation: Overview of NASA's Vision of Services (UMM-S), Migration of IDN SERFs to UMM-S, Metadata Management Tool (MMT): Successor to DocBuilder, DocBuilder-10: Create/Update Dataset Records, Transition of DIF-9 to DIF-10, Schedule for UMM-C and GCMD/IDN Keywords, IDN's WGISS/WGCV Joint Session: Collaborative Actions

Morahan, Michael

NASA Technical Reports Server

SESIP-0718-SC
WGISS-46 ACCESS Section:
IDN Report
CEOS WGISS-46
DLR, Oberpfaffenhofen, Germany
October 2018
This work was supported by NASA/GSFC under Raytheon Co. contract number NNG15HZ39C.
This document does not contain technology or Technical Data controlled under either the U.S. International Traffic 
in Arms Regulations or the U.S. Export Administration Regulations.
Michael Morahan
IDN Coordinator
Michael.P.Morahan@nasa.gov
https://ntrs.nasa.gov/search.jsp?R=20180007357 2019-08-31T18:12:57+00:00Z
SESIP-0718-SC
2
I. Overview of NASA’s Vision of Services (UMM-S)
II. Migration of IDN SERFs to UMM-S
III. Metadata Management Tool (MMT): Successor to 
DocBuilder
IV. DocBuilder-10: Create/Update Dataset Records
V. Transition of DIF-9 to DIF-10
VI. Schedule for UMM-C and GCMD/IDN Keywords
VII. IDN‘s WGISS/WGCV Joint Session: Collaborative 
Actions
Outline 
SESIP-0718-SC
I. Overview of NASA’s Vision 
of Services (UMM-S)
SESIP-0718-SC
4
What is the UMM-S? (Unified Metadata Model for 
Services)
How to I Discover and Utilize a Service?
In order to locate web services or portals, software or tools, we can use the GCMD client to search 
and return metadata for services. https://gcmd.nasa.gov/
In order to discover and transform data, we can use the EDSC client to search and return transformed 
data via services. https://earthdata.nasa.gov/
What is a Service?
We need to recognize SERF legacy for services, but extend NASA web services to include 
future support for access of NASA data from the cloud. These services provide a method of transforming 
the data (e.g. subsetting, reprojection or reformatting, or a combination of these). 
How to I Describe a Service?
Services are described by their attributes, e.g.
Service Name: “SERVIR”
Service LongName: “Mesoamerican 
Visualization and Monitoring 
System (SERVIR)”
Service Type: “WEB PORTAL”
Service RelatedURL: 
“https://www.servirglobal.net/default.aspx”
Service Version: “1.8”
SESIP-0718-SC
5
UMM-S Required Fields
Field Name Description
Name The name of the service, software, or tool.
(Example: "AIRS_L3_OPENDAP".)
LongName The long name of the service, software, or tool. It provides a human readable name for the service.
(Example: "OPENDAP (Hyrax) framework for AIRS Level 3 data products".)
Type The type of the service, software, or tool. 
(Example: OPeNDAP)
Version The edition or version of the service, software, or tool. The version should be defined in the form x, y, and 
z. where 'x.y.z' means 'major.minor.incremental' version numbers. Typically, 'x' and 'y' are numbers (0 
through 9) and 'z' is a number (0 through 99).
(Example: 1.1.1) 
Description A brief description of the service, software, or tool. For example, a description might contain information 
about what is the service, the purpose of the service, and the parameters (or variables) being invoked, 
and what are the sources of these data.
RelatedURLS This element contains important information about the Uniform Resource Locator (URL) for the service. 
ServiceKeywords Allows for the specification of Earth Science Service keywords that are representative of the service, 
software, or tool being described. The controlled vocabulary for Service Keywords is maintained in the 
Keyword Management System (KMS). (Example: "ServiceCategory: Earth Science Services, 
ServiceTopic: Data Management/Data Handling, ServiceTerm: Data Search and Retrieval". )
ServiceOrganizations The service provider, or organization, or institution responsible for developing, archiving, and/or 
distributing the service, software, or tool.
(Example: "Role: SERVICE PROVIDER, ShortName: INPE, LongName:  National Institute for Space 
Research, Brazil".)
SESIP-0718-SC
6
[ {
"meta" : {
"native-id" : "NOAA_Shoreline_GIS",
"provider-id" : "SCIOPS",
"concept-type" : "service",
"concept-id" : "S1535675684-SCIOPS",
"revision-date" : "2018-07-10T19:32:05Z",
"user-id" : "tstevens",
"deleted" : false,
"revision-id" : 2,
"format" : "application/vnd.nasa.cmr.umm+json"
},
"umm" : {
"AncillaryKeywords" : [ "GIS", "Hydrologic Data", "Environmental Risk Data", "Shoreline", "Sea", "Vector", "Coast" ],
"RelatedURLs" : [ {
"Description" : "Access the NOAA Shoreline Data Explorer.",
"URLContentType" : "DistributionURL",
"Type" : "GET SERVICE",
"Subtype" : "ACCESS MAP VIEWER",
"URL" : "http://www.ngs.noaa.gov/newsys_ims/shoreline/index.cfm"
} ],
"Type" : "TOOL",
"ServiceKeywords" : [ {
"ServiceCategory" : "EARTH SCIENCE SERVICES",
"ServiceTopic" : "DATA ANALYSIS AND VISUALIZATION",
"ServiceTerm" : "GEOGRAPHIC INFORMATION SYSTEMS",
"ServiceSpecificTerm" : "WEB-BASED GEOGRAPHIC INFORMATION SYSTEMS"
}, {
"ServiceCategory" : "EARTH SCIENCE SERVICES",
"ServiceTopic" : "DATA MANAGEMENT/DATA HANDLING",
"ServiceTerm" : "DATA SEARCH AND RETRIEVAL"
}, {
"ServiceCategory" : "EARTH SCIENCE SERVICES",
"ServiceTopic" : "METADATA HANDLING",
"ServiceTerm" : "DATA DISCOVERY"
} ],
"ServiceOrganizations" : [ {
"Roles" : [ "SERVICE PROVIDER" ],
"ShortName" : "DOC/NOAA/NOS/NGS",
"LongName" : "National Geodetic Survey, National Ocean Service, NOAA, U.S. Department of Commerce",
"Service "ContactPersons" : [ {
"Roles" : [ "SERVICE PROVIDER" ],
"ContactInformation" : {
"ContactMechanisms" : [ {
"Type" : "Email",
"Value" : "ngs.infocenter@noaa.gov"
}, {
"Type" : "Fax",
"Value" : "301-713-4172"
}, {
"Type" : "Telephone",
"Value" : "301-713-3242"
} ],
"Addresses" : [ {
"StreetAddresses" : [ "National Geodetic Survey, NOAA", "Information Services Branch", "1315 East-West Highway" ],
"City" : "Silver Spring",
"StateProvince" : "MD",
"PostalCode" : "20910-3282"
} ]
},
"FirstName" : "INFORMATION SERVICES BRANCH",
"LastName" : "NATIONAL GEODETIC SURVEY"
} ],
"ContactInformation" : {
"RelatedUrls" : [ {
"URLContentType" : "DataCenterURL",
"Type" : "HOME PAGE",
"URL" : "https://www.ngs.noaa.gov/"
} ]
}
} ],
"ScienceKeywords" : [ {
"Category" : "EARTH SCIENCE",
"Topic" : "SOLID EARTH",
"Term" : "GEOMORPHIC LANDFORMS/PROCESSES",
"VariableLevel1" : "COASTAL LANDFORMS"
}, {
"Category" : "EARTH SCIENCE",
"Topic" : "TERRESTRIAL HYDROSPHERE",
"Term" : "SURFACE WATER"
}, {
"Category" : "EARTH SCIENCE",
"Topic" : "TERRESTRIAL HYDROSPHERE",
"Term" : "SURFACE WATER",
"VariableLevel1" : "SURFACE WATER FEATURES",
"VariableLevel2" : "LAKES/RESERVOIRS"
}, {
"Category" : "EARTH SCIENCE",
"Topic" : "TERRESTRIAL HYDROSPHERE",
"Term" : "SURFACE WATER",
"VariableLevel1" : "SURFACE WATER FEATURES",
"VariableLevel2" : "RIVERS/STREAMS"
} ],
"Description" : "The NOAA Shoreline Data Explorer system is an online management and sharing system of shoreline data 
that cartographically depicting the dynamic interface between land and water at the time of survey. The tool provides high-
resolution digital shoreline from multi-temporal surveys of our nation's coastline. It provides the following capabilities:\r\n\r\n-View 
available shoreline project boundaries\r\n-View vector shoreline data\r\n-View and download FGDC compliant metadata\r\n-Make 
printable maps\r\n-Download vector shoreline shapefiles",
"Version" : "NOT PROVIDED",
"UseConstraints" : "The data contained on this site should NOT TO BE USED FOR NAVIGATION.",
"Name" : "NOAA_Shoreline_GIS",
"ContactPersons" : [ {
"Roles" : [ "AUTHOR" ],
"ContactInformation" : {
"ContactMechanisms" : [ {
"Type" : "Email",
"Value" : "Tyler.B.Stevens@nasa.gov"
}, {
"Type" : "Telephone",
"Value" : "301-851-8113"
} ],
"Addresses" : [ {
"StreetAddresses" : [ "5700 Rivertech Court" ],
"City" : "Riverdale",
"StateProvince" : "MD",
"Country" : "USA",
"PostalCode" : "20737"
} ]
},
"FirstName" : "TYLER",
"MiddleName" : "B.",
"LastName" : "STEVENS"
} ],
"LongName" : "NOAA Shoreline Data Explorer"
}
} ]
}
Example: NOAA_Shoreline_GIS Service
SESIP-0718-SC
7
• Service records can be search using the 
CMR SEARCH API
Search for all
curl ‘https://cmr.earthdata.nasa.gov/search/services’
Search by Keyword
curl ‘https://cmr.earthdata.nasa.gov/search/services?keyword=OpenDAP&pretty=true’
Search for specific service
curl 
‘https://cmr.earthdata.nasa.gov/search/services.umm_json?name=AIRX3STD.006&pretty=true’
CMR Search API Documentation: 
https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html#searching-for-services
CMR API SEARCH
SESIP-0718-SC
8
• RelatedURLs increased cardinality to 1 to N
• Options class expanded to include supported Input and Output 
projections, Input and Output formats.
• ServiceQuality class added
• Platforms and Instruments classes added
• Addition of OperationMetadata class to support a wide variety of 
server-side operations (e.g. GetCapabilities, GetMap, 
GetCoverage)
• UMM-S schema 1.2 
https://git.earthdata.nasa.gov/projects/EMFD/repos/unified-
metadata-model/browse/service/v1.2
• UMM-S document: 
https://wiki.earthdata.nasa.gov/download/attachments/49448405/U
MM-S_V1.2_20180530.docx
Latest Capabilities and Features
SESIP-0718-SC
II. Migration of IDN SERFs to 
UMM-S
SESIP-0718-SC
10
• Metadata standard used to 
describe Earth science tools, 
software, and models (e.g. 
EDSC, Giovanni, Panoply, 
USGS Model Viewer)
• Supports the discovery and 
access of these tools, 
software, and models
• Originated (in year 2000) as a 
prototype within NASA’s Earth 
Science Technology Office 
(ESTO) and supported by 
NASA and the Committee on 
Earth Observation Satellites 
(CEOS)
What is the SERF? (Service Entry Resource Format)
Data Service Types
SESIP-0718-SC
11
• Use the Expanded Model and Infrastructure To Make 
Services More Useful
– Extended service capabilities (service invocation, subsetting, 
reprojection, time aggregation, etc...)
• Link Services With Applicable Data Sets 
– Allows for service capabilities and specific tools to be 
accessible to/from their applicable data sets, which renders 
the services discoverable, available, and useful to users that 
might not already be aware of them
• Search Across a Common Repository for Services
– Centralized catalogue of all services, tools, software, models 
across U.S. and international agencies that work with Earth 
science data
Why Migrate to UMM-S in CMR?
SESIP-0718-SC
12
• All Viable SERFs in GCMD/IDN
– SCIOPS/IDN provider records
– EOSDIS provider records
• Should I Stay or Should I Go
– Does the service describe a tool or software?
– Is the service still viable/supported?
– Is the service being deprecated
– Do the links still work?
– Is the service from a commercial provider?
– Are the other attributes still accurate?
What’s Being Migrated?
SESIP-0718-SC
13
Migration Schedule: Fall 2018
• The IDN staff will email IDN Providers with their list of 
SERF records.
– The Providers are to review the SERF records.
• Determine to migration or deprecation the records.
• Work with IDN staff to cleanup/migrate or deprecate/archive SERF 
records
– The IDN staff will ingest viable UMM-S records into CMR.
The CMR Metadata Quality Team will be assisting with the migration of 
the service records. Please reach out to IDN User services (gsfc-
gcmduso@mail.nasa.gov) and/or myself 
(Michael.P.Morahan@nasa.gov) . 
SESIP-0718-SC
III. Metadata Management 
Tool (MMT): Successor to 
DocBuilder
SESIP-0718-SC
15
• Login using same ID and password as docBUILDER
(NASA Agency login Not required)
• Submit new/updated data and services descriptions to 
the IDN for review and ingest by the CMR Metadata 
Team.
• Assess quality of metadata using inline validation.
• Export metadata in DIF-10, NASA MENDs ISO, ATOM 
formats.
• View metadata using HTML “Pretty View” permanent 
link.
Metadata Management Tool 
(MMT): Coming Attraction (2019)
Features my change pending design review.
SESIP-0718-SC
16
CMR
Draft
2. Send draft
3. Edit draft
4. Publish record
Revisions
(up to 10)
5. Edit record
(Optional) Revert to
previous revision
1. Create draft record
Metadata Management Tool 
(MMT): Data Flow Diagram
Metadata 
Quality 
Coordinator
SESIP-0718-SC
17
Metadata Management Tool 
(MMT): Manage Collections
Search in CMR
Create new draft
User and Provider
SESIP-0718-SC
18
Metadata Management Tool 
(MMT): Draft Collection
Short Name_Version
Long Name
Publish CMR record Delete Draft
Progress Panel
Preview Panel
Spatial Preview
SESIP-0718-SC
19
Metadata Management Tool 
(MMT): Editing Fields
Required section
Required field
Information Select value from 
drop-down
SESIP-0718-SC
20
MMT allows users to manage and curate UMM-S metadata records in the Common 
Metadata Repository (CMR). 
Metadata Management Tool (MMT)
Manage Services Page Edit Service Record Page
SESIP-0718-SC
IV. DocBuilder-10: 
Create/Update Dataset 
Records
SESIP-0718-SC
22
• DocBuilder-10 is a web-based metadata authoring tool that allows metadata authors to add 
(or modify) dataset descriptions (DIFs) records for the IDN. 
https://idn.ceos.org/DocumentBuilder/Home.do?Portal=ceos_idn
Create/Update Dataset Records
SESIP-0718-SC
V. Transition of DIF-9 to DIF-10
SESIP-0718-SC
24
(2017-2018)
Sent Provider DIF-
9 to DIF-10 
QA/Triage 
Reports.
May 1, 2018
Start migrating 
existing non-NASA 
DIF-9 records to 
DIF-10.
2019 (First 
quarter)
Plan to have all 
metadata records 
transition.
DIF-9 to DIF-10 Transition Schedule 
• Providers need to submit new metadata in DIF-10 or 
any format compatible with NASA’s Common 
Metadata Repository (CMR).
– Compatible formats: DIF-10, NASA ISO (MENDs), and 
UMM-JSON.
• docBUILDER supports DIF-10 format.
SESIP-0718-SC
25
Provider # of records
# of records 
transition
# of records 
not transition
Completion 
Precentage State
SCIOPS 17282 10797 6485 62% On-going
AU_AADC 2772 0 2772 0% Under Review
CNES 20 20 0 100% Completed
ESA 112 20 92 18% On-going
EUMETSAT 62 0 0 0% Under Review
INPE 43 43 0 100% Completed
ISRO 34 34 0 100% Completed
JAXA 340 340 0 100% Completed
NOAA_NCEI 5578 0 5578 0% Testing
USGS_EROS 142 13 129 9% On-going
Total 26385 11267 15056 42.70%
Transitioned Status
SESIP-0718-SC
VI. Schedule for UMM-C and 
GCMD/IDN Keywords
SESIP-0718-SC
27
UMM-C: Proposed New Fields (2019)
average size of a 
downloadable file (granule)
total size of all of the 
downloadable collection files
More information on UMM: https://earthdata.nasa.gov/umm 
SESIP-0718-SC
28
Keyword Version 8.7 Proposed Topics 
(2019)
Cryosphere
Earth Science Services/
Web Services
If you are interested in becoming a keyword reviewer, 
please contact the ESO at eso-staff@lists.nasa.gov. 
SESIP-0718-SC
VII. IDN‘s WGISS/WGCV 
Joint Session: Collaborative 
Actions
SESIP-0718-SC
30
• Quality Indicators in Discovery Metadata
– Action: WGISS (Michael) to start defining best approach for 
representing and including QIs for the selected test case in 
discovery metadata searchable by end users.
• WGISS need one specific and one broader example of SST QI to 
start analysis. Awaiting input from WGCV
• CEOS Data Cubes and CEOS Test Sites Data Access 
in support to WGCV Activities
– Action: WGISS (Michael) to define how to get this info into the 
IDN for discover and possibly access by August.
• Will be started after receiving initial input by WGCV, completed by 
end November
IDN Actions Status
SESIP-0718-SC
31
Questions/Discussion
Please Provide feedback to:
gsfc-gcmduso@mail.nasa.gov
Or
Michael.P.Morahan@nasa.gov
SESIP-0718-SC
Backup SLIDES
SESIP-0718-SC
33
UMM-S Design (v1.2)
SESIP-0718-SC
34
UMM-S Fields
Name [R] ConnectPoint
DataResourceSpatialExtent 
[R] ParameterDescription [R] AncillaryKeywords SupportedOutputProjections
LongName [R] ResourceName
DataResourceSpatialExtentT
ype [R] ParameterOptionality [R] ServiceOptions ProjectionName
Type [R] ResourceLinkage [R] SpatialResolution [R] ParameterRepeatability [R] SubsetTypes ProjectionLatitudeOfCenter
Version [R] ResourceDescription SpatialResolutionUnit [R] ScienceKeywords VariableAggregation ProjectionLongitudeOfCenter
Description [R] OperationChainMetadata
DataResourceTemporalExte
nt [R] ServiceContacts [R] SupportedInputProjections ProjectionFalseEasting
RelatedURLs [R] OperationChainName [R]
DataResourceTemporalExte
ntType [R] ContactPersons ProjectionName ProjectionFalseNorthing
ServiceKeywords [R] OperationChainDescription TemporalResolution [R] ContactGroups ProjectionLatitudeOfCenter ProjectionAuthority
ServiceOrganizations [R] CoupledResource TemporalResolutionUnit [R] Platforms ProjectionLongitudeOfCenter ProjectionUnit
OperationMetadata ScopedName RelativePath Instruments ProjectionFalseEasting ProjectionDatumName
OperationName DataResourceDOI CouplingType ServiceQuality ProjectionFalseNorthing InterpolationTypes
DistributedComputingPlatfor
m DataResource Parameter ServiceCitation [R] ProjectionAuthority SupportedInputFormats
OperationDescription DataResourceIdentifier [R] ParameterName [R] AccessConstraints ProjectionUnit SupportedOutputFormats
InvocationName DataResourceType [R] ParameterDirection [R] UseConstraints ProjectionDatumName MaxGranules
SESIP-0718-SC
35
Metadata Management Tool 
(MMT): Preview Draft Record
SESIP-0718-SC
36
This work was supported by NASA/GSFC under 
Raytheon Co. contract number NNG15HZ39C.
in partnership with


WGISS-46 ACCESS Section: IDN Report

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server