Constable, Heather

Guralnick, Robert P.

Peterson, A. Townsend

Spencer, Carol

Weiczorek, John

English

KU ScholarWorks

Community PageVertNet: A New Model for Biodiversity Data SharingHeather Constable1*, Robert Guralnick2, John Wieczorek1, Carol Spencer1, A. Townsend Peterson3, TheVertNet Steering Committee"1 Museum of Vertebrate Zoology, University of California Berkeley, Berkeley, California, United Stated of America, 2 Department of Ecology and Evolutionary Biology,University of Colorado, Boulder, Colorado, United States of America, 3 Biodiversity Research Center, University of Kansas, Lawrence, Kansas, United States of AmericaBiodiversity is in a crisis caused bymultiple human impacts on the environ-ment [1–3]. The immediate and criticaltasks in addressing this crisis are toexamine global biodiversity patterns anddocument changes through time and spacein order to understand the factors contrib-uting to loss of biodiversity [4]. Meetingthis challenge has emerged as a globalpriority [5–7] that requires approaches tomobilize data across broad geographic andtaxonomic ranges.The community of vertebrate naturalhistory collections has begun to meet thischallenge by establishing social and tech-nological infrastructures that provide openaccess to species occurrence data throughbroad participation and funding from theUS National Science Foundation (NSF)and the Global Biodiversity InformationFacility (GBIF). One result is VertNet, apublicly accessible database of vertebratebiodiversity data from natural historycollections around the world. VertNetcurrently consists of four existing globalvertebrate networks: Mammal NetworkedInformation System (MaNIS) (http://manisnet.org – mammals [8]); Ornitho-logical Information System (ORNIS)(http://ornisnet.org – birds); HerpNET(http://herpnet.org – amphibians andreptiles); and FishNet 2 (http://www.fishnet2.net – fishes). These networkscollectively mobilize over 52 million re-cords from over 70 institutions, whichrepresent about 70% of all the vertebratespecies occurrence data that are accessi-ble through GBIF. VertNet was createdto develop the tools and infrastructurenecessary to make the data in thesedistributed networks available in a stan-dard format to maximize their potentialfor understanding and protecting biodi-versity. GBIF and VertNet work syner-gistically to enhance biodiversity datamobilization efforts. GBIF has identifiedthe important role that VertNet will playin its new emphasis on decentralizationof services and applications [5]. Inparticular, VertNet provides importantdata maintenance services, includingdata cleaning and indexing, thus remov-ing development and deployment bur-dens for many fundamental tasks fromGBIF.Data from VertNet are currentlyaccessed through the four networks bya broad audience at a high frequency ofabout 2.5 million records per week. Thenetworks continue to grow, even thosewhose extramural funding have expired;MaNIS, for example, has grown from 17contributing institutions under the orig-inal grant to 38 institutions, with awaiting list of 31 more that will be addedto the network as time and resourcespermit. The same enthusiasm is mir-rored across the other three vertebratenetworks. This rapid growth demon-strates an important sociological shiftfrom skepticism to enthusiasm for datasharing.Ironically, the success of these networkshas become their biggest challenge,straining original architectures and de-manding a scalable and more sustainablesolution. Below, we provide perspectiveson the sociological and technical devel-opments that brought vertebrate biodi-versity networks to this point and discusssolutions to the immediate and anticipat-ed challenges.Developing Data-SharingTechnologyThe fundamental concept underlyingthe vertebrate biodiversity networks is thatdata contributors are the primary andauthoritative source for information aboutthe occurrence data over which they havecustody. The networks merely facilitateaccess and sharing of these distributedprimary resources. A fully decentralizedarchitecture, with all requests distributeddirectly to the primary sources, highlight-ed the primacy of the contributing institu-tions and was an essential phase inpromoting participation, instilling confi-dence and a sense of control within thecommunity.The current system relies on a datastandard [9] and a distributed queryprotocol [10]. The Darwin Core specifiesterms—such as scientific name, date, andlocality descriptions—provide informationabout species’ occurrences in nature.Distributed generic information retrieval(DiGIR) specifies the messaging systemthat allows questions to be asked andanswered across the network of primarydata sources. The networks grow byhaving prospective contributors establishaccess to their Darwin Core–compliantdata through networked computers outfit-ted with DiGIR and a Web server, andthen by requesting that these servers beregistered to participate in one or more ofthe networks. Successful installation andThe Community Page is a forum for organizationsand societies to highlight their efforts to enhancethe dissemination and value of scientific knowledge.Citation: Constable H, Guralnick R, Wieczorek J, Spencer C, Peterson AT, et al. (2010) VertNet: A New Model forBiodiversity Data Sharing. PLoS Biol 8(2): e1000309. doi:10.1371/journal.pbio.1000309Published February 16, 2010Copyright:  2010 Constable et al. This is an open-access article distributed under the terms of the CreativeCommons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium,provided the original author and source are credited.Funding: The authors received no specific funding for this article.Competing Interests: The authors have declared that no competing interests exist.Abbreviations: API, application programming interface; DiGIR, distributed generic information retrieval; GBIF,Global Biodiversity Information Facility; IWGSC, Interagency Working Group on Scientific Collections; MaNIS,Mammal Networked Information System; NAS, National Academy of Sciences; NSF, National ScienceFoundation; ORNIS, Ornithological Information System; REST, Representational State Transfer* E-mail: hconstable@berkeley.edu" Membership of the VertNet Steering Committee is provided in the Acknowledgments.PLoS Biology | www.plosbiology.org 1 February 2010 | Volume 8 | Issue 2 | e1000309registration makes the new resourcesavailable, allowing them to be accessedsimultaneously with those of their contrib-uting peers so that each network simulatesaccess to a single data store of all of thecontributors’ data.Collaboration, Training, andData ImprovementOne of the strongest incentives fororganizations to participate in the verte-brate biodiversity networks has been thepromise of improved data quality, focus-ing particularly on converting textuallocality descriptions into spatial formatsfor mapping [11,12]. Through remotecollaborations, 64 institutions to date haveshared the enormous task of georeferen-cing the network contents. Collectively,nearly 4.5 million occurrence recordsfrom 867,000 distinct locations have beengeoreferenced following best practices[13] by leveraging geographic resourcesand expertise at each institution. Since2003, at least 175 undergraduates and282 higher level researchers from 161institutions in 40 countries have beentrained directly through project activities,including 14 international georeferencingworkshops. In addition to increasingtechnical understanding and capacityamong contributors, these collaborationshave produced effective economies ofscale and provided a vibrant exchange ofexpertise among participants. Whereasgeoreferencing was an intentional deliver-able of grants supporting the vertebratenetworks and an obvious benefit forcontributors, participation has also re-vealed unforeseen benefits to the primarydata custodians through data qualityimprovement from user feedback. It hasbecome clear that the vastly increasedexposure and use of the data haverevealed and motivated correction oferroneous information that otherwisemay have gone undetected.New ChallengesThe current networks were designedto rely on live connections to contribu-tors’ server installations to achieve twoFigure 1. Interactions with the data store. The proposed cloud-based architecture will allow contributors to publish primary data to the cloud,augmented with equivalent standard vocabulary values (in green), and with data quality assessments, which add annotations about potential errorsor updates (in blue). Applications can also interact with the data store and contribute recommended improvements, which are then accessible tocontributors and other users alike. For example, the collaborative georeferencing tool GeoLocate (http://www.museum.tulane.edu/geolocate/) couldbe used to add coordinates and uncertainty estimates to the data store as annotations. Data consumers can view data products, such as maps, andthey can also make annotations to the data store. All information is accessible by consumers and data contributors, who have the option of updatingthe primary data with all added value features.doi:10.1371/journal.pbio.1000309.g001PLoS Biology | www.plosbiology.org 2 February 2010 | Volume 8 | Issue 2 | e1000309goals: (1) to support contributor confi-dence that their participation was undertheir own control; and (2) to emphasizethe primacy of the original source. Theunforeseen growth of the networks hasrevealed performance bottlenecks, aswell as other scalability and sustainabilityproblems. Because every request for datais propagated to the networked sources,aggregate responses are limited by theslowest responder, and data from con-tributors not connected at the moment ofthe query are simply unavailable untilthey come back online. These limitationsmake it impractical to provide users withdynamic information about networkcontent in advance of a specific search.Users have no way of knowing theexpected content of individual fields orof overall content of the network, such aswhich countries or taxa are represented,or how many total records are available.Network responsiveness is clearly at themercy of each of its distributed compo-nents.The potential of the networks toimprove the quality of their holdings isalso hampered by the current architec-ture. Although data sets comply with theagreed-upon schema (Darwin Core), datacontent is inconsistent among contribu-tors and has limited quality-assessmentinformation. These deficits could beovercome most effectively with collabora-tive tools and feedback mechanismsassociated with the networks—tools suchas collaborative georeferencing work-benches, vocabulary look-up services,and taxonomic authorities.We have managed the current networksthrough concerted efforts from both net-work and participant personnel. TheNational Biological Information Infra-structure of the US Geological Surveyhas established a programmer position toprovide network support to meet thegrowing demands on the systems. In thefirst nine months following the inception ofthis position in 2008, the programmerprovided support to 73% of the contrib-uting installations. The estimated totalannual operating cost (people and hard-ware) across the four vertebrate networksat the 2009 participation level was$195,600.VertNet as a New Model forBiodiversity NetworksVertNet proposes a new model forbiodiversity networks in which the com-puting resources are consolidated on ‘‘thecloud’’ utilizing the Google App Engineplatform as a service. Cloud computing istypically a pay-per-use model utilizing anInternet-based third party, a dynamicallyscalable and often virtualized computingresource. Such a model removes therequirement and cost to contributors tobuy or maintain their own servers whileleveraging all the data integrity andreplication services provided by the cloud.Under the new model, contributors woulduse a Web-based administrative interfaceto create a ‘‘provider’’ in the cloud. Theprocess would allow contributors to de-scribe (i.e., provide metadata for) theirdata sets, define usage restrictions andcitation information, add contact infor-mation, and later access informationabout usage statistics. After creating theirprovider, contributors would downloadand use a local application to publishnetwork-ready data—data conforming tothe Darwin Core—to the cloud. Subse-quent updates would use the same localapplication to publish only differences(additions, changes, deletions) since theprevious publishing act. The data store inthe cloud will contain the primary datapublished from all contributors as persis-tently available records, uniquely identifi-able by their data store key. The datastore will also contain summary informa-tion about the aggregate of all data andassociated data from other sources, suchas auxiliary data look-ups, user feedback,and data quality assessments. All of thesewould be accessible and downloadablefrom the data store through an Applica-tion Programming Interface (API) andthrough Representational State Transfer(REST).VertNet is a radical departure from thecurrent model, in which Web portalsquery data from contributors’ serverinstallations on demand to one in whichcontributors publish to a cloud-baseddata store. Network performance andscalability issues will be alleviated underthis model, while the ‘‘traditional’’ pri-macy of the sources will be maintainedthrough contributor-mediated updates.Further, cloud-based annotation toolswill enable users to flag suspect recordsso that collection-based curators cancheck and correct data at the source.Consolidation in the cloud will create avariety of improvement opportunitiesthat are impractical under the currentmodel. Contributor-mediated publishingintroduces the feasibility of adding auto-mated data quality improvement servicesto the publishing workflow. The publish-ing activity will make it possible for thefirst time to alert users when data ofinterest change or enter the network (newrecords in an area or place of interest).The consolidated data store will make itpossible to determine the nature of thecontent of the network as a whole(vocabularies, record counts, net rate ofchange of information, uniqueness mea-sures of given contributor’s data in thecontext of the entire network). Thecombined data store also will facilitatecollaboration by providing not only aplatform on which to store the results ofcollaborations (georeferences, user-pro-vided annotations) associated directlywith the primary data they are meant toimprove, but also a platform on which tobuild innovative applications (e.g., anal-ysis, visualizations, workflows; Figure 1).The new model supports all of thesebenefits at an estimated 16-fold reductionin annual operating costs.ConclusionThe vertebrate networks represent asocial and technological success story. Inonly eight years, a global community ofeager contributors has managed tomobilize an impressive contribution ofpublicly accessible biodiversity data in astandard format. The success began byunderstanding the scope, sociologicalrequirements, and technological con-straints of the community we were tryingto serve and was made manifest throughhard work and contributions by asupportive community. By ensuring thatdata remain curated at the source, andby showing the importance of datasharing to promote data citation andusage, we have grown past our originaltechnology implementation and areready to move into a long-term produc-tion environment that departs from theoriginal model. The new cloud-basedarchitecture promises to be sustainableand scalable far into the future. Webelieve the development process here isnot unique to biodiversity data, rather,past successes, current challenges, andnew solutions may all provide usefullessons and approaches to other com-munities that are coalescing to sharedata.AcknowledgmentsWe’d like to thank National Biological Infor-mation Infrastructure for support of the Vert-Net project and the VertNet Steering Commit-tee for leadership and guidance. Besides theauthors listed in the authorship list, the SteeringCommittee includes Hank Bart, John Bates,Gladys Cotter, James Hanken, Craig Moritz,Nancy Simmons, and Linda Trueb.PLoS Biology | www.plosbiology.org 3 February 2010 | Volume 8 | Issue 2 | e1000309References1. Wilson EO (1999) The diversity of life. New York,NY: W.W. Norton & Co. 432 p.2. Jenkins M (2003) Prospects in biodiversity. Science302: 1175–1177. doi:10.1126/science.1088666.3. Loreau M, Oteng-Yeboah A, Arroyo MTK,Babin D, Barbault R, et al. (2006) Diversitywithout representation. Nature 442: 245–246.4. National Academy of Sciences (2001) Grandchallenges in environmental sciences. WashingtonDC: National Academy Press. 106 p.5. Global Biodiversity Information Facility (2008)GBIF Work Programme 2009-2010. Availa-ble:http://www2.gbif.org/WP2009-10.pdf. Ac-cessed 20 Jan 2010.6. Interagency Working Group on Scientific Col-lections (IWGSC), National Science and Tech-nology Council, Committee on Science (2009)Scientific collections: mission-critical infrastruc-ture of federal science agencies. Office ofScience and Technology Policy, Washington,DC. 50 p.7. Pennisi E (2005) Boom in digital collections makesa muddle of management. Science 308: 187–189.doi:10.1126/science.308.5719.187.8. Stein BR, Wieczorek J (2004) Mammals of theworld: MaNIS as an example of data integrationin a distributed network environment. Biodiver-sity Informatics 1: 14–22.9. Taxonomic Data Working Group (2009) Darwincore task group, biodiversity information stan-dards - Darwin Core. Available: http://rs.tdwg.org/dwc. Accessed 14 Oct 2009.10. Distributed Generic Information Retrieval - Di-GIR (2001) Available: http://digir.net/. Accessed20 Jan 2010.11. Wieczorek JR, Guo Q, Hijmans RJ (2004) Thepoint-radius method for georeferencing localitydescriptions and calculating associated uncertain-ty. Int J Geogr Inf Sci 18: 745–767.12. Guralnick R, Wieczorek J, Hijmans RJ,Beaman R, the Biogeomancer Working Group(2006) Biogeomancer: automated georeferencingto map the world;s biodiversity data. PLoSBiology 4(11): e381. doi:10.1371/journal.pbio.0040381.13. Chapman AD, Wieczorek J (2006) Guide to bestpractices in georeferencing. Copenhagen: GlobalBiodiversity Information Facility. 86 p.PLoS Biology | www.plosbiology.org 4 February 2010 | Volume 8 | Issue 2 | e1000309

VertNet: A New Model for Biodiversity Data Sharing

https://kuscholarworks.ku.edu/bitstream/1808/13190/1/Constable_2010.pdf

VertNet: A New Model for Biodiversity Data Sharing

Abstract

Similar works

Full text

Available Versions

KU ScholarWorks