This is the final version. Available from Wiley via the DOI in this record. National Institute for Health ResearchScience for Nature and People Partnershi

Anzaroot, S

Augustin, C

Bethel, A

Brun, J

Cheng, SH

DeWilde, B

Garside, R

Gill, D

Masuda, YJ

McKinnon, MC

Miller, DC

Minnich, RC

Wilkie, D

Wongbusarakum, S

Conservation Biology

English

Open Research Exeter

LetterUsingmachine learning to advance synthe-sis and use of conservation and environ-mental evidenceRapid growth in environmental research (Li & Zhao 2015)presents a potential wealth of information for use in con-servation decision making. Evidence synthesis methods(e.g., systematic maps, reviews, meta-analyses) (Pullin &Knight 2009) are critical for garnering actionable insightfrom published research, yet they require levels of timeand funding that are prohibitive for meeting short pol-icy windows (Elliott et al. 2014) and balancing trade-offsbetween conservation planning and implementation.In response, interest in machine learning to make syn-theses faster and more efficient is growing (O’Mara-Eveset al. 2015). Machine learning (ML) is based on the ideathat computers can be programmed to automatically per-form a set of tasks by learning from a set of rules andtraining data (Alpaydin 2014). For example, ML couldbe used to synthesize information by semiautomaticallyfinding articles relevant to users and even to summarizeinformation—potentially reducing time and bias and im-proving overall cost-effectiveness. Machine learning hasbeen widely applied in public health and syntheses ofmedical information and is beginning to be trialed inconservation and environmental topics (Westgate et al.2015; Roll et al. 2017). Bearing the challenges inmind,weendeavored to design a platform, powered by machinelearning, to improve on themanual synthesis process.Wepartnered with DataKind, a data-science nonprofit orga-nization, to create an open-access platform, Colandr, toaddress 2 laborious stages of information synthesis: find-ing relevant articles and extracting desired information.Colandr has 2 learning systems, the first iteratively sortsarticles by relevance as specified by users and the secondaids in categorizing article topics (Fig. 1). To illustrateColandr’s functionality, we used data from a systematicmap on linkages between conservation and human well-being (McKinnon et al. 2016).Reviewers typically sort through thousands of searchresults to find relevant articles, an inefficient process thatoften takes several months. For example, our search onconservation and human well-being recovered 35,000hits, of which only 1,000 were relevant. System 1 inArticle impact statement: Machine learning optimizes processes of systematic evidence synthesis and improves its utility for evidence-basedconservation.Colandr aims to speed up this process by dynamicallysorting the wheat from the chaff based on user input.As users indicate whether citations are relevant or irrel-evant, system 1 calculates the expected relevance of theremaining search results and dynamically pushes morerelevant citations to the top. Colandr does this by search-ing for patterns in the words around search terms (e.g., itidentifies the words and the order of those words aroundprotected area) and learning which of these combina-tions are more relevant to the user, a method calledword2vec (Mikolov et al. 2013). System 1 achieved a5-fold reduction in effort with our systematic map dataset. Manually, reviewers had to read 1,436 citations be-fore they recovered 100 relevant articles. Using Colandr,reviewers recovered 100 relevant citations after readingonly 250 citations.After finding relevant studies, reviewers embark onpulling out desired information (e.g., bibliographic, top-ical, results) from each article. For example, we cate-gorized articles according to topic area (e.g., types ofconservation and human well-being). Typically, review-ers read entire articles to categorize them, a very time-consuming process. System 2 is designed to deducethese categories faster by pulling sentences from articlesthat it identifies as relevant to each category with theglobal vectors for word representations (GLoVE) model(Pennington et al. 2014). For example, Colandr will dis-play sentences related to the law and policy categoryfor users to read and help them decide whether to ac-cept or reject that category. As users continue to cat-egorize articles, model confidence improves. Althoughthis approach does not necessarily improve speed, itcan improve accuracy by catching missing or mislabeledcategories.Colandr semiautomates the synthesis process, but itcontinues to retain significant user oversight to ensuretransparency. This is critical because conservation andenvironmental terms often have alternativemeanings andmany synonyms. For example, there are many differ-ent types of protected areas (e.g., key biodiversity ar-eas, reserves, no-take zones, biospheres, national parks),whereas there are only 2 ways to refer to influenza, in-fluenza or flu. This heterogeneity makes it harder (butnot impossible) to make automatic predictions with high762Conservation Biology, Volume 32, No. 4, 762–764C© 2018 Society for Conservation BiologyDOI: 10.1111/cobi.13117Cheng et al. 763SEARCH RESULTS    Bibtex format (.bib)SYSTEM 1: CITATION SCREENINGRELEVANCE RANKING: 2nd STAGE= INPUTS= OUTPUTS= = DOWNLOADScolandrLIST OF INCLUDED ARTICLES FOR FULL TEXT ASSESSMENT CODING RECORD FOR ALL CITATIONSCSV format (.csv) RELEVANCE RANKING: 1st STAGEARTICLES FOR FULL TEXT ASSESSMENT    Portable document format (.pdf )RANKED LIST OF CITATIONSRE-RANKED LIST OF CITATIONS== UPLOADSUser accepts or rejects predicted categoriesDESIRED DATA FROM INCLUDED ARTICLESComma-separated values (.csv)Excel & R compatible    User manually categorizes at least 25 articles to train modelDISPLAYS SENTENCES RELATED TO PREDICTED CATEGORIESLEGENDCOMPUTER ASSISTANCEINFORMING MODELinputoutputSYSTEM 2: DATA EXTRACTIONUser decides when to stop screeningFOR EXAMPLE:Land/water protectionLand/water managementSpecies managementResource managementWHAT CONSERVATION INTERVENTION(S) IS THIS ARTICLE ABOUT?WHAT HUMAN WELL-BEING OUTCOME(S)?Education/awarenessLaw & policyEconomic incentivesExternal capacity buildingEconomic living standardsMaterial living standardsHealthEducationSocial relationsSecurity & safetyGovernanceSubjective well-beingCultural/spiritual valuesFreedom of choice/actionMODEL TRAININGINFORMS MACHINE LEARNING MODEL(GLoVE)MODEL TRAININGUsers screen list and mark citations as included, excluded, and unknown. At least INFORMS MACHINE LEARNING MODEL(Word2Vec)Program actively learns with user input to provide a dynamic list ranked by relevance Users continue to screen citations as included, excluded, and unknown, continually informing the model and dynamically re-ranking list of citationsRELEVANCE TO TOPICFigure 1. Colandr is composed of 2 systems in which user input iteratively trains the machine-learning models. Insystem 1, articles are ranked by relevance. In system 2, topic labels are predicted for each article. Arrows illustratethe flow of inputs and outputs during a systematic evidence synthesis project and where model training occurs.levels of confidence. Thus, because this process is lessprecise, Colandr aims to retain user oversight to ensurethat relevant articles are not missed.Preliminary tests of Colandr demonstrate significantimprovements over a manual process. Although such as-sessments have not been exhaustive, they demonstrateColandr’s potential to help advance evidence-based deci-sion making in conservation by removing resource barri-ers to conducting comprehensive evidence syntheses.AcknowledgmentsWe thank the Science for Nature and People Partnership(SNAPP), a partnership of The Nature Conservancy, theWildlife Conservation Society, and the National Centerfor Ecological Analysis and Synthesis (NCEAS) at the Uni-versity of California, Santa Barbara, for providing fundingfor the Evidence-Based Conservation working group. De-velopment and deployment of Colandr was conductedin partnership with DataKind and Conservation Interna-tional. We especially thank S. Sagalovsky for front-enddevelopment of Colandr and E. Fegraus at ConservationInternational. Finally, we thank A. Pullin, M, Balisi, A.Fritts-Penniman, and S. Bittick for providing commentson earlier drafts of this manuscript. R.G. is partially sup-ported by the National Institute for Health Research(NIHR) Collaboration for Leadership in Applied HealthResearch and Care (CLAHRC) for the South West Penin-sula (PenCLAHRC).S.H. Cheng ,1 C. Augustin,2 A. Bethel,3 D. Gill,4,5S. Anzaroot,2 J. Brun,1 B. DeWilde,2 R.C. Minnich,2R. Garside,6 Y.J. Masuda,7 D.C. Miller,8 D. Wilkie,9S. Wongbusarakum,10 and M.C. McKinnon111National Center for Ecological Analysis and Synthesis, University ofCalifornia, Santa Barbara. 735 State Street, Suite 300, Santa Barbara, CA93101, U.S.A., email samantha.cheng@asu.edu2DataKind, 156 5th Avenue, Suit 502, New York, NY 10010, U.S.A.3University of Exeter Medical School, Heavitree Road, Exeter, EX1 2LU,U.K.4Conservation International, 2011 Crystal Drive, Suite 500, Arlington,VA 22202, U.S.A.5Environmental Science and Policy, George Mason University, Fairfax,VA 22030, U.S.A.6European Centre for Environment and Human Health, University ofExeter, Truro, Cornwall, TR1 3HD, U.K.7The Nature Conservancy, 4245 Fairfax Drive, Arlington, VA 22203,U.S.A.8Department of Natural Resources and Environmental Sciences, Uni-versity of Illinois, S-406 Turner Hall, 1102 S. Goodwin Avenue, Urbana,IL 61801, U.S.A.9Wildlife Conservation Society, 2300 Southern Boulevard, Bronx, NY10460, U.S.A.10Social Science Research Institute, University of Hawaii, 2424 MaileWay #704, Honolulu, HI 96822, U.S.A.11Vulcan, Inc., 505 Fifth Avenue S, Seattle, WA 98104, U.S.A.Literature CitedAlpaydin E. 2014. Introduction to machine learning. MIT Press, Cam-bridge.Elliott JH, Turner T, Clavisi O, Thomas J, Higgins JPT, MavergamesC, Gruen RL. 2014. Living systematic reviews: an emerging op-portunity to narrow the evidence-practice gap. PLoS Medicine11(2):e1001603. https://doi.org/10.1371/journal.pmed.1001603.Conservation BiologyVolume 32, No. 4, 2018764 LetterLi W, Zhao Y. 2015. Bibliometric analysis of global environmental as-sessment research in a 20-year period. Environmental Impact As-sessment Review 50:158–166.McKinnon MC, et al. 2016. What are the effects of nature conservationon human well-being? A systematic map of empirical evidence fromdeveloping countries. Environmental Evidence 5:8.Mikolov T, Sutskever I, Chen K, Corrado G, Dean J, Chen K, Dean J,Mikolov T, Chen K. 2013. Distributed representations of words andphrases and their compositionality. NIPS’13 Proceedings of the 26thInternational Conference on Neural Information Processing Systems2:3111–3119.O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. 2015.Using text mining for study identification in systematic reviews: asystematic review of current approaches. Systematic Reviews 4:5.Pennington J, Socher R, Manning CD. 2014. GloVe: global vectors forword representation. Pages 1532–1543. Proceedings of the 2014Conference on empirical methods in natural language processing(EMNLP). Association for Computational Linguistics, Doha, Qatar.Pullin AS, Knight TM. 2009. Doing more good than harm - Buildingan evidence-base for conservation and environmental management.Biological Conservation 142:931–934.Roll U, Correia RA, Berger-Tal O. 2017. Using machine learning to dis-entangle homonyms in large text corpora. Conservation Biology32:716–724.Westgate MJ, Barton PS, Pierson JC, Lindenmayer DB. 2015. Textanalysis tools for identification of emerging topics and researchgaps in conservation science. Conservation Biology 29:1606–1614.Conservation BiologyVolume 32, No. 4, 2018

Using machine learning to advance synthesis and use of conservation and environmental evidence

https://ore.exeter.ac.uk/repository/bitstream/10871/121588/1/Using%20machine%20leraning%20to%20advance%20synthesis.pdf

Using machine learning to advance synthesis and use of conservation and environmental evidence

Abstract

Similar works

Full text

Available Versions

Open Research Exeter