In this talk, I will discuss open science practices that aim to foster sustainability and transparency in computational cognitive neuroscience. First, I will review recent community efforts that aim to ease data sharing and analytical reproducibility, such as the reports of the OHBM Committees on Best Practice in Data Analysis and Sharing (COBIDAS) and the Brain Imaging Data Structures (BIDS). Second, I will discuss neuroimaging data sharing strategies in the light of ethical and legal constraints, such as the European General Data Protection Regulation (GDPR). Finally, I will discuss some common-sense guidelines for day-to-day research practice that aim to maximize the societal impact of computational cognitive neuroscience

Ostwald, D.

English

MPG.PuRe

OPEN DATA Sustainability and transparency in computational cognitive neuroscienceOPEN DATAQuality improvement • Reproducibility, methodological rigor, error discovery, …Cost reduction• Redundancy decrease, resource re-allocation, …Interdisciplinarity • Mathematics, statistics, computer science, …Progress acceleration• Meta-analyses, multi-site generalization, power, ....Zeitgeist• Funding agency demands, ...Poline et al. 2012; Brakewood & Poldrack, 2013⇒ Cognitive neuroimaging has to adopt an open science standard!OutlineCommunity efforts: COBIDAS and BIDSEthical and legal challengesScientific common senseMUSTSHOULDDISCUSSOutlineCommunity efforts: COBIDAS and BIDSEthical and legal challengesScientific common senseCOBIDAS backgroundPreemptive reaction of OHBM to perceived „reproducibility crisis“OHBM Council Statement on Neuroimaging Research and Data Integrity (2014) Committee on Best Practices in Data Analysis and Sharing (COBIDAS)• Identify best practices of data analysis and data sharing in brain mapping• Preparation of a white paper organizing and describing these practices• Seeking input from the OHBM community• Publishing the recommendationsCOBIDAS backgroundCommittee on Best Practices in Data Analysis and Sharing (COBIDAS)• „Proposed“ version of the report posted in October 2015• Comments collected via the COBIDAS blog (https://cobidas.wordpress.com/ )• Ratification in May 2016 with 96% positive ballots• Voting turnout 153 yes, 6 no (≈3,000 attendees of OHBM conference/year)COBIDAS backgroundCommittee on Best Practices in Data Analysis and Sharing (COBIDAS)• Increase transparency by comprehensive sharing of data, methods, and results• Recommendations and checklists to increase reproducibility• Tool, not law!• Poldrack et al. (2008) Guidelines for reporting an fMRI study NeuroimageCOBIDAS backgroundDefining reproducibilityISO* RepeatabilityPrecision of measurements under conditions where independent test/measurements results areobtained with the same method on identical test/measurement items in the same test ormeasuring facility by the same operator using the same equipment within short intervals of time.ISO ReproducibilityPrecision of measurements under conditions where independent test/measurement results areobtained with the same method on identical test/measurement items in different test ormeasurement facilities with different operators using different equipment.Computational ReproducibilityIndependent researchers use the exact same data and code to arrive at the original result (Peng(2011) Reproducible research in computational science, Science).ReplicabilityIndependent researchers use independent data and possibly distinct methods to arrive at thesame original conclusion.*International Standards OrganizationCOBIDAS content1) Experimental design reporting2) Acquisition reporting3) Preprocessing reporting4) Statistical modelling & inference5) Results reporting6) Data sharing7) ReproducibilityCOBIDAS content1) Experimental design reportingCOBIDAS content2) Acquisition reporting3) Preprocessing reporting4) Statistical modelling & inference5) Results reportingKnow exactly whatyour tool is doing!Transparency= Methodological knowledgeCOBIDAS content6) Data sharing… does not mean „available upon request“ … should adhere to the FAIR principles and use a standardized data format… should be an essential project component (data management plan)… may use appropriate licenses, such as Creative Commons… should include data analysis pipeline scripts… should happen at the time of study publication… should respect the appropriate ethical and legal constraintsCOBIDAS content7) Reproducibility… stresses „computational reproducibility“ … requires the analysis scripts to be documented, archived, and citable… may involve creation and release of virtual machines (VM)… should us Digital Object Identifiers for permanent citabilityCOBIDAS summary• Best practices in data analysis and sharing in neuroimaging using MRI • Comment in Nature Neuroscience Vol 20, pages299–303 (2017)• www.biorxiv.org/content/biorxiv/early/2016/07/10/054262.full.pdf• A good starting point for writing neuroimaging methods sections• Fairly silent on practices of data sharing• Relatively low community engagementOutlineCommunity efforts: COBIDAS and BIDSEthical and legal constraints: GDPRScientific common sense: Personal viewsBIDSBIDS = Brain Imaging Data StructureA standard for organizing data of human neuroimaging experimentshttp://bids.neuroimaging.io/Gorgolewski, K, … , Poldrack, R (2016) The brain imaging data structure, a format for organizing anddescribing outputs of neuroimaging experiments Scientific Data Volume 3, Article number: 160044http://reproducibility.stanford.edu/BIDSCore principlesFoster wide adoption by simplicity and familiarity (openfMRI)Some metadata is better than no metadataIndependence of external software or complicated file formatsCapture 80% of experiments, but allow extension for the restBIDSImplementationSome metadata encoded in folder structure and replicated in filenamesTab separated files for tabular dataCompressed NIFTI files for neuroimaging data JSON files for dictionary-type metadataText file formats for behavioral and physiological dataFew musts, many can dosBIDSFolder organizationsub-control001/anat/func/sub-control001_T1w.nii.gzsub-control001_T1w.json← Data← Participant data folder← Anatomical data folder← Metadatasub-control001_nback_bold_01.nii.gzsub-control001_nback_bold_01.jsonsub-control001_nback_bold_02.nii.gzsub-control001_nback_bold_02.json← Data← BOLD data folder← Metadata← Data← MetadataBIDSFolder organizationBIDSEvents fileOnset Duration Trial_Type Response Time1.2 0.7 easy 1.245.6 0.7 hard 2.109.2 0.7 hard 1.9211.1 0.7 easy 0.9815.0 0.7 easy 1.3418.3 0.7 hard 1.97… … … …BIDSMetadata JSON file (key-value data/structure format){„Repetition Time“ : 2 ,„EchoTime“ : 0.03 ,„FlipAngle“ : 79 ,„Slice Timing“ : [0.0, 0.2, …, 1.8,2.0] ,„MultibandAcceleration“ : 4 ,……}More BIDS …BIDSSummaryJUST DO IT.OutlineCommunity efforts: COBIDAS and BIDSEthical and legal challengesScientific common senseEthical and legal challengesData privacyprotectionData sharing in basic research≠ ≠Ethical and legal challengesData privacyprotectionData sharing in basic researchOur task!Ethical and legal challengesEuropean General Data Protection RegulationWhat is it?• Regulation 2016/679 by EU parliament, council, commission• Legally binding and applicable from May 25th, 2018• EU reaction to global surveillance disclosuresWhat are its aims?• Strengthen and unify data protection for individuals within EU• Empower citizens in control of their digital dataEthical and legal challengesEuropean General Data Protection RegulationWhat is its relation to neuroimaging data sharing?• GDPR applies to EU data collectors, processors, or subjects• Personal data := Any information relating to an individual• Name, address, photo, medical information, posts, ...• Sanctions and fines• Designed independently of open science principles Ethical and legal challengesEuropean General Data Protection Regulation What are its main principles?• Processing of personal data is prohibited, unless consent is provided• Data collection only for specified, explicit, and legitimate purposes• Data processing has to ensure appropriate security• The data controller is responsible for GDPR compliance• Science has a degree of priviledge for processing personal data [GDPR Articles 4, 5, 9, 89]Ethical and legal challengesElena Pavlenko & Dirk Ostwald Cognitive Neuroimaging Data Sharing: A European PerspectiveParticipant consent Data storage Data sharing• What does the GDPR specify?• What does the GDPR not specify?Cognitive Neuroimaging Data LifecycleEthical and legal challenges• The GDPR does not apply to anonymous data• Can functional neuroimaging data be anonymized?• “… [Anonymous data is] information which does not relate to an identified oridentifiable natural person or to personal data rendered anonymous in such amanner that the data subject is not or no longer identifiable. […] To ascertainwhether means are reasonably likely to be used to identify the natural person,account should be taken of all objective factors, such as the costs of and theamount of time required for identification, taking into consideration the availabletechnology at the time of the processing and technological developments.”[GDPR Article 4, Recital 26]Data storage | Anonymous dataEthical and legal challengesAnonymity and anatomical dataAnonymity/Identifiability and functional (BOLD) data (e.g. Finn et al. (2015))Data storage | Anonymous dataCan functional neuroimagingdata be anonymized? Ethical and legal challengesSome GDPR legal uncertainties for cognitive neuroimaging• Can functional neuroimaging data be anonymized?• How broad can the purpose of data processing be specified? • How specific do possible data users be named?• How can participant rights be maintained upon data sharing?• Can participant consents partially overrule participant right?Public SharingRestricted SharingDynamic SharingOpen data modesDisadvantages• No form of data usage control• GDPR-compatible consent difficult• Risk of data abuse• Data processing for all purposesAdvantages• Simplicity• Objective data usage rights• Small cost and effort• Data processing for all purposesPublic SharingUnrestricted access for anyone with an internet connectionOpen data modesResearcher-regulated password protected data access + data use agreementAdvantages• Researcher in control• Low risk of data abuse• Subjective data usage rights• Data use agreementsDisadvantages• Researcher effort• Researcher responsibility• Subjective data usage rights• Ressource intensiveRestricted SharingOpen data modesRestricted SharingOpen data modesVirtual labResearcher-regulated password protected data access + data use agreementAdvantages• Empowerment• Subjective data usage rights• Researchers not in charge• Participant engagementDisadvantages• Digital divide• Subjective data usage right• Effort for participants• Ressource intensive⇒ 21st century value of and responsibility for one‘s own digital dataDynamic SharingOpen data modesParticipant-regulated password protected data access + data use agreementPublic SharingRestricted SharingDynamic SharingOpen data modesMax PlanckOutlineCommunity efforts: COBIDAS and BIDSEthical and legal constraints: GDPRScientific common senseScientific common senseSustainability in (computational) cognitive neuroscience Day-to-day practice Strategic choices Scientific philosophya.k.a. my personal vi wsScientific common sense Day-to-day practice Use standard data organization Do not use toolboxes for central analyses Write clean, understandable code Set up result-independent win-win situations Sign your reviews Don’t believe in papers, because they are peer-reviewed Acquire the language to talk about computational modelsScientific common sense Strategic decisions Work with supervisors who work transparently Start university teaching Emphasize Science over Academia Shape the open research movementScientific common sense Scientific philosophy Science is communication, not the pursuit of absolute truth There is no “correct” statistical model or analysis Embrace uncertainty and modesty Don’t buy into the competitiveness noiseData Model„Statistics“„Probability modelling“„Multivariate models“„Dynamical models“„Quantitative theories“...ScienceResulte.g. p < 0.05 Scientific common sense46RealityData Model„Statistics“„Probability modelling„Multivariate models“„Dynamical m dels“„Quantitative theories“...ScienceResultሶ𝑥 𝑡 = 𝑎𝑥 𝑡ො𝑎 = 2.1Communication Scientific philosophySummaryCommunity efforts: COBIDAS and BIDSEthical and legal challengesScientific common senseMUSTSHOULDDISCUSSSeptember 2019

Sustainability and transparency in computational cognitive neuroscience

Abstract

Similar works

Full text

Available Versions

MPG.PuRe