Prereg posters are conference posters that present planned scientific projects. We provide preliminary evidence for their value in receiving constructive feedback, promoting open science, and supporting early career researchers

Brouwers, Kimberly

Chambers, Christopher D

Cooke, Anne

Henson, Richard

Tibon, Roni

English

Prereg posters are conference posters that present planned scientific projects. We provide preliminary evidence for their value in receiving constructive feedback, promoting open science and supporting early-career researchers

Chambers, Christopher D.

Online Research @ Cardiff

This is a n  Op e n  Acces s  doc u m e n t  dow nloa d e d  fro m  ORCA, Ca r diff U nive r si ty 'sins ti t u tion al r e posi to ry: h t t p s://o rc a .c a r diff.ac.uk/id/e p rin t/13 1 2 5 8/This  is t h e  a u t ho r’s ve r sion  of a  wo rk  t h a t  w as  s u b mi t t e d  to  / a c c e p t e d  forp u blica tion.Cit a tion  for  final p u blish e d  ve r sion:Brouw e r s ,  Kimb e rly, Cooke,  Anne,  Ch a m b e r s,  Ch ris top h e r  D. ORCID:h t t p s://o rcid.o r g/00 0 0-0 0 0 1-6 0 5 8-4 1 1 4,  H e n so n,  Rich a r d  a n d  Tibon,  Roni2 0 2 0.  Evide nc e  for  p r e r e g  pos t e r s  a s  a  pla tfo r m  for  p r e r e gis t r a tion.  N a t u r eH u m a n  Beh aviou r  4  , p p.  8 8 4-8 8 6.  1 0.10 3 8/s4 1 5 6 2-0 2 0-0 8 6 8-z file  P u blish e r s  p a g e:  h t t p://dx.doi.or g/10.10 3 8/s4 1 5 6 2-0 2 0-0 8 6 8-z< h t t p://dx.doi.o rg/10.10 3 8/s4 1 5 6 2-0 2 0-0 8 6 8-z >Ple a s e  no t e:  Ch a n g e s  m a d e  a s  a  r e s ul t  of p u blishing  p roc e s s e s  s uc h  a s  copy-e di ting,for m a t ting  a n d  p a g e  n u m b e r s  m ay no t  b e  r eflec t e d  in t his  ve r sion.  For  t h ed efini tive  ve r sion  of t his  p u blica tion,  ple a s e  r ef e r  to  t h e  p u blish e d  sou rc e.  Youa r e  a dvise d  to  cons ul t  t h e  p u blish e r’s ve r sion  if you  wish  to  ci t e  t his  p a p er.This ve r sion  is b ein g  m a d e  av ailable  in  a cco r d a n c e  wit h  p u blish e r  policie s.S e e  h t t p://o rc a .cf.ac.uk/policies.h t ml for  u s a g e  policies.  Copyrigh t  a n d  m o r al  r i gh t sfor  p u blica tions  m a d e  available  in ORCA a r e  r e t ain e d  by t h e  copyrig h thold e r s .1   Evidence for prereg posters as a platform for preregistration Kimberly Brouwers1, Anne Cooke2, Christopher D. Chambers3, Richard Henson4,5 & *Roni Tibon4 1School of Medicine, Medical Sciences and Nutrition, University of Aberdeen, UK, AB24 3FX 2British Neuroscience Association, The Dorothy Hodgkin Building, University of Bristol, Whitson Street, Bristol, UK, BS1 3NY  3Cardiff University Brain Research Imaging Centre (CUBRIC), School of Psychology, Cardiff University, UK, CF10 3AT 4MRC Cognition & Brain Sciences Unit, University of Cambridge, UK, CB2 7EF 5Department of Psychiatry, University of Cambridge, UK, CB2 0SZ  *Corresponding author:  Dr. Roni Tibon MRC Cognition & Brain Sciences Unit University of Cambridge 15 Chaucer Road Cambridge, CB2 7EF UNITED KINGDOM Telephone: +44 1223 355 294 Email: roni.tibon@mrc-cbu.cam.ac.uk  Abstract Prereg posters are conference posters that present planned scientific projects. We provide preliminary evidence for their value in receiving constructive feedback, promoting open science, and supporting early career researchers.    .CC-BY 4.0 International licensewas not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprint (whichthis version posted November 11, 2019. . https://doi.org/10.1101/833640doi: bioRxiv preprint 2   In recent years, it has been repeatedly shown that the results of many scientific studies fail to replicate1,2,3. The growing awareness of this problem has prompted several attempts to tackle it. One way is to submit a registered report to a journal, which is peer-reviewed and has the opportunity to be revised before any data are collected3,4,5,6. Not only does the peer-review potentially improve the study design and analyses, but once accepted, the design and analyses are “locked”, preventing researchers from adjusting their hypotheses or analyses post hoc, i.e., after having seen the data (reporting such additional analyses is allowed, but only when transparently labelled as “exploratory”). Moreover, the journal is committed to publishing the results, whatever the outcome, which reduces the “file-drawer” problem of many null results not being published7. Registered reports, first offered by the journal Cortex, are now supported by over 200 peer-reviewed journals, including Nature Human Behaviour. Another approach is to preregister studies on a publically-accessible platform such as the Open Science Framework6,8. Though they can be locked, such preregistered studies are not normally peer-reviewed prior to data collection, and they are not binding for any subsequent publications. Indeed, substantial deviations from preregistered plans have been observed9. Two likely reasons for the large number of deviations are immature preregistration (e.g., of a protocol that was not yet finalised) and acceptance of post-preregistration feedback (e.g., following presentation of data at a conference). Recently, we proposed another possible avenue to help preregistration, namely presenting posters at academic conferences about planned research, before data are collected10; so-called “prereg posters”. This allows presenters to receive feedback on their hypotheses, design and analyses from their colleagues (conference attendees), which is likely to improve the study. In turn, this can improve more formal preregistration, reducing the chances of subsequent deviation, and/or facilitate submission of the work as a registered report. Moreover, colleagues with shared scientific interests become aware of the study early on, which can open the door to collaborations. Prereg posters were recently adopted by the BNA2019 Festival of Neuroscience (https://meetings.bna.org.uk/bna2019/), a biennial event organised by the British Neuroscience Association (BNA). Such posters have also been allowed by at least three other conferences since then: FLUX (https://fluxsociety.org/2019-new-york), BACN (https://www.bacn.co.uk/conferences) and ICON (https://www.helsinki.fi/en/conferences/international-conference-of-cognitive-neuroscience-2020). The BNA2019 Festival organisers additionally collected informal survey data about prereg posters, which we report here as preliminary evidence for their value.  Nearly a fifth (100/491) of all submitted posters at the BNA2019 Festival conformed to the new prereg format, covering a diverse range of neuroscience topics and disciplines, and the overall impression .CC-BY 4.0 International licensewas not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprint (whichthis version posted November 11, 2019. . https://doi.org/10.1101/833640doi: bioRxiv preprint 3   was that they were enthusiastically welcomed. The informal survey data were collected at two time points (before and after the event), via two different online surveys. The full set of questions and response data are available here: https://osf.io/3h6w8/. The first informal survey (pre-conference survey) was administered three months prior to the event, and offered to all 445 participants whose poster was accepted. Of the 200 who responded, 151 were presenters of traditional posters and 49 of prereg posters. The second informal survey (post-conference survey) was administered one month after the event, and was completed by 95 participants, 66 of whom were presenters of traditional posters, and 29 of prereg posters. Below, we present data from the questionnaire items that address three main themes: prereg posters as means to (1) receive valuable feedback, (2) promote open science, and (3) support early career researchers (ECRs).  Getting valuable feedback  As a presenter, the motivation to present a prereg poster is clear: it allows one to get feedback on work at early stages. Indeed, 33% of the 123 responses to the pre-conference question “why did you submit a prereg abstract instead of a traditional abstract?” indicated that this choice was made in order to get feedback (either in general or on specific analyses) before completing the research. As shown in Figure 1, the groups differed in the type of feedback that they subsequently received at the conference. Interestingly, presenters of traditional posters mostly received feedback regarding future projects, whereas presenters of prereg posters mostly received feedback regarding methods and experimental design.   -- Insert Figure 1 about here --  Furthermore, the groups gave similar estimations of the likelihood of the presented research being published in a peer-reviewed journal upon completion (prereg: Median = 6 [scale: 1 – “highly unlikely” to 7 – “highly likely”], IQR = 2; traditional: Median = 6, IQR = 2). This is even though projects presented as prereg posters were at their early stages, and in most cases were still prior to data collection. While it could be the case that researchers’ estimation of publication likelihood remains stable throughout the course of a project, we speculate that it could also reflect the presenters’ trust in the preregistration system, where publication does not depend on results, and/or the presenters’ impression that they received valuable feedback that would promote their ability to publish the work later.   .CC-BY 4.0 International licensewas not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprint (whichthis version posted November 11, 2019. . https://doi.org/10.1101/833640doi: bioRxiv preprint 4   One potential objection to prereg posters is that conference attendees may not bother to discuss “half-baked” work and therefore there is no point in “wasting” conference space on posters that would not attract visitors. However, responses to two questions in the post-conference survey—“did you receive feedback on your work during the poster session?” and “how many people talked to you about your poster during the poster session?”—suggest that this is not the case: prereg posters did not receive less feedback than traditional posters (prereg = 75%; traditional = 68%), nor fewer visitors (prereg: Median category = 2 [5-8 people], IQR = 1; traditional: Median Category = 2 [5-8 people], IQR = 1), suggesting that conference attendees do not avoid prereg posters, despite the fact that these posters present planned or preliminary work.  Promoting open science  Our second theme concerns attitudes among the participants towards open science in general, and preregistration in particular. Two questions from the post-conference survey were indicative of such attitudes: (1) “to what degree would you say that preregistration of work is necessary in today's neuroscience community?” and (2) “to what degree did BNA2019 increase your awareness of preregistration?”. In general, presenters of both prereg and traditional posters agreed that preregistration is necessary (prereg: Median = 4 [scale: 1 - “not at all” to 5 – “incredibly necessary”], IQR = 1; traditional: Median = 4, IQR = 2). Moreover, participants in both groups indicated that the event increased their awareness of preregistration, and the increment was somewhat greater for prereg poster presenters (prereg: Median = 6 [scale: 1 - “not at all” to 7 – “completely”], IQR = 2; traditional: Median = 5, IQR = 2.75). Furthermore, in the pre-conference survey, 61% of prereg posters submitters indicated that one reason they chose this form of presentation was to gain experience and confidence with preregistration and with Registered Reports. These results suggest that the attendees of BNA2019 acknowledged the importance of preregistration and that the event was helpful in promoting this understanding further, particularly for those presenting prereg posters.   Supporting early career researchers Although all scientists can benefit from their colleagues’ comments, these can be particularly beneficial to ECRs who are still developing their networks and expertise in the field. Because on most occasions (especially for ECRs), travel funding is only available when work is presented, and given that ECRs often have fewer existing datasets to present compared to more senior colleagues, prereg posters can provide ECRs additional opportunities to attend academic events. Therefore, the .CC-BY 4.0 International licensewas not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprint (whichthis version posted November 11, 2019. . https://doi.org/10.1101/833640doi: bioRxiv preprint 5   opportunity to present a prereg poster may be especially compelling for ECRs. Data from the present informal surveys that are relevant to this question are shown in Figure 2. While the data do not support strong claims regarding the relations between career stage and the choice to present a prereg poster, they do indicate that presenters of BNA2019 prereg posters had more limited research experience and had presented fewer posters throughout their career compared to traditional poster presenters.   -- Insert Figure 2 about here --  Conclusions and outlook The qualitative evidence reviewed above supports our prior claim that prereg posters can be a useful tool in promoting academic discussion of planned and on-going research, encouraging open science, and benefiting early career researchers. We hope this encourages the adoption of prereg posters by an increasing number of future scientific conferences. Notably, unlike some forms of study registration, whose main goal is to commit researchers to a registered plan, the main goal of prereg posters is to provide an opportunity for adjusting and improving studies before they are formally registered, following feedback from colleagues. Prereg posters can be routinely archived in a public repository, and then cited in any subsequent formal preregistrations that arise from the nascent protocol. This would capture the history and thus provenance of the idea generation. Indeed, it may be helpful to distinguish between “unlocked preregistration”, which refers to research plans that are presented as posters or uploaded to a public website, but are still subject to change, and “locked (pre)registration”, such as a registered report, which refers to a finalised research plan.  As we previously discussed10, and as with any new initiative, we believe it is vital to instruct all the people involved in conferences (organisers, presenters, poster reviewers and attendees) about the aim and value of these posters, otherwise the initiative might flounder (e.g, if reviewers score prereg posters lower owing to their lack of results, because the reviewers were not properly informed of the aims). Additionally, at least until they become more customary, we recommend highlighting prereg posters in the conference program and at the stand (e.g., by open science badges, as done at BNA2019). Further practical advice and useful tips can be found on the BNA website: www.bna.org.uk/mediacentre/news/pre-reg-posters/. Importantly, we believe that conferences and academic events should not only support specific scientific topics, but also act as venues that increase the quality of scientific research.  .CC-BY 4.0 International licensewas not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprint (whichthis version posted November 11, 2019. . https://doi.org/10.1101/833640doi: bioRxiv preprint 6    References 1. Baker, M. Nature 533, 452-455 (2016). 2. Munafò, M.R. Nicotine Tob. Res. 19, 773-773 (2017). 3. Munafò, M.R., et al. Nat Hum Behav. 1, 1-9 (2017). 4. Chambers, C.D. Cortex 49, 609-610 (2013). 5. Chambers, C.D., Dienes, Z., McIntosh, R.D., Rotshtein, P., & Willmes, K. Cortex 66, A1-A2 (2015).    6. The COS Team. Retrieved from: https://cos.io/rr/. (2013) 7. Rosenthal, R. Psychol. Bull. 86, 638-641 (1979). 8. Nosek, B. A., et al. Science 348, 1422-1425 (2015). 9. Claesen, A., Gomes, S.L.B.T., Tuerlinckx, F., & Vanpaemel, W. Preprint at https://psyarxiv.com/d8wex (2019). 10. Tibon, R., CBU Open Science Committee & Henson, R. Trends Cogn Sci 22, 271-274 (2018).    Acknowledgements Authors KB and AC are/were employed by the British Neuroscience Association (BNA) and its credibility in neuroscience programme, supported by the Gatsby Charitable Foundation (GAT3674), and played a key role in the design, conceptualization, data collection, analysis, and preparation of this manuscript. The BNA further provided the venue for data collection (at the BNA2019 Neuroscience Festival). Other funders contributed to the time of the authors, namely a British Academy Postdoctoral Fellowship (SUAI/028 RG94188) to RT, UK Medical Research Council grant (SUAG/010 RG91365) to RH, and ERC Consolidator grant (647893-CCT) to CDC, but had no role in the conceptualization, data collection and analysis, decision to publish or preparation of the manuscript.  Author contribution K.B: designed, planned, and carried out the survey; collected the data; provided input on the paper. A.C: conceived the idea; designed and planned the survey; provided input on the paper. C.D.C: provided input on the paper. R.H: directed the project together with R.T; commented on structure and analysis approach since early drafts; provided input on the paper. R.T: directed the project together with R.H; analysed the data; wrote the paper with input from all authors; shared the data on OSF. .CC-BY 4.0 International licensewas not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprint (whichthis version posted November 11, 2019. . https://doi.org/10.1101/833640doi: bioRxiv preprint 7    Competing interests The authors declare no competing interests  Figure captions Figure 1: Kind of feedback received for posters Distribution of responses to the post-conference survey question “what kind of feedback did you receive”, among presenters of prereg posters (left) and traditional posters (right). Nr represents the total number of responses. Figure 2. Indications of presenters’ career stage Distribution (around the median) of the years of research experience (left) and number of presented posters in academic conferences (right) among the presenters of prereg posters (cyan) and traditional posters (amber). Within each plot, boxplots on the left represent data obtained from the pre-conference survey (N=200) and boxplots on the right represent data obtained from the post-conference survey (N=95) .CC-BY 4.0 International licensewas not certified by peer review) is the author/funder. It is made available under aThe copyright holder for this preprint (whichthis version posted November 11, 2019. . https://doi.org/10.1101/833640doi: bioRxiv preprint 

Evidence for prereg posters as a platform for preregistration

Apollo (Cambridge)

1   Evidence for prereg posters as a platform for preregistration Kimberly Brouwers1, Anne Cooke2, Christopher D. Chambers3, Richard Henson4,5 & *Roni Tibon4 1School of Medicine, Medical Sciences and Nutrition, University of Aberdeen, UK, AB24 3FX 2British Neuroscience Association, The Dorothy Hodgkin Building, University of Bristol, Whitson Street, Bristol, UK, BS1 3NY  3Cardiff University Brain Research Imaging Centre (CUBRIC), School of Psychology, Cardiff University, UK, CF10 3AT 4MRC Cognition & Brain Sciences Unit, University of Cambridge, UK, CB2 7EF 5Department of Psychiatry, University of Cambridge, UK, CB2 0SZ  *Corresponding author:  Dr. Roni Tibon MRC Cognition & Brain Sciences Unit University of Cambridge 15 Chaucer Road Cambridge, CB2 7EF UNITED KINGDOM Telephone: +44 1223 355 294 Email: roni.tibon@mrc-cbu.cam.ac.uk  Standfirst Prereg posters are conference posters that present planned scientific projects. We provide preliminary evidence for their value in receiving constructive feedback, promoting open science, and supporting early career researchers.    2   In recent years, it has been repeatedly shown that the results of many scientific studies fail to replicate1,2,3. The growing awareness of this problem has prompted several attempts to tackle it. One way is to submit a registered report to a journal, which is peer-reviewed and has the opportunity to be revised before any data are collected3,4,5,6. Not only does the peer-review potentially improve the study design and analyses, but once accepted, the design and analyses are “locked”, preventing researchers from adjusting their hypotheses or analyses post hoc, i.e., after having seen the data (reporting such additional analyses is allowed, but only when transparently labelled as “exploratory”). Moreover, the journal is committed to publishing the results, whatever the outcome, which reduces the “file-drawer” problem of many null results not being published7. Registered reports, first offered by the journal Cortex, are now supported by over 200 peer-reviewed journals, including Nature Human Behaviour. Another approach is to preregister studies on a publically-accessible platform such as the Open Science Framework6,8. Though they can be locked, such preregistered studies are not normally peer-reviewed prior to data collection, and they are not binding for any subsequent publications. Indeed, substantial deviations from preregistered plans have been observed9. Two likely reasons for the large number of deviations are immature preregistration (e.g., of a protocol that was not yet finalised) and acceptance of post-preregistration feedback (e.g., following presentation of data at a conference). Recently, we proposed another possible avenue to help preregistration, namely presenting posters at academic conferences about planned research, before data are collected10; so-called “prereg posters”. This allows presenters to receive feedback on their hypotheses, design and analyses from their colleagues (conference attendees), which is likely to improve the study. In turn, this can improve more formal preregistration, reducing the chances of subsequent deviation, and/or facilitate submission of the work as a registered report. Moreover, colleagues with shared scientific interests become aware of the study early on, which can open the door to collaborations. Prereg posters were recently adopted by the BNA2019 Festival of Neuroscience (https://meetings.bna.org.uk/bna2019/), a biennial event organised by the British Neuroscience Association (BNA). Such posters have also been allowed by at least three other conferences since then: FLUX (https://fluxsociety.org/2019-new-york), BACN (https://www.bacn.co.uk/conferences) and ICON (https://www.helsinki.fi/en/conferences/international-conference-of-cognitive-neuroscience-2020). The BNA2019 Festival organisers additionally collected informal survey data about prereg posters, which we report here as preliminary evidence for their value.  Nearly a fifth (100/491) of all submitted posters at the BNA2019 Festival conformed to the new prereg format, covering a diverse range of neuroscience topics and disciplines, and the overall impression 3   was that they were enthusiastically welcomed. The informal survey data were collected at two time points (before and after the event), via two different online surveys. The full set of questions and response data are available here: https://osf.io/3h6w8/. The first informal survey (pre-conference survey) was administered three months prior to the event, and offered to all 445 participants whose poster was accepted. Of the 200 who responded, 151 were presenters of traditional posters and 49 of prereg posters. The second informal survey (post-conference survey) was administered one month after the event, and was completed by 95 participants, 66 of whom were presenters of traditional posters, and 29 of prereg posters. Below, we present data from the questionnaire items that address three main themes: prereg posters as means to (1) receive valuable feedback, (2) promote open science, and (3) support early career researchers (ECRs).  Getting valuable feedback  As a presenter, the motivation to present a prereg poster is clear: it allows one to get feedback on work at early stages. Indeed, 33% of the 123 responses to the pre-conference question “why did you submit a prereg abstract instead of a traditional abstract?” indicated that this choice was made in order to get feedback (either in general or on specific analyses) before completing the research. As shown in Figure 1, the groups differed in the type of feedback that they subsequently received at the conference. Interestingly, presenters of traditional posters mostly received feedback regarding future projects, whereas presenters of prereg posters mostly received feedback regarding methods and experimental design.   -- Insert Figure 1 about here --  Furthermore, the groups gave similar estimations of the likelihood of the presented research being published in a peer-reviewed journal upon completion (prereg: Median = 6 [scale: 1 – “highly unlikely” to 7 – “highly likely”], IQR = 2; traditional: Median = 6, IQR = 2). This is even though projects presented as prereg posters were at their early stages, and in most cases were still prior to data collection. While it could be the case that researchers’ estimation of publication likelihood remains stable throughout the course of a project, we speculate that it could also reflect the presenters’ trust in the preregistration system, where publication does not depend on results, and/or the presenters’ impression that they received valuable feedback that would promote their ability to publish the work later.   4   One potential objection to prereg posters is that conference attendees may not bother to discuss “half-baked” work and therefore there is no point in “wasting” conference space on posters that would not attract visitors. However, responses to two questions in the post-conference survey—“did you receive feedback on your work during the poster session?” and “how many people talked to you about your poster during the poster session?”—suggest that this is not the case: prereg posters did not receive less feedback than traditional posters (prereg = 75%; traditional = 68%), nor fewer visitors (prereg: Median category = 2 [5-8 people], IQR = 1; traditional: Median Category = 2 [5-8 people], IQR = 1), suggesting that conference attendees do not avoid prereg posters, despite the fact that these posters present planned or preliminary work.  Promoting open science  Our second theme concerns attitudes among the participants towards open science in general, and preregistration in particular. Two questions from the post-conference survey were indicative of such attitudes: (1) “to what degree would you say that preregistration of work is necessary in today's neuroscience community?” and (2) “to what degree did BNA2019 increase your awareness of preregistration?”. In general, presenters of both prereg and traditional posters agreed that preregistration is necessary (prereg: Median = 4 [scale: 1 - “not at all” to 5 – “incredibly necessary”], IQR = 1; traditional: Median = 4, IQR = 2). Moreover, participants in both groups indicated that the event increased their awareness of preregistration, and the increment was somewhat greater for prereg poster presenters (prereg: Median = 6 [scale: 1 - “not at all” to 7 – “completely”], IQR = 2; traditional: Median = 5, IQR = 2.75). Furthermore, in the pre-conference survey, 61% of prereg posters submitters indicated that one reason they chose this form of presentation was to gain experience and confidence with preregistration and with Registered Reports. These results suggest that the attendees of BNA2019 acknowledged the importance of preregistration and that the event was helpful in promoting this understanding further, particularly for those presenting prereg posters.   Supporting early career researchers Although all scientists can benefit from their colleagues’ comments, these can be particularly beneficial to ECRs who are still developing their networks and expertise in the field. Because on most occasions (especially for ECRs), travel funding is only available when work is presented, and given that ECRs often have fewer existing datasets to present compared to more senior colleagues, prereg posters can provide ECRs additional opportunities to attend academic events. Therefore, the 5   opportunity to present a prereg poster may be especially compelling for ECRs. Data from the present informal surveys that are relevant to this question are shown in Figure 2. While the data do not support strong claims regarding the relations between career stage and the choice to present a prereg poster, they do indicate that presenters of BNA2019 prereg posters had more limited research experience and had presented fewer posters throughout their career compared to traditional poster presenters.   -- Insert Figure 2 about here --  Conclusions and outlook The qualitative evidence reviewed above supports our prior claim that prereg posters can be a useful tool in promoting academic discussion of planned and on-going research, encouraging open science, and benefiting early career researchers. We hope this encourages the adoption of prereg posters by an increasing number of future scientific conferences. Notably, unlike some forms of study registration, whose main goal is to commit researchers to a registered plan, the main goal of prereg posters is to provide an opportunity for adjusting and improving studies before they are formally registered, following feedback from colleagues. Prereg posters can be routinely archived in a public repository, and then cited in any subsequent formal preregistrations that arise from the nascent protocol. This would capture the history and thus provenance of the idea generation. Indeed, it may be helpful to distinguish between “unlocked preregistration”, which refers to research plans that are presented as posters or uploaded to a public website, but are still subject to change, and “locked (pre)registration”, such as a registered report, which refers to a finalised research plan.  As we previously discussed10, and as with any new initiative, we believe it is vital to instruct all the people involved in conferences (organisers, presenters, poster reviewers and attendees) about the aim and value of these posters, otherwise the initiative might flounder (e.g, if reviewers score prereg posters lower owing to their lack of results, because the reviewers were not properly informed of the aims). Additionally, at least until they become more customary, we recommend highlighting prereg posters in the conference program and at the stand (e.g., by open science badges, as done at BNA2019). Further practical advice and useful tips can be found on the BNA website: www.bna.org.uk/mediacentre/news/pre-reg-posters/. Importantly, we believe that conferences and academic events should not only support specific scientific topics, but also act as venues that increase the quality of scientific research.  6    References 1. Baker, M. Nature 533, 452-455 (2016). 2. Munafò, M.R. Nicotine Tob. Res. 19, 773-773 (2017). 3. Munafò, M.R., et al. Nat Hum Behav. 1, 1-9 (2017). 4. Chambers, C.D. Cortex 49, 609-610 (2013). 5. Chambers, C.D., Dienes, Z., McIntosh, R.D., Rotshtein, P., & Willmes, K. Cortex 66, A1-A2 (2015).    6. The COS Team. Retrieved from: https://cos.io/rr/. (2013) 7. Rosenthal, R. Psychol. Bull. 86, 638-641 (1979). 8. Nosek, B. A., et al. Science 348, 1422-1425 (2015). 9. Claesen, A., Gomes, S.L.B.T., Tuerlinckx, F., & Vanpaemel, W. Preprint at https://psyarxiv.com/d8wex (2019). 10. Tibon, R., CBU Open Science Committee & Henson, R. Trends Cogn Sci 22, 271-274 (2018).    Acknowledgements Authors KB and AC are/were employed by the British Neuroscience Association (BNA) and its credibility in neuroscience programme, supported by the Gatsby Charitable Foundation (GAT3674), and played a key role in the design, conceptualization, data collection, analysis, and preparation of this manuscript. The BNA further provided the venue for data collection (at the BNA2019 Neuroscience Festival). Other funders contributed to the time of the authors, namely a British Academy Postdoctoral Fellowship (SUAI/028 RG94188) to RT, UK Medical Research Council grant (SUAG/010 RG91365) to RH, and ERC Consolidator grant (647893-CCT) to CDC, but had no role in the conceptualization, data collection and analysis, decision to publish or preparation of the manuscript.  Author contribution K.B: designed, planned, and carried out the survey; collected the data; provided input on the paper. A.C: conceived the idea; designed and planned the survey; provided input on the paper. C.D.C: provided input on the paper. R.H: directed the project together with R.T; commented on structure and analysis approach since early drafts; provided input on the paper. R.T: directed the project together with R.H; analysed the data; wrote the paper with input from all authors; shared the data on OSF. 7    Competing interests The authors declare no competing interests  Figure captions Figure 1: Kind of feedback received for posters Distribution of responses to the post-conference survey question “what kind of feedback did you receive”, among presenters of prereg posters (left) and traditional posters (right). Nr represents the total number of responses. Figure 2. Indications of presenters’ career stage Distribution (around the median) of the years of research experience (left) and number of presented posters in academic conferences (right) among the presenters of prereg posters (cyan) and traditional posters (amber). Within each plot, boxplots on the left represent data obtained from the pre-conference survey (N=200) and boxplots on the right represent data obtained from the post-conference survey (N=95) 

Evidence for prereg posters as a platform for preregistration.

https://www.repository.cam.ac.uk/bitstream/handle/1810/304550/preregPosters_acceptedVersion.pdf?sequence=1&isAllowed=y

Evidence for prereg posters as a platform for preregistration.

Abstract

Similar works

Full text

Available Versions

Online Research @ Cardiff

Apollo (Cambridge)