Reproducibility in Management Science

With the help of more than 700 reviewers, we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hardware and software requirements were not an obstacle for reviewers, the results of more than 95% of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29% of articles, at least part of the data set was not accessible to the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68%. These figures represent a significant increase compared with the period before the introduction of the disclosure policy, where only 12% of articles voluntarily provided replication materials, of which 55% could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in data set accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, and software and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies and suggest potential avenues for enhancing their effectivenes

Reproducibility in Management Science

With the help of more than 700 reviewers we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hard- and software requirements were not an obstacle for reviewers, the results of more than 95% of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29% of articles at least part of the dataset was not accessible to the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68%. These figures represent a significant increase compared to the period before the introduction of the disclosure policy, where only 12% of articles voluntarily provided replication materials, out of which 55% could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in dataset accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, but also soft- and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies, and suggest potential avenues for enhancing their effectiveness

Strategic environment effect and communication

We study the interaction of the effects of the strategic environment and communication on the observed levels of cooperation in two-person finitely repeated games with a Pareto-inefficient Nash equilibrium and replicate previous findings that point to higher levels of tacit cooperation under strategic complementarity than under strategic substitutability. We find that this is not because of differences in the levels of reciprocity as previously suggested. Instead, we demonstrate that slow learning coupled with noisy choices may drive this effect. When subjects are allowed to communicate in free-form online chats before making choices, cooperation levels increase significantly to the extent that the difference between strategic complements and substitutes disappears. A machine-assisted natural language processing approach then shows how the content of communication is dependent on the strategic environment and cooperative behavior, and indicates that subjects in complementarity games reach full cooperation by agreeing on gradual moves toward it.This version of the article has been accepted for publication, after peer review (when applicable) and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/s10683-022-09774-7

Strategic environment effect and communication

This version of the article has been accepted for publication, after peer review (when applicable) and is subject to Springer Nature’s AM terms of use, but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://doi.org/10.1007/s10683-022-09774-7.We study the interaction of the effects of the strategic environment and communication on the observed levels of cooperation in two-person finitely repeated games with a Pareto-inefficient Nash equilibrium and replicate previous findings that point to higher levels of tacit cooperation under strategic complementarity than under strategic substitutability. We find that this is not because of differences in the levels of reciprocity as previously suggested. Instead, we demonstrate that slow learning coupled with noisy choices may drive this effect. When subjects are allowed to communicate in free-form online chats before making choices, cooperation levels increase significantly to the extent that the difference between strategic complements and substitutes disappears. A machine-assisted natural language processing approach then shows how the content of communication is dependent on the strategic environment and cooperative behavior, and indicates that subjects in complementarity games reach full cooperation by agreeing on gradual moves toward it

Classification Aggregation without Unanimity

A classification is a surjective mapping from a set of objects to a set of categories. A classification aggregation function aggregates every vector of classifications into a single one. We show that every citizen sovereign and independent classification aggregation function is essentially a dictatorship. This impossibility implies an earlier result of Maniquet and Mongin (2016), who show that every unanimous and independent classification aggregation function is a dictatorship. The relationship between the two impossibilities is reminiscent to the relationship between Wilson's and Arrow's impossibilities in preference aggregation. Moreover, while the Maniquet-Mongin impossibility rests on the existence of at least three categories, we propose an alternative proof technique that covers the case of two categories, except when the number of objects is also two. We also identify all independent and unanimous classification aggregation functions for the case of two categories and two objects

Reproducibility in Management Science

FNEGE 1*, HCERES A, ABS 4*International audienceWith the help of more than 700 reviewers, we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hardware and software requirements were not an obstacle for reviewers, the results of more than 95% of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29% of articles, at least part of the data set was not accessible to the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68%. These figures represent a significant increase compared with the period before the introduction of the disclosure policy, where only 12% of articles voluntarily provided replication materials, of which 55% could be (largely) reproduced. Substantial heterogeneity in reproducibility rates across different fields is mainly driven by differences in data set accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, and software and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies and suggest potential avenues for enhancing their effectiveness.A complete list of the members of the Management Science Reproducibility Collaboration is included in Online Appendix A

Reproducibility in management science

With the help of more than 700 reviewers, we assess the reproducibility of nearly 500 articles published in the journal Management Science before and after the introduction of a new Data and Code Disclosure policy in 2019. When considering only articles for which data accessibility and hardware and software requirements were not an obstacle for reviewers, the results of more than 95% of articles under the new disclosure policy could be fully or largely computationally reproduced. However, for 29% of articles, at least part of the data set was not accessible to the reviewer. Considering all articles in our sample reduces the share of reproduced articles to 68%. These figures represent a significant increase compared with the period before the introduction of the disclosure policy, where only 12% of articles voluntarily provided replication materials, of which 55% could be (largely) reproduced. Substantial het-erogeneity in reproducibility rates across different fields is mainly driven by differences in data set accessibility. Other reasons for unsuccessful reproduction attempts include missing code, unresolvable code errors, weak or missing documentation, and software and hardware requirements and code complexity. Our findings highlight the importance of journal code and data disclosure policies and suggest potential avenues for enhancing their effectiveness