Appendix 10 Diatoms-user-results

Online Appendix 10. Diatom user score

Appendix 12 Lilies-user-results

Online Appendix 12. Lilies user scores

Appendix 15 all-users-results

line Appendix 15. Combined results for all users. Details for Figure 3

Appendix 20 Instructions to Crowd

Online Appendix 20. Instruction sheets for the study participants (undergraduate students at Ohio State University)

Appendix 5 Bats-character-taxon-results

Online Appendix 5. Bats character scores

Appendix 16 joint-predicted-difficulty

Online Appendix 16. Joint predicted difficulty. Predicted and Observed difficulty of each character based on a linear model fit to all of the character score data. Details for Figure 5

Appendix 3 Anemones-character-taxon-results

Online Appendix 3. Sea anemones character scores. For each character and taxon in the anemones matrix, we show the probability (“Estimate”) that a crowd member’s score would agree with the majority vote of the crowd. We also show the lower confidence interval on this probability (ci.lower), which is the crowd confidence score. Finally, we indicate whether the majority vote was correct, and compute an ROC curve for the crowd’s scores. The Threshold Plot worksheet provides a visualization of this information

Appendix 1 - List of six matrices developed for this study.

Online Appendix 1. List of six matrices developed for this study, including character names, types of characters, and assessment of character difficulty by experts. See also Morphobank (www.morphobank.org) projects P950, P2463, P2502, P2490, P2491, and P2577

Appendix 6 Bats-users-results

Online Appendix 6. Bats user scores

Appendix 18 final-results-summary_MAH edited

Online Appendix 18. Final results. Details for Figure 3