A Novel Tool for Online Community Moderator Evaluation

This study introduces a new instrument for leadership evaluation in online forums and other online communities which was developed using a grounded approach. Questions that emerged from the literature were then evaluated to create hypotheses that guided the development of an instrument for moderator evaluation. The Moderator Evaluation Contingency Scale (MECS) is modified from Fiedler’s contingency model to determine if a moderator is more task- or relationship-oriented in his or her approach to moderation and interactions with other members of a community. The MECS was developed and tested on Reddit in 2013–2014 using random sampling for Forum selection, moderator selection, and interactions with users. A content analysis using the MECS to evaluate posts was found to be a viable measure of a moderator’s ability to perform tasks like removing content as well as his or her ability to interact with users. Bots were analyzed using the MECS as well to determine bias. Next steps include making the instrument available for use by social media and niche community sites, administrators, and other moderators