All that glitters (in the lab) may not be gold (in the field)

Abstract. AI-ED community has hewed to rigorous evaluation of software tutors and their features. Most of these evaluations were done in-ovo or in-vivo. Can the results of these evaluations be replicated in in-natura evaluations? In our experience, the evidence for such replication has been mixed. We propose that the features of tutors that are found to be effective in-ovo/in-vivo might need motivational supports to also be effective in-natura. We speculate that some features may not transfer to in-natura use even with supports. Recognition of these issues might bridge the gap between AI-ED community and educational community at large