Research Methods for Education With Technology: Four Concerns, Examples, and Recommendations

The success of education with technology research is in part because the field draws upon theories and methods from multiple disciplines. However, drawing upon multiple disciplines has drawbacks because sometimes the methodological expertise of each discipline is not applied when researchers conduct studies outside of their research training. The focus here is on research using methods drawn largely from psychology, for example, evaluating the impact of different systems on how students perform. The methodological concerns discussed are: low power; not using multilevel modeling; dichotomization; and inaccurate reporting of the numeric statistics. Examples are drawn from a recent set of proceedings. Recommendations, which are applicable throughout the social sciences, are made for each of these

Modeling the Incubation Effect Among Students Playing an Educational Game for Physics

We attempted to model the Incubation Effect, a phenomenon in which a momentary break helps the generation of a solution to a problem, among students playing Physics Playground. We performed a logistic regression analysis to predict the outcome of the incubation using a genetic algorithm for feature selection. Out of 14 candidate features, those that significantly predicted the outcome were total badges earned prior to post-incubation, the problem’s level of difficulty, total attempts made prior to post-incubation, and time interval of post-incubation. We found evidence that incubation in the earlier part of the game is more beneficial than breaks at the later part where students may already be mentally exhausted

Modeling the incubation effect among students playing an educational game for physics

This research investigated the phenomenon called Incubation Effect (IE) in the context of Physics Playground, a computer-based learning environment, and extracted features that would predict the incidence of revisiting an unsolved problem and its positive outcome. A logistic regression model was developed and found coarse-grained level features that predict IE such as time of revisit, students productivity, and problem difficulty. Fine-grained analysis used LSTM, a deep learning technique, which improved the performance of the IE model. A combination of a dimension reduction and visualization technique called T-SNE and X-means clustering were used to interpret the learned features and found that the coarse-grained features are consistent with the fine-grained features but action level features were also discovered such as higher incidence of erase and hover tutorial, lower incidence of pause, and improvement in drawing ramp and springboard during the revisit after incubation. These features were discussed and how they could be translated into game mechanics that could improve students performance in computer-based learning environments