Design of a Prediction Game in the Domain of Computer Security

Prediction Games are games where players analyze historical data and make predictions about future events. The predictions are scored which gives the players an idea of where they stand. This game is appropriate for domains where data is coming in frequently and where there is a decent quantity of historical data for participants to explore. Knowledge in Computer Security carries high value personally and professionally and statistics in this data domain is collected by many organizations for various reasons. This thesis explores the design of a prediction game in the field of Computer Security. The goals of this project include identifying data sets that could be used for a prediction game, designing a prediction activity which will be helpful to players, and developing a prototype version of the prediction game. A heuristic evaluation of the prototype will provide feedback for improvements to the game mechanics such as user interface and data visualizations. At the end of the project, there will be a greater understanding of the availability of computer security data, how it can be used for developing prediction games, and tradeoffs in the design of computer security prediction games

Prediction Games: Encouraging Engagement with Data

Prediction games, data-driven games modeled after fantasy sports, are aimed to motivate people to explore, analyze, and develop their own understanding of large data sets. They revolve around activities where players examine historical data and information resources to make predictions about future events. As a result, they may help improve the players’ domain knowledge and data interpretation skills. But what matters in the design of such games? And, as we envision prediction games created by instructors in an educational environment, what forms of support aid the authoring of prediction activities yet involve very little to no programming? To answer these questions, we first conducted a survey of fantasy sports players which showed that many seek out information including news and data. They analyze this content to make predictions, resulting in them learning more about the sport. Next, we developed Fantasy Forecaster, a prediction game prototype to gather system requirements and user feedback. Lessons from the survey and development of the prototype informed our prediction games framework and its implementation in the climate domain: Fantasy Climate. Fantasy Climate is a prediction game based on weather data where players select a location among a set of choices based on whether their assessment of upcoming weather. In particular, they are asked to select which location will be warmest and coolest compared to their historic norms on an upcoming date. The game also featured communication tools, integrated climate-related news, and historical weather data with visualizations to make sense of them. User studies of Fantasy Climate revealed that social interaction, particularly asynchronous discussions made the game more engaging and helped players gather information for prediction making. Also, the in-game presentation of domain-related news had an effect on engagement and players' performance. From our prediction games framework and the implementation of Fantasy Climate, we identified a set of necessary and valuable prediction activity specifications which led to the development of the Activity Creation Wizard (ACW). The ACW is an environment that guides the author through a series of steps to author their prediction activity. Features of the ACW included a help system that provides the author with explanations, tutorials and examples during the authoring process. Also included were a template component that allows the author to reuse the customizations of a previously created prediction activity, and tools to automate repetitive and tedious tasks such as building the prediction schedule. The evaluation of the ACW showed no background knowledge was required to use the ACW to author a prediction activity. The help system was in general adequate in assisting the participants in their information needs, templates were found useful by many, and automation reduced the time taken for repetitive tasks. Some authors did not want to use templates or automation in order to have more control over the design of their activity. However, the help system, templates, and automation tools of the ACW were not sufficient in helping the participants understand the consequences of their customization on the prediction activity. Reasoning about the effects of their choices on gameplay was noted as the primary challenge during the authoring task by several participants. Additionally, the evaluation identified alternative ways of authoring the prediction activity that challenged our current design of the ACW, including the potential value of co-dependent customizations and collaborative authoring. Finally, the ACW evaluation also involved a task where participants created a prediction game in the domain of their choice. Interviews with participants on their created prediction games revealed two major findings. One finding was that educational, social, and socio-cultural factors play an important role in what makes prediction games engaging. The other finding was authoring resulted in a recognition by the participants of the educational benefits of prediction games which align well with the primary motives of this research work

Effect of Visualization of News Articles in Data Driven Games

The popularity of prediction games such as fantasy sports has been on the rise and the amount of data available for the players to make the prediction in such games is growing rapidly. Prediction games, an area of data driven games require the user to interpret archival data along with real-time data from a domain to make a prediction about a future event. This work is being done in the context of a prediction game where players select geographic locations as part the game. News articles can serve as a source of information and have the potential to improve engagement and learning. To make sense of the millions of news artifacts published online is not possible manually. Moreover, keyword-based search is very limited when it comes to exploring data rather than just searching for something particular. The proposed work will develop a visualization and user interface to represent the news articles in an activity-appropriate manner and allow the users to explore news data using approximate search along with keyword-based search. The first component of this system extracts news articles related to the game, then geotags and clusters them based on the geographic references in the articles. Displaying these clusters on a map takes advantage of the spatially referenced news data. This thesis will compare alternative visualizations of the geo-tagged and clustered news articles and their value to players of the game. A user study was conducted to evaluate the visualizations and their effect on the engagement of the players with a data driven game. The results show that the map visualization is very effective in engaging players with the game when compared to the regular list form of news representation. Moreover, the overall performance of the players who used the map visualization was better than the performance of the users who used the list visualization. Future work will explore more on fine-tuning data sources which provide the input to the map visualization as well as variations in the display and accessible features on the map interface to enable users to control data visualization according to their imagination and preference