Criteria and Scrutiny in Computing Education Research

In 2020, a working group was convened at the Innovation and Technology in Computer Science Education (ITiCSE) conference, led by Marian Petre, Kate Sanders and Robert McCartney on Mapping the Landscape of Peer Review in Computing Education Research (CER). The working group considered 17 venues, including CER conferences and journals, as well as overlapping conferences in Software Engineering and Human Factors. In this presentation, we consider some of the common review criteria observed across venues as well as some of the ethical concerns that emerge in peer-review and the process itself. These elements are considered through the lens of excerpts and vignettes drawn from conference chairs and journal editors interviewed by the working group that reflect aspects of the conversations and debates that have happened during the week at the present seminar

Game Development for Computer Science Education (Extended Abstract)

Educators have long used digital games as platforms for teaching. Games tend to have several qualities that aren’t typically found in homework: they situate problems within a compelling alternate reality that unfolds through intriguing narrative, they draw more upon a player’s intrinsic motivations than extrinsic ones, they facilitate deliberate low intensity practice, and they emphasize a spirit of play instead of work. At ITiCSE 2016, this working group convened to survey the landscape of existing digital games that have been used to teach and learn computer science concepts. Our group discovered that these games lacked explicitly defined learning goals and even less evaluation of whether or not the games achieved these goals. As part of this process, we identified and played over 120 games that have been released or described in literature as means for learning computer science concepts. In our report, we classified how these games support the learning objectives outlined in the ACM/IEEE Computer Science Curricula 2013. While we found more games than we expected, few games explicitly stated their learning goals and even fewer were evaluated for their capacity to meet these goals. Most of the games we surveyed fell into two categories: short-lived proof-of-concept projects built by academics or closed-source games built by professional developers. Gathering adequate learning data is challenging in either situation. Our original intent for the second year of our working group was to prepare a comprehensive framework for collecting and analyzing learning data from computer science learning games. Upon further discussion, however, we decided that a better next step is to validate the design and development guidelines that we put forth in our final report for ITiCSE 2016. We extend this working group to a second year—with a mission to collaboratively develop a game with clearly defined learning objectives and define a methodology for evaluating its capacity to meet its goals

Identification of genetic variants associated with Huntington's disease progression: a genome-wide association study

Background Huntington's disease is caused by a CAG repeat expansion in the huntingtin gene, HTT. Age at onset has been used as a quantitative phenotype in genetic analysis looking for Huntington's disease modifiers, but is hard to define and not always available. Therefore, we aimed to generate a novel measure of disease progression and to identify genetic markers associated with this progression measure. Methods We generated a progression score on the basis of principal component analysis of prospectively acquired longitudinal changes in motor, cognitive, and imaging measures in the 218 indivduals in the TRACK-HD cohort of Huntington's disease gene mutation carriers (data collected 2008–11). We generated a parallel progression score using data from 1773 previously genotyped participants from the European Huntington's Disease Network REGISTRY study of Huntington's disease mutation carriers (data collected 2003–13). We did a genome-wide association analyses in terms of progression for 216 TRACK-HD participants and 1773 REGISTRY participants, then a meta-analysis of these results was undertaken. Findings Longitudinal motor, cognitive, and imaging scores were correlated with each other in TRACK-HD participants, justifying use of a single, cross-domain measure of disease progression in both studies. The TRACK-HD and REGISTRY progression measures were correlated with each other (r=0·674), and with age at onset (TRACK-HD, r=0·315; REGISTRY, r=0·234). The meta-analysis of progression in TRACK-HD and REGISTRY gave a genome-wide significant signal (p=1·12 × 10−10) on chromosome 5 spanning three genes: MSH3, DHFR, and MTRNR2L2. The genes in this locus were associated with progression in TRACK-HD (MSH3 p=2·94 × 10−8 DHFR p=8·37 × 10−7 MTRNR2L2 p=2·15 × 10−9) and to a lesser extent in REGISTRY (MSH3 p=9·36 × 10−4 DHFR p=8·45 × 10−4 MTRNR2L2 p=1·20 × 10−3). The lead single nucleotide polymorphism (SNP) in TRACK-HD (rs557874766) was genome-wide significant in the meta-analysis (p=1·58 × 10−8), and encodes an aminoacid change (Pro67Ala) in MSH3. In TRACK-HD, each copy of the minor allele at this SNP was associated with a 0·4 units per year (95% CI 0·16–0·66) reduction in the rate of change of the Unified Huntington's Disease Rating Scale (UHDRS) Total Motor Score, and a reduction of 0·12 units per year (95% CI 0·06–0·18) in the rate of change of UHDRS Total Functional Capacity score. These associations remained significant after adjusting for age of onset. Interpretation The multidomain progression measure in TRACK-HD was associated with a functional variant that was genome-wide significant in our meta-analysis. The association in only 216 participants implies that the progression measure is a sensitive reflection of disease burden, that the effect size at this locus is large, or both. Knockout of Msh3 reduces somatic expansion in Huntington's disease mouse models, suggesting this mechanism as an area for future therapeutic investigation

Undergraduate game degree programs in the United Kingdom and United States: A comparison of the curriculum planning process

Digital games are marketed, mass-produced, and consumed by an increasing number of people and the game industry is only expected to grow. In response, post-secondary institutions in the United Kingdom (UK) and the United States (US) have started to create game degree programs. Though curriculum theorists provide insight into the process of creating a new program, no formal research contextualizes curriculum planning for game degree programs. The purpose of this research was to explore these processes when planning undergraduate game degree programs. The research methodology included an explanatory mixed-methods approach, using a quantitative survey of participants in the UK and the US, followed by an interview of several participants selected on the basis of their institution's demographics. The study provides insight into the curriculum planning process, including factors that influence the final program content, and a list of recommendations for educators, trade associations, and the games industry to improve game degree programs

Institutional support for computing faculty research productivity: does gender matter?

We address the question of how male and female computing faculty in the U.S. and Canada perceive research requirements and institutional support for promotion and tenure. Via a survey sent to approximately 7500 computing faculty at the 256 institutions that participate in the annual Taulbee Survey, our results identify differences in reported tenure and promotion requirements, including the number of publications required during the probationary period, the importance of the scope of publication venues, the importance of publishing in non-refereed journals, and the importance of collaborative presentations. Differences were also discovered in institutional support and the satisfaction levels with that support. The study finds that some misperceptions may exist about promotion and tenure requirements among female faculty while at the same time female faculty feel more supported by their institutions

Computing faculty tenure and promotion requirements at USA and Canadian post-secondary institutions

As a first step in understanding whether computing faculty are receiving the support necessary for them to achieve promotion and tenure at U.S. and Canadian institutions, we address the question of what requirements exist at such institutions. Via a survey created and sent to approximately 7500 computing faculty at the 256 institutions that participate in the annual Taulbee Survey, we examined characteristics of and differences in promotion and tenure requirements at U.S. and Canadian institutions. Our results identify several hidden tenure and promotion requirements that can be important for computing faculty to know.Our results also show significant differences in requirements for the number of publications, venue ranking and scope, refereed conferences and non-refereed journals, and collaborative publications and grants across various types of institutions. This work provides a basis for further study of whether faculty receive support congruent with their promotion and tenure requirements and allows computing administrators and faculty to compare institutional requirements to requirements in the broader community

Towards a Common Framework for Evaluating Computing Outreach Activities

In the past six years, dozens of conference papers and journal articles have been presented in Association of Computing Machinery (ACM) and Institute of Electrical and Electronics Engineers (IEEE) educational forums concerning computing outreach activities. Nearly half of these (47.5%) appeared in SIGCSE venues. In this study, we used the free-form question What type of data has been collected in formal, peer-reviewed research that has been conducted on computing outreach activities in recent years ? as a basis for a systematic literature review in these venues from 2009-2015. During the analysis of the articles, it was discovered that a majority of efforts focused on middle school and high school students, a majority of the reported events took place in the United States, and almost half had a goal of increasing gender diversity in computing. This paper summarizes the information about the studies, including their data collection techniques and the data that was collected. We also present a list of recommended practices for data collection, methodologies, and reporting for educational researchers engaged in these activities in an effort to provide comparative data and allow us as a community to more scientifically understand the impact that these activities are having on the participants

Demographics of undergraduates studying games in the United States: a comparison of computer science students and the general population

Our study gathered data to serve as a benchmark of demographics of undergraduate students in game degree programs. Due to the high number of programs that are cross-disciplinary with computer science programs or that are housed in computer science departments, the data is presented in comparison to data from computing students (where available) and the US population. Participants included students studying games at four nationally recognized postsecondary institutions. The results of the study indicate that there is no significant difference between the ratio of men to women studying in computing programs or in game degree programs, with women being severely underrepresented in both. Women, blacks, Hispanics/Latinos, and heterosexuals are underrepresented compared to the US population. Those with moderate and conservative political views and with religious affiliations are underrepresented in the game student population. Participants agree that workforce diversity is important and that their programs are adequately diverse, but only one-half of the participants indicated that diversity has been discussed in any of their courses

Computing outreach literature review

The lack of diversity in computing has existed for decades. It has garnered the attention of computing educators and private companies who have implemented a host of outreach and retention programs to draw more diverse students into the field and the workforce. A question that stands out is whether or not these programs are effective in the long term in helping to mitigate the lack of diversity in the field. To determine an answer to this question, the authors undertook a systematic literature review of reported computing outreach activities in relevant journals and conference proceedings for the years 2009- 2014 inclusive. Upon consideration of all relevant articles, 73 articles were coded for information about the type of outreach, target audience, and reported results. Summaries of the findings of the literature review are presented in this poster